I'm working on better SIMD support in Dart and one of the motivations for that work is specializing parser specifications based on schema specifications of that language.
Or more concretely, if we had a parser generator + json parser specification that can generate efficient simdjson style parsers, then we should be able to parse json restricted to a subset of json based on a schema at least as efficiently as simdjson can parse general purpose json.
That way we could have the best of both worlds, a (potentially lazy) statically type-safe model and simdjson performance!
I still think that this is definitely doable, it's just that a lot of infrastructure needs to be built for it.
I remember reading that Meta is doing something similar with binary formats. I find it weird that there's nothing like that for json yet.
I was working on metaparsing earlier this year (specifically over grpc because of its better support for types and better serialization format).
I think most people who use JSON don't want strong typing or to use schemas. If you do, you just use a different serialization format: JSON is designed to be a generic, human-readable way to basically represent any kind of data structure (since it deserializes to JS objects). So everything about JSON is working against you if you do try to enforce type/schemas on it.
A grpc-like protocol with more prescriptive type registries/discovery (over the network) and better interop with http (like grpc-gateway) would be the best approach IMO. Definitely not JSON schema
> think most people who use JSON don't want strong typing or to use schemas. If you do, you just use a different serialization format: JSON is designed to be a generic, human-readable way to basically represent any kind of data structure (since it deserializes to JS objects). So everything about JSON is working against you if you do try to enforce type/schemas on it.
I have been using serde without issues, and never resorted to `serde::Value`[0], the untyped option, so far.
I have yet to experience anything working against me.
> I think most people who use JSON don't want strong typing or to use schemas.
Not necessarily. It's biggest advantage is probably that it has very good interoperability. Basically every language has a readily accessible library for json, and even if you don't, it isn't that hard to write a parser and/or generator for it. And being human readable isn't something to sneeze at either.
I think it may be fairer to say, “most people pick up JSON because of the flexibility.” Those who continue to use it for anything where there’s a consumer of your API probably do come to want types. In my day job I am starting to enforce end-to-end typing as best I can (given it’s Python and JSON) because our payloads are partially content from agents and I want to be able to flag up weird shit if we can. Certainly not the only gate that we have for looking for prompt injection and not even a great one, but it provides some piece of mind.
> I think most people who use JSON don't want strong typing or to use schemas.
Pretty much anywhere I've come across JSON (and xml before that,) everyone expected strong typing. The fact that JSON (and xml) can represent different data than they anticipated "did not compute" in their heads.
In many cases, they were treating JSON (and xml) as dumb serializers and not even paying attention to the output and corner cases.
I definitely want jsonschema, it's a standard and i can use it for many tools.
openapi spec is also a standard and built on top of jsonschema. I can use jsonschema to generate models, types, servers, client sdks, automate forms etc.
I think it's better than grpc for some of these things, plus protobufs and grpc have bunch of tooling that i can pretend to understand but in depth i don't really.
I generally like `encoding/json/v2` and I'm glad they did it.
I wish they'd left `MarshalIndent` in there though. The equivalent code is super complicated and non-obvious, so it's a pain to migrate, and it's a pain when you're banging out some quick code and want to dump some readable JSON. I suppose these days the LLMs can write the complicated version for us, of course.
I used the jsontext package at my company when we needed to make a parser for challenging clients (specific errors for each kind of mistake) while still keeping performance, and it works like a charm.
There is no introspection and speed is superb. I would definitely recommend jsontext as a parser for anything non-straightforward.
I'm just getting the neuron-band back together where Golang is concerned. I have done some in the past for little system tasks, but I am switching away from interpreted languages entirely for web stuff, and initially to Go, as I switch away from hand-coding to at least LLM-supported coding.
So I am a bit out of date on the pain points (I never really felt pain points when I last used it, because the jobs I was doing had very small, quite Go-friendly shapes).
The fact that this took 6 years to land stood out to me.
I'm not entirely surprised, and I'm glad the Go team is carefully considering their interfaces before launching. But 6 years seems like a lot of work to land a JSON API.
For like 99% of use cases encoding/json was fine. Of course improvements are welcomed. But personally I've lead development of the backend of part one of the biggest sites of its kind in the world, and we just used encoding/json and net/http. Most of our use cases are just building APIs, client facing ones, internal ones, trusted third-parties. These systems have pretty high throughput, more than most people will ever work on.
Within our company I saw people in other teams writing their own optimised JSON libraries for whatever hellish reason. I always reiterated the point that as soon as that app made a single IO request all those nanosecond savings are invisible. Yes, it affects your total CPU time for processing a request but compute was never ever a limitation for us and it's likely not a limitation for most people. And if the trade off is your app needs scale slightly larger versus writing custom JSON code or pulling in a third party dependency, then I'm accepting the larger scaling.
The biggest benefit of encoding/json/v2 isn't the speed, it's the improved API.
I think in the case of JSON, it was “good enough”. People who found this to be a problem would use one of the many 3rd party options.
I kind of like that the standard library doesn’t move fast. It makes it easier to fit the ecosystem into my head (unlike, for example, the JavaScript ecosystem where I find code doesn’t survive 2 years before failing to compile).
But I do agree that slow and steady produces its own problems. Your example of generics is a good one. But think the example of Go modules is another, perhaps even better, example of how sometimes the Go team really should have accelerated their efforts on.
lol, I don't mean to be contrary, but I looked into dependency management (partly because of the modules implementation, partly because of early teething issues, and partly out of pure curiousity.
Dependency management is a freaking nightmare, nobody has managed to create something that is bulletproof, not cargo, npm, apt, yum, nix, homebrew, nothing.
The key issue is that you're managing 3rd party software that may or may not adhere to the prescribed lifecycle/versioning
The Linux variants do it by each having dedicated teams watching upstream sources, taking what they believe to be whatever they feel is right, building for their nominated target(s), then testing, before deploying (on top of that, all of these steps vary in importance and rigor depending on numerous variables)
For programming libraries (which is what depmod, goose, vgo, and modules are managing) it's just as bad, if not harder, because their very nature is to provide a base upon which other developers can turn into whatever their imagination.
Add to that, modules was (initially) written by people who were dealing with a slightly different ecosystem; Google rather famously has a monorepo, making their lives a little easier when it comes to managing dependencies.
So, with those things in mind, dependency management is always going to be a hard problem that is opinionated (eg. I have a firm opinion that there should be an easier way to "pin" a dependency, other than vendoring - note that this isn't an invitation to every junior Go developer to come and claim go.sum does that. It doesn't. go.sum is a log of what dependencies were used for the last build by whomever wrote to it)
I’ve written package management tools and I agree that dependency management is hard. Though I wouldn’t say it’s that hard compared to the other problems you encounter with language development.
But difficulty aside, my point was Go waited until there were multiple established 3rd party package management tools for Go already in widespread use before even releasing their own experimental implementation.
Their inaction caused far more disruption to the ecosystem because people got fed up of waiting, implemented their own, integrated it into their workflow and then had to reintegrate a whole new system all over again a year or two later.
Your point about Googles needs is the one often cited for Gos inaction. But I think that actually makes the situation worse. Because if the Go team cannot empathise with use cases outside of one bespoke enterprise, then it really shouldn’t be released as a general purpose language available for all to use.
And I say this as someone who generally likes the language.
Interesting, I came to the opposite conclusions, but for the same reasons. I like that they waited to see what alternative proposals/implementations were out there before they decided on their own.
Especially since "Go team cannot empathise with use cases outside of one bespoke enterprise" — it shows self-awareness of their shorcomings. I think it's wise to let the target audience explore the design space themselves a bit first. Considerably lowers the risk of cementing the wrong design.
I do somewhat remember the drama around this at the time, but I my impression was that of a petulent community of various dependency management projects with too much ego attached to them to see the bigger, long-term picture.
> generics (something that I still haven't used beyond the tutorials) took an age as well
Only about a year from when the Haskell guys lent their expertise in designing it and it landing in a release. That's actually fairly quick given that it was a pretty significant change under the hood and constrained by only two releases annually.
The Go team was always clear that they didn't have the necessary capabilities internally, despite a lot of failed efforts trying anyway, and were relying on the open source community to lend a hand. Unfortunately, open source doesn't mean open willingness to contribute. It turns out that for all the experts on HN, posting "go doesn't have generics lol" was preferable to helping out.
Before Featherweight Go, there was Featherweight Java.
So maybe a language designed in 2012 (v 1.0 release year), should learn from history instead of hand-waving it away as needless fluff.
> In retrospect, we were biased too much by experience with C++ without concepts and Java generics. We would have been well-served to spend more time with CLU and C++ concepts earlier.
And not all on the team were convinced even after Featherweight Go, here Rob Pike,
> I think the transition to getting polymorphism into the language through generics is has still got some years to work through there's still things that don't quite work right
> instead of hand-waving it away as needless fluff.
If you go back and watch the original Go announcement it was made abundantly clear that they believed Go did need generics but that they didn't have the expertise to do it. Yes it is true that the book was already written on it, but that doesn't mean anyone on the Go team was familiar with it or had the time to become familiar with it. Programming languages do not require gods to design them. Imperfect humans can too.
The experts on HN who know all the ins and outs of generics could have contributed but, you know, shitposting "go doesn't have generics lol" was a better use of their time. There are so many hours in the day.
Gonuts is full of content that they were not keen in generics, in fact gccgo has been abandoned, and I expect it to eventually follow gcj and be removed from GCC.
Go culture is not welcoming to experts contributions, Rust, D, C#, Swift and others, heck even Java, are more open arms to language designers.
Nobody is keen about language features. Each new one adds complexity. That isn't the same as failing to recognize a need, though. And we know the Go team recognized the need as they clearly kept working on generics all the way from the very beginnings up until the Haskell muscle showed up. The receipts are there. The results just weren't good, as one would expect from a team who weren't shy to say that they didn't have the expertise necessary.
You'd think one of the HN experts would have shown up to offer that expertise, but I guess there was a lot of "go doesn't have generics lol" needing to get out.
Some did try, and were welcomed with stones, clubs and pitchforks, until leadership had to reach out for help, from the same team that designed Java generics, years before Go v1.0 came to materialise.
The Haskell muscle had proven their skills already with Featherweight Java, in 2006.
The same people who were working on generics in Go prior to v1 were also still there to help out when it eventually landed, so whatever stones, clubs, and pitchforks you saw weren't compelling enough to scare one away from making it happen. Not that anyone would expect them to. Meaningless distractions don't get in the way when doing something is worthwhile.
But obviously "go doesn't have generics lol" comments on HN were even more important. Just think of what would have happened if that wasn't repeated multiple times daily for years and that effort was put into getting Go generics into place much sooner. Count your lucky stars or something.
And there was plenty of effort given by others who also weren't experts. There was no lack of trying within a sea of incapable people. Unfortunately, that wasn't what the project needed. It already had a team full of blub developers, it being a language for them.
At this stage to me Go is the new C compiler. Quite honestly I spent 10 years or more writing Go code. At times 18 hours a day. And now with AI, I get models to write my Go code. I STILL want it to be Go code. Is that strange? No because I understand the performance and safety of Go. But I rarely write it directly anymore. Probably not at all in 6-9 months. Yet I still occasionally look at the code to see what the output looks like.
So yea hopefully a nice new addition but only my prompts now matter. So need to make sure they pickup on the new API.
Yes. It's that thing — the fact that a language helps you read what a single function does really easily, but in so doing makes the entire codebase quite dense, is not optimal for us, but is good for them.
And the things that we do to manage this, that are symbolic or based on visceral real-world analogy, are good for us and bad for them.
Go is kind of in the middle, in the way that many vaguely Wirth-like and CSP-like languages possibly are, I guess.
Efficient code means code with a large amount of embedded assumptions. Code that's more generic is almost always less efficient.
Go is about as far as you can "go" to the efficiency end without becoming Rust. It's a low-overhead language (like C) but with a GC that ensures memory safety. If you want to manage memory yourself and still prove safety it gets very complicated, as we see in Rust.
It always feels like OCaml is on the verge of becoming the perfect language.
Expressive and type safety of a functional language, without the odd semantics of Haskell, where laziness can make performance difficult to predict and purity can make simple things like IO difficult to understand.
Does it still lack easy to use concurrency primitives?
If I want simplicity and maintainability, I’d pick Kotlin over Go. If I want speed, I’m picking Rust over Go. Why would I ever pick Go? It’s the best at nothing.
It's really great fun to build stuff in Crystal, I don't have anything prod-grade yet to say how it scores there but the community seems to be happy about that too.
I agree, the compiler is slow but I don't mind much, or better, the fact that I find building maintainable programs much easier outweights that downside massively in my case.
Im not saying this is what's happening here, it could totally be just that he was burning himself out at work, but it's a truly incredible thing to find something that you love working on and want to dive into to the point that you lose hours at it and have to pull yourself away from it to go to sleep (obviously outside of stuff designed to be addictive and exploitative like gambling that hijack those reward mechanisms). Programming is one of those artistic technical pursuits and a lot of programmers love it in that way. And are grieving losing that with A. At least so far as you felt you could only justify spending time on something you loved like that if you were also paid for it. Nobody is stopping you from opening up a text editor again on your own time. But you know, it's good to be passionate about things. Not being able to sleep until you figure something out, losing track of time while you're lost in whatever you're working in - all that stuff where it's from innate drive and the love of the game? that stuff rules. If it's burnout throw it out but don't throw out the electric stuff that makes us human
I do personally find myself most productive with a cycle of 20 hours awake / 12 hours asleep. Circadian rhythms vary by person.
Unfortunately, the world makes it hard to keep that cycle, and so instead I mostly do 17 hours awake / 7 asleep, which makes me feel constantly tired (though less tired than 16/8 at least).
No not a brag. I haven't done this in a long time. It's really unhealthy but I was trying to get a startup off the ground and became obsessed. Don't recommend it.
I still prefer Go over the alternatives because I'm reviewing the code carefully and trying to make it all look as good as if I'd written it very carefully by hand.
The few times I've needed a language without a GC and used Rust and C as target languages, I've reviewed the code less and enjoyed it less.
JIT and AOT compilers, REPL development, bleeding edge GC implementations, Java, Scala, Kotlin, Clojure, C#, F# to chose from, and modern IDEs inspired by Smalltalk/Lisp Machines experience, batteries included backed and frontend development frameworks.
As I do more agentic coding, I am starting to feel strongly that I need to set aside a project -- and time -- for strictly hand-coding, using AI (if at all) only to explain things, never to write any code.
First, I like programming and don't want to stop doing it just because a machine can do it too. Second, I'm pretty sure atrophied coding skills will make me worse at directing the machines, thus worse at what seems to be "the job" these days.
>First, I like programming and don't want to stop doing it just because a machine can do it too. Second, I'm pretty sure atrophied coding skills will make me worse at directing the machines, thus worse at what seems to be "the job" these days.
In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding? I mean isn’t whats left just design and architecture and testing?
Sure, if LLMs are producing shit code you need to have an understanding of what it should do instead. But I think that day is already gone.
Its hard to think of a skill that you’d need to build with an LLM that requires writing meaningful amounts of code by oneself. From my end all the complexity is business logic and system design.
> In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding? I mean isn’t whats left just design and architecture and testing?
No, because good understanding of design comes from programming experience. Atrophied skills mean you lose touch with the actual nuts and bolts of what you are building.
This is also the worst kind of lock-in you can imagine, by outsourcing all of your understanding of a codebase and your skills you are now completely at the mercy of LLM providers. Even if you switch to open weight models you're at the mercy of graphics cards producers and scalpers.
> Sure, if LLMs are producing shit code you need to have an understanding of what it should do instead. But I think that day is already gone.
Maybe your standards are too low. By my standard, they still produce shit code and require lots of iterations to get reasonable designs (I often end up just rewriting the bits that are obviously wrong rather than trying to figure out how to "prompt engineer" the right output).
They do often produce passable code but my experience is the longer you spend reading the code the more things you find wrong with it, almost like an uncanny valley.
At least that's my experience with Opus and Fable. Maybe it's too old-fashioned at this point to complain about code quality, but I find it very strange how many people have started proclaiming that their code quality is good.
>No, because good understanding of design comes from programming experience.
“Programming” is a bit loaded. Do you mean thinking through and typing everything out yourself?
What design skills specifically can you only develop from continually programming in this way? I fully grant that you need to know how to code yourself to build things with models but what specific ability to use models atrophies alongside your no AI coding abilities?
Personally I’ve found the opposite to be true. As writing out code myself has shrank to nearly 0, I now do way more designing. There is a much shorter feedback loop on design now. You get a lot better at design when thats basically all you are doing, and seeing feedback on your design in minutes rather than weeks or even months.
> seeing feedback on your design in minutes rather than weeks or even months.
Weeks and month for to wait for a feedback in developing software? Are you the sloth in zootopia?
The longest cycle for feedback for me has only been hours. And that was about redesigning the core architecture of an app. Otherwise, it’s like 10 minutes cycle if it’s edit-compile-run, 5 minutes if it’s REPL (lisp, bash, python,…).
> What design skills specifically can you only develop from continually programming in this way?
As I write code myself I continously take into consideration stuff like following the consistency of the entire design, whether what I'm doing is efficient, etc. LLMs are terrible at this and I don't see how you could develop such skills by prompting only.
For example, I would never produce designs that require degenerate db access patterns (linear scans, etc.). Everything I write myself always takes into account using db & network efficiently from the ground-up. Every query is an index lookup, every api request is rate limited, everything is batched and cached, so on and so forth.
LLMs meanwhile don't care about any of this. I just had Claude happily try to fetch youtube page html because the api didn't expose something. Great idea, I'd love to be banned by google for scraping.
Designing with performance in mind is black magic too, I need to notice myself that stuff that I think has no reason to be slow (pretty much nothing does) is slow and hammering the llm to finally come up with a non-stupid design that's not ridiculously slow and wasteful. Yeah, why don't we pull 2.3mb of html to update a list, sounds good.
The best part is, LLMs are very obedient. If you tell them to make you a design that's inherently wasteful of resources and slow they will happily oblige, and since you have no programming experience to know better, you won't even know how bad what you're asking for is.
Read my comment again. And the one I replied to. He claimed that his ability to code through LLMs has decreased as his coding ability atrophied. That is what im questioning. Im not questioning coding for the love of the game.
Do it! I have the same feeling and the hardest part is picking something that you are interested in. I have just kept the idea of "thinking of ideas" on my mind for a few days once I finish something. I usually come up with a few ideas if I'm always considering any little thing for a project. It has been fun and I have learned a few things along the way - more than I feel I do letting AI write most of the code.
I haven't actually tried it yet, but I think it'd be good to write a dsl for whatever you're doing. Then have ai make an interpreter to run it. Sort of as a way to stay... interested? Engaged?
That's exactly how i do all my personal projects. No agents, AI just for planning/research/debugging and occasionally for small bits of code i am not super interested in. For example I'm currently working in Go, but need to use a C library, and don't know C much at all. So when i get stuck with the C interop parts, sometimes i let the AI write bits of C for me, or let it write scripts for cross-compiling or other stuff i just find to be a pain. But I'm doing all the writing and logic myself.
Agents make it too easy to gloss over too much. They're great at work where i care about productivity but I've zero interest at home where the point is learning and having fun
People are sleeping on Go as a LLM language of choice. Everyone trying to rewrite in rust which is fine and rust is fine. However its compile times are several orders of magnitude greater than Go. You can have LLMs write and compile 20 apps in the time one rust app compiles. I've got a Go personal project that would take hours for rust to compile, I can't imagine making the same velocity in Rust. Though I do see the workplace appeal of bringing back the xkcd "compiling" meme.
However it kind of makes me feel like my professional life was wasted and I don't really know what kind of future I have. One of Go's main points was you can pick it up in a weekend. What career moat do people like us have?
isn't that one of TinyGo's raison d'etre's? I've never tried wasm, but executables are typically a magnitude smaller than go's (assuming your go code is compatible with tinygo, of course)
One of the best techniques for using LLM coding is to let it write programs that frontload and automate work in a repeatable and testable manner rather than letting it only do free form turns.
Letting it generate machine code directly would be the opposite of that.
Like a JIT compiler then, where the generated Assembly depends on various parameters, where not two runs are the same, unless you manage to replicate the whole system state across them.
As for proven wrong,
"Frame was created over the past month largely via Claude Code. The Assembly code in all its glory can be found on GitHub."
That is fighting the last war on the early days of 5 GLs, and taken care by agentic runtimes.
This isn't futurology, besides a few projects that already came up on Phoronix and around here, with Assembly generation, there are enough SIGPLAN papers on ML driven compilers already.
ML driven compilers are different than LLM assembly generation, and former is more sound than the latter one. I believe we both have enough experience to understand that.
However, starting with LLMs directly generating machine code and moving to ML directed optimization passes, you're moving the whole stadium with a crawler-transporter, not only the goalposts.
Helping compiler in a well defined way, and working on IR or driving it repeatedly to optimize the output is one thing, generating machine code from a training set is other. Let's not mix them.
There are other researchers if you interest yourself on the matter.
Then the whole iPaaS agentic coding environments like Workato and Boomi, that even if they might generate 3GL as intermediate step, you don't get to see it.
This is like saying that a strong man would rather dig a hole with his bare hands than use a shovel. Structured programming languages are force multipliers, it makes absolutely zero sense to forego them just so you can vibe assembly directly.
Optimizing compilers were faced with the same disbelief by Assembly developers.
Enterprise applications are already being vibe coded in iPaaS platforms, and as mentioned in other comments with LLMs in the picture, that is what compiler research is switching to.
In no sense is this "disbelief". You seem to be laboring under a fundamental misunderstanding of the role of structured programming languages. A man with a shovel can dig a hole for some amount of effort, and when that man gets stronger he can use that shovel to either dig the same hole for less effort or dig a larger hole for the same amount of effort. With LLMs, leveraging a structured programming language allows a user to either create the same program for less effort or to create a larger program for the same amount of effort. Getting rid of the programming language and emitting assembly directly sacrifices efficiency for no gain.
I am already using a bagger, while you are arguing for electric shovels.
In case you answer there don't fit the same purpose, there are tiny baggers that replace shovels altogether.
I think those arguing for structured language generation, never had that much experience with 4GLs or declarative programming on the previous AI Summer.
So like the Assembly developers required those -S compiler switches, or nowadays Compiler Explorer snippets, there is a tendency to argue for 3GL code generation, when low code/no code tools with AI runtimes are already being adopted at enterprise level.
What in the past took a bunch of developers using structured programming languages to write their frontend, and the myriad of microservices, is slowly being taken over by MCP tools, SaaS AI orchestration prompts and UI generation tooling, most of which doesn't even expose whatever code gets generated.
Any modern redesigned json library in 2026 should have at least a flag to serialize and unserialize json5. Sad that they had this opportunity here and blew it.
In Go, it’s a question of who owns the allocation. If you return a pointer type allocated within the function, the answer is “no one,” so the pointer escapes and has to be managed by the garbage collector.
If the allocation is owned by the calling function which uses it internally and then doesn’t return it, it can be a stack variable. Or the same allocation can be reused many times in a loop. Or an array of allocations can be made all at once with foreknowledge of the total number.
If you pass a pointer into a function, the Go compiler can (and sometimes does) allocate it on the heap. Go has no allocation/ownership semantics, tragically, so the best we can do is depend on undefined behavior regarding the escape analyzer and hope the Go team doesn’t break it (which is a relatively safe assumption, considering that it’s unlikely that the Go team would start allocating things on the heap that were previously allocated on the stack—Go rarely regresses performance like this).
The parameters named "out" here are not "output parameters" in the way you seem to be thinking. They are "input parameters" named "out", because they are the buffer or pipe you want the data written to.
`out` params in C# are params that must be set within the function. They are way to implement multiple return values and tell the caller "this is written to, not read from."
But you might be conflating that with in/out for Contra/Co-variance when specifying a generic.
This v2 idea in the stdlib should have started even earlier. So I am glad we are getting a better API for json/v2.
This pragmatic attitude towards software engineering is exactly why I still reach for Go for pretty much everything these days. I still read and review code.
So working with a language that doesn't throw a splash of sigil soup on my face is quite nice while reviewing clanker output. Also, being able to do all this without needing 5 crates is nice. Oh the fast build time helps too.
the marshal api has been very annoying. why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable. it's certainly a code bug. to me it's like int+str returning an error instead of compiler error
> why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable
Not necessarily. Among other things, MarshalJSON methods called during marshaling can return an error for whatever reason the developer sees fit - one possibility that comes to mind is that part of the object tree being marshalled is lazily loaded, and failed to do so.
And, as others have mentioned, there are types like "any" whose marshalability can't be determined at compile time. Those types aren't necessarily at the top level, either; they could appear anywhere in the object tree.
This isn’t a type system problem. It’s a data origin problem. Ie it could be any random JSON blob a user entered. The compiler doesn’t know this ahead of time.
If you wanted type safety with JSON in Go then don’t use the marshal interface. Period.
For unmarshalling, but the original complaint was about marshalling. And for that the type system could help if the marshal API only allowed you to marshal objects of types that supported marshalling.
And even for unmarshalling you could get a compile time error if you try to unmarshall to a type that doesn't support it.
I’d already addressed that when I said the marshal API is an interface that’s meant to be reusable across generalise use cases. it can be used for a variety of data formats and not just JSON.
Go has this concept baked into the language. For example read and write interfaces so you can write data to an endpoint without having to worry about whether it’s a file system object, TCP/UDP socket, gzip achieve, and a plethora of other options too.
It’s a nice convenience feature and makes up considerably for the restrictions that the Go type system places.
And if you need higher performance code, or compile time safety, or have any other constraints that limit your ability to use these generalised interfaces, then you can use different APIs from the Go standard library to achieve the same thing.
You don’t need to call json.Marshal() if you don’t want to. It’s an option, but it’s just one API into the JSON toolbox.
First of all, the json2 Marshall API is json specific.
Secondly, other languages like rust, haskell, scala, etc. Have json APIs that are typesafe and give you compile time errors if you try to use a type that doesn't support (de) serialization . In the case of rust at least, the API is also general and can be used for other data types as well.
Even in go, you could have some more type safety if Marshall took an interface (such as MarshallerTo) rather than `any`. Although then you have the problem of how you implement that interface. So I guess you probably want some kind of "derive" functionality, or macros that can approximate it for a convenient and type-safe API.
P.S. gophers like to talk about how explicit being better than implicit, but using reflection for marshalling is very implicit.
Except it’s not JSON specific. The article literally talks about how their function design was chosen for similarity with the v1 API. And I’d wager that the go team are planning on reusing that function signature for other encoders too. Or at least expecting it to be reused by the wider ecosystem.
If you want type safety then pass a struct to the marshal. Which is actually the idiomatic way to do it anyway. The type safety would be in that.
I’m not going to disagree with you that Gos type system is a rather blunt tool nor that Gos expressiveness is extremely rigid compared to many other popular languages. All of that is completely true. But I don’t agree that the v2 JSON API is a good example of that problem.
The function is in the `encoding/json/v2` package. It might have a similar design, and maybe they will use the same function signature for other encoders, but it the functions are very much json specific.
> If you want type safety then pass a struct to the marshal.
But if you use a struct that has a field that can't be marshalled, you don't get a compile error, you get a runtime SemanticError. That's the problem.
> The function is in the `encoding/json/v2` package. It might have a similar design, and maybe they will use the same function signature for other encoders, but it the functions are very much json specific.
But the point you keep missing is that the function signature needs to be encoder agnostic for it to work as an interface. That’s a hard requirement.
And since interfaces are a convenience to allow common abstractions to be reused at runtime, you cannot depend on things like generics (ie compile time abstractions) to generalise the data types for disparate serialisation formats.
I know this limitation well because I’ve built a shell with a typed pipeline and support for different serialisation formats. So make heavy use interfaces to achieve this. And I’ve spent a lot of time investigating alternative approaches. Each time, I’ve found interfaces to be a massive advantage despite any tradeoffs I end up making do to generalising a “class” (to borrow a term from OOP).
I’d honestly love to have a more sophisticated type system in Go. But that’s not a limitation in this specific instance.
> But if you use a struct that has a field that can't be marshalled, you don't get a compile error, you get a runtime SemanticError. That's the problem.
My point here was: if you know you’re creating a struct to pass to the mashaller at compile time, then you’re already going to use JSONAble fields. So it’s not an issue.
And if you don’t know what you’re passing to the mashaller a compile time, then you’re dependent on runtime checks regardless of the expressiveness of Gos type system.
I don’t disagree that Gos type system is inflexible. But in respect to this specific use case, your argument is a storm in a teacup. The type system isn’t a limiting factor here.
> But the point you keep missing is that the function signature needs to be encoder agnostic for it to work as an interface.
It's a function, not a method. So it can't be part of an interface.
I suppose you could make an argument for passing it as a function argument to a higher ordered function. But that is kind of an unusual way to use such a function. And isn't a common pattern in go.
> My point here was: if you know you’re creating a struct to pass to the mashaller at compile time, then you’re already going to use JSONAble fields. So it’s not an issue.
Maybe. Maybe not. For a simple struct where all the fields are primitive types sure. But if you have a complicated hierarchy, you might miss that some strict in the hierarchy has a field that can't be marshaled.
Say you are marshaling a struct that contains a struct from a third party library, and it works fine. Then you update to a new version of that library, and that added a new field for a type that doesn't work, say a map[A]string where A is a struct. Your program still compiles fine, but then you get an error at runtime when you try to serialize something that worked before. Hopefully you have tests that catch it.
> It's a function, not a method. So it can't be part of an interface.
The mashaller here is a function, yes. But having a consistent signature means you can interchange those marshallers. It’s a trick I use heavily in murex to enable typed pipelines.
And I do this via a pipe interface. So it’s not uncommon to reuse function signatures for interface types (fewer runtime abstractions) while retaining some level of dynamism.
> Say you are marshaling a struct that contains a struct from a third party library, and it works fine. Then you update to a new version of that library, and that added a new field for a type that doesn't work, say a map[A]string where A is a struct.
The first mistake there is to blindly forward an internal API struct that you don’t control, and depend on that business logic.
Which comes back to my point about if you want type safety then you’d create a struct to be marshelled rather than passing data you don’t control.
I do take your point that the compiler in smarter languages would catch this. But you’d equally be told not to do this sort of thing on those languages too. So it feels like a moot point.
I'm confused by this - you're saying that a "type system" would address this, even though there are two (incompatible) type systems at play here - Go's and JSON's
Compile time cannot check for runtime values (not even in Rust) - is your position that fields can only be populated at compile time?
In go, the Marshall and unmarshal functions take `any` for the source or destination. If you use a value of a type that doesn't support marshaling or unmarshalling (say because it contains an unserializable type like a channel, function pointer, etc) then you get a runtime error. Whereas in rust, Haskell, scala, etc. you can get a compile time error if you try to use a type that doesn't support serialization.
You don’t get runtime errors for marshalling pointers to JSON. I actually do this all the time. What you get is the address of the pointer marshalled.
I think the same is true for channels too but I’m not near a computer to check.
I’ve abused the v1 API heavily* and haven’t found anything it couldn’t marshal. Though I’m yet to apply the same “fuzzing” (for lack of a better term) to v2.
* the v1 JSON marshaller underpins some core functionality in my alternative $SHELL. So I’ve thrown a lot of exotic type structures (by Gos standards at least) at it.
That just shows there are pointer values that can be marshaled, not that all such values won't error. If you have a non-nil pointer that points to an object that transitively contains one of the unsupported types, you will get an error.
i meant i hoped it's panic as opposed to return an error. if there was an error, how am i supposed to handle it? can i tell the user hey sorry my bad it's likely a code error? crash is better than possible eating the error siliently
It is trivial to ignore an error in Go if you know your use case cannot result in one. But at the end of the day, json.Marshal takes an `any` as a parameter, so it must return an error.
It IS a simple format. And you can make a parser yourself very easily. Doesn’t mean that it will be the fastest or most memory efficient. Don’t confuse format and implementation
The spec itself is simple, but different libraries are written that optimize for different things: parsing speed, streaming parsing, minimal memory usage, very very large data, distributed storage, etc.
Some languages have a culture of doing things correctly and conforming strictly to the spec. These languages have indeed sorted it out years ago. Haskell for example. Any language that doesn’t have a special number type for JSON is a half-assed implementation, because 1e1000 is a perfectly valid JSON number that is beyond the typical IEEE754 double precision floating point numbers; does your language handle that correctly? Haskell does. This is but a small example of a strong culture bias towards correctness.
The spec (RFC 8259) leaves significant leeway to implementations:
A JSON parser MAY accept non-JSON forms or extensions.
An implementation may set limits on the size of texts that it accepts. An implementation may set limits on the maximum depth of nesting. An implementation may set limits on the range and precision of numbers. An implementation may set limits on the length and character contents of strings.
Add to that things like duplicate keys being allowed by the spec. There’s a lot of implementation choices to decide on.
And then things can get really complex with JSON Schema.
Yeah that’s what I mean by strictly. Does your implementation take shortcuts or does it hold itself to a higher standard and not do that? Does it only use available memory as the ultimate limit?
A lot of specs contain these optional limitations. The C standard allowed implementations to truncate variable names to 31 characters if they have external linkage and 63 if internal linkage. Would you be happy if your implementation actually does that? Of course not. Neither LLVM nor GCC does that. That’s because people writing compilers also hold themselves to a higher standard. Now imagine this kind of attitude being pervasive.
The thing is this goes both ways. For example, JSON with duplicate keys is valid JSON, and the standard specifies two ways to handle them. To support all ways JSON is used, an implementation will have to provide for both ways, and decide which one to pick as the default. The stricter default on the parsing side is at the same time the less interoperable one. Same for numeric precision: seamlessly allowing higher precision on the generating side lowers interoperability towards the receiver (but lessens demands on the driving application), while accepting it on the parsing side increases interoperability (but also places higher demands on the consuming application). So what do you do? There is no easy and simple answer, unless you want to limit the scope of applicability of your implementation.
I don't get your argument here. You say Haskell conforms strictly to the spec and that's a good thing, okay. But then if the C spec allows for truncated names, a compiler could strictly follow the spec and truncate names. So strictly following the spec is not in fact the whole story
Yeah very sorry for the confusion. I should not have used the word strictly. I explained my definition of strictly in the above comment and it’s not really strictly in the dictionary sense. I apologize for my bad writing.
Sure, if you ignore the massive difference in the rate such changes are happening at and that this new api is strictly better than the previous one, so there's no two ways that need to be decided between but rather the new go-to and the old-to-be-moved-away-from option, so there's no cognitive load here at all.
So yeah, if you ignore most of what matters, everything is the same.
I think it'll bite some people, but probably is the better default for JSON specifically. Most of the nil-maps-become-null thing is just an extension of Go defaulting to nil for empty arrays and maps, which is usually not what you expect when you're writing a JSON schema like
type ListUsersResponse struct {
Users []User `json:"users"`
}
in Go. It's pretty common have code like
userRows := db.Query(...)
var users []api.User
for _, user := range userRows {
users = append(users, api.User{...})
}
where you don't _really_ intend for the default nil-ness to leak out. (Apologies for any broken syntax, I'm realizing in real time that it's been a while since I've written Go)
> Most of the nil-maps-become-null thing is just an extension of Go defaulting to nil for empty arrays and maps, which is usually not what you expect when you're writing a JSON schema
This is definitely not what I expect in other language. I would argue this is a quick of Go though and not JSON's fault.
How is this a broken default? In Go, I almost always want a nil slice to be treated exactly as an empty slice, and JSONv1 was the ~only place this didn’t work as expected, leading to frequent bugs.
JSON often means data is crossing a boundary, and JSON has `null`. So if you have a PATCH endpoint that accepts a partial object, how do you determine what was actually set in the request vs what go has initialized to an empty value.
I don’t think there was a single canonical solution, but pointers to everything was a pretty common one. Everything in the AWS SDK was a `string` or a `float64` or similar—it was exceedingly annoying, not least of all because it basically forced tons of unnecessary allocations.
The root cause is this idea of "zero values". Any de/serialization in go will always be crutchwork of guessing and incorrect validation.
Just look at the inane rules for map and slice access, what gets nil, what gets the zero value, which built-in mist be used. Even if that were consistent, then you'd still left to wonder if json.field was truly zero or not there.
I'm working on better SIMD support in Dart and one of the motivations for that work is specializing parser specifications based on schema specifications of that language.
Or more concretely, if we had a parser generator + json parser specification that can generate efficient simdjson style parsers, then we should be able to parse json restricted to a subset of json based on a schema at least as efficiently as simdjson can parse general purpose json.
That way we could have the best of both worlds, a (potentially lazy) statically type-safe model and simdjson performance!
I still think that this is definitely doable, it's just that a lot of infrastructure needs to be built for it.
I remember reading that Meta is doing something similar with binary formats. I find it weird that there's nothing like that for json yet.
This approach is called "staged parsing":
https://dl.acm.org/doi/pdf/10.1145/2714064.2660241
https://dl.acm.org/doi/pdf/10.1145/3314221.3314625
I was working on metaparsing earlier this year (specifically over grpc because of its better support for types and better serialization format).
I think most people who use JSON don't want strong typing or to use schemas. If you do, you just use a different serialization format: JSON is designed to be a generic, human-readable way to basically represent any kind of data structure (since it deserializes to JS objects). So everything about JSON is working against you if you do try to enforce type/schemas on it.
A grpc-like protocol with more prescriptive type registries/discovery (over the network) and better interop with http (like grpc-gateway) would be the best approach IMO. Definitely not JSON schema
> think most people who use JSON don't want strong typing or to use schemas. If you do, you just use a different serialization format: JSON is designed to be a generic, human-readable way to basically represent any kind of data structure (since it deserializes to JS objects). So everything about JSON is working against you if you do try to enforce type/schemas on it.
I have been using serde without issues, and never resorted to `serde::Value`[0], the untyped option, so far.
I have yet to experience anything working against me.
[0]: https://docs.rs/serde_json/latest/serde_json/enum.Value.html
> I think most people who use JSON don't want strong typing or to use schemas.
Not necessarily. It's biggest advantage is probably that it has very good interoperability. Basically every language has a readily accessible library for json, and even if you don't, it isn't that hard to write a parser and/or generator for it. And being human readable isn't something to sneeze at either.
> I think most people who use JSON don't want strong typing or to use schemas.
We use JSON because it's standard, human-readable, mostly self-described and works natively with Javascript.
You can read JSON object without having to know the schema even if one exists.
Protobuf is unusablr garbage without a lot of supporting tooling.
I think it may be fairer to say, “most people pick up JSON because of the flexibility.” Those who continue to use it for anything where there’s a consumer of your API probably do come to want types. In my day job I am starting to enforce end-to-end typing as best I can (given it’s Python and JSON) because our payloads are partially content from agents and I want to be able to flag up weird shit if we can. Certainly not the only gate that we have for looking for prompt injection and not even a great one, but it provides some piece of mind.
> I think most people who use JSON don't want strong typing or to use schemas.
Pretty much anywhere I've come across JSON (and xml before that,) everyone expected strong typing. The fact that JSON (and xml) can represent different data than they anticipated "did not compute" in their heads.
In many cases, they were treating JSON (and xml) as dumb serializers and not even paying attention to the output and corner cases.
I definitely want jsonschema, it's a standard and i can use it for many tools. openapi spec is also a standard and built on top of jsonschema. I can use jsonschema to generate models, types, servers, client sdks, automate forms etc. I think it's better than grpc for some of these things, plus protobufs and grpc have bunch of tooling that i can pretend to understand but in depth i don't really.
I generally like `encoding/json/v2` and I'm glad they did it.
I wish they'd left `MarshalIndent` in there though. The equivalent code is super complicated and non-obvious, so it's a pain to migrate, and it's a pain when you're banging out some quick code and want to dump some readable JSON. I suppose these days the LLMs can write the complicated version for us, of course.
I'm not sure if I'm missing something, but isn't this functionality just moved into jsontext? https://pkg.go.dev/encoding/json/jsontext#Value.Indent
I love paying 10 cents to some billionaire to serialize some json. Screw using my brain!!! Ill rent me some competency
Not an insult to you, its just how I feel things are going, myself inculded at times.
Edit: why do yall downvote this lol, you know its true.
I used the jsontext package at my company when we needed to make a parser for challenging clients (specific errors for each kind of mistake) while still keeping performance, and it works like a charm.
There is no introspection and speed is superb. I would definitely recommend jsontext as a parser for anything non-straightforward.
To preserve the order when processing JSON a workaround has been introduced:
https://news.ycombinator.com/item?id=44893833
Still searching the true solution:
https://github.com/golang/go/issues/27179#issuecomment-53634...
A proposal of the missing datatype for this operation mode has been just accepted:
https://github.com/golang/go/issues/69559
I… think this is good?
I'm just getting the neuron-band back together where Golang is concerned. I have done some in the past for little system tasks, but I am switching away from interpreted languages entirely for web stuff, and initially to Go, as I switch away from hand-coding to at least LLM-supported coding.
So I am a bit out of date on the pain points (I never really felt pain points when I last used it, because the jobs I was doing had very small, quite Go-friendly shapes).
The fact that this took 6 years to land stood out to me.
I'm not entirely surprised, and I'm glad the Go team is carefully considering their interfaces before launching. But 6 years seems like a lot of work to land a JSON API.
Sometimes I wonder about Go's velocity - generics (something that I still haven't used beyond the tutorials) took an age as well
JSON processing, quite markedly, is renown in Go for being slow, so I don't really understand why it took so long
For like 99% of use cases encoding/json was fine. Of course improvements are welcomed. But personally I've lead development of the backend of part one of the biggest sites of its kind in the world, and we just used encoding/json and net/http. Most of our use cases are just building APIs, client facing ones, internal ones, trusted third-parties. These systems have pretty high throughput, more than most people will ever work on.
Within our company I saw people in other teams writing their own optimised JSON libraries for whatever hellish reason. I always reiterated the point that as soon as that app made a single IO request all those nanosecond savings are invisible. Yes, it affects your total CPU time for processing a request but compute was never ever a limitation for us and it's likely not a limitation for most people. And if the trade off is your app needs scale slightly larger versus writing custom JSON code or pulling in a third party dependency, then I'm accepting the larger scaling.
The biggest benefit of encoding/json/v2 isn't the speed, it's the improved API.
I think in the case of JSON, it was “good enough”. People who found this to be a problem would use one of the many 3rd party options.
I kind of like that the standard library doesn’t move fast. It makes it easier to fit the ecosystem into my head (unlike, for example, the JavaScript ecosystem where I find code doesn’t survive 2 years before failing to compile).
But I do agree that slow and steady produces its own problems. Your example of generics is a good one. But think the example of Go modules is another, perhaps even better, example of how sometimes the Go team really should have accelerated their efforts on.
lol, I don't mean to be contrary, but I looked into dependency management (partly because of the modules implementation, partly because of early teething issues, and partly out of pure curiousity.
Dependency management is a freaking nightmare, nobody has managed to create something that is bulletproof, not cargo, npm, apt, yum, nix, homebrew, nothing.
The key issue is that you're managing 3rd party software that may or may not adhere to the prescribed lifecycle/versioning
The Linux variants do it by each having dedicated teams watching upstream sources, taking what they believe to be whatever they feel is right, building for their nominated target(s), then testing, before deploying (on top of that, all of these steps vary in importance and rigor depending on numerous variables)
For programming libraries (which is what depmod, goose, vgo, and modules are managing) it's just as bad, if not harder, because their very nature is to provide a base upon which other developers can turn into whatever their imagination.
Add to that, modules was (initially) written by people who were dealing with a slightly different ecosystem; Google rather famously has a monorepo, making their lives a little easier when it comes to managing dependencies.
So, with those things in mind, dependency management is always going to be a hard problem that is opinionated (eg. I have a firm opinion that there should be an easier way to "pin" a dependency, other than vendoring - note that this isn't an invitation to every junior Go developer to come and claim go.sum does that. It doesn't. go.sum is a log of what dependencies were used for the last build by whomever wrote to it)
I’ve written package management tools and I agree that dependency management is hard. Though I wouldn’t say it’s that hard compared to the other problems you encounter with language development.
But difficulty aside, my point was Go waited until there were multiple established 3rd party package management tools for Go already in widespread use before even releasing their own experimental implementation.
Their inaction caused far more disruption to the ecosystem because people got fed up of waiting, implemented their own, integrated it into their workflow and then had to reintegrate a whole new system all over again a year or two later.
Your point about Googles needs is the one often cited for Gos inaction. But I think that actually makes the situation worse. Because if the Go team cannot empathise with use cases outside of one bespoke enterprise, then it really shouldn’t be released as a general purpose language available for all to use.
And I say this as someone who generally likes the language.
Interesting, I came to the opposite conclusions, but for the same reasons. I like that they waited to see what alternative proposals/implementations were out there before they decided on their own.
Especially since "Go team cannot empathise with use cases outside of one bespoke enterprise" — it shows self-awareness of their shorcomings. I think it's wise to let the target audience explore the design space themselves a bit first. Considerably lowers the risk of cementing the wrong design.
I do somewhat remember the drama around this at the time, but I my impression was that of a petulent community of various dependency management projects with too much ego attached to them to see the bigger, long-term picture.
That’s an interesting take. I appreciate the alternative view point and I can’t say I disagree with the way you’re looking at it.
Thanks for sharing.
> there should be an easier way to "pin" a dependency, other than vendoring
Put this in your go.mod:
Then go.sum will do its actual job (along with GOSUMDB) and ensure you only get exactly the revision of the code that you want.> there should be an easier way to "pin" a dependency, other than vendoring
You left out the part that explains why.
Go is an incomplete language masquerading as a simple one.
Meanwhile people are happily shipping widely used software in Go :shrug:
Back in the day we shipped software in ZX Spectrum and C64 BASIC, among other variants as well.
Wordpress powered by PHP is also the most widely deployed CMS.
That argument is not as great as you think it is.
Sounds like those technologies were pretty great for their day if they motivated a bunch of people to use them over boondoggle alternatives :shrug:
> That argument is not as great as you think it is.
But at least it is an argument in contrast to the comment I was replying to.
> generics (something that I still haven't used beyond the tutorials) took an age as well
Only about a year from when the Haskell guys lent their expertise in designing it and it landing in a release. That's actually fairly quick given that it was a pretty significant change under the hood and constrained by only two releases annually.
The Go team was always clear that they didn't have the necessary capabilities internally, despite a lot of failed efforts trying anyway, and were relying on the open source community to lend a hand. Unfortunately, open source doesn't mean open willingness to contribute. It turns out that for all the experts on HN, posting "go doesn't have generics lol" was preferable to helping out.
Those guys were the same that helped out with generics in Java, they even published a book in 2006,
https://homepages.inf.ed.ac.uk/wadler/topics/gj.html
Before Featherweight Go, there was Featherweight Java.
So maybe a language designed in 2012 (v 1.0 release year), should learn from history instead of hand-waving it away as needless fluff.
> In retrospect, we were biased too much by experience with C++ without concepts and Java generics. We would have been well-served to spend more time with CLU and C++ concepts earlier.
https://github.com/golang/proposal/blob/master/design/go2dra...
And not all on the team were convinced even after Featherweight Go, here Rob Pike,
> I think the transition to getting polymorphism into the language through generics is has still got some years to work through there's still things that don't quite work right
https://youtu.be/yE5Tpp2BSGw?t=2134
> instead of hand-waving it away as needless fluff.
If you go back and watch the original Go announcement it was made abundantly clear that they believed Go did need generics but that they didn't have the expertise to do it. Yes it is true that the book was already written on it, but that doesn't mean anyone on the Go team was familiar with it or had the time to become familiar with it. Programming languages do not require gods to design them. Imperfect humans can too.
The experts on HN who know all the ins and outs of generics could have contributed but, you know, shitposting "go doesn't have generics lol" was a better use of their time. There are so many hours in the day.
Gonuts is full of content that they were not keen in generics, in fact gccgo has been abandoned, and I expect it to eventually follow gcj and be removed from GCC.
Go culture is not welcoming to experts contributions, Rust, D, C#, Swift and others, heck even Java, are more open arms to language designers.
Nobody is keen about language features. Each new one adds complexity. That isn't the same as failing to recognize a need, though. And we know the Go team recognized the need as they clearly kept working on generics all the way from the very beginnings up until the Haskell muscle showed up. The receipts are there. The results just weren't good, as one would expect from a team who weren't shy to say that they didn't have the expertise necessary.
You'd think one of the HN experts would have shown up to offer that expertise, but I guess there was a lot of "go doesn't have generics lol" needing to get out.
Some did try, and were welcomed with stones, clubs and pitchforks, until leadership had to reach out for help, from the same team that designed Java generics, years before Go v1.0 came to materialise.
The Haskell muscle had proven their skills already with Featherweight Java, in 2006.
The same people who were working on generics in Go prior to v1 were also still there to help out when it eventually landed, so whatever stones, clubs, and pitchforks you saw weren't compelling enough to scare one away from making it happen. Not that anyone would expect them to. Meaningless distractions don't get in the way when doing something is worthwhile.
But obviously "go doesn't have generics lol" comments on HN were even more important. Just think of what would have happened if that wasn't repeated multiple times daily for years and that effort was put into getting Go generics into place much sooner. Count your lucky stars or something.
To offer help, the receiving end must be open to take it.
And there was plenty of effort given by others who also weren't experts. There was no lack of trying within a sea of incapable people. Unfortunately, that wasn't what the project needed. It already had a team full of blub developers, it being a language for them.
It is after all a language designed for not smart people, as per Rob Pike own words.
The key point here is our programming language designers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school…
Robert Griesemer, the youngest of Go's original authors (the other two are Rob Pike and Ken Thompson) was 45 when they started working on Go.
Haskell guys? Got a reference to this? That’s news to me.
I'm not sure about the haskell association, but search for "Featherweight Go"
Thanks for the pointer. The reference makes sense now, as Philip Wadler is a coauthor of that paper.
At this stage to me Go is the new C compiler. Quite honestly I spent 10 years or more writing Go code. At times 18 hours a day. And now with AI, I get models to write my Go code. I STILL want it to be Go code. Is that strange? No because I understand the performance and safety of Go. But I rarely write it directly anymore. Probably not at all in 6-9 months. Yet I still occasionally look at the code to see what the output looks like.
So yea hopefully a nice new addition but only my prompts now matter. So need to make sure they pickup on the new API.
This is mostly why I have come back to Go.
I am using local LLMs and Go has a bunch of strengths where smaller contexts are concerned; less complicated flow of control being one of them.
not a lot of overloading complexity, very verbose with a small total language structure too helps.
Yes. It's that thing — the fact that a language helps you read what a single function does really easily, but in so doing makes the entire codebase quite dense, is not optimal for us, but is good for them.
And the things that we do to manage this, that are symbolic or based on visceral real-world analogy, are good for us and bad for them.
Go is kind of in the middle, in the way that many vaguely Wirth-like and CSP-like languages possibly are, I guess.
Truly the argument can be made either way.
If you code in Go, the code is more high level, prettier, simpler.
If you code in Rust, the code is more robust, efficient, faster.
Both are important for LLM, why can't we have a high level, simple language that is also robust, efficient and fast? =)
Efficient code means code with a large amount of embedded assumptions. Code that's more generic is almost always less efficient.
Go is about as far as you can "go" to the efficiency end without becoming Rust. It's a low-overhead language (like C) but with a GC that ensures memory safety. If you want to manage memory yourself and still prove safety it gets very complicated, as we see in Rust.
OCaml fits your bill. Super fast compile times and also very fast runtime.
It always feels like OCaml is on the verge of becoming the perfect language.
Expressive and type safety of a functional language, without the odd semantics of Haskell, where laziness can make performance difficult to predict and purity can make simple things like IO difficult to understand.
Does it still lack easy to use concurrency primitives?
OCaml 5 + https://github.com/ocaml-multicore/eio
I' just starting OCaml, so can't really compare, but it seems a good progress there.
> Both are important for LLM, why can't we have a high level, simple language that is also robust, efficient and fast? =)
I think many variants from Rust can be stripped to make it simple, especially in type system, which is way more complicated than go.
I dont think we need simpler types for agentic coding though. We need compiler speed. I guess those are probably related to some degree though.
It depends if you are full vibe-coding mode or still try to understand your codebase.
LLM can generate very cryptic rust code.
Goose?
Swift
If I want simplicity and maintainability, I’d pick Kotlin over Go. If I want speed, I’m picking Rust over Go. Why would I ever pick Go? It’s the best at nothing.
It's surely better than Kotlin in simplicity and maintainability. It also doesn't tie you to the JVM.
What if you want a single language that's pretty fast while simple and maintainable?
Recently I stumbled upon https://crystal-lang.org/ and I love it!
It's really great fun to build stuff in Crystal, I don't have anything prod-grade yet to say how it scores there but the community seems to be happy about that too.
Crystal is slow to compile. There is a variant of Crystal that compiles a lot faster. But I forgot its name. Need to dig it up when I get back home.
I agree, the compiler is slow but I don't mind much, or better, the fact that I find building maintainable programs much easier outweights that downside massively in my case.
Then you pick C# which is both faster and simpler than Go.
Simpler it definitely is not. Better, yes I would accept that. C# is complex though.
I like C# but simpler? I don’t believe so.
C# may be easier but it is not simpler than Go
didn't that language have like 150 keywords 10 years ago, it probably has 500 keywords now
> At times 18 hours a day
I hope you're not still doing this, it's nothing to brag about.
As long as you do it because you want to and not because you're forced to for work or something who cares? Whatever gives people energy
Sleep deprivation does not give you energy.
Im not saying this is what's happening here, it could totally be just that he was burning himself out at work, but it's a truly incredible thing to find something that you love working on and want to dive into to the point that you lose hours at it and have to pull yourself away from it to go to sleep (obviously outside of stuff designed to be addictive and exploitative like gambling that hijack those reward mechanisms). Programming is one of those artistic technical pursuits and a lot of programmers love it in that way. And are grieving losing that with A. At least so far as you felt you could only justify spending time on something you loved like that if you were also paid for it. Nobody is stopping you from opening up a text editor again on your own time. But you know, it's good to be passionate about things. Not being able to sleep until you figure something out, losing track of time while you're lost in whatever you're working in - all that stuff where it's from innate drive and the love of the game? that stuff rules. If it's burnout throw it out but don't throw out the electric stuff that makes us human
I do personally find myself most productive with a cycle of 20 hours awake / 12 hours asleep. Circadian rhythms vary by person.
Unfortunately, the world makes it hard to keep that cycle, and so instead I mostly do 17 hours awake / 7 asleep, which makes me feel constantly tired (though less tired than 16/8 at least).
It actually can if you ride that dragon just the right way! https://en.wikipedia.org/wiki/Wake_therapy
No not a brag. I haven't done this in a long time. It's really unhealthy but I was trying to get a startup off the ground and became obsessed. Don't recommend it.
Did you succeed?
I still prefer Go over the alternatives because I'm reviewing the code carefully and trying to make it all look as good as if I'd written it very carefully by hand.
The few times I've needed a language without a GC and used Rust and C as target languages, I've reviewed the code less and enjoyed it less.
Go is lovely. Best language to generate backend services with.
JVM and .NET platforms.
JIT and AOT compilers, REPL development, bleeding edge GC implementations, Java, Scala, Kotlin, Clojure, C#, F# to chose from, and modern IDEs inspired by Smalltalk/Lisp Machines experience, batteries included backed and frontend development frameworks.
> generate
Yes, and my comment still applies, I usually read what I reply to.
Very generous of you to assume "generate" guys are reading code, so IDE and REPL are useless to them.
Very generous of you to assume Go matters at all, in such scenarios the text being generated is a markdown paragraph description.
As I do more agentic coding, I am starting to feel strongly that I need to set aside a project -- and time -- for strictly hand-coding, using AI (if at all) only to explain things, never to write any code.
First, I like programming and don't want to stop doing it just because a machine can do it too. Second, I'm pretty sure atrophied coding skills will make me worse at directing the machines, thus worse at what seems to be "the job" these days.
>First, I like programming and don't want to stop doing it just because a machine can do it too. Second, I'm pretty sure atrophied coding skills will make me worse at directing the machines, thus worse at what seems to be "the job" these days.
In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding? I mean isn’t whats left just design and architecture and testing?
Sure, if LLMs are producing shit code you need to have an understanding of what it should do instead. But I think that day is already gone.
Its hard to think of a skill that you’d need to build with an LLM that requires writing meaningful amounts of code by oneself. From my end all the complexity is business logic and system design.
> In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding? I mean isn’t whats left just design and architecture and testing?
No, because good understanding of design comes from programming experience. Atrophied skills mean you lose touch with the actual nuts and bolts of what you are building.
This is also the worst kind of lock-in you can imagine, by outsourcing all of your understanding of a codebase and your skills you are now completely at the mercy of LLM providers. Even if you switch to open weight models you're at the mercy of graphics cards producers and scalpers.
> Sure, if LLMs are producing shit code you need to have an understanding of what it should do instead. But I think that day is already gone.
Maybe your standards are too low. By my standard, they still produce shit code and require lots of iterations to get reasonable designs (I often end up just rewriting the bits that are obviously wrong rather than trying to figure out how to "prompt engineer" the right output).
They do often produce passable code but my experience is the longer you spend reading the code the more things you find wrong with it, almost like an uncanny valley.
At least that's my experience with Opus and Fable. Maybe it's too old-fashioned at this point to complain about code quality, but I find it very strange how many people have started proclaiming that their code quality is good.
>No, because good understanding of design comes from programming experience.
“Programming” is a bit loaded. Do you mean thinking through and typing everything out yourself?
What design skills specifically can you only develop from continually programming in this way? I fully grant that you need to know how to code yourself to build things with models but what specific ability to use models atrophies alongside your no AI coding abilities?
Personally I’ve found the opposite to be true. As writing out code myself has shrank to nearly 0, I now do way more designing. There is a much shorter feedback loop on design now. You get a lot better at design when thats basically all you are doing, and seeing feedback on your design in minutes rather than weeks or even months.
> seeing feedback on your design in minutes rather than weeks or even months.
Weeks and month for to wait for a feedback in developing software? Are you the sloth in zootopia?
The longest cycle for feedback for me has only been hours. And that was about redesigning the core architecture of an app. Otherwise, it’s like 10 minutes cycle if it’s edit-compile-run, 5 minutes if it’s REPL (lisp, bash, python,…).
> What design skills specifically can you only develop from continually programming in this way?
As I write code myself I continously take into consideration stuff like following the consistency of the entire design, whether what I'm doing is efficient, etc. LLMs are terrible at this and I don't see how you could develop such skills by prompting only.
For example, I would never produce designs that require degenerate db access patterns (linear scans, etc.). Everything I write myself always takes into account using db & network efficiently from the ground-up. Every query is an index lookup, every api request is rate limited, everything is batched and cached, so on and so forth.
LLMs meanwhile don't care about any of this. I just had Claude happily try to fetch youtube page html because the api didn't expose something. Great idea, I'd love to be banned by google for scraping.
Designing with performance in mind is black magic too, I need to notice myself that stuff that I think has no reason to be slow (pretty much nothing does) is slow and hammering the llm to finally come up with a non-stupid design that's not ridiculously slow and wasteful. Yeah, why don't we pull 2.3mb of html to update a list, sounds good.
The best part is, LLMs are very obedient. If you tell them to make you a design that's inherently wasteful of resources and slow they will happily oblige, and since you have no programming experience to know better, you won't even know how bad what you're asking for is.
If it helps, imagine they were talking about something like drawing or painting or stitching or sculpting instead of coding.
You don't expect everyone to stop wanting to do those just because they can be automated more or less effectively, right?
> You don't expect everyone to stop wanting to do those just because they can be automated more or less effectively, right?
Please reread my comment and the one im replying to. That’s not what I’m addressing.
> In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding?
I am begging HN users to understand that some people enjoy writing software themselves
Read my comment again. And the one I replied to. He claimed that his ability to code through LLMs has decreased as his coding ability atrophied. That is what im questioning. Im not questioning coding for the love of the game.
Only in the same sense that painting skills not matter
I still catch Claude writing some shit code sometimes
Do it! I have the same feeling and the hardest part is picking something that you are interested in. I have just kept the idea of "thinking of ideas" on my mind for a few days once I finish something. I usually come up with a few ideas if I'm always considering any little thing for a project. It has been fun and I have learned a few things along the way - more than I feel I do letting AI write most of the code.
I haven't actually tried it yet, but I think it'd be good to write a dsl for whatever you're doing. Then have ai make an interpreter to run it. Sort of as a way to stay... interested? Engaged?
That's exactly how i do all my personal projects. No agents, AI just for planning/research/debugging and occasionally for small bits of code i am not super interested in. For example I'm currently working in Go, but need to use a C library, and don't know C much at all. So when i get stuck with the C interop parts, sometimes i let the AI write bits of C for me, or let it write scripts for cross-compiling or other stuff i just find to be a pain. But I'm doing all the writing and logic myself.
Agents make it too easy to gloss over too much. They're great at work where i care about productivity but I've zero interest at home where the point is learning and having fun
This is sort of the deal with any language. I’m using go for personal projects and dotnet for paid projects.
Why? Because I don’t give time to wait for code to compile on my time and I don’t have time for people to learn a new language for review at work.
Life is good! Would recommend that split.
>> to me Go is the new C compiler
In what sense? Go has a GC and binary sizes are relatively pretty huge. Its closer to Java and C# than C.
Which are your favorite models for writing Go? Any models for editing/improving written documentation?
People are sleeping on Go as a LLM language of choice. Everyone trying to rewrite in rust which is fine and rust is fine. However its compile times are several orders of magnitude greater than Go. You can have LLMs write and compile 20 apps in the time one rust app compiles. I've got a Go personal project that would take hours for rust to compile, I can't imagine making the same velocity in Rust. Though I do see the workplace appeal of bringing back the xkcd "compiling" meme.
However it kind of makes me feel like my professional life was wasted and I don't really know what kind of future I have. One of Go's main points was you can pick it up in a weekend. What career moat do people like us have?
My only issue with it is that you can't ship a wasm go app in under 2mb. With rust you can make it under 100kb.
isn't that one of TinyGo's raison d'etre's? I've never tried wasm, but executables are typically a magnitude smaller than go's (assuming your go code is compatible with tinygo, of course)
Jeez, and I thought 100 KB was absurd.
Do you do a lot of work in WASM? What is a reasonable size? Or just against the every increasing internet bloat in general?
Think about how many instructions fit in 100 KB. Why are they in your hello world?
Binary bloat is not just about size; it impacts startup time and icache. It’s basically the speed of light launch time.
So you’re telling me rust is not good for UNIX style tools.
I would rather have LLM directly generate machine code, and best of all, macro Assemblers are blazing fast for decades.
One of the best techniques for using LLM coding is to let it write programs that frontload and automate work in a repeatable and testable manner rather than letting it only do free form turns.
Letting it generate machine code directly would be the opposite of that.
Like a JIT compiler then, where the generated Assembly depends on various parameters, where not two runs are the same, unless you manage to replicate the whole system state across them.
As for proven wrong,
"Frame was created over the past month largely via Claude Code. The Assembly code in all its glory can be found on GitHub."
https://www.phoronix.com/news/Frame-X11-Server-Assembly
You can't PGO that code, though. Also, I'm not sure it can generate architecture specific optimized (-march, -mtune) code as well.
Considering it just pulls and remixes blocks, this is a very tall order for today's LLMs, and maybe a couple of generations as well.
Calling GCC/clang at the back and emitting ASM is both cheating and has no advantages whatsoever as well.
That is fighting the last war on the early days of 5 GLs, and taken care by agentic runtimes.
This isn't futurology, besides a few projects that already came up on Phoronix and around here, with Assembly generation, there are enough SIGPLAN papers on ML driven compilers already.
ML driven compilers are different than LLM assembly generation, and former is more sound than the latter one. I believe we both have enough experience to understand that.
However, starting with LLMs directly generating machine code and moving to ML directed optimization passes, you're moving the whole stadium with a crawler-transporter, not only the goalposts.
Helping compiler in a well defined way, and working on IR or driving it repeatedly to optimize the output is one thing, generating machine code from a training set is other. Let's not mix them.
Yeah, well as mentioned, that is where it is going.
"Compiler 2.0: Building the Next Generation of Compilers with Machine Learning"
https://eecs.berkeley.edu/research/colloquium/compiler-2-0-b...
There are other researchers if you interest yourself on the matter.
Then the whole iPaaS agentic coding environments like Workato and Boomi, that even if they might generate 3GL as intermediate step, you don't get to see it.
Thanks for the link. I have also found a small trove as a warm-up. If anyone interested, here it is:
https://github.com/zwang4/awesome-machine-learning-in-compil...
Using small, purpose built ML "things" to make things better makes sense. We use them everywhere. I never argued against it, as well.
Like inform7 but irreproducible and shitter on every iteration.
Like JIT and PGO machine learning optimizer passes more likely.
This is like saying that a strong man would rather dig a hole with his bare hands than use a shovel. Structured programming languages are force multipliers, it makes absolutely zero sense to forego them just so you can vibe assembly directly.
Optimizing compilers were faced with the same disbelief by Assembly developers.
Enterprise applications are already being vibe coded in iPaaS platforms, and as mentioned in other comments with LLMs in the picture, that is what compiler research is switching to.
In no sense is this "disbelief". You seem to be laboring under a fundamental misunderstanding of the role of structured programming languages. A man with a shovel can dig a hole for some amount of effort, and when that man gets stronger he can use that shovel to either dig the same hole for less effort or dig a larger hole for the same amount of effort. With LLMs, leveraging a structured programming language allows a user to either create the same program for less effort or to create a larger program for the same amount of effort. Getting rid of the programming language and emitting assembly directly sacrifices efficiency for no gain.
I am already using a bagger, while you are arguing for electric shovels.
In case you answer there don't fit the same purpose, there are tiny baggers that replace shovels altogether.
I think those arguing for structured language generation, never had that much experience with 4GLs or declarative programming on the previous AI Summer.
So like the Assembly developers required those -S compiler switches, or nowadays Compiler Explorer snippets, there is a tendency to argue for 3GL code generation, when low code/no code tools with AI runtimes are already being adopted at enterprise level.
What in the past took a bunch of developers using structured programming languages to write their frontend, and the myriad of microservices, is slowly being taken over by MCP tools, SaaS AI orchestration prompts and UI generation tooling, most of which doesn't even expose whatever code gets generated.
Use a working Go program as the prompt+oracle for cross-language port or direct generation.
I rather not, unless I have to.
If 3 GL code has to be generated, my option list won't certainly have Go on it.
May I ask what your option list will be.
There is the superset of JSON known as JSON5. But A.I. tells me that nobody has (yet) built a JSON5 implementation that builds on encoding/json/v2 .
Any modern redesigned json library in 2026 should have at least a flag to serialize and unserialize json5. Sad that they had this opportunity here and blew it.
> in any, out any
Completely unrelated: can anyone tell me why language authors introduce in/out/inout function parameters (see also C#).
It feels like this is some rather bad workaround for something that breaks assumptions about parameters being passed into the function.
In Go, it’s a question of who owns the allocation. If you return a pointer type allocated within the function, the answer is “no one,” so the pointer escapes and has to be managed by the garbage collector.
If the allocation is owned by the calling function which uses it internally and then doesn’t return it, it can be a stack variable. Or the same allocation can be reused many times in a loop. Or an array of allocations can be made all at once with foreknowledge of the total number.
If you pass a pointer into a function, the Go compiler can (and sometimes does) allocate it on the heap. Go has no allocation/ownership semantics, tragically, so the best we can do is depend on undefined behavior regarding the escape analyzer and hope the Go team doesn’t break it (which is a relatively safe assumption, considering that it’s unlikely that the Go team would start allocating things on the heap that were previously allocated on the stack—Go rarely regresses performance like this).
The parameters named "out" here are not "output parameters" in the way you seem to be thinking. They are "input parameters" named "out", because they are the buffer or pipe you want the data written to.
`out` params in C# are params that must be set within the function. They are way to implement multiple return values and tell the caller "this is written to, not read from."
But you might be conflating that with in/out for Contra/Co-variance when specifying a generic.
> They are way to implement multiple return values
They are one way to implement multiple return values. For multiple returns, tuples can be used. In fact, async methods can’t declare out params
"in" and "out" are the names of the parameters, not qualifiers.
C# `out` is kinda nice because it can be `var`d and used directly.
Erm... what do you mean?
This v2 idea in the stdlib should have started even earlier. So I am glad we are getting a better API for json/v2.
This pragmatic attitude towards software engineering is exactly why I still reach for Go for pretty much everything these days. I still read and review code.
So working with a language that doesn't throw a splash of sigil soup on my face is quite nice while reviewing clanker output. Also, being able to do all this without needing 5 crates is nice. Oh the fast build time helps too.
It WAS started earlier, math/rand/v2 was added in 2024 (Go 1.22)
I love it too, it’s a great way to keep the stdlib moving without actually breaking anything. Simple and efficient design.
This is how you roll out features, C++. Congrats to Go team and users.
wohoo, good news indeed better later than never :D as an active Golong user I'm very happy
I like it. I never really liked marshalling anyways, It was weird.
I'm happy it's getting fixed!
the marshal api has been very annoying. why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable. it's certainly a code bug. to me it's like int+str returning an error instead of compiler error
> why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable
Not necessarily. Among other things, MarshalJSON methods called during marshaling can return an error for whatever reason the developer sees fit - one possibility that comes to mind is that part of the object tree being marshalled is lazily loaded, and failed to do so.
And, as others have mentioned, there are types like "any" whose marshalability can't be determined at compile time. Those types aren't necessarily at the top level, either; they could appear anywhere in the object tree.
Types may be mostly marshallabble, have a subset of values that are unmarshallable. E.g. non-finite floats.
AFAIK it's mostly because of floats. JSON doesn't support `NaN` and `infinity` for example.
Because they are common types, makes sense to precautionary have the error return.
I wouldn’t class unmarshallable data as a compile time error.
Plus the point of the marshal interface is that it’s reusable.
It could be if go had a more powerful type system.
This isn’t a type system problem. It’s a data origin problem. Ie it could be any random JSON blob a user entered. The compiler doesn’t know this ahead of time.
If you wanted type safety with JSON in Go then don’t use the marshal interface. Period.
For unmarshalling, but the original complaint was about marshalling. And for that the type system could help if the marshal API only allowed you to marshal objects of types that supported marshalling.
And even for unmarshalling you could get a compile time error if you try to unmarshall to a type that doesn't support it.
I’d already addressed that when I said the marshal API is an interface that’s meant to be reusable across generalise use cases. it can be used for a variety of data formats and not just JSON.
Go has this concept baked into the language. For example read and write interfaces so you can write data to an endpoint without having to worry about whether it’s a file system object, TCP/UDP socket, gzip achieve, and a plethora of other options too.
It’s a nice convenience feature and makes up considerably for the restrictions that the Go type system places.
And if you need higher performance code, or compile time safety, or have any other constraints that limit your ability to use these generalised interfaces, then you can use different APIs from the Go standard library to achieve the same thing.
You don’t need to call json.Marshal() if you don’t want to. It’s an option, but it’s just one API into the JSON toolbox.
First of all, the json2 Marshall API is json specific.
Secondly, other languages like rust, haskell, scala, etc. Have json APIs that are typesafe and give you compile time errors if you try to use a type that doesn't support (de) serialization . In the case of rust at least, the API is also general and can be used for other data types as well.
Even in go, you could have some more type safety if Marshall took an interface (such as MarshallerTo) rather than `any`. Although then you have the problem of how you implement that interface. So I guess you probably want some kind of "derive" functionality, or macros that can approximate it for a convenient and type-safe API.
P.S. gophers like to talk about how explicit being better than implicit, but using reflection for marshalling is very implicit.
Except it’s not JSON specific. The article literally talks about how their function design was chosen for similarity with the v1 API. And I’d wager that the go team are planning on reusing that function signature for other encoders too. Or at least expecting it to be reused by the wider ecosystem.
If you want type safety then pass a struct to the marshal. Which is actually the idiomatic way to do it anyway. The type safety would be in that.
I’m not going to disagree with you that Gos type system is a rather blunt tool nor that Gos expressiveness is extremely rigid compared to many other popular languages. All of that is completely true. But I don’t agree that the v2 JSON API is a good example of that problem.
> Except it’s not JSON specific.
The function is in the `encoding/json/v2` package. It might have a similar design, and maybe they will use the same function signature for other encoders, but it the functions are very much json specific.
> If you want type safety then pass a struct to the marshal.
But if you use a struct that has a field that can't be marshalled, you don't get a compile error, you get a runtime SemanticError. That's the problem.
> The function is in the `encoding/json/v2` package. It might have a similar design, and maybe they will use the same function signature for other encoders, but it the functions are very much json specific.
But the point you keep missing is that the function signature needs to be encoder agnostic for it to work as an interface. That’s a hard requirement.
And since interfaces are a convenience to allow common abstractions to be reused at runtime, you cannot depend on things like generics (ie compile time abstractions) to generalise the data types for disparate serialisation formats.
I know this limitation well because I’ve built a shell with a typed pipeline and support for different serialisation formats. So make heavy use interfaces to achieve this. And I’ve spent a lot of time investigating alternative approaches. Each time, I’ve found interfaces to be a massive advantage despite any tradeoffs I end up making do to generalising a “class” (to borrow a term from OOP).
I’d honestly love to have a more sophisticated type system in Go. But that’s not a limitation in this specific instance.
> But if you use a struct that has a field that can't be marshalled, you don't get a compile error, you get a runtime SemanticError. That's the problem.
My point here was: if you know you’re creating a struct to pass to the mashaller at compile time, then you’re already going to use JSONAble fields. So it’s not an issue.
And if you don’t know what you’re passing to the mashaller a compile time, then you’re dependent on runtime checks regardless of the expressiveness of Gos type system.
I don’t disagree that Gos type system is inflexible. But in respect to this specific use case, your argument is a storm in a teacup. The type system isn’t a limiting factor here.
> But the point you keep missing is that the function signature needs to be encoder agnostic for it to work as an interface.
It's a function, not a method. So it can't be part of an interface.
I suppose you could make an argument for passing it as a function argument to a higher ordered function. But that is kind of an unusual way to use such a function. And isn't a common pattern in go.
> My point here was: if you know you’re creating a struct to pass to the mashaller at compile time, then you’re already going to use JSONAble fields. So it’s not an issue.
Maybe. Maybe not. For a simple struct where all the fields are primitive types sure. But if you have a complicated hierarchy, you might miss that some strict in the hierarchy has a field that can't be marshaled.
Say you are marshaling a struct that contains a struct from a third party library, and it works fine. Then you update to a new version of that library, and that added a new field for a type that doesn't work, say a map[A]string where A is a struct. Your program still compiles fine, but then you get an error at runtime when you try to serialize something that worked before. Hopefully you have tests that catch it.
> It's a function, not a method. So it can't be part of an interface.
The mashaller here is a function, yes. But having a consistent signature means you can interchange those marshallers. It’s a trick I use heavily in murex to enable typed pipelines.
And I do this via a pipe interface. So it’s not uncommon to reuse function signatures for interface types (fewer runtime abstractions) while retaining some level of dynamism.
> Say you are marshaling a struct that contains a struct from a third party library, and it works fine. Then you update to a new version of that library, and that added a new field for a type that doesn't work, say a map[A]string where A is a struct.
The first mistake there is to blindly forward an internal API struct that you don’t control, and depend on that business logic.
Which comes back to my point about if you want type safety then you’d create a struct to be marshelled rather than passing data you don’t control.
I do take your point that the compiler in smarter languages would catch this. But you’d equally be told not to do this sort of thing on those languages too. So it feels like a moot point.
I'm confused by this - you're saying that a "type system" would address this, even though there are two (incompatible) type systems at play here - Go's and JSON's
Compile time cannot check for runtime values (not even in Rust) - is your position that fields can only be populated at compile time?
In go, the Marshall and unmarshal functions take `any` for the source or destination. If you use a value of a type that doesn't support marshaling or unmarshalling (say because it contains an unserializable type like a channel, function pointer, etc) then you get a runtime error. Whereas in rust, Haskell, scala, etc. you can get a compile time error if you try to use a type that doesn't support serialization.
You don’t get runtime errors for marshalling pointers to JSON. I actually do this all the time. What you get is the address of the pointer marshalled.
I think the same is true for channels too but I’m not near a computer to check.
I’ve abused the v1 API heavily* and haven’t found anything it couldn’t marshal. Though I’m yet to apply the same “fuzzing” (for lack of a better term) to v2.
* the v1 JSON marshaller underpins some core functionality in my alternative $SHELL. So I’ve thrown a lot of exotic type structures (by Gos standards at least) at it.
> You don’t get runtime errors for marshalling pointers to JSON
https://pkg.go.dev/encoding/json/v2#Marshal says otherwise:
> The Go map key must encode as a JSON string, otherwise this results in a SemanticError
> All other Go types (e.g., complex numbers, channels, and functions) have no default representation and result in a SemanticError.
v1 has the same constraints.
> A Go pointer is encoded as a JSON null if nil, otherwise it is the recursively JSON-encoded representation of the underlying value.
The doc you linked to literally proves me right.
That just shows there are pointer values that can be marshaled, not that all such values won't error. If you have a non-nil pointer that points to an object that transitively contains one of the unsupported types, you will get an error.
I read that differently. If you have a non-nil pointer then you get the address of the pointer irrespective of what that pointer points too.
Then write yourself a MustMarshal wrapper that panics.
A generic Must[T] you mean ;)
The GP is talking about compile time errors. You’re discussing exceptions, which is still a runtime check
Their complaint is about the signature. The comment about the compile time checking was a misguided justification for their position.
JSON is a runtime input of unknown type but in your example those are known types so the compiler can catch them.
i meant i hoped it's panic as opposed to return an error. if there was an error, how am i supposed to handle it? can i tell the user hey sorry my bad it's likely a code error? crash is better than possible eating the error siliently
It is trivial to ignore an error in Go if you know your use case cannot result in one. But at the end of the day, json.Marshal takes an `any` as a parameter, so it must return an error.
do not know at compile time that input is not marshalable, every language marshal must return error for this reason, not only golang
> "JavaScript Object Notation (JSON) is a simple data interchange format [...]"
Clearly it isn't, otherwise every language would have sorted it out by... a long time ago.
It IS a simple format. And you can make a parser yourself very easily. Doesn’t mean that it will be the fastest or most memory efficient. Don’t confuse format and implementation
The spec itself is simple, but different libraries are written that optimize for different things: parsing speed, streaming parsing, minimal memory usage, very very large data, distributed storage, etc.
Some languages have a culture of doing things correctly and conforming strictly to the spec. These languages have indeed sorted it out years ago. Haskell for example. Any language that doesn’t have a special number type for JSON is a half-assed implementation, because 1e1000 is a perfectly valid JSON number that is beyond the typical IEEE754 double precision floating point numbers; does your language handle that correctly? Haskell does. This is but a small example of a strong culture bias towards correctness.
Ironically Go has(d) json.Number which almost no one used, so it got dropped from v2: https://pkg.go.dev/encoding/json#Number
You can still use it though, and I built my tiny decimal library around it: https://github.com/ncruces/decimal
The spec (RFC 8259) leaves significant leeway to implementations:
A JSON parser MAY accept non-JSON forms or extensions.
An implementation may set limits on the size of texts that it accepts. An implementation may set limits on the maximum depth of nesting. An implementation may set limits on the range and precision of numbers. An implementation may set limits on the length and character contents of strings.
Add to that things like duplicate keys being allowed by the spec. There’s a lot of implementation choices to decide on.
And then things can get really complex with JSON Schema.
Yeah that’s what I mean by strictly. Does your implementation take shortcuts or does it hold itself to a higher standard and not do that? Does it only use available memory as the ultimate limit?
A lot of specs contain these optional limitations. The C standard allowed implementations to truncate variable names to 31 characters if they have external linkage and 63 if internal linkage. Would you be happy if your implementation actually does that? Of course not. Neither LLVM nor GCC does that. That’s because people writing compilers also hold themselves to a higher standard. Now imagine this kind of attitude being pervasive.
The thing is this goes both ways. For example, JSON with duplicate keys is valid JSON, and the standard specifies two ways to handle them. To support all ways JSON is used, an implementation will have to provide for both ways, and decide which one to pick as the default. The stricter default on the parsing side is at the same time the less interoperable one. Same for numeric precision: seamlessly allowing higher precision on the generating side lowers interoperability towards the receiver (but lessens demands on the driving application), while accepting it on the parsing side increases interoperability (but also places higher demands on the consuming application). So what do you do? There is no easy and simple answer, unless you want to limit the scope of applicability of your implementation.
I don't get your argument here. You say Haskell conforms strictly to the spec and that's a good thing, okay. But then if the C spec allows for truncated names, a compiler could strictly follow the spec and truncate names. So strictly following the spec is not in fact the whole story
Yeah very sorry for the confusion. I should not have used the word strictly. I explained my definition of strictly in the above comment and it’s not really strictly in the dictionary sense. I apologize for my bad writing.
Which C standard? ANSI C? C99?
C99 and all subsequent versions.
Simple data format does not imply that API to work with it is easy to get right.
Meet the "Simple Object Access Protocol": https://www.w3.org/TR/2000/NOTE-SOAP-20000508/
My company uses JSUN (/sarcasm) where O = object, and U = union.
Who knew the same object could have completely different definitions based on some flags?
Ok so nothing is simple then?
I mean, it is better than the disaster that is CSV...
Now guess what will happen when they add monadic error handling. Yes, correct, there will be another JSON Go API
json/v3 ...so what? The v* idea in the stdlib allows things to move when needed.
"There only ONE way to do each thing! Only ONE! No exception"
"What if it sucks?"
"There are now TWO ways to do each thing! No more than TWO! I promise!"
It's like python again.
Sure, if you ignore the massive difference in the rate such changes are happening at and that this new api is strictly better than the previous one, so there's no two ways that need to be decided between but rather the new go-to and the old-to-be-moved-away-from option, so there's no cognitive load here at all.
So yeah, if you ignore most of what matters, everything is the same.
I bet if I was unemployed enough go through python mails, I would find a similar argument.
The problem with “Batteries Included” is that batteries run out of juice eventually.
Python introduces new abstractions for the same problem. Whereas this is an updated API version.
It’s versioned because the function signature differs but the abstraction itself doesn’t.
That’s not to say that Go doesn’t have multiple different abstractions for common problems too. Just that your specific argument here is incorrect.
It was more a joke than an argument. Learning Go was like that Nate Bargatze sketch from SNL where he plays George Washington.
"Sir we need a loop" "We will have for loop, and nothing else." "Only one lo.." "Only for loop"
> nil slices and maps are marshaled as an empty JSON array or object by default
I feel that is the kind of broken default that will come back and bite a bunch of people, and will eventually need to be fixed in a v3.
I think it'll bite some people, but probably is the better default for JSON specifically. Most of the nil-maps-become-null thing is just an extension of Go defaulting to nil for empty arrays and maps, which is usually not what you expect when you're writing a JSON schema like
in Go. It's pretty common have code like where you don't _really_ intend for the default nil-ness to leak out. (Apologies for any broken syntax, I'm realizing in real time that it's been a while since I've written Go)> Most of the nil-maps-become-null thing is just an extension of Go defaulting to nil for empty arrays and maps, which is usually not what you expect when you're writing a JSON schema
This is definitely not what I expect in other language. I would argue this is a quick of Go though and not JSON's fault.
In Golang a nil value is semantically an empty container. You wouldn't want this translation in Java but in Golang it makes sense.
Can you explain why? This seems like the appropriate expectation to me, given how JSON is constructed and consumed.
Not the OP, but serializing both nil and empty arrays to the same json string means round tripping will not work for all objects.
That seems like a bad default to me.
How is this a broken default? In Go, I almost always want a nil slice to be treated exactly as an empty slice, and JSONv1 was the ~only place this didn’t work as expected, leading to frequent bugs.
JSON often means data is crossing a boundary, and JSON has `null`. So if you have a PATCH endpoint that accepts a partial object, how do you determine what was actually set in the request vs what go has initialized to an empty value.
Use the same `Option[T]` type that you will need to define for strings and numbers and structs.
What was the solution before go had generics? I thought it was nil pointers.
I don’t think there was a single canonical solution, but pointers to everything was a pretty common one. Everything in the AWS SDK was a `string` or a `float64` or similar—it was exceedingly annoying, not least of all because it basically forced tons of unnecessary allocations.
The root cause is this idea of "zero values". Any de/serialization in go will always be crutchwork of guessing and incorrect validation.
Just look at the inane rules for map and slice access, what gets nil, what gets the zero value, which built-in mist be used. Even if that were consistent, then you'd still left to wonder if json.field was truly zero or not there.
I think I can build a new web tools serving for this new API