> External/world effects are visible in function signatures
Brilliant. I think Jai has something like that? Each function declares what it's going to touch (both read/write) globally, and I think you can specify that per block even.
I haven't used Jai (I think it's not out yet) but I remember the author talking about this and it sounded like a great idea.
It's related to the idea of pure functions being easy to reason about. Right now most languages don't even have a concept of pure functions, but the ones that do, just have two categories. When a function is messing with global state you do actually want to know what it's doing.
I think that can be statically computed (and displayed as IDE annotations or whatever) but specifying it explicitly sounds like a good practice either way.
I like that you're including other side effects (e.g. network) there too though, that's pretty cool!
Another thing I'm really interested in is proofs. Not necessarily total proofs (though we seem to be moving in that direction, at least for subsets of the codebase), but just normalizing rudimentary pre and post conditions checks.
I was thinking of setting it up so code can't even compile in release mode if those are missing. (Not every function would need them, but you at least want to state their absence explicitly.)
I was also thinking of setting up strictness levels per function, using hashing or something, so if a function is modified, you'd have to go through a process of double checking it again. And then functions labeled e.g. level 7 strict couldn't call ones with a lower proven strictness level, and so on. I'm told that I've basically been reinventing Ada from first principles so I should probably go and take a look at that...
Wait, your thing is doing hashing too... Woah. (I think that comes from proofs land or something?)
Thanks for the pointer to Jai! I will check it out.
Yes, Jacquard uses content-addressed definitions and it should be possible to set up a process for 'review again if this changes' on top of it. Warp, the testing framework, already uses this to avoid rerunning pure tests when neither the definition or dependencies have changed.
Jacquard does not currently implement proof or strictness levels, but binding those to a definition’s content identity is interesting and definitely worth exploring.
What are you building that people keep comparing to Ada?
Haven't built anything yet as far as language / tooling goes, but my experiences with poorly designed languages and tooling have basically made me paranoid as far as "what touches this variable", "what does this function actually do", ended up arriving in Rust / Haskell-ish territory just by recoiling from the pain of PHP/WordPress, where everything is global, mutable, dynamic and designed to make you go completely insane.
(In a nutshell: the more dynamic a programming language is, the more impossible it becomes to reason about what the program is doing. That's fine for throwaway scripts and game jams (#pragma JamMode), but the "the program is probably seriously wrong" should be explicitly opt-in, not the default ...)
So my current approach is everything should be as local as possible, as immutable as possible, etc. Basically Rust except I'm not a fan of low level programming. (I basically want Rust With GC (C# memory model), which sends Rust folks into paroxysms!)
I want it stricter than Rust in many respects, but also less annoying. (Better ergonomics and higher level, with a low level escape hatch when needed.)
Swift is apparently close to that, and might deserve another look.
I've also been obsessed with formal verification and proofs except, today I had a very revealing experience. I had an LLM incorrectly implement a major feature, and "verify" it with several thousand lines of tests. It was, as implemented, backwards, and also completely pointless — but all tests came back green!
I laughed when I realized, if it had been done in Rust, and with a layer of formal verification... well all the proofs (of the stupid and wrong and backwards thing) would have come back green too... It would have mathematically proven that the incorrect thing was correct...
Ah, that’s a tough one. My hope is that Jacquard, or some similar language, can make intent explicit through readable, high-level constructs, giving both the human reviewer and the model a clearer target while the details are filled in. But it still can’t tell us whether the original intent was right.
Coincidentally, you reminded me of one of my favorite Charles Babbage quotes:
> “On two occasions I have been asked, ‘Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?’ I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.”
I suppose catching that kind of mistake is how humans prove to our future AI overlords that we should be kept around.
> Algebraic effects with deep, multi-shot handlers. A handler can resume a computation zero, one, or many times, which is what makes exhaustive search and exact inference ordinary library code.
I love me some algebraic effects, but this sets off alarm bells for me.
Multi-shot effects are neat and powerful but are incredibly difficult to reason about for humans, and given that there are few languages that have implemented them (and they're so niche with very little training data) I'm skeptical that LLMs are sufficiently better at them to make up the gap.
One-shot effects are a cross between `throw` and a function call. Multi-shot are full-on delimited continuations and come with all the complexity that entails.
That’s a fair concern. I’d expect multi-shot handlers to be relatively rare and mostly something library code uses rather than a day-to-day utility. But an LLM given free rein may decide to behave differently.
For now, I am looking into making a clearer distinction between one-shot and multi-shot handlers so we can reject cases where resuming again would be unsafe, like around filesystem writes.
> a C-emitting native AOT backend that currently compiles the kernel .jqd carrier
This human reviewer gave up at line 4 of the Readme. What is the kernel? What does the carrier carry? Why does it need to be carried? Where does the .jqd come from?
Also, for whatever reason, from the submission here:
> I had AI analyze the ASTs of several mainstream languages plus a few of the conceptually groundbreaking but esoteric ones (listed in the README)
No, they are not listed in the Readme. Why lie to us right off the bat?
I like the general idea, but the human responsible for this seems to demonstrate no interest in actually reviewing their system's output. Which does not bode well.
Just in case the above is not considered constructive enough, how about this: In my experience, with some prompting and iterations, AIs do reasonably well at writing solid, "boring" code. They do really badly at writing useful documentation, and this is a major pain point when reviewing their output. A system that claims to be oriented towards human review must address this somehow. It must talk about how it addresses it, and it must demonstrate that its own documentation is significantly better than unfiltered LLM output.
This system does not demonstrate this. It needs clear guidelines: Start with the big picture. Show an example. Show comments in that example. Don't get bogged down in the details of the internal workings of the compiler. Don't use jargon like "kernel" to mean "lowered intermediate representation". Don't use "carrier" when you mean "file". It's just a file, FFS. But again, don't tell me at all about the fact that there is an intermediate representation; that's not "big picture", and anyone who cares about compiler internals will already have guessed that you have an intermediate representation. Do tell me about the effect system in the first two paragraphs! That seems to be what sets this language apart. Do give me a brief argument why an effect system is a good match for a system where humans don't have to type everything by hand. And so on and so forth.
I always feel like I'm losing my mind when reading LLM comments or prose for code. It's so esoteric and 'i-am-very-smart' and makes me feel like I'm not supposed to read the comments in the first place
Once, during an especially tedious review session, I told my agent something like "comments are not meant to demonstrate how well you understand the system, they are meant to help others understand the system". It felt like that helped.
Given how poorly LLMs do with writing prompts for LLMs, I'm not sure I'd trust their judgement in designing a language for LLMs.
> and the runtime requires explicit permission to touch the filesystem, network, etc
This feels like more of an OS problem (or library problem) than a language problem.
> Run one program against many worlds. The same code can run against the real network, a scripted fake, a recording of last week's traffic, or a probability model of how servers usually behave
How is the "world" model different from plain dependency injection?
> How is the "world" model different from plain dependency injection?
In addition to what the other comment said, this "world" model is great for hermetic testing of complex code, LLM written or not. We've seen existing projects that intercept the OS level syscall for testing, replayability, etc. Building it into the language runtime, hopefully with better ergonomics from the start than a syscall, would be a welcome addition broadly.
Dependency injection provides the same hermetic-testing capability, so that's not really an answer to OP's question. Effect-as-world-model is, in this specific way, just a special argument slot that only accepts dependency-injected functions.
What effects provide beyond DI is entirely in their ability to abort (resume zero times) or (in the case of multi-shot effects) resume multiple times. An effect that resumes exactly once is structurally the same as a dependency injected lambda.
> What effects provide beyond DI is entirely in their ability to abort (resume zero times) or (in the case of multi-shot effects) resume multiple times
Thanks, this answers my question and does sound handy. Also I do see the value in having more problem-shaped versions of a general construct.
> Given how poorly LLMs do with writing prompts for LLMs, I'm not sure I'd trust their judgement in designing a language for LLMs.
Yeah agree, what I really want to see now, is a programming language for LLMs, designed by a human (although code could still be LLM-made I suppose), and see how both of them fare in various scenarios.
Why do you think LLMs write prompts for LLMs badly? I use LLMs to write and refine prompts all the time. The prompts seem to come out very good. What are you basing this on?
Generally excessively verbose and failing to account for LLM failure modes. Example: when dispatching a subagent with an instruction prompt, I see the agent refer to "phase 5 of the plan" without actually pointing to the plan file it's referencing. The subagent says something to the tune of "phase 5, sure thing boss" and goes and generates garbage because it never reads the plan to find out what "phase 5" entails.
If I ask an agent to maintain agent-facing docs like AGENTS.md then similarly it ends up with low information density and not really saying what I want it to say. Something I thought was unequivocal ends up leaving the agent with far too much scope for interpretation and judgement calls.
Perhaps I'm just Prompting It Wrong (TM) but I do find LLMs lack the theory of mind to realise that other LLM instances don't know the things that the current instance knows, and make poor calls on what to include and what not to include. It's a bit hand-wavey, but I was wondering if the same issue might transfer to language design.
But basically world is a bit more narrow, that moment where your code touches the outside world (logging, http, etc), you can swap that. It's sorta like DI but deliberately narrower, only the moments where code touches the outside world are swappable.. With DI in theory you can replace anything, which has its benefits, but at least personally I am not a big fan of mocks, except when they touch the outside world. So that's what's replaceable.
>This feels like more of an OS problem (or library problem) than a language problem.
And from my perspective it's the exact opposite. You can apply a sandbox on the OS level and the library you call will crash at runtime. In your model you now get to harass the library author "hey you didn't tell me you do I/O" after it is already too late.
Crashing instead of raising an exception or bubbling a return code sounds like a legitimate bug ticket IMO.
One reason I'm in favour of handling capabilities in the OS is so we can stop having trivial symlink traversal and /.. traversal bugs in path filters (or indeed more complex bugs in the face of FS-specific linking primitives).
The name is a bit of a misnomer, though. It describes the statistical mode and implies the token selection, but what actually collapses is internal semantic mapping: if you give the model the same input concept, it will nearly always respond with the same or a very similar output concept, even if it has a wide range of valid options to select from. Sometimes it's not even 1:1, but many-to-one, which is what you see in case of the Jacquard collision. Even if you ban/zero some logits, it will try responding with the same idea in synonyms, and be really stubborn in it. The fact that the instruction-tuned model can and will vaguely predict the response before writing it makes longer thinking and longer replies introduce much less entropy than you'd think. Which means even less variance and more baked-in stereotypes.
Very cool, this is the first language I've seen that has effects/permissions annotated.
When reasoning about a bit of code, types and effects are two things that help me grok it. However, there are a lot more. Like: how much memory can this use, does it process any PII, how quickly should it respond, which users are allowed to run it. Our "type" system could then alert us when we call a function that takes 20 seconds straight from a button without a loading spinner.
I'm very curious to see whether a language with many annotations this could work. Probably a language stored in a database approach like Unison is needed.
Other than that, I'm also curious to know if Jacquard works well with LLMs in practice. How well does providing the language docs into the prompt actually work? The models are trained on mainstream languages so I've been doubtful whether new languages will be able to compete.
I’ve been very fascinated by the concept of a programming language designed for LLMs, both to that advantage of their strengths and to try and minimize their weaknesses.
I don't understand the code in the examples after simply reading it. That's a problem. You need to have comments to clarify or pick a better example of code.
You are in competition with Python and Javascript which has lots of code for training data. I wouldn't touch esoteric features unless it improves readability or is under the hood. The bigger problem with esoteric features is that its hard for humans in general to understand unless you cover it with enough syntactic sugar for people to write it down.
I love how people create so many new things with AI, but to think how much tokens, and in turn money we all have collectively burned for these side projects is crazy.
We only come to understand idea quality via evolutionary fitness tests, which means we have to MAKE things in order to find out if they're any good. Everything you make is gonna do hill-climbing in a [potentially unstable] environment in search of an ecological niche. This is true of species a la Darwin, but it's also true of products & ideas.
Anything we make gets to participate in this evolutionary process, even if it sounded dumb at the outset. In fact, doing dumb things is how we get "unstuck" from outdated ways of thinking.
it pales in comparison to all the brain cells burned chasing dumb ideas. Don't sweat it, and keep burning tokens in an effort to discover something great.
Streaming videos and playing heavy online games has a similar effect. Which do you think should be preferable: watching ~3 hours of video streaming, or generating a respectably-sized project that perfectly scratches an itch?
I don't really have a dog in this race, but "3 hours of streaming" is several orders of magnitude off using a frontier model to write a project like this. Like, 3000 hours of streaming would be closer. Then you're talking about one person consuming the resources for a useless project they'll abandon after a week vs. the resources consumed providing years of entertainment.
How did you get 3000? That's a wildly inaccurate and out-of-this-world figure. I did some preliminary research recently that I'm not fully recalling (would have to look for it), but I had a baseline of something like 9 minutes of streaming video using the equivalent resource (electricity, etc) of just over 5 average prompts. So 3 hours of streaming video is a bit over 1000 prompts.
I can get an OK tool that still goes a good distance in scratching an itch in say ~3-8 prompts, so with 1000 I'm looking at a small app. With 100000... I'm finding it a bit hard to imagine TBH; somewhere in the region of the Bun port to Rust[0] is my likely-off-base-but-not-as-bad-as-yours guess. Heck just consider that there are devs out there running systems with local models that are probably more than half the size of a frontier model, and how that 3000-hours-equivalent cost would've been eating into their pockets in the form of electricity bills.
Streaming is not computationally expensive at all. The computing and networking cost is so trivial it's difficult to measure, meaning most of the cost is in the cost of electricity your TV, computer monitor, or phone consumes with the display on. Some of this cost may even be rendered redundant if you were leaving the display on anyways; I certainly don't turn my PC monitor off when I'm done watching a video, even if I'm letting it idle for a while. At any rate, the typical device will be in the range of 0.1kwh per hour of streaming. Perhaps substantially less if most people watch on their phones these days.
"Average prompts" is a meaningless measurement. An 8x B200 GPU node, which is capable of running Deepseek V4-Pro, consumes in the ballpark of 300kwh per day while producing over 300 million tokens per day when operating at production loads, so we can get a roughly clean estimate of 1kwh per million tokens from a near-frontier open model. The project in the OP, which we'll take to be "respectably-sized", is 100k loc. How many tokens it takes to arrive at 100k loc is the most hand-wavy, difficult part to quantify here, but looking at my own usage, I've spent about 800 million tokens in the past two weeks on a 25k loc side project[1], and that was with full human-in-the-loop management, single agent usage. If we extrapolate that to 100k loc, we're looking at ~3.2 billion tokens, around 320kwh. Note this is for Deepseek, which is much cheaper to run than the frontier models. Inference on Fable or GPT 5.6 is likely another order of magnitude higher power cost, but those numbers are not disclosed, and we can only make rough inferences from their API token pricing (which also includes undisclosed amount of profit margin).
Anyways, 320kwh for a 100k loc project vs. 0.1kwh per hour of streaming comes out to my ballpark estimate of 3000 hours, but that was at Deepseek rates. It might actually be closer to 30,000 hours of streaming if this was done on Fable.
Note that the Bun project you cited burned $165,000 in tokens at their API rates. Perhaps you can look up the price of electricity in your region and do the math for yourself on how many hours having your TV on that would buy you. I'm estimating around 10 million hours, which sounds in line with my previous math: it's 1m loc written with Fable, so 10x the 30k number for 100k loc, with another magnitude for wasting tokens on swarms of agents checking each other and consuming many more tokens per loc than you would with human management and review. Even if you assume Fable API rate comes baked in with a 50% profit margin on inference, that would still leave you with 5 million hours of streaming for the electricity usage of the Bun rewrite.
[1] To be honest, I'm kind of shocked at how much electricity I've blown on this project, after having done the math! It actually resulted in something I'm shipping and maintaining with hundreds of real-world users, at least. The project would have been easier without LLMs, which have bungled so many things and taken so much effort to correct that it would have been significantly easier to do 90% of the work myself and leave only some data-munging to the bots, but I try to stay in tune with the absolute limits of what frontier models are capable of in real-world scenarios.
This is an interesting direction. I like the premise that the language is designed around AI-written, human-reviewed code rather than treating AI as just another code generator.
I'm curious about one thing though: what kinds of bugs or review tasks does Jacquard make substantially easier than existing languages? In other words, what's the smallest example where reviewing Jacquard is noticeably easier than reviewing Python or Rust? Thank you.
Why not use an existing language with proper algebraic effects? This language seems to be a language with something that looks like algebraic effects, but it doesn't look like it properly implemented those.
These effect systems seem nice enough when all your I/O is going through a narrow interface, but I'm wondering what happens when you have a very large API surface, like everything available to a web page running in a browser?
a feature that i thought would be good for llm-centric languages would be making something like python's doctest prominent where you put simple little unit tests in the docstring of the function rather than in some testing module someplace else.
it would make it easy for humans to easily stub out tests with a docstring description and the tests that would guarantee certain behaviors. for the machine, it'd make it easy to add in new tests because the function+tests are in the same context window
Can we train a model to understand machine code better so that we can direclty create application with machine code which can remove lot of abstractions of programming language.
The obvious problem with any new AI-centric programming language (I think this is the third I’ve seen at this point) is that any LLM you try to use with it will be starting from zero - it has nothing to copy from and very little documentation available to find on its own, if it’s even capable of doing web searches. Java or C++ or whatever may be “worse” by some standards for LLMs (or humans) to work with, but all LLMs in common use have already been trained on decades of existing code which it can crib from.
You're right, an LLM doesn't have preferences. As shorthand, I thought it a useful concept though, as they do have particular ways of writing that can be tuned for.
The effect system here is part of a type system (technically a type-and-effect system). Where a value type typically describes what value a computation returns, an effect describes which operations it can perform during evaluation.
The difference is that effects propagate through the call graph, so lower-level code cannot access disallowed resources without that authority appearing higher up. At runtime, world effects also require an explicit grant. In a typical value type system, lower-level code can introduce side effects without them appearing in the caller’s type.
Yea, LLM remove the burden of typing - so if token cost don’t explode the new high-level languages will be above the current - and the new low-level language will maybe just weights…
Like to believe we will one day use a low level language that is non-deterministic to build everything on top off seems to me in violation of such fundamental laws of information theory it's on the level of a belief in telekinesis.
> External/world effects are visible in function signatures
Brilliant. I think Jai has something like that? Each function declares what it's going to touch (both read/write) globally, and I think you can specify that per block even.
I haven't used Jai (I think it's not out yet) but I remember the author talking about this and it sounded like a great idea.
It's related to the idea of pure functions being easy to reason about. Right now most languages don't even have a concept of pure functions, but the ones that do, just have two categories. When a function is messing with global state you do actually want to know what it's doing.
I think that can be statically computed (and displayed as IDE annotations or whatever) but specifying it explicitly sounds like a good practice either way.
I like that you're including other side effects (e.g. network) there too though, that's pretty cool!
Another thing I'm really interested in is proofs. Not necessarily total proofs (though we seem to be moving in that direction, at least for subsets of the codebase), but just normalizing rudimentary pre and post conditions checks.
I was thinking of setting it up so code can't even compile in release mode if those are missing. (Not every function would need them, but you at least want to state their absence explicitly.)
I was also thinking of setting up strictness levels per function, using hashing or something, so if a function is modified, you'd have to go through a process of double checking it again. And then functions labeled e.g. level 7 strict couldn't call ones with a lower proven strictness level, and so on. I'm told that I've basically been reinventing Ada from first principles so I should probably go and take a look at that...
Wait, your thing is doing hashing too... Woah. (I think that comes from proofs land or something?)
Thanks for the pointer to Jai! I will check it out.
Yes, Jacquard uses content-addressed definitions and it should be possible to set up a process for 'review again if this changes' on top of it. Warp, the testing framework, already uses this to avoid rerunning pure tests when neither the definition or dependencies have changed.
Jacquard does not currently implement proof or strictness levels, but binding those to a definition’s content identity is interesting and definitely worth exploring.
What are you building that people keep comparing to Ada?
Haven't built anything yet as far as language / tooling goes, but my experiences with poorly designed languages and tooling have basically made me paranoid as far as "what touches this variable", "what does this function actually do", ended up arriving in Rust / Haskell-ish territory just by recoiling from the pain of PHP/WordPress, where everything is global, mutable, dynamic and designed to make you go completely insane.
(In a nutshell: the more dynamic a programming language is, the more impossible it becomes to reason about what the program is doing. That's fine for throwaway scripts and game jams (#pragma JamMode), but the "the program is probably seriously wrong" should be explicitly opt-in, not the default ...)
So my current approach is everything should be as local as possible, as immutable as possible, etc. Basically Rust except I'm not a fan of low level programming. (I basically want Rust With GC (C# memory model), which sends Rust folks into paroxysms!)
I want it stricter than Rust in many respects, but also less annoying. (Better ergonomics and higher level, with a low level escape hatch when needed.)
Swift is apparently close to that, and might deserve another look.
I've also been obsessed with formal verification and proofs except, today I had a very revealing experience. I had an LLM incorrectly implement a major feature, and "verify" it with several thousand lines of tests. It was, as implemented, backwards, and also completely pointless — but all tests came back green!
I laughed when I realized, if it had been done in Rust, and with a layer of formal verification... well all the proofs (of the stupid and wrong and backwards thing) would have come back green too... It would have mathematically proven that the incorrect thing was correct...
Ah, that’s a tough one. My hope is that Jacquard, or some similar language, can make intent explicit through readable, high-level constructs, giving both the human reviewer and the model a clearer target while the details are filled in. But it still can’t tell us whether the original intent was right.
Coincidentally, you reminded me of one of my favorite Charles Babbage quotes:
> “On two occasions I have been asked, ‘Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?’ I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.”
I suppose catching that kind of mistake is how humans prove to our future AI overlords that we should be kept around.
Not just effects, multi-shot effects:
> Algebraic effects with deep, multi-shot handlers. A handler can resume a computation zero, one, or many times, which is what makes exhaustive search and exact inference ordinary library code.
I love me some algebraic effects, but this sets off alarm bells for me.
Multi-shot effects are neat and powerful but are incredibly difficult to reason about for humans, and given that there are few languages that have implemented them (and they're so niche with very little training data) I'm skeptical that LLMs are sufficiently better at them to make up the gap.
One-shot effects are a cross between `throw` and a function call. Multi-shot are full-on delimited continuations and come with all the complexity that entails.
> Multi-shot effects are neat and powerful but are incredibly difficult to reason about for humans
I had a very similar thought and ended up building a capability system[1] in my language to capture side effects.
[1] https://march-lang.org/docs/cookbook/capabilities/ * docs are a work in progress
That’s a fair concern. I’d expect multi-shot handlers to be relatively rare and mostly something library code uses rather than a day-to-day utility. But an LLM given free rein may decide to behave differently.
For now, I am looking into making a clearer distinction between one-shot and multi-shot handlers so we can reject cases where resuming again would be unsafe, like around filesystem writes.
More to explore here, certainly.
AST hashing & effects are both used in Unison, if you want to check that out. I don't know that they have any proof-land stuff, though.
> a C-emitting native AOT backend that currently compiles the kernel .jqd carrier
This human reviewer gave up at line 4 of the Readme. What is the kernel? What does the carrier carry? Why does it need to be carried? Where does the .jqd come from?
Also, for whatever reason, from the submission here:
> I had AI analyze the ASTs of several mainstream languages plus a few of the conceptually groundbreaking but esoteric ones (listed in the README)
No, they are not listed in the Readme. Why lie to us right off the bat?
I like the general idea, but the human responsible for this seems to demonstrate no interest in actually reviewing their system's output. Which does not bode well.
Just in case the above is not considered constructive enough, how about this: In my experience, with some prompting and iterations, AIs do reasonably well at writing solid, "boring" code. They do really badly at writing useful documentation, and this is a major pain point when reviewing their output. A system that claims to be oriented towards human review must address this somehow. It must talk about how it addresses it, and it must demonstrate that its own documentation is significantly better than unfiltered LLM output.
This system does not demonstrate this. It needs clear guidelines: Start with the big picture. Show an example. Show comments in that example. Don't get bogged down in the details of the internal workings of the compiler. Don't use jargon like "kernel" to mean "lowered intermediate representation". Don't use "carrier" when you mean "file". It's just a file, FFS. But again, don't tell me at all about the fact that there is an intermediate representation; that's not "big picture", and anyone who cares about compiler internals will already have guessed that you have an intermediate representation. Do tell me about the effect system in the first two paragraphs! That seems to be what sets this language apart. Do give me a brief argument why an effect system is a good match for a system where humans don't have to type everything by hand. And so on and so forth.
I always feel like I'm losing my mind when reading LLM comments or prose for code. It's so esoteric and 'i-am-very-smart' and makes me feel like I'm not supposed to read the comments in the first place
Once, during an especially tedious review session, I told my agent something like "comments are not meant to demonstrate how well you understand the system, they are meant to help others understand the system". It felt like that helped.
Given how poorly LLMs do with writing prompts for LLMs, I'm not sure I'd trust their judgement in designing a language for LLMs.
> and the runtime requires explicit permission to touch the filesystem, network, etc
This feels like more of an OS problem (or library problem) than a language problem.
> Run one program against many worlds. The same code can run against the real network, a scripted fake, a recording of last week's traffic, or a probability model of how servers usually behave
How is the "world" model different from plain dependency injection?
> How is the "world" model different from plain dependency injection?
In addition to what the other comment said, this "world" model is great for hermetic testing of complex code, LLM written or not. We've seen existing projects that intercept the OS level syscall for testing, replayability, etc. Building it into the language runtime, hopefully with better ergonomics from the start than a syscall, would be a welcome addition broadly.
Dependency injection provides the same hermetic-testing capability, so that's not really an answer to OP's question. Effect-as-world-model is, in this specific way, just a special argument slot that only accepts dependency-injected functions.
What effects provide beyond DI is entirely in their ability to abort (resume zero times) or (in the case of multi-shot effects) resume multiple times. An effect that resumes exactly once is structurally the same as a dependency injected lambda.
> What effects provide beyond DI is entirely in their ability to abort (resume zero times) or (in the case of multi-shot effects) resume multiple times
Thanks, this answers my question and does sound handy. Also I do see the value in having more problem-shaped versions of a general construct.
> Given how poorly LLMs do with writing prompts for LLMs, I'm not sure I'd trust their judgement in designing a language for LLMs.
Yeah agree, what I really want to see now, is a programming language for LLMs, designed by a human (although code could still be LLM-made I suppose), and see how both of them fare in various scenarios.
Why do you think LLMs write prompts for LLMs badly? I use LLMs to write and refine prompts all the time. The prompts seem to come out very good. What are you basing this on?
Generally excessively verbose and failing to account for LLM failure modes. Example: when dispatching a subagent with an instruction prompt, I see the agent refer to "phase 5 of the plan" without actually pointing to the plan file it's referencing. The subagent says something to the tune of "phase 5, sure thing boss" and goes and generates garbage because it never reads the plan to find out what "phase 5" entails.
If I ask an agent to maintain agent-facing docs like AGENTS.md then similarly it ends up with low information density and not really saying what I want it to say. Something I thought was unequivocal ends up leaving the agent with far too much scope for interpretation and judgement calls.
Perhaps I'm just Prompting It Wrong (TM) but I do find LLMs lack the theory of mind to realise that other LLM instances don't know the things that the current instance knows, and make poor calls on what to include and what not to include. It's a bit hand-wavey, but I was wondering if the same issue might transfer to language design.
I've done a version of "world" for Sigil, a programming language I was kinda doing but stopped, also for agents. LLM generated article here https://inerte.github.io/sigil/articles/worlds-not-mocks/
But basically world is a bit more narrow, that moment where your code touches the outside world (logging, http, etc), you can swap that. It's sorta like DI but deliberately narrower, only the moments where code touches the outside world are swappable.. With DI in theory you can replace anything, which has its benefits, but at least personally I am not a big fan of mocks, except when they touch the outside world. So that's what's replaceable.
>This feels like more of an OS problem (or library problem) than a language problem.
And from my perspective it's the exact opposite. You can apply a sandbox on the OS level and the library you call will crash at runtime. In your model you now get to harass the library author "hey you didn't tell me you do I/O" after it is already too late.
Crashing instead of raising an exception or bubbling a return code sounds like a legitimate bug ticket IMO.
One reason I'm in favour of handling capabilities in the OS is so we can stop having trivial symlink traversal and /.. traversal bugs in path filters (or indeed more complex bugs in the face of FS-specific linking primitives).
lol I had started a “plugin compiler” that Claude also wanted to name after the Jacquard loom https://github.com/jondwillis/jacq
Fricking training distribution/same-y ness coming for us all… cool project though.
The distribution is actually pretty diverse, but this diversity is constrained by mode collapse.
Thanks for the clarification. “Mode collapse” explains so much about how I have been experiencing output convergence and repetition for a while now.
The name is a bit of a misnomer, though. It describes the statistical mode and implies the token selection, but what actually collapses is internal semantic mapping: if you give the model the same input concept, it will nearly always respond with the same or a very similar output concept, even if it has a wide range of valid options to select from. Sometimes it's not even 1:1, but many-to-one, which is what you see in case of the Jacquard collision. Even if you ban/zero some logits, it will try responding with the same idea in synonyms, and be really stubborn in it. The fact that the instruction-tuned model can and will vaguely predict the response before writing it makes longer thinking and longer replies introduce much less entropy than you'd think. Which means even less variance and more baked-in stereotypes.
It's a good name regardless! Don't stop on my account.
Very cool, this is the first language I've seen that has effects/permissions annotated.
When reasoning about a bit of code, types and effects are two things that help me grok it. However, there are a lot more. Like: how much memory can this use, does it process any PII, how quickly should it respond, which users are allowed to run it. Our "type" system could then alert us when we call a function that takes 20 seconds straight from a button without a loading spinner.
I'm very curious to see whether a language with many annotations this could work. Probably a language stored in a database approach like Unison is needed.
Other than that, I'm also curious to know if Jacquard works well with LLMs in practice. How well does providing the language docs into the prompt actually work? The models are trained on mainstream languages so I've been doubtful whether new languages will be able to compete.
Very cool!
I’ve been very fascinated by the concept of a programming language designed for LLMs, both to that advantage of their strengths and to try and minimize their weaknesses.
Here’s a take I experimented with last year (feel very free to steal anything useful): https://github.com/GoogleCloudPlatform/aether/tree/feature/v...
(Note: main has a version that was an ugly syntax, the branch in the link was for a prettier attempt)
I don't understand the code in the examples after simply reading it. That's a problem. You need to have comments to clarify or pick a better example of code.
You are in competition with Python and Javascript which has lots of code for training data. I wouldn't touch esoteric features unless it improves readability or is under the hood. The bigger problem with esoteric features is that its hard for humans in general to understand unless you cover it with enough syntactic sugar for people to write it down.
I love how people create so many new things with AI, but to think how much tokens, and in turn money we all have collectively burned for these side projects is crazy.
We only come to understand idea quality via evolutionary fitness tests, which means we have to MAKE things in order to find out if they're any good. Everything you make is gonna do hill-climbing in a [potentially unstable] environment in search of an ecological niche. This is true of species a la Darwin, but it's also true of products & ideas.
Anything we make gets to participate in this evolutionary process, even if it sounded dumb at the outset. In fact, doing dumb things is how we get "unstuck" from outdated ways of thinking.
As long as token costs are being subsidized, I don't mind that this is the result of the consumer surplus.
it pales in comparison to all the brain cells burned chasing dumb ideas. Don't sweat it, and keep burning tokens in an effort to discover something great.
Not just money. A lot of CO2 was dumped into our atmosphere and tons of clean water were evaporated as well.
Streaming videos and playing heavy online games has a similar effect. Which do you think should be preferable: watching ~3 hours of video streaming, or generating a respectably-sized project that perfectly scratches an itch?
I don't really have a dog in this race, but "3 hours of streaming" is several orders of magnitude off using a frontier model to write a project like this. Like, 3000 hours of streaming would be closer. Then you're talking about one person consuming the resources for a useless project they'll abandon after a week vs. the resources consumed providing years of entertainment.
How did you get 3000? That's a wildly inaccurate and out-of-this-world figure. I did some preliminary research recently that I'm not fully recalling (would have to look for it), but I had a baseline of something like 9 minutes of streaming video using the equivalent resource (electricity, etc) of just over 5 average prompts. So 3 hours of streaming video is a bit over 1000 prompts.
I can get an OK tool that still goes a good distance in scratching an itch in say ~3-8 prompts, so with 1000 I'm looking at a small app. With 100000... I'm finding it a bit hard to imagine TBH; somewhere in the region of the Bun port to Rust[0] is my likely-off-base-but-not-as-bad-as-yours guess. Heck just consider that there are devs out there running systems with local models that are probably more than half the size of a frontier model, and how that 3000-hours-equivalent cost would've been eating into their pockets in the form of electricity bills.
[0] https://bun.com/blog/bun-in-rust
Streaming is not computationally expensive at all. The computing and networking cost is so trivial it's difficult to measure, meaning most of the cost is in the cost of electricity your TV, computer monitor, or phone consumes with the display on. Some of this cost may even be rendered redundant if you were leaving the display on anyways; I certainly don't turn my PC monitor off when I'm done watching a video, even if I'm letting it idle for a while. At any rate, the typical device will be in the range of 0.1kwh per hour of streaming. Perhaps substantially less if most people watch on their phones these days.
"Average prompts" is a meaningless measurement. An 8x B200 GPU node, which is capable of running Deepseek V4-Pro, consumes in the ballpark of 300kwh per day while producing over 300 million tokens per day when operating at production loads, so we can get a roughly clean estimate of 1kwh per million tokens from a near-frontier open model. The project in the OP, which we'll take to be "respectably-sized", is 100k loc. How many tokens it takes to arrive at 100k loc is the most hand-wavy, difficult part to quantify here, but looking at my own usage, I've spent about 800 million tokens in the past two weeks on a 25k loc side project[1], and that was with full human-in-the-loop management, single agent usage. If we extrapolate that to 100k loc, we're looking at ~3.2 billion tokens, around 320kwh. Note this is for Deepseek, which is much cheaper to run than the frontier models. Inference on Fable or GPT 5.6 is likely another order of magnitude higher power cost, but those numbers are not disclosed, and we can only make rough inferences from their API token pricing (which also includes undisclosed amount of profit margin).
Anyways, 320kwh for a 100k loc project vs. 0.1kwh per hour of streaming comes out to my ballpark estimate of 3000 hours, but that was at Deepseek rates. It might actually be closer to 30,000 hours of streaming if this was done on Fable.
Note that the Bun project you cited burned $165,000 in tokens at their API rates. Perhaps you can look up the price of electricity in your region and do the math for yourself on how many hours having your TV on that would buy you. I'm estimating around 10 million hours, which sounds in line with my previous math: it's 1m loc written with Fable, so 10x the 30k number for 100k loc, with another magnitude for wasting tokens on swarms of agents checking each other and consuming many more tokens per loc than you would with human management and review. Even if you assume Fable API rate comes baked in with a 50% profit margin on inference, that would still leave you with 5 million hours of streaming for the electricity usage of the Bun rewrite.
[1] To be honest, I'm kind of shocked at how much electricity I've blown on this project, after having done the math! It actually resulted in something I'm shipping and maintaining with hundreds of real-world users, at least. The project would have been easier without LLMs, which have bungled so many things and taken so much effort to correct that it would have been significantly easier to do 90% of the work myself and leave only some data-munging to the bots, but I try to stay in tune with the absolute limits of what frontier models are capable of in real-world scenarios.
Doesn't stop people from mass-producing and consuming beef lol, people just don't care aside from performative lip-service.
This is an interesting direction. I like the premise that the language is designed around AI-written, human-reviewed code rather than treating AI as just another code generator.
I'm curious about one thing though: what kinds of bugs or review tasks does Jacquard make substantially easier than existing languages? In other words, what's the smallest example where reviewing Jacquard is noticeably easier than reviewing Python or Rust? Thank you.
Why not use an existing language with proper algebraic effects? This language seems to be a language with something that looks like algebraic effects, but it doesn't look like it properly implemented those.
These effect systems seem nice enough when all your I/O is going through a narrow interface, but I'm wondering what happens when you have a very large API surface, like everything available to a web page running in a browser?
a feature that i thought would be good for llm-centric languages would be making something like python's doctest prominent where you put simple little unit tests in the docstring of the function rather than in some testing module someplace else.
it would make it easy for humans to easily stub out tests with a docstring description and the tests that would guarantee certain behaviors. for the machine, it'd make it easy to add in new tests because the function+tests are in the same context window
> for the machine, it'd make it easy to add in new tests because the function+tests are in the same context window
Given how many useless unit tests Codex just loves to write, no, thank you.
Thankfully, its training data completely misses this. So at least for now it completely ignores doctests (at least in Elixir).
Very cool -- I often thought that Unisons caching approach should be adopted more widely. Look forward to poking at it later!
Can we train a model to understand machine code better so that we can direclty create application with machine code which can remove lot of abstractions of programming language.
The obvious problem with any new AI-centric programming language (I think this is the third I’ve seen at this point) is that any LLM you try to use with it will be starting from zero - it has nothing to copy from and very little documentation available to find on its own, if it’s even capable of doing web searches. Java or C++ or whatever may be “worse” by some standards for LLMs (or humans) to work with, but all LLMs in common use have already been trained on decades of existing code which it can crib from.
This is definitely true. The language is intentionally small enough to fit into a single SKILL.md file, for what it's worth:
https://github.com/jbwinters/jacquard-lang/blob/main/docs/SK...
Agents I've tested with have had been able to pick up the language from that, at least to the extent that I've been able to test so far.
> very little documentation available to find on its own
Building something like haskel's hoogle seems like a smart approach to this problem.
An llm doesnt 'prefer' to use anything.
Whats different to the effects system just being a type system?
Isnt a function that opens a particular file still just a type? Perhaps dependent?
Reminds me of mercury which has determinism of the function like type signatures.
You're right, an LLM doesn't have preferences. As shorthand, I thought it a useful concept though, as they do have particular ways of writing that can be tuned for.
The effect system here is part of a type system (technically a type-and-effect system). Where a value type typically describes what value a computation returns, an effect describes which operations it can perform during evaluation.
The difference is that effects propagate through the call graph, so lower-level code cannot access disallowed resources without that authority appearing higher up. At runtime, world effects also require an explicit grant. In a typical value type system, lower-level code can introduce side effects without them appearing in the caller’s type.
"Esperanto for Clankers"
Yea, LLM remove the burden of typing - so if token cost don’t explode the new high-level languages will be above the current - and the new low-level language will maybe just weights…
Like to believe we will one day use a low level language that is non-deterministic to build everything on top off seems to me in violation of such fundamental laws of information theory it's on the level of a belief in telekinesis.
All of human technology resulted from human brains, which are non-deterministic at the level you're talking about, right?
More like all of human technology is dedicated to make deterministic systems out of our feeble non-deterministic minds.