Dynamic linking with a safe ABI, where if you change and recompile one library then the outcome has to obey some definition of safety, and ABI stability is about as good as C or Objective-C or Swift.
Until that happens, it'll be hard to adopt Rust in a lot of C/C++ strongholds where C's ABI and dynamic linking are the thing that enables the software to get huge.
> Until that happens, it'll be hard to adopt Rust in a lot of C/C++ strongholds where C's ABI and dynamic linking are the thing that enables the software to get huge.
Wait, Rust can already communicate using the C ABI. In fact, it offers exactly the same capabilities as C++ in this regard (dynamic linking).
As unsafe as C or C++. In fact, safer, because only the ABI surface is unsafe, the rust code behind it can be as safe or unsafe as you want it to be.
I was addressing this portion of your comment: "C's ABI and dynamic linking are the thing that enables the software to get huge". If the C ABI is what enables software to get huge then Rust is already there.
There is a second claim in your comment about a "safe ABI", but that is something that neither C or C++ offers right now.
Here's the problem. If you told me that you rebuilt the Linux userland with Rust but you used C ABI at all of the boundaries, then I would be pretty convinced that you did not create a meaningful improvement to security because of how many dynamic linking boundaries there are. So many of the libraries involved are small, and big or small they expose ABIs that involve pointers to buffers and manual memory management.
> There is a second claim in your comment about a "safe ABI", but that is something that neither C or C++ offers right now.
Of course C and C++ are no safer in this regard. (Well, with Fil-C they are safer, but like whatever.)
But that misses the point, which is that:
- It would be a big deal if Rust did have a safe dynamic linking ABI. Someone should do it. That's the main point I'm making. I don't think deflecting by saying "but C is no safer" is super interesting.
- So long as this problem isn't fixed, the upside of using Rust to replace a lot of the load bearing stuff in an OS is much lower than it should be to justify the effort. This point is debatable for sure, but your arguments don't address it.
> - It would be a big deal if Rust did have a safe dynamic linking ABI. Someone should do it. That's the main point I'm making. I don't think deflecting by saying "but C is no safer" is super interesting.
I think we all agree that it would be a huge deal.
> - So long as this problem isn't fixed, the upside of using Rust to replace a lot of the load bearing stuff in an OS is much lower than it should be to justify the effort. This point is debatable for sure, but your arguments don't address it.
As you point out, this is the debatable part, and I'm not sure I get your justification here.
This might end up being the forcing function (quoting myself from another reply in this discussion):
> It can't be that replacing 20 C/C++ shared objects with 20 Rust shared objects results in 20 copies of the Rust standard library and other dependencies that those Rust libraries pull in. But, today, that is what happens. For some situations, this is too much of a memory usage regression to be tolerable.
If memory was cheap, then maybe you could say, "who cares".
What you are asking for is to make a library definition replacement to .h-files that contain sufficient information to make rust safe. That is a big, big step and would be fantastic not only for rust but for any other language trying to break out of the C tar pit.
So you're calling for dynamic linking for rust native code? Because rust's safety doesn't come from runtime, it comes from the compiler and the generated code. An object file generated from a bit of rust source isn't some "safe" object file, it's just generated in a safe set of patterns. That safety can cross the C ABI perfectly fine if both things on either side came from rust to begin with. Which means rust dynamic linking.
I don’t think GP is moving the goalposts at all, rather I think a lot of people are willfully misrepresenting GP’s point.
Rust-to-rust code should be able to be dynamically linked with an ABI that has better safety guarantees than the C ABI. That’s the point. You can’t even express an Option<T> via the C ABI, let alone the myriad of other things rust has that are put together to make it a safe language.
It would be very hard to accomplish. Apple was extremely motivated to make Swift have a resilient/stable ABI, because they wanted to author system frameworks in swift and have third parties use them in swift code (including globally updating said frameworks without any apps needing to recompile.) They wanted these frameworks to feel like idiomatic swift code too, not just be a bunch of pointers and manual allocation. There’s a good argument that (1) Rust doesn’t consider this an important enough feature and (2) they don’t have enough resources to accomplish it even if they did. But if you could wave a magic wand and make it “done”, it would be huge for rust adoption.
No fundamental reason, that I know of, why Rust or any other safe language can't also have some kind of story here.
> I think you're moving the goalposts significantly here.
No. I'm describing a problem worth solving.
Also, I think a major chasm for Rust to cross is how defensive the community gets. It's important to talk about problems so that the problems can be solved. That's how stuff gets better.
A safe ABI would be cool, for sure, but in the market (specifically addressing your prediction) I don't know if it's really that big a priority for adoption. The market is obviously fine with an unsafe ABI, seeing how C/C++ is already dominant. Rust with an unsafe ABI might then not be as big an improvement as we would like, but it's still an improvement, and I feel like you're underestimating the benefits of safe Rust code as an application-level frontline of security, even linked to unsafe C code.
C++ ABI stability is the main reason improvements to the language get rejected.
You cannot change anything that would affect the class layout of something in the STL. For templated functions where the implementation is in the header, ODR means you can't add optimizations later on.
Maybe this was OK in the 90s when companies deleted the source code and laid off the programmers once the software was done, but it's not a feature Rust should ever support or guarantee.
The "stable ABI" is C functions and nothing else for a very good reason.
I think if Rust wants to evolve even more aggressively than C++ evolves, then that is a chasm that needs to be crossed.
In lots of domains, having a language that doesn't change very much, or that only changes very carefully with backcompat being taken super seriously, is more important than the memory safety guarantees Rust offers.
As a C++ developer, I regularly deal with people that think creating a compiled object file and throwing away the source code is acceptable, or decide to hide source code for "security" while distributing object files. This makes my life hell.
Rust preventing this makes my life so much better.
> As a C++ developer, I regularly deal with people that think creating a compiled object file and throwing away the source code is acceptable, or decide to hide source code for "security" while distributing object files. This makes my life hell.
I mean yeah that's bad.
> Rust preventing this makes my life so much better.
I'm talking about a different issue, which is: how do you create software that's in the billions of lines of code in scale. That's the scale of desktop OSes. Probably also the scale of some other things too.
At that scale, you can't just give everyone the source and tell them to do a world compile. Stable ABIs fix that. Also, you can't coordinate between all of the people involved other than via stable ABIs. So stable ABIs save both individual build time and reduce cognitive load.
This is true even and especially if everyone has access to everyone else's source code
> At that scale, you can't just give everyone the source and tell them to do a world compile. Stable ABIs fix that. Also, you can't coordinate between all of the people involved other than via stable ABIs. So stable ABIs save both individual build time and reduce cognitive load.
Rust supports ABI compatibility if everyone is on the same compiler version.
That means you can have a distributed caching architecture for your billion line monorepo where everyone can compile world at all times because they share artifacts. Google pioneered this for C++ and doesn't need to care about ABI as a result.
What Rust does not support is a team deciding they don't want to upgrade their toolchains and still interoperate with those that do. Or random copy and pasting of `.so` files you don't know the provenance of. Everyone must be in sync.
In my opinion, this is a reasonable constraint. It allows Rust to swap out HashMap implementations. In contrast, C++ map types are terrible for performance because they cannot be updated for stability reasons.
My understanding: Even if everyone uses the same toolchain, but someone changes the code for a module and recompiles, then you're in UB land unless everyone who depends on that recompiles
If your key is a hash of the code and its dependencies, for a given toolchain and target, then any change to the code, its dependencies, the toolchain or target will result in a new key unique to that configuration. Though I am not familiar with these distributed caching systems so I could be overlooking something.
> Except as you well know, C might not change as fast, but it does change, including the OS ABI.
I don't know that.
Here's what I know: the most successful OSes have stable OS ABIs. And their market share is positively correlated with the stability of their ABIs.
Most widely used: Windows, which has a famously stable OS ABI. (If you wanted to be contrarian you could say that it doesn't because the kernel ABI is not stable, but that misses the point - on Windows you program against userland ABIs provided by DLLs, which are remarkably stable.)
Second place: macOS, which maintains ABI stability with some sunsetting of old CPU targets. But release to release the ABI provides solid stability at the framework level, and used to also provide stability at the kernel ABI level (not sure if that's still true - but see above, the important thing is userland framework ABI stability at the end of the day).
Third place: Linux, which maintains excellent kernel ABI stability. Linux has the stablest kernel ABI right now AFAIK. And in userland, glibc has been investing heavily in ABI stability; it's stable enough now that in practice you could ship a binary that dynlinks to glibc and expect it to work on many different Linuxes today and in the future.
So it would seem that OS ABIs are stable in those OSes that are successful.
Speaking of Windows alone, there are the various calling conventions (pascal, stdcall, cdecl), 16, 32, 64 bits, x86, ARM, ARM64EC, DLLs, COM in-proc and ext-proc, WinRT within Win32 and UWP.
Leaving aside the platforms it no longer supports.
So there are some changes to account for depending on the deployment scenario.
What's the stat of single-compiler version ABI? I mean - if the compiler guaranteed that for the same version of the compiler the ABI can work, we could potentially use dynamic linking for a lot of things (speed up iterative development) without committing to any long term stable API or going through C ABI for everything.
Dynamic linking is also great for compile time of debug builds. If a large library or application is split up into smaller shared libraries, ones unaffected by changes don't need to be touched at all. Runtime dynamic linking has a small overhead, but it's several orders of magnitude faster than compile-time linking, so not a problem in debug builds.
for developer turnaround time, it is huge. we explicitly do not statically link Ardour because as developers we are in the edit-compile-debug cycle all day every day, and speeding up the link step (which dynamic linking does dramatically, especially with parallel linkers like lld) is a gigantic improvement to our quality of life and productivity.
1) It can't be that replacing 20 C/C++ shared objects with 20 Rust shared objects results in 20 copies of the Rust standard library and other dependencies that those Rust libraries pull in. But, today, that is what happens. For some situations, this is too much of a memory usage regression to be tolerable.
2) If you really have 20 libraries calling into one another using C ABI, then you end up with manual memory management and manual buffer offset management everywhere even if you rewrite the innards in Rust. So long as Rust doesn't have a safe ABI, the upside of a Rust rewrite might be too low in terms of safety/security gained to be worth doing
I found c ABI a bit too difficult in rust compared to c or zig. Mainly because of destructors. I am guessing c++ would be difficult in a similar way.
Also unsafe rust has always on strict-aliasing, which makes writing code difficult unless you do it in certain ways.
Having glue libraries like pyo3 makes it good in rust. But that introduces bloat and other issues. This has been the biggest issue I had with rust, it is too hard to write something so you use a dependency. And before you know it, you are bloating out of control
Not really. The foreign ABI requires a foreign API, which adds friction that you don't have with C exporting a C API / ABI. I've never tried, but I would guess that it adds a lot of friction.
COM is interesting as it implements interfaces using the C++ vtable layout, which can be done in C. Dynamic COM (DCOM) is used to provide interoperability with Visual Basic.
You can also access .NET/C# objects/interfaces via COM. It has an interface to allow you to get the type metadata but that isn't necessary. This makes it possible to e.g. get the C#/.NET exception stack trace from a C/C++ application.
One particular chasm to keep an eye on, possibly even more relevant than Ubuntu using Rust: When it comes to building important stuff, Ubuntu sticks to curl|YOLO|bash instead of trusting trust in their own distributions.
When people say "curl|bash", this usually means secondary fetches, random system config changes, likely adding stuff to user's .bashrc
But it's not quite that bad in this particular case - they are fetching pre-built static toolchain, and running old-school install script, just like in 1990s. The social convention for those is quite safer.
(Although I agree, it is pretty ironic that they prefer this to using ppa or binary packaged into deb...)
In practice, very rarely. Lots of 'curl | sh' do secondary fetches, and those don't come with hash checks. And even if they come with hash checks _today_, there is no guarantee next version won't quietly remove them.
Aren't the versions of Rust in stable Linux distributions like, a century old? Or at least they were last I checked what Debian and Ubuntu LTS were distributing. I think it's because they don't like static linking.
I believe Rust is typically only used through `apt` as a dependency for system packages written in Rust, or for building system packages that are written in Rust, so that they can link against a single shared instance of the Rust Standard Library.
Unrelated to the language debate, but it seems a lot of people here missed the fact that Rust Coreutils project is licensed under MIT, and I am not sure if I feel that it is the appropriate license for such project. As much as FSF's philosophy has bad PR at times with Stallman, the GPL licenses really do protect open source. Who knows what Canonical would do when all parts of Ubuntu become MIT...
They did, until the automatic copyright laundering machine was invented. Pretty much every piece of GPL code ever written is now being magically transmuted into MIT/BSD or proprietary code, and the FSF has no solution.
A discussion on licenses will go sideways very quickly. GPL does limit the adoption of software in certain environments. So it really depends on your goals. Do you want an OSS project that will be useable by everyone (including corporations) or do you want to guarantee that the software will always be OSS and guarantee that Corporations can’t benefit from it without contributing back (potentially requiring them to open their own proprietary code).
There’s a lot of moral perspective that people apply to this decision, but not all developers have the same goals for their software. MIT is more flexible in its use than GPL, but doesn’t help ensure that software remains open.
Really good references to "crossing the chasm" between early adopter needs and mainstream needs. In addition to the Ubuntu coreutils use case, I wonder what other chasms Rust is attempting to cross. I know Rust for Linux (though I think that's still relegated to drivers?) and automotive (not sure where that is).
If you want to take a look at some of the "big drivers", the Project Goals[1] is the right place. These are goals proposed by the community and the language developers put together, they are not explicit milestones or must-haves, but they do serve as a guideline to what the project tries to put its time and effort on.
There are big pushes in pretty much every direction. The projects that really stand out to me are pyo3 (Replace c++ python modules with rust), Dioxus (react-like web framework), The ferrocine qualified compiler (automotive)
I think right now the ecosystem is pretty ripe and with DARPA TRACTOR there are only more and more reasons every day to put rust on your toolbelt.
I am secretly hoping that eventually we break free from the cycle of "hire a senior dev and he likes rust so the company switches" over to hey let's hire some good mid-level and junior rust developers
Are mid level and junior developers being hired anywhere for any reason right now? I don't mean specifically rust developers. I mean software developers.
Sure. There was an article a week or two ago about IBM aggressively hiring juniors. Of course the fact that is noteworthy probably means something in itself....
The author refers to a few things that he thinks will appeal to the "early majority," but I feel like that's a weakness of the article. Is the author part of the "early majority?" (doesn't seem like it). Does he have the same problems that they have? How does he know?
.NET has a _huge_ platform library and you know what? It’s a pleasure. So many things are just the standard way of doing things. When things are done weirdly, you can usually get a majority in favour of standardising it.
Yes, there’s always a couple of people who really push the boat out…
I've been a fan of all rust-based utilities that I've used. I am worried that 20+ (??) years of bug fixes and edge-case improvements can't be accounted for by simply using a newer/better code-base.
A lot of bug fixes/exploits are _CAUSED_ by the C+ core, but still... Tried & true vs new hotness?
I do get what you mean, but Rust has been baking for a decade, finally took off after 10 years of baking, and now that is been repeatedly tried and tested it is eating the world, as some developers suggested it could eventually do so. I however do think this shows a different problem:
If nobody writes unit tests, how do you write them when you port over projects to ensure your new language doesn't introduce regressions. All rewrites should be preceded by strong useful unit tests.
I was born in 1990 so I get it! I still say 21 when people ask me how old I am... Aka how old do I need to say I am to be able to drink alcohol LOL I don't drink that often mind you. I just don't really think about my age a whole lot...
Ideally, but if a project wasn't written with tests at the time then finding a working time machine can be a challenge. If you try to add them later you won't capture all the nuance that went into the original program. After all, if the implementation code was expressive enough to capture that nuance, you'd already have your test suite, so to speak. Tests are written to fill in the details that the rest of the code isn't able to express.
Rust has editions for strong stability guarantees, and has had them for nearly a decade i believe. Besides, tech backing has grown way past the risky point.
FWIW, the GP comment's claim that you're lucky if you can compile 2-year-old code is exaggerated, but so is yours. Rust does not offer "strong stability guarantees". Adding a new method to a standard type or trait can break method inference, and the Rust standard library does that all the time.
In C or C++, this isn't supposed to happen: a conformant implementation claiming to support e.g. C++17 would use ifdefs to gate off new C++20 library functions when compiling in C++17 mode.
> and the Rust standard library does that all the time.
I don't doubt this is true, but do you have an example? I think I haven't run into a build breaking like this in std in like maybe seven/eight years. In my experience breaking changes/experimental apis are typically ensconced in features or gated by editions.
Granted, it'd be nice to be able to enforce abi stability at the crate level, but managing that is its own can of worms.
I did find that the breakage rfc allows for breaking inference, which tbh seems quite reasonable... inference is opt-in.
Almost every major release of rust stabilizes new library methods. For example, the latest major release (1.93) stabilized Vec::into_raw_parts. This isn’t gated by an edition. So if you had a trait with a method “into_raw_parts” which you had defined on Vec, after updating to 1.93 or later your code will either fail to compile, or start running different code when that method is called.
Sorry, I meant to write “method resolution”, not inference. This isn’t the same issue as type inference (though indeed, stdlib changes can break that too)
> years of bug fixes and edge-case improvements can't be accounted for by simply using a newer/better code-base.
Partially is in fact true: Just because the Rust use a better type system (after ML) + better resource model (aka borrow checker), and if you are decently good, you eliminate, forever!, tons of problems.
It can't solve things that arise by complex interactions or just lack of port subtle details like in parsing poor inputs (like html) but is true that changing the language in fact solve tons of things.
> Jon made the provocative comment that we needed to revisit our policy around having a small standard library. He’s not the first to say something like that, it’s something we’ve been hearing for years and years
It sounds to me like you "cross the chasm" a little too early. As a user I don't care about your "chasms" I care about high quality durable systems. This isn't the first time I've heard the "we'll change the std lib later" logic. I've yet to see it actually work.
Most of the platforms were successfully petitioned to have rust sdk mandatory added so that rust code can be added to the platforms. The previously situation was rust was not allowed because the external dependency of the rust sdk was blocked.
Note that the rust having no stable api is not fixed, so I think there's a bunch of internal systems on each platform to hard lock the rust dependencies across multiple rust users.
There's some friction between platform packagers and the code that the author wrote exactly as it was written.
Indeed. With AI lifting legacy code bases into Rust got a whole lot easier, and purging the blight of C from the world, excepting the most deeply embedded of applications, got a whole lot closer.
Really? You think AI writes better Rust than Python? Can you give me some examples? I strictly code Django, and Claude Code is really good at following my lead with it.
Rust has a very strict type system and an ecosystem that often utilizes the type system well.
Many things that would only be caught at runtime in other languages are caught at compile time in Rust, making coding agents iterate until things compile and work well.
Rust also has great error messages, which help the agents in fixing compilation errors.
I can't give you examples, but my experience is that AI does very well with Rust except for cases where a library has a constantly changing API/ has had recent breaking changes. I find that AI does extremely well at "picking up" a Rust codebase, I suspect due to the type information providing context but I couldn't say.
The compile errors are great. I can change one function signature and have my output fill up with compile errors (that would all be runtime errors in python). Then I just let claude cook on fixing them. Any time you have to run your program and tell claude what’s wrong with it you’re wasting time, but because claude can run the compiler itself and iterate it’s much more able to complete a task without intervention.
I think relative to the typical Rust code it likely does worse than AI relative to the typical Python code. But due to the compiler, it's possible you might get more correctness out of AI-generated rust code on average.
I think the argument is more that working rust code is better than working Python, and AI assistance makes it more tenable for average developers to successfully produce working rust code, and in particular is helpful for navigating the gap between "code written" and "code compiling" (eg why is the borrow checked mad at me).
Even if it writes the same or even somewhat worse rust than python, assuming the output is the same you are likely to get a speedup + a better distribution story.
I don't care that the non-gnu coreutils are using rust. I care that they aren't GPL licensed.
This means Canonical can offer proprietary patches on top of these packages and sell them as part of their "enterprise" offerings and this gives me the ick.
Here's the chasm I want to see Rust cross:
Dynamic linking with a safe ABI, where if you change and recompile one library then the outcome has to obey some definition of safety, and ABI stability is about as good as C or Objective-C or Swift.
Until that happens, it'll be hard to adopt Rust in a lot of C/C++ strongholds where C's ABI and dynamic linking are the thing that enables the software to get huge.
> Until that happens, it'll be hard to adopt Rust in a lot of C/C++ strongholds where C's ABI and dynamic linking are the thing that enables the software to get huge.
Wait, Rust can already communicate using the C ABI. In fact, it offers exactly the same capabilities as C++ in this regard (dynamic linking).
That's an unsafe ABI.
As unsafe as C or C++. In fact, safer, because only the ABI surface is unsafe, the rust code behind it can be as safe or unsafe as you want it to be.
I was addressing this portion of your comment: "C's ABI and dynamic linking are the thing that enables the software to get huge". If the C ABI is what enables software to get huge then Rust is already there.
There is a second claim in your comment about a "safe ABI", but that is something that neither C or C++ offers right now.
Here's the problem. If you told me that you rebuilt the Linux userland with Rust but you used C ABI at all of the boundaries, then I would be pretty convinced that you did not create a meaningful improvement to security because of how many dynamic linking boundaries there are. So many of the libraries involved are small, and big or small they expose ABIs that involve pointers to buffers and manual memory management.
> There is a second claim in your comment about a "safe ABI", but that is something that neither C or C++ offers right now.
Of course C and C++ are no safer in this regard. (Well, with Fil-C they are safer, but like whatever.)
But that misses the point, which is that:
- It would be a big deal if Rust did have a safe dynamic linking ABI. Someone should do it. That's the main point I'm making. I don't think deflecting by saying "but C is no safer" is super interesting.
- So long as this problem isn't fixed, the upside of using Rust to replace a lot of the load bearing stuff in an OS is much lower than it should be to justify the effort. This point is debatable for sure, but your arguments don't address it.
> - It would be a big deal if Rust did have a safe dynamic linking ABI. Someone should do it. That's the main point I'm making. I don't think deflecting by saying "but C is no safer" is super interesting.
I think we all agree that it would be a huge deal.
> - So long as this problem isn't fixed, the upside of using Rust to replace a lot of the load bearing stuff in an OS is much lower than it should be to justify the effort. This point is debatable for sure, but your arguments don't address it.
As you point out, this is the debatable part, and I'm not sure I get your justification here.
This might end up being the forcing function (quoting myself from another reply in this discussion):
> It can't be that replacing 20 C/C++ shared objects with 20 Rust shared objects results in 20 copies of the Rust standard library and other dependencies that those Rust libraries pull in. But, today, that is what happens. For some situations, this is too much of a memory usage regression to be tolerable.
If memory was cheap, then maybe you could say, "who cares".
Unfortunately memory isn't cheap these days
What you are asking for is to make a library definition replacement to .h-files that contain sufficient information to make rust safe. That is a big, big step and would be fantastic not only for rust but for any other language trying to break out of the C tar pit.
The argument for unsafe ABI not being that big of a deal is that ABI boundaries often reflect organizational boundaries as well.
E.g. the kernel wouldn't really benefit from a "safe ABI" because users calling into the kernel need to be considered malicious by default.
Would a safe ABI work with sandboxing the C code? I'm a bit unsure how one would construct a safe C ABI from Rust's side,
So you're calling for dynamic linking for rust native code? Because rust's safety doesn't come from runtime, it comes from the compiler and the generated code. An object file generated from a bit of rust source isn't some "safe" object file, it's just generated in a safe set of patterns. That safety can cross the C ABI perfectly fine if both things on either side came from rust to begin with. Which means rust dynamic linking.
How could a safe dynamic linking API ever work?
I think you're moving the goalposts significantly here.
I don’t think GP is moving the goalposts at all, rather I think a lot of people are willfully misrepresenting GP’s point.
Rust-to-rust code should be able to be dynamically linked with an ABI that has better safety guarantees than the C ABI. That’s the point. You can’t even express an Option<T> via the C ABI, let alone the myriad of other things rust has that are put together to make it a safe language.
You can look to Swift for prior art on how this can be done: https://faultlore.com/blah/swift-abi/
It would be very hard to accomplish. Apple was extremely motivated to make Swift have a resilient/stable ABI, because they wanted to author system frameworks in swift and have third parties use them in swift code (including globally updating said frameworks without any apps needing to recompile.) They wanted these frameworks to feel like idiomatic swift code too, not just be a bunch of pointers and manual allocation. There’s a good argument that (1) Rust doesn’t consider this an important enough feature and (2) they don’t have enough resources to accomplish it even if they did. But if you could wave a magic wand and make it “done”, it would be huge for rust adoption.
> I don’t think GP is moving the goalposts at all
Thank you :-)
> It would be very hard to accomplish.
Yeah it's a super hard problem especially when you provide safety using the type system!
The work the Swift team did here is hella impressive.
> But if you could wave a magic wand and make it “done”, it would be huge for rust adoption.
Yeah!
> How could a safe dynamic linking API ever work?
Fil-C solves it. I think Swift solves it, too.
So it's solvable.
No fundamental reason, that I know of, why Rust or any other safe language can't also have some kind of story here.
> I think you're moving the goalposts significantly here.
No. I'm describing a problem worth solving.
Also, I think a major chasm for Rust to cross is how defensive the community gets. It's important to talk about problems so that the problems can be solved. That's how stuff gets better.
A safe ABI would be cool, for sure, but in the market (specifically addressing your prediction) I don't know if it's really that big a priority for adoption. The market is obviously fine with an unsafe ABI, seeing how C/C++ is already dominant. Rust with an unsafe ABI might then not be as big an improvement as we would like, but it's still an improvement, and I feel like you're underestimating the benefits of safe Rust code as an application-level frontline of security, even linked to unsafe C code.
C++ ABI stability is the main reason improvements to the language get rejected.
You cannot change anything that would affect the class layout of something in the STL. For templated functions where the implementation is in the header, ODR means you can't add optimizations later on.
Maybe this was OK in the 90s when companies deleted the source code and laid off the programmers once the software was done, but it's not a feature Rust should ever support or guarantee.
The "stable ABI" is C functions and nothing else for a very good reason.
I think if Rust wants to evolve even more aggressively than C++ evolves, then that is a chasm that needs to be crossed.
In lots of domains, having a language that doesn't change very much, or that only changes very carefully with backcompat being taken super seriously, is more important than the memory safety guarantees Rust offers.
Isn’t this solution solved by just compiling your libraries with your main app code? Computers are fast enough that this shouldn’t be a huge issue.
In my view, this is a good thing.
As a C++ developer, I regularly deal with people that think creating a compiled object file and throwing away the source code is acceptable, or decide to hide source code for "security" while distributing object files. This makes my life hell.
Rust preventing this makes my life so much better.
> As a C++ developer, I regularly deal with people that think creating a compiled object file and throwing away the source code is acceptable, or decide to hide source code for "security" while distributing object files. This makes my life hell.
I mean yeah that's bad.
> Rust preventing this makes my life so much better.
I'm talking about a different issue, which is: how do you create software that's in the billions of lines of code in scale. That's the scale of desktop OSes. Probably also the scale of some other things too.
At that scale, you can't just give everyone the source and tell them to do a world compile. Stable ABIs fix that. Also, you can't coordinate between all of the people involved other than via stable ABIs. So stable ABIs save both individual build time and reduce cognitive load.
This is true even and especially if everyone has access to everyone else's source code
> At that scale, you can't just give everyone the source and tell them to do a world compile. Stable ABIs fix that. Also, you can't coordinate between all of the people involved other than via stable ABIs. So stable ABIs save both individual build time and reduce cognitive load.
Rust supports ABI compatibility if everyone is on the same compiler version.
That means you can have a distributed caching architecture for your billion line monorepo where everyone can compile world at all times because they share artifacts. Google pioneered this for C++ and doesn't need to care about ABI as a result.
What Rust does not support is a team deciding they don't want to upgrade their toolchains and still interoperate with those that do. Or random copy and pasting of `.so` files you don't know the provenance of. Everyone must be in sync.
In my opinion, this is a reasonable constraint. It allows Rust to swap out HashMap implementations. In contrast, C++ map types are terrible for performance because they cannot be updated for stability reasons.
My understanding: Even if everyone uses the same toolchain, but someone changes the code for a module and recompiles, then you're in UB land unless everyone who depends on that recompiles
Am I wrong?
If your key is a hash of the code and its dependencies, for a given toolchain and target, then any change to the code, its dependencies, the toolchain or target will result in a new key unique to that configuration. Though I am not familiar with these distributed caching systems so I could be overlooking something.
That's not the issue I'm worried about
> At that scale, you can't just give everyone the source and tell them to do a world compile.
Firstly, of course you could.
Secondly, you don't even need to, as NixOS shows.
C++ is still changing quite a lot though, just not in ways that fix the existing issues (often because doing so would break ABI stability).
That is a reason why a lot of folks stick with C.
In some sense, the chasm I'm describing hasn't been crossed by C++ yet
Except as you well know, C might not change as fast, but it does change, including the OS ABI.
Those folks think it doesn't.
> Except as you well know, C might not change as fast, but it does change, including the OS ABI.
I don't know that.
Here's what I know: the most successful OSes have stable OS ABIs. And their market share is positively correlated with the stability of their ABIs.
Most widely used: Windows, which has a famously stable OS ABI. (If you wanted to be contrarian you could say that it doesn't because the kernel ABI is not stable, but that misses the point - on Windows you program against userland ABIs provided by DLLs, which are remarkably stable.)
Second place: macOS, which maintains ABI stability with some sunsetting of old CPU targets. But release to release the ABI provides solid stability at the framework level, and used to also provide stability at the kernel ABI level (not sure if that's still true - but see above, the important thing is userland framework ABI stability at the end of the day).
Third place: Linux, which maintains excellent kernel ABI stability. Linux has the stablest kernel ABI right now AFAIK. And in userland, glibc has been investing heavily in ABI stability; it's stable enough now that in practice you could ship a binary that dynlinks to glibc and expect it to work on many different Linuxes today and in the future.
So it would seem that OS ABIs are stable in those OSes that are successful.
Speaking of Windows alone, there are the various calling conventions (pascal, stdcall, cdecl), 16, 32, 64 bits, x86, ARM, ARM64EC, DLLs, COM in-proc and ext-proc, WinRT within Win32 and UWP.
Leaving aside the platforms it no longer supports.
So there are some changes to account for depending on the deployment scenario.
The most stable would be FreeBSD with compaNx libraries/modules for old binaries, where N = FreeBSD version number.
What's the stat of single-compiler version ABI? I mean - if the compiler guaranteed that for the same version of the compiler the ABI can work, we could potentially use dynamic linking for a lot of things (speed up iterative development) without committing to any long term stable API or going through C ABI for everything.
The big question is does Rust want to play being adopted by those vendors, or it would leave them alone with languages that embrace native libraries.
Dynamic linking is also great for compile time of debug builds. If a large library or application is split up into smaller shared libraries, ones unaffected by changes don't need to be touched at all. Runtime dynamic linking has a small overhead, but it's several orders of magnitude faster than compile-time linking, so not a problem in debug builds.
for developer turnaround time, it is huge. we explicitly do not statically link Ardour because as developers we are in the edit-compile-debug cycle all day every day, and speeding up the link step (which dynamic linking does dramatically, especially with parallel linkers like lld) is a gigantic improvement to our quality of life and productivity.
A common pattern is dynamic linking for development and static linking for production-ready releases.
We considered doing both, but it turned out that the GUI toolkit we use was really, really not designed to be statically linked, so we stopped trying.
Yes, that's a good way to do it.
The C ABI can already be used, it comes with all the existing safety guarantees that C will provide. Isn’t this as good as C?
It is as good as C.
It's also as bad as C.
I'm saying that the chasm to cross is a safe ABI.
There is no existing safe ABI, so this cannot be an adoption barrier.
Lots of reasons why it is. I'll give you two.
1) It can't be that replacing 20 C/C++ shared objects with 20 Rust shared objects results in 20 copies of the Rust standard library and other dependencies that those Rust libraries pull in. But, today, that is what happens. For some situations, this is too much of a memory usage regression to be tolerable.
2) If you really have 20 libraries calling into one another using C ABI, then you end up with manual memory management and manual buffer offset management everywhere even if you rewrite the innards in Rust. So long as Rust doesn't have a safe ABI, the upside of a Rust rewrite might be too low in terms of safety/security gained to be worth doing
I found c ABI a bit too difficult in rust compared to c or zig. Mainly because of destructors. I am guessing c++ would be difficult in a similar way.
Also unsafe rust has always on strict-aliasing, which makes writing code difficult unless you do it in certain ways.
Having glue libraries like pyo3 makes it good in rust. But that introduces bloat and other issues. This has been the biggest issue I had with rust, it is too hard to write something so you use a dependency. And before you know it, you are bloating out of control
Not really. The foreign ABI requires a foreign API, which adds friction that you don't have with C exporting a C API / ABI. I've never tried, but I would guess that it adds a lot of friction.
Indeed, Victor Ciura from Microsoft DevDiv has several talks on how this is currently an adoption problem at Microsoft.
They have been working around it with DLLs, and COM/WinRT, but still the tooling isn't ideal.
COM is interesting as it implements interfaces using the C++ vtable layout, which can be done in C. Dynamic COM (DCOM) is used to provide interoperability with Visual Basic.
You can also access .NET/C# objects/interfaces via COM. It has an interface to allow you to get the type metadata but that isn't necessary. This makes it possible to e.g. get the C#/.NET exception stack trace from a C/C++ application.
One particular chasm to keep an eye on, possibly even more relevant than Ubuntu using Rust: When it comes to building important stuff, Ubuntu sticks to curl|YOLO|bash instead of trusting trust in their own distributions.
https://github.com/canonical/firefox-snap/blob/90fa83e60ffef...
When people say "curl|bash", this usually means secondary fetches, random system config changes, likely adding stuff to user's .bashrc
But it's not quite that bad in this particular case - they are fetching pre-built static toolchain, and running old-school install script, just like in 1990s. The social convention for those is quite safer.
(Although I agree, it is pretty ironic that they prefer this to using ppa or binary packaged into deb...)
I don't get it. What's the chasm here?
You can curl stuff and run it just gotta have hashes in place.
In theory, yes.
In practice, very rarely. Lots of 'curl | sh' do secondary fetches, and those don't come with hash checks. And even if they come with hash checks _today_, there is no guarantee next version won't quietly remove them.
Aren't the versions of Rust in stable Linux distributions like, a century old? Or at least they were last I checked what Debian and Ubuntu LTS were distributing. I think it's because they don't like static linking.
Hasn’t the right way to install rust has always been using rust up? I am an Ubuntu user and never once tried apt for rust.
I believe Rust is typically only used through `apt` as a dependency for system packages written in Rust, or for building system packages that are written in Rust, so that they can link against a single shared instance of the Rust Standard Library.
Unrelated to the language debate, but it seems a lot of people here missed the fact that Rust Coreutils project is licensed under MIT, and I am not sure if I feel that it is the appropriate license for such project. As much as FSF's philosophy has bad PR at times with Stallman, the GPL licenses really do protect open source. Who knows what Canonical would do when all parts of Ubuntu become MIT...
> the GPL licenses really do protect open source.
They did, until the automatic copyright laundering machine was invented. Pretty much every piece of GPL code ever written is now being magically transmuted into MIT/BSD or proprietary code, and the FSF has no solution.
A discussion on licenses will go sideways very quickly. GPL does limit the adoption of software in certain environments. So it really depends on your goals. Do you want an OSS project that will be useable by everyone (including corporations) or do you want to guarantee that the software will always be OSS and guarantee that Corporations can’t benefit from it without contributing back (potentially requiring them to open their own proprietary code).
There’s a lot of moral perspective that people apply to this decision, but not all developers have the same goals for their software. MIT is more flexible in its use than GPL, but doesn’t help ensure that software remains open.
What evil deeds are you worried about in particular? What are you afraid people will do now that coreutils is MIT?
Just today I found that rust-coreutils makes installing cuda toolkit impossible, related to use of `dd`: https://forums.developer.nvidia.com/t/cuda-runfile-wont-extr...
Do you have more details? The thread you linked was about gzip, not dd.
The .run file is a shell script with a compressed archive appended:
Where Edit: this is apparently packaged with Makeself, and various sources report issues with rust-coreutils. For example https://bugs.launchpad.net/ubuntu/+source/rust-coreutils/+bu...Really good references to "crossing the chasm" between early adopter needs and mainstream needs. In addition to the Ubuntu coreutils use case, I wonder what other chasms Rust is attempting to cross. I know Rust for Linux (though I think that's still relegated to drivers?) and automotive (not sure where that is).
If you want to take a look at some of the "big drivers", the Project Goals[1] is the right place. These are goals proposed by the community and the language developers put together, they are not explicit milestones or must-haves, but they do serve as a guideline to what the project tries to put its time and effort on.
[1]: https://rust-lang.github.io/rust-project-goals/
There are big pushes in pretty much every direction. The projects that really stand out to me are pyo3 (Replace c++ python modules with rust), Dioxus (react-like web framework), The ferrocine qualified compiler (automotive)
I think right now the ecosystem is pretty ripe and with DARPA TRACTOR there are only more and more reasons every day to put rust on your toolbelt.
I am secretly hoping that eventually we break free from the cycle of "hire a senior dev and he likes rust so the company switches" over to hey let's hire some good mid-level and junior rust developers
Are mid level and junior developers being hired anywhere for any reason right now? I don't mean specifically rust developers. I mean software developers.
Sure. There was an article a week or two ago about IBM aggressively hiring juniors. Of course the fact that is noteworthy probably means something in itself....
IBM is also not known for holding on to bodies -- IBM layoff stories abound.
a glut of junior hires now does not a pretty picture make in the long-term sense
The author refers to a few things that he thinks will appeal to the "early majority," but I feel like that's a weakness of the article. Is the author part of the "early majority?" (doesn't seem like it). Does he have the same problems that they have? How does he know?
He is the Rust project lead, and the Rust project has been doing quite a bit of user, adopter, and non-adopter interviews over the past few years.
.NET has a _huge_ platform library and you know what? It’s a pleasure. So many things are just the standard way of doing things. When things are done weirdly, you can usually get a majority in favour of standardising it.
Yes, there’s always a couple of people who really push the boat out…
I've been a fan of all rust-based utilities that I've used. I am worried that 20+ (??) years of bug fixes and edge-case improvements can't be accounted for by simply using a newer/better code-base.
A lot of bug fixes/exploits are _CAUSED_ by the C+ core, but still... Tried & true vs new hotness?
Don't hate me for this, but... is 20 years of Rust really new?
https://en.wikipedia.org/wiki/Rust_(programming_language)
I do get what you mean, but Rust has been baking for a decade, finally took off after 10 years of baking, and now that is been repeatedly tried and tested it is eating the world, as some developers suggested it could eventually do so. I however do think this shows a different problem:
If nobody writes unit tests, how do you write them when you port over projects to ensure your new language doesn't introduce regressions. All rewrites should be preceded by strong useful unit tests.
But the 90s was only 20-years ago!
lol, you got me. Stupid old brain not calculating time correctly.
I was born in 1990 so I get it! I still say 21 when people ask me how old I am... Aka how old do I need to say I am to be able to drink alcohol LOL I don't drink that often mind you. I just don't really think about my age a whole lot...
Ideally, but if a project wasn't written with tests at the time then finding a working time machine can be a challenge. If you try to add them later you won't capture all the nuance that went into the original program. After all, if the implementation code was expressive enough to capture that nuance, you'd already have your test suite, so to speak. Tests are written to fill in the details that the rest of the code isn't able to express.
Rust does not even have a specification, and stable release yet! You are lucky if current version, compiles two years old code!
Rust will be "repeatedly tried and tested" maybe in year 2040!
Rust has editions for strong stability guarantees, and has had them for nearly a decade i believe. Besides, tech backing has grown way past the risky point.
FWIW, the GP comment's claim that you're lucky if you can compile 2-year-old code is exaggerated, but so is yours. Rust does not offer "strong stability guarantees". Adding a new method to a standard type or trait can break method inference, and the Rust standard library does that all the time.
In C or C++, this isn't supposed to happen: a conformant implementation claiming to support e.g. C++17 would use ifdefs to gate off new C++20 library functions when compiling in C++17 mode.
> and the Rust standard library does that all the time.
I don't doubt this is true, but do you have an example? I think I haven't run into a build breaking like this in std in like maybe seven/eight years. In my experience breaking changes/experimental apis are typically ensconced in features or gated by editions.
Granted, it'd be nice to be able to enforce abi stability at the crate level, but managing that is its own can of worms.
I did find that the breakage rfc allows for breaking inference, which tbh seems quite reasonable... inference is opt-in.
Almost every major release of rust stabilizes new library methods. For example, the latest major release (1.93) stabilized Vec::into_raw_parts. This isn’t gated by an edition. So if you had a trait with a method “into_raw_parts” which you had defined on Vec, after updating to 1.93 or later your code will either fail to compile, or start running different code when that method is called.
Sorry, I meant to write “method resolution”, not inference. This isn’t the same issue as type inference (though indeed, stdlib changes can break that too)
Adding a new method can change the behavior of C++ code as well due to templates. Does the standard library never add new methods because of that?
Yes. All the time. Subscribe to the std-proposals mailing list and you'll see so many obvious improvements get rejected due to ABI compat guarantees.
> Adding a new method can change the behavior of C++ code as well due to templates.
Yes, but the code can be gated off with ifdefs to only be present when compiling for a particular version of the standard.
> Rust does not even have a specification
Neither do most programming languages.
> You are lucky if current version, compiles two years old code!
That's not true.
> Neither do most programming languages.
My favorite nemesis and friend JavaScript does, which always gives me a laugh. Such a mess of a wonderful language.
You and me both; never change you beautiful bastard of a language <3
> Neither do most programming languages.
Rust is trying to replace C++ and C in particular. Those languages have specifications.
> years of bug fixes and edge-case improvements can't be accounted for by simply using a newer/better code-base.
Partially is in fact true: Just because the Rust use a better type system (after ML) + better resource model (aka borrow checker), and if you are decently good, you eliminate, forever!, tons of problems.
It can't solve things that arise by complex interactions or just lack of port subtle details like in parsing poor inputs (like html) but is true that changing the language in fact solve tons of things.
I think it's worth trying!
It absolutely is worth trying. I look forward to it being battle tested and proven. I just don't want to be the one doing the testing.
rg, fzf, and several others that I can't think have proven to me that rust is the direction going forward.
Is Rust still considered "new hotness"? I feel like the industry has long-since moved past that perceived "blocker".
It seems like Rust is now just the default in all manner of critical systems.
Rust - no. sudo-rs not hotness, but relatively new.
Sudo no longer supporting path inheritance kinda sucks
> Jon made the provocative comment that we needed to revisit our policy around having a small standard library. He’s not the first to say something like that, it’s something we’ve been hearing for years and years
It sounds to me like you "cross the chasm" a little too early. As a user I don't care about your "chasms" I care about high quality durable systems. This isn't the first time I've heard the "we'll change the std lib later" logic. I've yet to see it actually work.
a few weeks ago it was all about Zig, now it's all about Rust, Clojure or Elixir next?
Rust was first
Why am I hearing about Rust a lot these days? Did anything significant happen?
What do you mean by “these days”? To me, it seems like rust is a pretty constant factor on HN for at least two years now.
Most of the platforms were successfully petitioned to have rust sdk mandatory added so that rust code can be added to the platforms. The previously situation was rust was not allowed because the external dependency of the rust sdk was blocked.
Note that the rust having no stable api is not fixed, so I think there's a bunch of internal systems on each platform to hard lock the rust dependencies across multiple rust users.
There's some friction between platform packagers and the code that the author wrote exactly as it was written.
there has been a few adopters of rust... linux formally choosing it for some of their systems being the most notable recently(maybe a few months ago).
AI has made it exceptionally easy to program with.
I've switched to using Rust from Python simply because of AI development
Indeed. With AI lifting legacy code bases into Rust got a whole lot easier, and purging the blight of C from the world, excepting the most deeply embedded of applications, got a whole lot closer.
Really? You think AI writes better Rust than Python? Can you give me some examples? I strictly code Django, and Claude Code is really good at following my lead with it.
Rust has a very strict type system and an ecosystem that often utilizes the type system well.
Many things that would only be caught at runtime in other languages are caught at compile time in Rust, making coding agents iterate until things compile and work well.
Rust also has great error messages, which help the agents in fixing compilation errors.
I can't give you examples, but my experience is that AI does very well with Rust except for cases where a library has a constantly changing API/ has had recent breaking changes. I find that AI does extremely well at "picking up" a Rust codebase, I suspect due to the type information providing context but I couldn't say.
The compile errors are great. I can change one function signature and have my output fill up with compile errors (that would all be runtime errors in python). Then I just let claude cook on fixing them. Any time you have to run your program and tell claude what’s wrong with it you’re wasting time, but because claude can run the compiler itself and iterate it’s much more able to complete a task without intervention.
I think relative to the typical Rust code it likely does worse than AI relative to the typical Python code. But due to the compiler, it's possible you might get more correctness out of AI-generated rust code on average.
I think the argument is more that working rust code is better than working Python, and AI assistance makes it more tenable for average developers to successfully produce working rust code, and in particular is helpful for navigating the gap between "code written" and "code compiling" (eg why is the borrow checked mad at me).
Even if it writes the same or even somewhat worse rust than python, assuming the output is the same you are likely to get a speedup + a better distribution story.
I don't care that the non-gnu coreutils are using rust. I care that they aren't GPL licensed.
This means Canonical can offer proprietary patches on top of these packages and sell them as part of their "enterprise" offerings and this gives me the ick.