I love this work and everything else Ratatui does but I have to ask: if you're changing the terminal protocol and the terminal itself to do this, why not do something more ambitious; a full monospaced information-dense UI framework, a terminal designed for user interfaces, rather than one designed to replicate the limitations of dumb terminals?
I've written about it before, but in brief the mountain of standards is worth ripping out wholesale and doing write as a semantic protocol. It's needed to solve several things that work really poorly in terminals because their unit of compatibility is cells rather than a higher primitive. It's hard to bake something on top that works well.
The need is most strong in Accessibility, Text wrapping, Scrollback handling (i.e. making the scrollback editable and not dead text), and various other bits that make writing highly functional TUIs. Coding agents tend to find all those edge cases.
This often gets push back that it's easy, to which I can point nay sayers at the various many months long issues on things in Codex, Gemini, Claude etc. on flickering, scrolling, key handline, IME, ... doing anything moderately complex involves modeling a terminal in a terminal app effectively.
> a full monospaced information-dense UI framework, a terminal designed for user interfaces, rather than one designed to replicate the limitations of dumb terminals?
I can only speak for myself but I use the CLI and TUIs specifically because of the limitations those modalities impose on software that uses them as the primary method for interfacing with the end user. If you want a full featured UI and all that comes with it why not just use QT or GTK or modern HTML or etc?
I think the only "improvements" I'd like to see are thing such as metadata regarding what the different streams are for, an official way to communicate using structured data as opposed to raw bytes, that sort of thing. Spiritually in the same ballpark as the difference between HTML before and after CSS.
I actually am exploring exactly this. Ironically from the perspective of someone that was trying to leave the Terminal.
How i got here i find kinda interesting. Originally i was trying to leave the terminal. Using GPUI/etc frameworks. I kept running into the terminal though, because no matter how nice the UI is - the terminal still needs to work excellently.
Eventually i decided to try an abomination. To use a custom terminal which, not unlike Kitty images, allows generic apps to pass escape codes to control the UI components. Where the UI components have no restriction on "dumb terminals". So my text editor is a two part app, an editor and a tty to render the actual custom UI APC codes.
This paved the way for true smooth scrolling from within a terminal editor, as the editor effectively paginates but the tty scrolls within the buffered pages. It allows for different UI components to mix fonts and font sizes, eg popup docs being a different size than the other text, and .. well, anything i suppose.
I have to say it's working excellently and is slowly becoming my daily driver. I won't link it though, it's a fully vibe coded and even claude designed horror.
I'm surprised i've not seen more of this in the past to be honest.
Text terminal with every keystroke round-tripped through the server is not a design that makes sense for any reason other than backwards compatibility. It's kind of annoying to design for and atrocious for performance.
A much better alternative to this would be to for more terminal emulators to implement overstrike mode (DECSET 20, as available on the VK100). That way you could just draw border characters over existing text cells to achieve the same thing.
Unfortunately mintty is the only terminal emulator I'm aware of which supports overstrike [0], as this would open up a world of new possibilities for TUI apps if more widely supported.
There's some downsides to this - I can't recall where I called this out off the top of my head, but this also adds a bunch more things that that wouldn't do that are covered in the spec.
I’m not against overstrike, but I can think of one drawback of that solution: it becomes significantly harder to have rounded corners with a filled interior.
But I’m not trying to argue for nor against either solution.
I appreciate the innovation happening with TUIs in these modern times. But, aesthetically, I have to say, I like my borders to have some breathing room, which gets lost in this particular treatment.
Why wouldn’t this work over tmux? The authors claims don’t make a whole lot of sense because if this were a cell property, like any other CSI, then it’ll get rendered in the terminal emulator exactly as intended. And we know this because none of the other CSI sequences get garbled over multiplextors.
But for there to be a dedicated section on how multiplexors wouldn’t work suggests to me that the authors already tried this via tmux.
So I’m left wondering what they’ve done with the implementation of this to make that a problem.
I spotted they are using DCS sequences. But even that shouldn’t be an issue itself.
Tmux drops DCS that it doesn't know like this. That aside, this is extra state per cell that needs to be tracked, handled, understood from a rendering perspective etc. It can work, but would require maybe a thousand lines of code or so to support storing and handling state, redraw etc. and all the various edge cases.
Nothing really - Ghostty has a bit of padding that's unused per cell - 16 bits. The implementation stores an ID into a lookup table with border combinations. If all of those were used and never cleaned up, 65535 at around 100 for each combination used, order of magnitude 25MB absolute worst case assuming no real optimization in place for this.
Doing it through port forwarding is kinda bad though because generally access to network ports is not very well access controlled. Doing it through unix sockets (like x) would be much better but annoyingly browsers do not support http over uds
UI in a browser is a massive hack in itself, a clean design would be better. For example a compact semantic remote access protocol to use with UIs, something like SPICE but better optimized for networks, fully presentation-independent, and less janky. This would let everyone use a GUI framework of their choice with an adapter. But people building terminal GUIs are culturally not ready for this, they'd rather continue reinventing the wheel and piling up hacks in terminals. I wonder when they realize that proportional fonts are more readable when vertical alignment is handled separately, and introduce terminals with variable-width fonts. And the presentation layer (I suggest calling it CSS).
Afraid that’s a contradiction. In this role, HTML is a pile of hacks upon hacks that was not originally intended for UIs.
But, yeah, people are trying to reinvent GUIs. By layering hacks over TTYs. If only there’d be some protocol for widgets and events over a stream connection…
the thing with X and wayland etc is, that they are low level. which is fine if you want full control over the presentation. what HTML was trying to do is to go higher level, send over some data and let the browser figure out how to present it. but it went wrong and people are abusing it as a low level presentation interface like X or wayland.
what we are missing is a really high level protocol/system that takes really high level data and figures out the presentation on its own.
take a list of files for example. you can run ls -l in a terminal and it will attempt a crude text based formatting of the output. or you can run a full gui filemanager.
what if there was a command that would produce ls output as a table data structure, run in a sort of terminal that knows how to format a table nicely without the ls command having to include any formatting instructions? this terminal would have support for all sorts of data types and structures to display all of them nicely.
> what if there was a command that would produce ls output as a table data structure, run in a sort of terminal that knows how to format a table nicely without the ls command having to include any formatting instructions? this terminal would have support for all sorts of data types and structures to display all of them nicely.
So, powershell? For some reason only IT jockeys on Microsoft's death grip remember and use its full capability after learning it.
I love this work and everything else Ratatui does but I have to ask: if you're changing the terminal protocol and the terminal itself to do this, why not do something more ambitious; a full monospaced information-dense UI framework, a terminal designed for user interfaces, rather than one designed to replicate the limitations of dumb terminals?
Yep,and I want to do that too :)
I've written about it before, but in brief the mountain of standards is worth ripping out wholesale and doing write as a semantic protocol. It's needed to solve several things that work really poorly in terminals because their unit of compatibility is cells rather than a higher primitive. It's hard to bake something on top that works well.
The need is most strong in Accessibility, Text wrapping, Scrollback handling (i.e. making the scrollback editable and not dead text), and various other bits that make writing highly functional TUIs. Coding agents tend to find all those edge cases.
This often gets push back that it's easy, to which I can point nay sayers at the various many months long issues on things in Codex, Gemini, Claude etc. on flickering, scrolling, key handline, IME, ... doing anything moderately complex involves modeling a terminal in a terminal app effectively.
> a full monospaced information-dense UI framework, a terminal designed for user interfaces, rather than one designed to replicate the limitations of dumb terminals?
I can only speak for myself but I use the CLI and TUIs specifically because of the limitations those modalities impose on software that uses them as the primary method for interfacing with the end user. If you want a full featured UI and all that comes with it why not just use QT or GTK or modern HTML or etc?
I think the only "improvements" I'd like to see are thing such as metadata regarding what the different streams are for, an official way to communicate using structured data as opposed to raw bytes, that sort of thing. Spiritually in the same ballpark as the difference between HTML before and after CSS.
I actually am exploring exactly this. Ironically from the perspective of someone that was trying to leave the Terminal.
How i got here i find kinda interesting. Originally i was trying to leave the terminal. Using GPUI/etc frameworks. I kept running into the terminal though, because no matter how nice the UI is - the terminal still needs to work excellently.
Eventually i decided to try an abomination. To use a custom terminal which, not unlike Kitty images, allows generic apps to pass escape codes to control the UI components. Where the UI components have no restriction on "dumb terminals". So my text editor is a two part app, an editor and a tty to render the actual custom UI APC codes.
This paved the way for true smooth scrolling from within a terminal editor, as the editor effectively paginates but the tty scrolls within the buffered pages. It allows for different UI components to mix fonts and font sizes, eg popup docs being a different size than the other text, and .. well, anything i suppose.
I have to say it's working excellently and is slowly becoming my daily driver. I won't link it though, it's a fully vibe coded and even claude designed horror.
I'm surprised i've not seen more of this in the past to be honest.
Text terminal with every keystroke round-tripped through the server is not a design that makes sense for any reason other than backwards compatibility. It's kind of annoying to design for and atrocious for performance.
Teletypes and dumb terminals managed to implement local echo. You don't need to round trip everything.
A much better alternative to this would be to for more terminal emulators to implement overstrike mode (DECSET 20, as available on the VK100). That way you could just draw border characters over existing text cells to achieve the same thing.
Unfortunately mintty is the only terminal emulator I'm aware of which supports overstrike [0], as this would open up a world of new possibilities for TUI apps if more widely supported.
[0] https://github.com/mintty/mintty/wiki/CtrlSeqs#overstrike
There's some downsides to this - I can't recall where I called this out off the top of my head, but this also adds a bunch more things that that wouldn't do that are covered in the spec.
Expecting Terminal Emulators to support overstrike is a high bar even many text editors don't even support strikethrough. Would love to see it though
I’m not against overstrike, but I can think of one drawback of that solution: it becomes significantly harder to have rounded corners with a filled interior.
But I’m not trying to argue for nor against either solution.
There's a nice thread about this on lobsters, but the author is getting piled on.
I appreciate the innovation happening with TUIs in these modern times. But, aesthetically, I have to say, I like my borders to have some breathing room, which gets lost in this particular treatment.
Why wouldn’t this work over tmux? The authors claims don’t make a whole lot of sense because if this were a cell property, like any other CSI, then it’ll get rendered in the terminal emulator exactly as intended. And we know this because none of the other CSI sequences get garbled over multiplextors.
But for there to be a dedicated section on how multiplexors wouldn’t work suggests to me that the authors already tried this via tmux.
So I’m left wondering what they’ve done with the implementation of this to make that a problem.
I spotted they are using DCS sequences. But even that shouldn’t be an issue itself.
Tmux drops DCS that it doesn't know like this. That aside, this is extra state per cell that needs to be tracked, handled, understood from a rendering perspective etc. It can work, but would require maybe a thousand lines of code or so to support storing and handling state, redraw etc. and all the various edge cases.
This is gorgeous and I need it in trunk for both kitty and ghostty
How much RAM does this use?
Nothing really - Ghostty has a bit of padding that's unused per cell - 16 bits. The implementation stores an ID into a lookup table with border combinations. If all of those were used and never cleaned up, 65535 at around 100 for each combination used, order of magnitude 25MB absolute worst case assuming no real optimization in place for this.
Would be cool to get borders and lines out of copy/paste
What if, dare I say it, we just served HTML over ssh, and let the browser render a proper UI, instead of layering hacks on top of hacks?
Honestly why? It is not designed at all for that.
In fact it just makes it less secure.
You mean serving html over SSH?
It can already port forward so you can basically already do this, I don’t think that’s insecure per se.
Doing it through port forwarding is kinda bad though because generally access to network ports is not very well access controlled. Doing it through unix sockets (like x) would be much better but annoyingly browsers do not support http over uds
https://bugzilla.mozilla.org/show_bug.cgi?id=1688774 etc
Ssh lets you forward arbitrary ports, and forward X11 directly. Why is HTML more of a problem?
UI in a browser is a massive hack in itself, a clean design would be better. For example a compact semantic remote access protocol to use with UIs, something like SPICE but better optimized for networks, fully presentation-independent, and less janky. This would let everyone use a GUI framework of their choice with an adapter. But people building terminal GUIs are culturally not ready for this, they'd rather continue reinventing the wheel and piling up hacks in terminals. I wonder when they realize that proportional fonts are more readable when vertical alignment is handled separately, and introduce terminals with variable-width fonts. And the presentation layer (I suggest calling it CSS).
> HTML
> instead of hacks on top of hacks
Afraid that’s a contradiction. In this role, HTML is a pile of hacks upon hacks that was not originally intended for UIs.
But, yeah, people are trying to reinvent GUIs. By layering hacks over TTYs. If only there’d be some protocol for widgets and events over a stream connection…
While we're figuring it out let's call this hypothetical protocol X.
the thing with X and wayland etc is, that they are low level. which is fine if you want full control over the presentation. what HTML was trying to do is to go higher level, send over some data and let the browser figure out how to present it. but it went wrong and people are abusing it as a low level presentation interface like X or wayland.
what we are missing is a really high level protocol/system that takes really high level data and figures out the presentation on its own.
take a list of files for example. you can run ls -l in a terminal and it will attempt a crude text based formatting of the output. or you can run a full gui filemanager.
what if there was a command that would produce ls output as a table data structure, run in a sort of terminal that knows how to format a table nicely without the ls command having to include any formatting instructions? this terminal would have support for all sorts of data types and structures to display all of them nicely.
> what if there was a command that would produce ls output as a table data structure, run in a sort of terminal that knows how to format a table nicely without the ls command having to include any formatting instructions? this terminal would have support for all sorts of data types and structures to display all of them nicely.
So, powershell? For some reason only IT jockeys on Microsoft's death grip remember and use its full capability after learning it.
[delayed]
We could even use `ssh -X` to send it over SSH if that flag is otherwise unused.
I think it would make sense if it had a client-server architecture.
And for the ecosystem's sake, punish (harshly) GUI toolkit developers that use bitmap rendering (GTK4 and Qt Quick applications are slow on ssh -X).
plan 9?