Yes, basically all buffering should use work-or-time based flushing.
Work-based flushing exists to batch work to amortize overhead. It allows you to tune how much time you spend in overhead in a work period. However, this can result in unbounded delay.
Time-based flushing targets a worst-case latency in low throughput environments. This ensures you do not get unbounded delays. Interestingly, this also allows you to tune how much time you spend in overhead in a work period.
So, with work-or-time based flushing what you are really tuning is how much time you are willing to spend in overhead in a work period and balancing that against latency.
As a nit, the article calls this a derivative of Nagle’s algorithm, but it is basically the opposite. Nagle’s algorithm is about batching until acknowledgement with no mention of time. In fact, the lack of time-based flushing in Nagle’s algorithm is why it interacts so poorly with TCP Delayed Ack.
It's worth noting that there are more that two kinds of buffering, contrary to the introduction here, and the one that's actually used in several C libraries is so-called smart buffering. This also flushes standard output when standard input needs to do an underlying read and it is reading the beginning of a line.
And since musl libc is in the news, it is worth nothing that musl's fwrite() is another of those that looks for the last linefeed in the buffer, rather than the first, and so should emit multiple lines in one underlying write(). Although this is only done in fwrite() specifically.
Doesn't the code as written potentially keep extending the deadline each time the deadline_flush is called even if nothing is written to the buffer? So long as you call the flush often enough which would depend on the work in the while loop.
Yeah, that is a pitfall one shouldn't fall into even the first time programming something like that - it's pretty obvious, I think? You need to start the deadline timer if it's not running, then keep it running - don't restart it when more input arrives.
Also, this:
> Where DEADLINE is something appropriate for human perception, like 100ms or so
is way off. You easily notice a hitch of one frame in 60 fps content, and this is comparable. The timer should be something like 1-10 ms. Outputting multiple lines is really fast, so even 1 ms would merge most output into a single write() call. I'd start with 1 ms and raise it if there's a strong reason, which there probably isn't.
One more thing to consider is atomicity of write interleaving to the same pipe. Say you fork two unzips in the terminal like `unzip a & unzip b`. With naive line buffering the outputs mix cleanly, but if aggregating multiple lines per write the chunk size needs to stay below PIPE_BUF for this to stay true.
In C, you can use setvbuf to control buffering on stdout. There are also Linux command line utilities you can use to control buffering, unbuffer and stdbuf.
Yes, basically all buffering should use work-or-time based flushing.
Work-based flushing exists to batch work to amortize overhead. It allows you to tune how much time you spend in overhead in a work period. However, this can result in unbounded delay.
Time-based flushing targets a worst-case latency in low throughput environments. This ensures you do not get unbounded delays. Interestingly, this also allows you to tune how much time you spend in overhead in a work period.
So, with work-or-time based flushing what you are really tuning is how much time you are willing to spend in overhead in a work period and balancing that against latency.
As a nit, the article calls this a derivative of Nagle’s algorithm, but it is basically the opposite. Nagle’s algorithm is about batching until acknowledgement with no mention of time. In fact, the lack of time-based flushing in Nagle’s algorithm is why it interacts so poorly with TCP Delayed Ack.
It's worth noting that there are more that two kinds of buffering, contrary to the introduction here, and the one that's actually used in several C libraries is so-called smart buffering. This also flushes standard output when standard input needs to do an underlying read and it is reading the beginning of a line.
And since musl libc is in the news, it is worth nothing that musl's fwrite() is another of those that looks for the last linefeed in the buffer, rather than the first, and so should emit multiple lines in one underlying write(). Although this is only done in fwrite() specifically.
* https://git.musl-libc.org/cgit/musl/tree/src/stdio/fwrite.c#...
I have seen versions of this many times in my life. Unfortunately, that progress report can hurt a lot, is something we devs often miss.
From my horror cabinet: 1. Running count * on all sqlite tables 4x per second to show sync progress.
2. Progress bounce animation eating 80% of cpu (windows phone bizarro silverlight)
3. Sending myself 800 emails in 10 minutes…
The rule is quite simple, we are just bad at it. Computer to human communication should run at human speed.
> flush the buffer if it’s been idle a short time
I've thought this was the right default for a good while. 50 years ago when Unix was designed, maybe not.
This would withhold a line if the writer paused for a while without calling into buffer code. Another case for Golang supremacy?
We can also do better than terminals. 7-Zip was the GOAT.
Doesn't the code as written potentially keep extending the deadline each time the deadline_flush is called even if nothing is written to the buffer? So long as you call the flush often enough which would depend on the work in the while loop.
Yeah, that is a pitfall one shouldn't fall into even the first time programming something like that - it's pretty obvious, I think? You need to start the deadline timer if it's not running, then keep it running - don't restart it when more input arrives.
Also, this:
> Where DEADLINE is something appropriate for human perception, like 100ms or so
is way off. You easily notice a hitch of one frame in 60 fps content, and this is comparable. The timer should be something like 1-10 ms. Outputting multiple lines is really fast, so even 1 ms would merge most output into a single write() call. I'd start with 1 ms and raise it if there's a strong reason, which there probably isn't.
One more thing to consider is atomicity of write interleaving to the same pipe. Say you fork two unzips in the terminal like `unzip a & unzip b`. With naive line buffering the outputs mix cleanly, but if aggregating multiple lines per write the chunk size needs to stay below PIPE_BUF for this to stay true.
I strongly suspect that that is the reason for the current behavior. The people behind glibc aren't stupid.
The number of times I've thought a Python process was broken because it wasn't printing anything...
In C, you can use setvbuf to control buffering on stdout. There are also Linux command line utilities you can use to control buffering, unbuffer and stdbuf.
This is true, although neither of those by them selves will achieve what the author is aiming for.
If you want to learn more, I liked this blog post: https://hmarr.com/blog/how-stdbuf-works/