"Scale" isn't a binary, it's a continuum. "Scales to 60K/s" can be 5 orders of magnitude more than one system needs and 5 orders of magnitude too small for another. Personally I'd knock the general "premature optimization" off the list of "most common developer errors" and put in its place "using techs with the wrong scaling factors". If you use something too small and you exceed its needs, the failure is obvious, but the other way around is a problem too. Bringing in the overhead and management issues of the super scalable techs, as well as their limitations they impose so that they can scale, to a system that would actually be better off with a richer model whose richness prevents it from scaling but would save a lot of effort is also a bad choice.
The ceiling of LISTEN/NOTIFY is small enough that you need to pay attention, and I personally like to have at least an order of magnitude of slack left over even after my most pessimistic load numbers are accounted for, but it's still plenty for a lot of projects, and the integration with the rest of the DB, its availability, its not being another service you have to devops, it's definitely not something that should be simply dismissed out of hand as an option. Even the original 2K/s number they cite is a lot of messages for some systems that are more properly measured in seconds per message.
Nitpick on an otherwise good post, but I don’t think there are very many 6billion RPS systems out there, and those that do exist are almost certainly using bespoke, purpose-built tools
DynamoDB scales much more than that. In one of their dynamodb papers they claimed that the amazon us retail website alone made like 89 millions rps during prime day, a few years ago.
For reference, I've seen Visa marketing materials that suggest their network can do ~70k TPS. There are not very many systems one could conceive of that could do useful work 1/s for ~every human on the planet.
I think if you expect to be under 60K/s and suddenly find yourself at 20K/s heading for 200K/s-- you have a better problem than if you built for 1M/s and actual load is 20K/s. The unexpected success of the former will pay for a lot of band-aids and scaling, while you're pretty stuck with the cost structure and upfront spent capital in the latter.
IMO one should design for actual anticipated scale with moderate margin, only exceeding this when it's relatively "free" to do so. (If you can buy bigger hardware for a few K, or if solutions are equivalent other than scalability, pick the bigger solution).
This is conventional wisdom, but I wonder. Labor is a material cost against most software businesses. Doing the Whatsapp thing where you just make extremely good choices upfront so that you can reach $10B scale with hardly any employees or infra footprint seems like a pretty good deal for the founding team. Plenty of other 2010s darlings are in structural trouble with stock-based comp because they are locked into these insanely overcomplicated architectures due to bad decisions from the startup and hypergrowth phases + path dependency.
I continue to love DBOS for how it just leverages Postgres (and now SQLite) properly. It's effortless to drop into an existing CRUD stack.
Once you start down the "durable workflows" path, you start seeing them everywhere.
My latest experiments are treating individual emails as durable workflows, where you, the people you're communicating with, agents and tools like GitHub or Attio all take turns in the flow.
I recall that in the first release that supported LISTEN/NOTIFY there was a performance issue around it (poor locking IIRC), which today the "bad post" mentioned in here corrects in a errata just after their first paragraph.
Since the correction apparently dates from May 8th, I think that a post from July 24th might want to acknowledge that the popular post asserting this feature doesn't (didn't?) scale was not made in bad faith or was even wrong about their claims at the time.
If you mean the optimizations coming in Postgres 19, the original post addresses this:
> As an aside, there’s been some online discussion of a Postgres patch (https://github.com/postgres/postgres/commit/282b1cde9dedf456...) related to this issue. This patch (to be released in Postgres 19) does not remove the global lock or fix the bottleneck we observed. Instead, it optimizes the narrower case where there are many notification channels and each listener is waiting only on a specific channel.
I think a lot of these kind of posts are standalone assessment of your problems, understanding and solutions. Its debatable to term something as lack of expertise if one is trying to work with default settings of a tool and expecting a certain performance. Everyone is doing a continuous learning with the failures.
1. What I find interesting is that the experiment seems to be using a DB server with 96 cores, 384 GB RAM (https://github.com/dbos-inc/dbos-postgres-benchmark/blob/mai...). This is very critical part of any such experiment, it should have been called out. The database is vertically scalable and that too has its limits
2. Who is making connection, and from where has its own impact on performance and overall latency
3. 60k may seem big number, however in real world the things which bring the systems down are the bursts of traffic, not the regular traffic.
Personally I would never start with such a big server unless I am a big business. Its > 100K cost for one production DB cluster if I include read replicas and cross region redundancy
I went through this process when I was designing the sync server for Digital Carrot.
In the end, I decided to just go with the simplest solution possible. In my case it's just a barebones Go gRPC service that uses an in memory channel to send notifications between connected clients.
The reality is that this simple Go server will scale up to about 1000 simultaneously connected customers on about 2gb of RAM. I don't expect to have more than that many paying customers, and if I do I can always just throw a bigger VM at the problem.
Engineers love to over complicate things in the name of infinite scalability, when in reality you can save a lot of time and effort by just understanding the scope of the actual problem you're trying to solve. Fingers crossed that this will become an issue for me some day, but until then most of us just don't need to worry about it!
I had a similar setup, scaled pretty well with some GOGC tuning. I had a small, simple "router" using channels https://github.com/urjitbhatia/gopipe and except for the per connection 16ish kb network overhead per socket, you can get away with a lot of performance with a small hand rolled service.
One way it explicitly doesn't scale (unless something has changed since I last checked and my quick search failed me) is the hard limit on 8000 bytes of data in a notification. If your notification doesn't make sense to exist as a row (where you can just give an ID), then that makes it hard to use. I had a web game and the events were transient descriptions of changes in state that didn't make sense to store in the database, and could be bigger than that, so it didn't work for that use case.
As an implementer of a scalable notification system I'd be sure to cap the notification size.
- Keeps messages O(1) so I can focus on scaling in the amount of notifications
- Tells whoever runs into this that,
- I didn't planned for arbitrarily large messages as they *might* otherwise grind performance to a halt.
- They *might* be misusing my notification system
The core optimization is to buffer notifications in-memory and send them in a batch instead of sending them as part of every transaction. So that's a general-purpose optimization for Postgres apps using LISTEN/NOTIFY.
Much of the FUD around using LISTEN/NOTIFY in production comes from people who have never done so. It of course has its limits, and we should remain aware of them. Much like the limits of every piece of tech in our stacks.
"Scale" isn't a binary, it's a continuum. "Scales to 60K/s" can be 5 orders of magnitude more than one system needs and 5 orders of magnitude too small for another. Personally I'd knock the general "premature optimization" off the list of "most common developer errors" and put in its place "using techs with the wrong scaling factors". If you use something too small and you exceed its needs, the failure is obvious, but the other way around is a problem too. Bringing in the overhead and management issues of the super scalable techs, as well as their limitations they impose so that they can scale, to a system that would actually be better off with a richer model whose richness prevents it from scaling but would save a lot of effort is also a bad choice.
The ceiling of LISTEN/NOTIFY is small enough that you need to pay attention, and I personally like to have at least an order of magnitude of slack left over even after my most pessimistic load numbers are accounted for, but it's still plenty for a lot of projects, and the integration with the rest of the DB, its availability, its not being another service you have to devops, it's definitely not something that should be simply dismissed out of hand as an option. Even the original 2K/s number they cite is a lot of messages for some systems that are more properly measured in seconds per message.
> 5 orders of magnitude too small for another.
Nitpick on an otherwise good post, but I don’t think there are very many 6billion RPS systems out there, and those that do exist are almost certainly using bespoke, purpose-built tools
The highest I could think of is WhatsApp, which gets ~1.6 million RPS on average, and they use MQTT
DynamoDB scales much more than that. In one of their dynamodb papers they claimed that the amazon us retail website alone made like 89 millions rps during prime day, a few years ago.
Could be base 2. ~1.3M RPS is a lot higher than most will ever require, but still within the realm of possibility.
Maybe there would be more if it were more straightforward to do so?
For reference, I've seen Visa marketing materials that suggest their network can do ~70k TPS. There are not very many systems one could conceive of that could do useful work 1/s for ~every human on the planet.
Fair.
I think if you expect to be under 60K/s and suddenly find yourself at 20K/s heading for 200K/s-- you have a better problem than if you built for 1M/s and actual load is 20K/s. The unexpected success of the former will pay for a lot of band-aids and scaling, while you're pretty stuck with the cost structure and upfront spent capital in the latter.
IMO one should design for actual anticipated scale with moderate margin, only exceeding this when it's relatively "free" to do so. (If you can buy bigger hardware for a few K, or if solutions are equivalent other than scalability, pick the bigger solution).
This is conventional wisdom, but I wonder. Labor is a material cost against most software businesses. Doing the Whatsapp thing where you just make extremely good choices upfront so that you can reach $10B scale with hardly any employees or infra footprint seems like a pretty good deal for the founding team. Plenty of other 2010s darlings are in structural trouble with stock-based comp because they are locked into these insanely overcomplicated architectures due to bad decisions from the startup and hypergrowth phases + path dependency.
Related, presumably:
Postgres LISTEN/NOTIFY does not scale - https://news.ycombinator.com/item?id=44490510 - July 2025 (321 comments)
I continue to love DBOS for how it just leverages Postgres (and now SQLite) properly. It's effortless to drop into an existing CRUD stack.
Once you start down the "durable workflows" path, you start seeing them everywhere.
My latest experiments are treating individual emails as durable workflows, where you, the people you're communicating with, agents and tools like GitHub or Attio all take turns in the flow.
https://housecat.com/blog/gmail-durable-workflows-sandbox-vm
I recall that in the first release that supported LISTEN/NOTIFY there was a performance issue around it (poor locking IIRC), which today the "bad post" mentioned in here corrects in a errata just after their first paragraph.
Since the correction apparently dates from May 8th, I think that a post from July 24th might want to acknowledge that the popular post asserting this feature doesn't (didn't?) scale was not made in bad faith or was even wrong about their claims at the time.
If you mean the optimizations coming in Postgres 19, the original post addresses this:
> As an aside, there’s been some online discussion of a Postgres patch (https://github.com/postgres/postgres/commit/282b1cde9dedf456...) related to this issue. This patch (to be released in Postgres 19) does not remove the global lock or fix the bottleneck we observed. Instead, it optimizes the narrower case where there are many notification channels and each listener is waiting only on a specific channel.
I think a lot of these kind of posts are standalone assessment of your problems, understanding and solutions. Its debatable to term something as lack of expertise if one is trying to work with default settings of a tool and expecting a certain performance. Everyone is doing a continuous learning with the failures.
1. What I find interesting is that the experiment seems to be using a DB server with 96 cores, 384 GB RAM (https://github.com/dbos-inc/dbos-postgres-benchmark/blob/mai...). This is very critical part of any such experiment, it should have been called out. The database is vertically scalable and that too has its limits
2. Who is making connection, and from where has its own impact on performance and overall latency
3. 60k may seem big number, however in real world the things which bring the systems down are the bursts of traffic, not the regular traffic.
Personally I would never start with such a big server unless I am a big business. Its > 100K cost for one production DB cluster if I include read replicas and cross region redundancy
I went through this process when I was designing the sync server for Digital Carrot.
In the end, I decided to just go with the simplest solution possible. In my case it's just a barebones Go gRPC service that uses an in memory channel to send notifications between connected clients.
The reality is that this simple Go server will scale up to about 1000 simultaneously connected customers on about 2gb of RAM. I don't expect to have more than that many paying customers, and if I do I can always just throw a bigger VM at the problem.
Engineers love to over complicate things in the name of infinite scalability, when in reality you can save a lot of time and effort by just understanding the scope of the actual problem you're trying to solve. Fingers crossed that this will become an issue for me some day, but until then most of us just don't need to worry about it!
I had a similar setup, scaled pretty well with some GOGC tuning. I had a small, simple "router" using channels https://github.com/urjitbhatia/gopipe and except for the per connection 16ish kb network overhead per socket, you can get away with a lot of performance with a small hand rolled service.
1000 connections is a fairly low number by modern standards, c10k “challenges” are like twenty years old by now.
The real questions are:
- how many messages per second are you processing on that 2gb machine (and using how many cpus)?
- does your message processing involve transaction handling, including saving data ti disk durably?
No offense but it really seems you’re comparing apples and oranges, with your use case being much much simpler than the one described.
One way it explicitly doesn't scale (unless something has changed since I last checked and my quick search failed me) is the hard limit on 8000 bytes of data in a notification. If your notification doesn't make sense to exist as a row (where you can just give an ID), then that makes it hard to use. I had a web game and the events were transient descriptions of changes in state that didn't make sense to store in the database, and could be bigger than that, so it didn't work for that use case.
As an implementer of a scalable notification system I'd be sure to cap the notification size.
- Keeps messages O(1) so I can focus on scaling in the amount of notifications
- Tells whoever runs into this that,
I'm reading the code for this and I'm like yeah it doesn't scale. Like forget about debugging this code, does anybody even know what it is doing?
So in summary, Postgres NOTIFY actually scales if you make your application not write to the database? That's a bit of a tough sell...
Sure it "scales" if clients batch commands to amortize the per-request cost.
The word "scale" does a lot of load-bearing, maybe it's just not a useful or productive word in practice.
Is the optimization only possible using dbos? is not clear to me if this mean a way to tune normal PG
The core optimization is to buffer notifications in-memory and send them in a batch instead of sending them as part of every transaction. So that's a general-purpose optimization for Postgres apps using LISTEN/NOTIFY.
So this is not inside a trigger but on the app connected to pg?
If I understnad right, they are saying it scales (to theri needs) with a custom patch to pg changing their semantics, right?
It does seem interesting, and possibly welcome if there were a configuration option or even a way to set individual notifies as serialized or not.
To be clear, it's not a custom patch to pg itself, but an application-side buffering and batching optimization.
Much of the FUD around using LISTEN/NOTIFY in production comes from people who have never done so. It of course has its limits, and we should remain aware of them. Much like the limits of every piece of tech in our stacks.
Choose database queue technology https://news.ycombinator.com/item?id=37636841