> Instead of joining the tables completely and sorting at the end, a Rank Join streams its inputs in descending score order (sorted access, or SA), building candidate result-sets as it goes.
Your algorithm feels like a variant of the WAND algorithm the Snowflake Top-K pruning paper [1] pulls from.
You can go distributed with this approach if you have a "gossip" protocol between workers to distribute the heap boundaries periodically.
Before the AI age, a "papers we love" group was a great thing within any company.
Now, Claude or Codex will know the SOTA literature around a topic like this and "uniformly distribute" the past into every single implementation of a similar idea. The future isn't uniformly distributed, but the past is getting the treatment & that's good.
This looks exciting for some searches (top k), but the headline doesn't really fit. Yes - we still have to denormalize. Just not for some of the use cases paradedb targets. Impressive work.
> Instead of joining the tables completely and sorting at the end, a Rank Join streams its inputs in descending score order (sorted access, or SA), building candidate result-sets as it goes.
Your algorithm feels like a variant of the WAND algorithm the Snowflake Top-K pruning paper [1] pulls from.
You can go distributed with this approach if you have a "gossip" protocol between workers to distribute the heap boundaries periodically.
Before the AI age, a "papers we love" group was a great thing within any company.
Now, Claude or Codex will know the SOTA literature around a topic like this and "uniformly distribute" the past into every single implementation of a similar idea. The future isn't uniformly distributed, but the past is getting the treatment & that's good.
[1] - https://arxiv.org/html/2504.11540v1#S5
Any questions please go for it, ParadeDB team is watching.
This looks exciting for some searches (top k), but the headline doesn't really fit. Yes - we still have to denormalize. Just not for some of the use cases paradedb targets. Impressive work.