Trying to guess what's going on from what is there... looks like conflicts are raised on a per row basis, and the app resolves the conflicts.
Per-row is going to make it hard to handle a lot of data models. Maybe there's some way to see the whole transaction, but how? And how can you see the server vs local data? You'll want that for many cases.
It's right to leave it to the app to resolve conflicts -- there is no one-size-fits-all answer to this. But by providing no tools and, seemingly pushing everything through a per-row API, it's going to be essentially impossible for apps to get things right. You'll end up with incoherent data -- data that isn't valid within one node, and nodes different from each other. Generally, not what you want when you implement sync.
Looks interesting, was looking for something like this as currently for Flutter I use Loro via flutter_rust_bridge as the CRDT data store, but it's not SQL so some things like big queries are annoying. How are you handling CRDTs in a SQL database? I thought those were notoriously hard as relational wasn't built for CRDT style syncing?
That’s kinda the tension I wanted to avoid. I’m not trying to make the whole SQL database a CRDT.
CRDTs are opt-in per column. A row can keep normal, queryable fields like project_id, status, and title, while one doc column stores Yjs/yrs bytes. Concurrent edits to that column are merged by the server; normal columns still use versions and explicit conflicts. Flutter uses yrs through the shared Rust core.
The trade-off is that SQL can’t query inside the CRDT document. Anything you need for filtering, joins, or aggregates should remain an ordinary column. That boundary gives you relational queries without giving up CRDTs where they’re really useful.
I’d be interested in your feedback on the Flutter client. I chose a very small C ABI—five hand-written dart:ffi bindings into the Rust core, rather than generating a large FFI surface. The app-specific schema, row types, subscriptions, and typed SQL/SYQL queries are generated from the shared schema IR using syncular generate. I’d love to hear how that feels compared with your flutter_rust_bridge setup.
Got it. Many things I do need to be queryable though, like for example if I'm making a to-do list app the todos need to be queryable, so not sure your approach solves much for my use cases. For the Flutter side it's not too bad using FRB, I'd say AI takes care of a lot of the details.
This is cool. Reminds of Lotus Notes. In a strange way a concept that felt "outdated" is something that I would love to be the default behavior for many apps today.
It’s great to see apps being shipped with offline first (and flawless syncing as an optional feature) from ground up, sad to see many modern notes apps like notion or capacities treating offline as afterthought. But alas, it’s still a non trivial nut to crack.
Hope one day offline first will be a default and a solved problem.
Agreed. Also, pretty confident this is one of the reasons Obsidian is able to compete (and win) against Notion despite the huge difference in capitalization.
How do you handle browser storage eviction? I shipped a DuckDB-Wasm in-browser and found that survives reloads isn’t isnt really reliable.
Since pending writes live in the OPFS SQLite database, does the recommended setup request persistent storage or warn the user that an unsynced outbox could disappear if the origin is evicted?
Both use local SQLite and keep the backend authoritative. The main difference is that PowerSync mostly handles downstream replication and gives you an upload queue; your API still owns the write semantics. Syncular specifies the whole round trip. Durable commits, auth scopes, conflicts, replay, revocation and bootstrap.
Syncular also goes further up the stack: schemas plus SQL/SYQL generate typed queries for TS, Swift, Kotlin, Dart and Rust, and blobs, CRDT/encrypted columns and windowed sync are built into the protocol.
In my separate benchmark harness, Syncular currently does considerably better than the tested PowerSync setup. For example, 100k bootstrap is ~0.57s vs ~6.5s and offline replay ~35ms vs ~5.1s. Those are early results from one environment, though, not a universal claim (https://github.com/bkniffler/offline-sync-bench).
I've used PowerSync in the past with a huge initial bootstrap sync and was very frustrated by the performance back then (ca 1 year ago), which was one of the reasons to create my own library.
PowerSync is much more mature today obviously and still a great and proven tool.
Firstly, it's awesome that you are using @sqlite.org/sqlite-wasm for your client-side SQLite database. We really like wa-sqlite ourselves.
We're currently implementing this proposal https://github.com/orgs/powersync-ja/discussions/376, which will speedup the sync throughput a lot - this was a natural next step for us as we have users with massive datasets (syncing a million plus rows).
Sidebar: What has your experience with OPFS+SQLite been like on sqlite-wasm? We have seen our fair share of issues on Safari/web kit.
Hey, nice, looks like I’ll have to rerun the benchmarks soon!
sqlite-wasm with opfs-sahpool has mostly behaved. No OPFS in private mode, storage may be evicted, handles can’t be passed to workers, and webviews may close them when backgrounded.
We open SQLite inside one dedicated worker and proxy other tabs through it, so that avoids some of the mess. Tauri uses native SQLite.
Nice! Would recommend to decouple as much as possible and have tons of testing, every issue thats catched before having to go to the length of debugging client side issues is gonna save some sleep. Have been there, learned the hard way..
Yeah it was a deceptively complicated issue. I’m not sure if my implementation is as comprehensive, but it sounds similar: optimistic local writes, outbound queue, server as source of truth, each client has local SQLite.
The issue I found most difficult architecturally was how to deal with staying responsive on first sign in with a large DB. I ended up with a system I called “materialisation on demand” where it will search the server and download only those rows ahead of time.
EDIT: I also ended up using multiple local databases to ensure responsive writes during heavy synching activity.
The README has had plenty of LLM help, yes. I’ve described how I use them here: https://syncular.dev/llms/
I’ll keep refining it (talking about docs, tests, benchmarks etc). LLMs are involved throughout Syncular, as they are in my other projects, but this sure wasn't prompted into existence, its been built on and off over a lot of time. Have spent years deep in the offline-first rabbit hole. Most code, design and its trade-offs come from that experience and predate LLMs.
Hi HN! I’ve been circling this problem for a long time.
In 2019 I built an offline-first datastore called debe. It used CRDTs and multi-master replication and never became something I could confidently ship. It did teach me that convergence is only a small part of sync. The difficult parts are durable writes, authorization changes, lost acknowledgements, schema upgrades, bounded bootstrap, and explaining what happened after something fails.
I eventually came back to the problem and built Syncular over the last year.
Every client has a real SQLite database: sqlite-wasm on OPFS in the browser and native SQLite elsewhere. Writes apply locally and enter a durable outbox in the same transaction. A server-authoritative commit log validates and orders them, using explicit scopes resolved inside the application backend.
The protocol is written down independently of the implementations. SSP2 is currently a draft, with canonical binary encoding and golden byte vectors. There are independent TypeScript and Rust client cores, both run against the same 95-scenario conformance catalog. The Rust core is exposed through Swift, Kotlin, Flutter, React Native, Tauri, Rust, and a small C API.
It also includes realtime WebSocket sync, resumable bootstrap segments, explicit conflict evidence, authorization revocation and local purge, windowed replicas, blobs, optional CRDT columns, per-column encryption, and typesafe generated queries for five languages.
Important boundaries: it is pre-1.0, self-hosted, and there is no managed service or peer-to-peer mode. Native packaging maturity still varies by ecosystem.
Let me know what you think, I'm happy about feedback!
There was a local-first conv just a few weeks ago in Berlin; if you're not yet part of that community I suggest reaching out. https://www.localfirst.fm/ for example, or https://lofi.so/
I wish there was an established way to build local first software with a robust syncing in the background, either P2P or to a server.
I feel projects like this for local first and projects like Iroh for connectivity over NAT and through firewalls need to come together more.
It would be could to have a drop in framework for this.
Have you looked at https://duckdb.org/quack/ ?
Loro along with their protocol crate works great. Automerge has something similar.
[dead]
No mention of conflict resolution? I suspect it's far too naive to be useful for anything real.
https://github.com/syncular/syncular/blob/0ff4fed6b7e538293e...
There's not much there.
Trying to guess what's going on from what is there... looks like conflicts are raised on a per row basis, and the app resolves the conflicts.
Per-row is going to make it hard to handle a lot of data models. Maybe there's some way to see the whole transaction, but how? And how can you see the server vs local data? You'll want that for many cases.
It's right to leave it to the app to resolve conflicts -- there is no one-size-fits-all answer to this. But by providing no tools and, seemingly pushing everything through a per-row API, it's going to be essentially impossible for apps to get things right. You'll end up with incoherent data -- data that isn't valid within one node, and nodes different from each other. Generally, not what you want when you implement sync.
[dead]
I'm not sure I would entirely trust a tricky algorithm not written by a real developer. Still, interesting.
(If it is manually written I don't trust the LLM-written docs.)
[dead]
[flagged]
Looks interesting, was looking for something like this as currently for Flutter I use Loro via flutter_rust_bridge as the CRDT data store, but it's not SQL so some things like big queries are annoying. How are you handling CRDTs in a SQL database? I thought those were notoriously hard as relational wasn't built for CRDT style syncing?
That’s kinda the tension I wanted to avoid. I’m not trying to make the whole SQL database a CRDT.
CRDTs are opt-in per column. A row can keep normal, queryable fields like project_id, status, and title, while one doc column stores Yjs/yrs bytes. Concurrent edits to that column are merged by the server; normal columns still use versions and explicit conflicts. Flutter uses yrs through the shared Rust core.
The trade-off is that SQL can’t query inside the CRDT document. Anything you need for filtering, joins, or aggregates should remain an ordinary column. That boundary gives you relational queries without giving up CRDTs where they’re really useful.
I’d be interested in your feedback on the Flutter client. I chose a very small C ABI—five hand-written dart:ffi bindings into the Rust core, rather than generating a large FFI surface. The app-specific schema, row types, subscriptions, and typed SQL/SYQL queries are generated from the shared schema IR using syncular generate. I’d love to hear how that feels compared with your flutter_rust_bridge setup.
More detail: https://syncular.dev/concepts-crdt/ https://syncular.dev/platform-flutter/
Got it. Many things I do need to be queryable though, like for example if I'm making a to-do list app the todos need to be queryable, so not sure your approach solves much for my use cases. For the Flutter side it's not too bad using FRB, I'd say AI takes care of a lot of the details.
This is cool. Reminds of Lotus Notes. In a strange way a concept that felt "outdated" is something that I would love to be the default behavior for many apps today.
It’s great to see apps being shipped with offline first (and flawless syncing as an optional feature) from ground up, sad to see many modern notes apps like notion or capacities treating offline as afterthought. But alas, it’s still a non trivial nut to crack.
Hope one day offline first will be a default and a solved problem.
Agreed. Also, pretty confident this is one of the reasons Obsidian is able to compete (and win) against Notion despite the huge difference in capitalization.
Pretty cool.
How do you handle browser storage eviction? I shipped a DuckDB-Wasm in-browser and found that survives reloads isn’t isnt really reliable.
Since pending writes live in the OPFS SQLite database, does the recommended setup request persistent storage or warn the user that an unsynced outbox could disappear if the origin is evicted?
[flagged]
In which way this is different from PowerSync?
Both use local SQLite and keep the backend authoritative. The main difference is that PowerSync mostly handles downstream replication and gives you an upload queue; your API still owns the write semantics. Syncular specifies the whole round trip. Durable commits, auth scopes, conflicts, replay, revocation and bootstrap. Syncular also goes further up the stack: schemas plus SQL/SYQL generate typed queries for TS, Swift, Kotlin, Dart and Rust, and blobs, CRDT/encrypted columns and windowed sync are built into the protocol.
In my separate benchmark harness, Syncular currently does considerably better than the tested PowerSync setup. For example, 100k bootstrap is ~0.57s vs ~6.5s and offline replay ~35ms vs ~5.1s. Those are early results from one environment, though, not a universal claim (https://github.com/bkniffler/offline-sync-bench).
I've used PowerSync in the past with a huge initial bootstrap sync and was very frustrated by the performance back then (ca 1 year ago), which was one of the reasons to create my own library.
PowerSync is much more mature today obviously and still a great and proven tool.
Hey PowerSyncer here :)
Firstly, it's awesome that you are using @sqlite.org/sqlite-wasm for your client-side SQLite database. We really like wa-sqlite ourselves.
We're currently implementing this proposal https://github.com/orgs/powersync-ja/discussions/376, which will speedup the sync throughput a lot - this was a natural next step for us as we have users with massive datasets (syncing a million plus rows).
Sidebar: What has your experience with OPFS+SQLite been like on sqlite-wasm? We have seen our fair share of issues on Safari/web kit.
Hey, nice, looks like I’ll have to rerun the benchmarks soon!
sqlite-wasm with opfs-sahpool has mostly behaved. No OPFS in private mode, storage may be evicted, handles can’t be passed to workers, and webviews may close them when backgrounded.
We open SQLite inside one dedicated worker and proxy other tabs through it, so that avoids some of the mess. Tauri uses native SQLite.
It’s a bit weird to see every comment here talks about apps she or he has built but never really give a shit about this app itself.
[dead]
Interesting! I wrote something similar for my app recently, it was the first thing I coded using LLMs.
Nice! Would recommend to decouple as much as possible and have tons of testing, every issue thats catched before having to go to the length of debugging client side issues is gonna save some sleep. Have been there, learned the hard way..
Whats your app?
Yeah it was a deceptively complicated issue. I’m not sure if my implementation is as comprehensive, but it sounds similar: optimistic local writes, outbound queue, server as source of truth, each client has local SQLite.
The issue I found most difficult architecturally was how to deal with staying responsive on first sign in with a large DB. I ended up with a system I called “materialisation on demand” where it will search the server and download only those rows ahead of time.
EDIT: I also ended up using multiple local databases to ensure responsive writes during heavy synching activity.
The app is https://www.benko.app/ it’s available on iOS, android, Mac and windows.
[dead]
The README phrasing has vibecoding tells right away
The README has had plenty of LLM help, yes. I’ve described how I use them here: https://syncular.dev/llms/
I’ll keep refining it (talking about docs, tests, benchmarks etc). LLMs are involved throughout Syncular, as they are in my other projects, but this sure wasn't prompted into existence, its been built on and off over a lot of time. Have spent years deep in the offline-first rabbit hole. Most code, design and its trade-offs come from that experience and predate LLMs.
Be warned, AI Slop ahead
[dead]
Hi HN! I’ve been circling this problem for a long time.
In 2019 I built an offline-first datastore called debe. It used CRDTs and multi-master replication and never became something I could confidently ship. It did teach me that convergence is only a small part of sync. The difficult parts are durable writes, authorization changes, lost acknowledgements, schema upgrades, bounded bootstrap, and explaining what happened after something fails.
I eventually came back to the problem and built Syncular over the last year.
Every client has a real SQLite database: sqlite-wasm on OPFS in the browser and native SQLite elsewhere. Writes apply locally and enter a durable outbox in the same transaction. A server-authoritative commit log validates and orders them, using explicit scopes resolved inside the application backend.
The protocol is written down independently of the implementations. SSP2 is currently a draft, with canonical binary encoding and golden byte vectors. There are independent TypeScript and Rust client cores, both run against the same 95-scenario conformance catalog. The Rust core is exposed through Swift, Kotlin, Flutter, React Native, Tauri, Rust, and a small C API.
It also includes realtime WebSocket sync, resumable bootstrap segments, explicit conflict evidence, authorization revocation and local purge, windowed replicas, blobs, optional CRDT columns, per-column encryption, and typesafe generated queries for five languages. Important boundaries: it is pre-1.0, self-hosted, and there is no managed service or peer-to-peer mode. Native packaging maturity still varies by ecosystem.
Let me know what you think, I'm happy about feedback!
Demo: https://demo.syncular.dev Protocol: https://github.com/syncular/syncular/blob/main/docs/SPEC.md Background: https://syncular.dev/blog/offline-first-writes/ Docs: https://syncular.dev/
There was a local-first conv just a few weeks ago in Berlin; if you're not yet part of that community I suggest reaching out. https://www.localfirst.fm/ for example, or https://lofi.so/
Thanks for pointing me there! Posted to their discord, but seems not so active.
Very interesting, have you experimented with getting it working with Crux in Rust?
[dead]
[flagged]
[flagged]