Possible new architecture

[dai-shi]

from: #59 (comment)

[dai-shi]

#59 (comment)

@hellozyemlya

Actually, I am interested, since init issues happens to me as well :)

Nice.
If we were to be free from any past requirements, what would you do?
(In the end, we would like to keep our tests as much as possible, but as I said some breaking changes should be acceptable.)

[hellozyemlya]

First of all, I am not familiar with yjs well enough to figure out why it overrides data sometimes like in #45 (comment) . If someone have suggestions - I will be happy to hear them.

[hellozyemlya]

Answering my previous question - I can imagine that Y.Doc get value from remote, but then toYValue just override that with new Y.Map\Y.Array instance, which is different from remote and yields data-loss. I can guess that at least at init time, it must call has(key)+get(key) instead of new Y.Map, e.g just merge proxy initial value to destination yjs type.

[hellozyemlya]

Seems like Y.encodeStateAsUpdate and Y.applyUpdate can solve data-loss issue(creating temp doc that contains proxy value transformed by toYValue), but that requires utilizing Y.Doc instance, which will definitely require rewrite a way of binding(e.g. we will bind not to y.map or y.array, but to y.doc directly, with some extra selector to get target y.map or y.array from y.doc)...

I suppose Y.applyUpdate will merge initial value if Y.Doc already synced with remote, if not synced - it will just copy state and connector will merge it later with remote.

[dai-shi]

we will bind not to y.map or y.array, but to y.doc directly,

That sounds like a reasonable change.

with some extra selector to get target y.map or y.array from y.doc

Ah, that sounds a bit tricky.

[hellozyemlya]

Also, I found some important side-effect - when y->p sync happens, empty yjs transaction generated, because subscribeY writes to p, which triggers subscribeP logic and yields empty yjs transaction. That makes Y.UndoManager configuration a little bit untrivial, because requires manually filter empty transaction. Wonder if that is serious issue for other yjs infrastructure and if that possible to fix without extra checks.

[agcty]

So the problem currently seems to be that valtio-yjs uses a naive two-way binding (proxy → Yjs and Yjs → proxy) that

• can overwrite existing Y structures on init (toYValue always creates new containers),
• triggers circular updates and even empty transactions, and
• lacks proper granularity (every change can cause broad re-syncs).

right?

Only possible solution then seems a re-architecture, as this issue already implies.

I’ve been thinking about it also in comparison to e.g. SyncedStore (which looks abandoned, but has some interesting ideas we could borrow like per-key/index reactivity).

Possible new architecture:

• Bind at the document level instead of individual Y types.
• Route all writes through a mediator that batches V→Y changes into one transaction with origin tags.
• Listen to Yjs changes and update proxies granularly (per key/index) instead of resyncing whole objects.
• Use Y’s native update API (encodeStateAsUpdate / applyUpdate) for init & reconciliation, so remote state isn’t overwritten.

Example usage could look like this:

const { proxy: state, dispose } = bindYDoc({
  doc: ydoc,
})

state.todos.push({ id: 't1', title: 'Hello' })

This would avoid the circular updates and empty transactions by design

[dai-shi]

I think I like the idea in general. I don't call it "bind". We basically create a new proxy from ydoc. I don't follow the yjs detail, but other than that, it sounds good.

[agcty]

So took me quite a while to figure some things out but this PR introduces a new architecture #62. Would love to hear your thoughts and happy to collab to make this the default for simple collaborative state

[willfeerr]

Some advance? I'm really need it, and possibly I'll do a lexical Collaboration Provider with it, cause lexical bindings has the same issue, cleaning the whole document

[dai-shi]

@willfeerr Check out the new repo https://github.com/valtiojs/valtio-y by @agcty .

[agcty]

@willfeerr valtio-y has now been released, however it does not support Y.Text and Y.XML for various reasons outlined in the docs. What is your use case exactly? Would love to understand your needs better.