Description
In #94, I was talking about some ways to improve redis implementation performance, mostly when there are a large number of events (or snapshots).
The problem with actual the implementation is the scan
which usually need to scan every events (or snapshots) of an aggregate, and then make the necessary calculations to only take needed events.
I have made a quick test with my production data.
I have an aggregate with +46K events with a total amount of +117K
I tried to scan all the keys (only scanning, no get
or mget
) using 2 strategies:
- I made a common
scan
on all the keys. It took 6181ms. - I put all the aggregate keys in a sorted set (members score was the event version), and made a
zrange
on all the keys. It took 415ms
I think that sorted sets could be a good option to index aggregates keys, at least for 2 reasons:
- It seems more efficient to scan all the keys at once, for an aggregate
- It will be more efficient to get keys between 2 revisions, for an aggregate (like you need to do when you reload state from a snapshot and an event)
This means that the database need to be indexed at first, and the index maintained. So whenever a client connects, we need to check if the store has been indexed, and if it's not the case, scan all the aggregates, and scan for their events and snapshots keys, and finally index the keys (events and snapshots separated obviously). Once done, each time an event is added, its key should be added to the corresponding sorted set.
What do you think about this solution? I'll try to work on an implementation. But if you have any remarks, objections or suggestions, don't hesitate!
Activity