We are excited to announce that Event Store 4.1.0 has just been released! It is available for the following operating systems:
This version was originally set to be version 4.0.4, and is based on the 4.0.4 release candidate. We decided to bump up the version to 4.1.0 due to the large number of changes in this release, and because the new index version is not backwards compatible (more on that below.)
Loading indexes have a large impact on the startup times of Event Store. This is due to two processes that happen every time Event Store starts :
The improvements in this release focus specifically on these two processes.
Firstly, index verification can now be disabled. This means that the entire index file no longer needs to be read on startup. Some extra checks have instead been added to check for possible corruption of the indexes. If any potential corruption is discovered while running, a file is created to indicate that verification must be run at the next startup. If there is anything wrong, then the indexes will be rebuilt as per usual.
Secondly, midpoints are now written to the footer of new index files. This allows Event Store to load up the midpoints without recalculating them on every run.
These improvements won’t always be immediately apparent. There are some steps required to benefit from these changes:
--skip-index-verify option.Because of the changes mentioned above, the indexes are not backward compatible. Any index written in 4.1.0 will be considered corrupt by any versions before it. You might want to back up your indexes before upgrading so that if you do want to roll back, you can restore the indexes from backup rather than rebuilding them from scratch. If you wish to roll back and do not have your previous indexes backed up, you can rebuild them as recommended below.
The indexes will upgrade naturally as more data is written to Event Store and the index files are upgraded during merges. This can take a considerable amount of time if you have a lot of data.
If you would like to force the indexes to upgrade, you can do this by rebuilding all your indexes upfront. You can do this by simply stopping a node, deleting its index files, and restarting it. Please be aware that the index rebuild is a time-consuming process.
If you would like to rebuild your indexes with minimal downtime, you can rebuild the indexes on a separate, faster node and then restore them to the original cluster.
To do that, you would need to do the following :
This release also changes the way the index is updated with a new write, as well as how events are read from $all.
Previously the index was updated as soon as possible after an event had been written to disk on a node. This means that an event could be added to the index before it has been replicated to other nodes in the cluster. Consequently if a client were to read or subscribe to that stream, it could receive an event that does not yet exist in the entire cluster.
As an example, consider the following scenario :
Because the event was never replicated to any of the other nodes, that event no longer exists. If that event is not retried, then our client may have acted on an event that essentially never happened.
Another common result of the above scenario is the client resubscribing to one of the other nodes, but the position it tries to subscribe from is no longer valid. This causes the subscription to throw errors, and means that the client will need to resubscribe from a known valid position.
We are still working on improvements for projections and aiming to make them more stable and reliable.
There are two fixes in this RC for projections not starting up correctly, as well as a performance improvement for multi-stream projections. Additionally, the issue where projections may be deleted if they take a few minutes to start up has been fixed.