EventStoreDB 3.9.3 Released

Happy thanksgiving to everyone that celebrates it. We have just released Event Store 3.9.3. This release includes a set of fixes and will likely be the last 3.x release before Event Store 4.0.0. The fixes in the release are mainly focussed around an issue that could have resulted in duplicate entries being written due to a hash conflict between stream names occurring.

When will Event Store 4.0.0 be released?

As this is a major version, and the anticipated release of projections, we are working very hard to ensure that we meet our expectations and release before the end of the year. You can expect to receive an email as we get closer to the release which will detail what this release contains, as well as possible pre-release versions.

More details on the hash collision and duplicate entry fix

A stream that has been affected by this particular issue is considered unreadable. Event Store 3.9.3 will restore the stream to a readable state and a scavenge will remove the erroneous events from all the affected streams.

The message stems from a validation that is performed as a last set of checks that are performed before the read operation returns to the client. In particular, one of the checks ensures that the events that were read are in the correct order. The message is indicating that they were not in order, due to another event at position 0 appearing in the slice of events that were read.

The conditions that need to be in place for the above issue to occur is all of the following:

  • The last correct event number for the destination stream should not be cached. (When you write to a stream, or the in memory index is built up on a node restart, the event numbers for the streams are cached).
  • 2 streams have the same hash. i.e. a hash collision needs to occur.
  • The hash collision read limit has to be exhausted. For example:
    • stream-x has a 1 events.
    • stream-y has a 100 events.
    • We try and find the last event for stream-x, but keep reading stream-y’s events as we are going through the index, we eventually exhaust the hash collision read limit and return NoStream.

By upgrading to 64bit indexes, you are unlikely to run into a hash collision which is why it will reduce the likelihood of this issue re-occurring. Also, if you are running with 3.9.x, new indexes that are created will always be 64bit indexes, also when a merge takes place, the merge will result in a 64bit index (This is how Event Store will eventually upgrade all your indexes to 64bit indexes).

In the cases where a normal scavenge does not remove the duplicate events due to the scavenged chunk not being compacted, we have added an option (--always-keep-scavenged, AlwaysKeepScavenged: true) that will ensure that the scavenged chunk is kept. You don’t want to always run with the option turned on and therefore we suggest that you only run a scavenge with this option turned on once.

Event Store Server 3.9.3 release notes

  • #1099 (All Platforms) Fix 2 additional duplicate entry issues.
    • when the duplicate entry is in the memory table and when rebuilding the memory table on node startup.
    • when the duplicate entry is cached as the last event number.
  • #1080 (All Platforms) Fix performance counters for a process.
  • #1078 (All Platforms) Fix expectations from exhausting the hash collision read limit.
  • #1075 (All Platforms) Start the persistent subscription from the event after the checkpoint.
  • #1104 (All Platforms) Add option to always keep scavenged chunks.

Photo of Pieter Germishuys

Written by: Pieter Germishuys

Pieter is a developer at Event Store and works in our Netherlands office. He originally joined Event Store in early 2014 and came back in late 2019. He brings a strong customer-focused approach to building reliable software. Outside of work, Pieter is an avid pc gamer and spends the days he isn't playing games learning new programming languages, technologies and techniques.

Related posts

EventStoreDB 20.6.0 Release

Release Notes
Hayley Campbell,

We are pleased to announce the official release of [EventStoreDB]( 20.6.0! Thank you to everyone who tried out the previews and release candidates and provided us with some great feedback. It is available for the following operating systems: - Windows - Ubuntu 16.04 - Ubuntu 18.04 - CentOS 7 (Commercial version) - Amazon Linux 2 (Commercial version) - Oracle Linux 7 (Commercial version) ## Where can I get the packages? Downloads are available on our...

EventStoreDB 20.6 Release Candidate

Release Notes
Pieter Germishuys,

We are pleased to announce the release candidate of [Event Store]( 20.6, which contains mainly bug fixes and improvements on the previous release This release is not intended to be used in production, we welcome your feedback as we prepare for final release. If you encounter any issues, please don’t hesitate to [open an issue]( on GitHub if there isn’t one already or head over to our [community forum]( to start a discussion. You can...

EventStoreDB 5.0.8

Release Notes
Mat McLoughlin,

Version 5.0.8 has been released which contains a number changes and bug fixes. [EventStoreDB](/eventstoredb/) 5.0.8 is out! This release contains some important bug fixes related to TCP connections for both the server and the .NET client. The fixes bring an overall improvement to the stability of subscriptions. It is available for the following operating systems: - Windows - Ubuntu 18.04 - Ubuntu 16.04 - CentOS 7 (Commercial version) - Amazon Linux AMI (Commercial version) -...