Event Store Blog

EventStoreDB 24.10 Preview

Written by Event Store Team | Oct 9, 2024 8:58:09 PM

Today, we are thrilled to announce a preview release of EventStoreDB 24.10 is available. This release is a significant update introducing important operational features and a mature set of connectors to help you build event-native applications. The highlights below represent what EventStoreDB 24.10 will offer.

In addition, 24.10 introduces significant changes to how EventStoreDB is packaged and delivered to users, described below.

Distribution and license keys

Until today, EventStoreDB was distributed in two packages:

EventStore-OSS, built from the Open Source codebase, containing core database features
EventStore-Commercial, based on the OSS package, including features only available for Event Store customers, as well as (partially) in Event Store Cloud

Starting with 24.10, EventStore-EE (Enterprise Edition) will be the only package available. The package is freely available for anyone to use. Providing a license or trial key enables additional enterprise features.

The new packages are available via a new Event Store package registry. The PackageCloud registry will only contain previous versions of EventStoreDB.

Based on the changes to the package name, upgrades from previous ESDB versions require you to uninstall the previous version, add the new package registry to the operating system, and install the latest version using the new package name.

Additional details on the upgrade process can be found here.

To activate enterprise features, request a license key here.

Highlights

The 24.10 release represents a milestone for Event Store beyond updating how it is licensed, packaged, and delivered. The list of new features available in the preview release was driven directly from customer feedback and continues to establish EventStoreDB as enterprise-grade. The following are a list of the new features:

  • Connector HTTP sink (available in the free version)
  • New Connectors (egress) supporting HTTP and Kafka (license key required)
  • Fully automated scheduled scavenge (license key required)
  • Policy-based authorization for streams (license key required)
  • Encryption at rest (license key required)

New connectors

The Connectors Preview was released in EventStoreDB 24.2 as part of Event Store’s Commercial Edition.  The connector preview allowed you to send events from ESDB directly to an HTTP endpoint for additional processing. Starting with the 24.10 preview, the HTTP sink connector is freely available to everyone. We are also adding three more destinations: Kafka, RabbitMQ, and MongoDB sink connectors have been added for users with a valid license key.

The connectors allow you to integrate ESDB with other systems without writing code or with a small amount of code focused on what needs to be done rather than how. For example, the HTTP sink allows you to create an HTTP API application that processes events however desired. The connector can be configured to send events to that API. The application behind that API can project events to other databases, execute complex event processing logic, or integrate with other systems. You can run the application in a serverless function. EventStoreDB will call the provided HTTP API, so there’s no need to implement any infrastructure logic for hosting and maintaining subscriptions or, in that regard, use the EventStoreDB client library.

The new Kafka connector allows you to apply simple event transformations and send events directly from ESDB to Apache Kafka. The Kafka sink requires a license key.

We plan to include more sinks in the 24.10 GA release.

All connector sinks will also be available in Event Store Cloud when 24.10 GA is released.

Connectors documentation

Automated Scavenge

A common operational challenge with EventStoreDB is correctly automating the scavenging process across cluster nodes. It’s not always obvious how to properly automate the process, as it might require knowing the node role in the cluster and whether the scavenge is already running on other nodes.

The 24.10 automated scavenge is a fantastic new feature that removes this operational complexity. After the auto-scavenge configuration is provided, the cluster will execute scavenge operations on a specified schedule. It monitors progress to enforce two nodes not to be scavenged simultaneously and ensures the cluster changes its leader before trying to run scavenge on the node. The cluster remains clean while maintaining its transactional performance.

The feature is available for EventStoreDB and Event Store Cloud customers with a valid license key.

Documentation

Stream Authorization Policies

EventStoreDB historically provided powerful authorization features. Each stream can be protected by an access control list (ACL), allowing only specific users to read and write to it.

However, ACLs are very granular and must be set for each stream. It’s common for streams with the same prefix or category, for example, “Order,” to all require the same write and read permissions. It's also common that there may be millions of streams in one category. Changing authorization policies for the whole category of streams quickly becomes extremely complex and time-consuming. As a result, many users do not use ACLs for authorization and don’t maintain strong authorization policies for accessing data.

Policy-based authorization replaces ACLs. The two authorization methods cannot be used at the same time. The new process allows you to configure a policy and apply it to an entire category of streams. When the policy changes, it is immediately applied to all relevant streams.  No longer are users required to adjust individual streams in the category. This feature allows our customers to introduce proper authorization for accessing data without the high maintenance burden. Scenarios where data segregation is required can now be easily implemented. For example, you can have specific policies per tenant, microservice, or any other logical boundary if the context name is captured in stream names.

Using policies also speeds up authorized reads and writes because ESDB doesn’t need to read ACLs for specific streams.

In the future, we plan to support authorized reads and subscriptions to $all stream, which currently require administrator access.

The feature is available for customers running EventStoreDB and Event Store Cloud with a valid license key

Documentation

Encryption-at-rest

This new feature adds additional protection to data stored in EventStoreDB. It is common to use filesystem-based encryption of data volumes, as done in Event Store Cloud. Still, encrypted volumes are accessible by anyone with access to the physical or virtual machine where the volume is mounted and read data directly from database chunk files. 

With encryption at rest, data files are protected by encryption inside EventStoreDB If a malicious user manages to retrieve data files from a production database, the data remains inaccessible without the encryption key.

EventStore DB only supports master keys stored in files. While providing value when the key file is mounted from a protected volume, based on the security levels of file-based key storage we don’t currently recommend using this feature for production environments that fall under strict compliance regulations.

We aim to evolve this feature over time. We encourage our customers to provide feedback on where they wish to store their keys so we can add integrations to third-party key management systems.

The feature is available for customers running EventStoreDB with a valid license key.  It is not available for Event Store Cloud customers.  

Documentation

Installing the Preview

This preview is intended for testing purposes only and is not supported in production environments.

We recommend installing a new cluster to access the new features. Follow the installation guidelines in our documentation. To activate enterprise features, request a license key here.

It’s also possible to use the preview version with production-like data by creating a copy of a production or staging cluster, and upgrading it to the preview version. The upgrade process is different from previous versions.  Please read the upgrade guide carefully.

Feedback

Please feel free to contact us and give feedback about the new version. You can use one of the following feedback options:

We greatly appreciate your feedback; it helps us shape EventStoreDB's future!