Playing with Projections

Tutorials
,

Projections are common concept in Event Sourcing that allow you to create queries of your events and streams of events. Last year I attended Michel Grootjans’s “playing with projections” workshop that taught attendees how to create projections in a variety of programming languages. I decided to convert the workshop exercises to use Event Store’s internal projections engine and show how to use our projections API.

The data set is player interactions with a quiz. Visit the workshop wiki to see the full event list, and the projection exercises. In this first part we setup Event Store, import our data, and create two projections. Future posts will create the other projections from the workshop.

Install and setup Event Store

If you don’t have EventStore installed already, find the instructions for your development platform in our documentation.

Start EventStore with the following command to enable projections:

Windows

EventStore.ClusterNode.exe --run-projections=all --start-standard-projections=true

Linux

Add EVENTSTORE_RUN_PROJECTIONS=All and EVENTSTORE_START_STANDARD_PROJECTIONS=true to your environment variables, or the /etc/eventstore/eventstore.conf configuration file and start Event Store:

sudo systemctl start eventstore

Docker

The Event Store Docker image has projections enabled by default, but you need to enable standard projections:

docker run --name eventstore-node -it -p 2113:2113 -p 1113:1113 -e EVENTSTORE_RUN_PROJECTIONS=All -e EVENTSTORE_START_STANDARD_PROJECTIONS=true eventstore/eventstore

macOS

eventstore --run-projections=all --start-standard-projections=true

Finding Data for your Event Store Dashboard

The “Playing with projections” workshop has a series of data files in the data folder you can use for test data. They vary in size and event types, and we recommend you start with a smaller one such as 1.json that contains a couple of hundred events, and work through the larger files if you wish. We have changed the original data to suit the EventStore schema, and you can find that fork on GitHub.

Create the stream (called “quiz”) and events, for example with the HTTP API:

curl -i -d "@1.json" "http://127.0.0.1:2113/streams/quiz" -H "Content-Type:application/vnd.eventstore.events+json"

Open the admin dashboard ({SERVER_IP}:2113) and click the streams tab to see the newly created stream.

Quiz stream

Writing Your Projection

For this post we use the Projections tab, and the New Projections button of the admin UI. You can also use the .NET API or HTTP API to create a projection.

Count all events

The first projection counts all the unique events in the quiz stream. Add the code below into the text area, give it an appropriate name, and click Save:

fromStream('quiz')
.when({
    $init: function(){
        return {
            count: 0
        };
    },
    $any: function(s,e){
        s.count += 1;
    }
}).outputState();

Create projection

By default EventStoreDB disables the trackemittedstreams setting for projections. When enabled, an event written records the stream name (in $projections-{projection_name}-emittedstreams) of each event emitted by the projection. This means that write amplification is a possibility, as each event that the projection emits writes a separate event. As such, this option is not recommended for projections that emit a lot of events, and you should enable only where necessary.

A Projection starts with a selector, in this case fromStream('quiz'), which pulls events from the specified stream. Find more details on projections selector options in the documentation.

The second part of a projection is a set of filters, in this case a .when filter that matches the filter parameters you define within the filter.

Inside the filter are handlers. In this case, $init that sets up an initial state, which is a counter from 0, and the $any handler that increments the counter each time Event Store observes any event.

Finally, the outputState() filter outputs the state to a stream method, which by default produces a $projections-{projection-name}-result stream.

Projection result

Count all occurrences of a specific event type

The next projection counts all occurrences of a particular event type, for example PlayerHasRegistered, indicating that a new player has registered to take part in the quiz.

fromStream('quiz')
.when({
    $init: function(){
        return {
            count: 0
        };
    },
    PlayerHasRegistered: function(s,e){
        s.count += 1;
    }
}).outputState();

Create projection

The only difference in this code example is that we changed the $any handler to match that of the event type we are looking for.

Create, save, and run this projection.

Projection result


Related posts

Data ingestion with Logstash and EventStore

Tutorials
,

In this post our challenge is to load a CSV file and ingest it using the [EventStoreDB](/eventstoredb/) HTTP API to be ingested. To be precise, we want to convert this: ```csv Europe,Italy,Clothes,Online,M,12/17/2013,278155219,1/10/2014,1165,109.28,35.84,127311.20,41753.60,85557.60 ``` To this (which is an example of an HTTP POST to the Event Store HTTP API): ```shell [ { "eventId": "fbf4b1a1-b4a3-4dfe-a01f-ec52c34e16e4", "eventType": "InboundDataReceived", "data": { "message": "Europe,Italy,Clothes,Online,M,12/17/2013,278155219,1/10/2014,1165,109.28,35.84,127311.20,41753.60,85557.60" }, "metadata": { "host": "box-1", "path": "/usr/data/sales.csv" } } ] ``` In this example we...


Easier Development Builds on Unix-like Operating Systems

Tutorials
,

A common complaint we’ve heard about Event Store since it was originally open sourced is that it’s complex to make development builds locally - and they were slow - and thus the project was hard to contribute to. As part of our recent work on Event Store v3.1.0 (more on this in another post) we decided to rectify this. Event Store is primarily written in C#, and builds on Mono on Linux and MacOS and...


Catch-up subscriptions with the Event Store

Tutorials
,

We have had many questions about how to use the catch-up subscriptions in the C# Event Store Client API. In this post I’ll build out a simple example of a console application which receives a callback each time an event is written to the built-in statistics stream. Contrary to what some have thought, catch-up subscriptions are implemented as part of the client, and have been available in the client API version 1.1.0 which has been...