Event Sourcing

Note: This was adapted from an email newsletter from Nick Chapsas.

Traditional databases typically store an entity's latest state as a row in a table. But what if, instead of just storing the end result, we could record each change, like a history log, and rebuild the current state from these changes? Well, this is event sourcing.

What is Event Sourcing?

Event sourcing is an architectural pattern where the state of a system is stored as a series of immutable events rather than maintaining the current state directly in a database. Instead of saving just the final state of an object, we record every change as an event. Over time, this forms a history that can be replayed to reconstruct the current state of the system.

Let's see a concrete example. In a banking application, instead of updating an account's balance every time a transaction occurs, you store each transaction as an event. You would capture events like AccountOpened, FundsDeposited, FundsWithdrawn, and so on. To calculate the current balance, you sum up all of the FundsDeposited and FundsWithdrawn events for that account.

Key Benefits of Event Sourcing

The event-sourcing approach comes with several advantages over traditional state-based storage:

Immutability: Events are immutable and append-only. Once an event is stored, it cannot be modified or deleted. This guarantees an audit trail, giving you a full history of what happened in your system.
No Updates or Deletes: In a typical CRUD system, you update or delete records directly. However, with event sourcing, you never modify data. Each new event represents a state change and is appended to the history. This ensures you always have a clear timeline of events without losing any data.
Auditability: Every change to an object is explicitly captured as an event. You can easily trace why an object is in its current state when each change occurred and what the system looked like at any point in time.
Rebuilding State: By replaying events, you can reconstruct the current state of an object at any time, as well as see how it evolved. This is extremely useful for debugging or recovering from failures.
Time Travel: You can replay events to not only rebuild the current state but also to see what the state of the system was at any given point in history. This is particularly valuable in financial and regulatory contexts where a full audit trail is required.
Scalability and CQRS: Event sourcing often pairs well with the CQRS (Command Query Responsibility Segregation) pattern. The idea is to split the write model (commands that change state) from the read model (queries that fetch data). This approach allows you to scale your reads and writes independently.

As an example, this is a diagram to showcase the flow of a command when using CQRS:

CQRS Event Source Flow Diagram

Implementing Event Sourcing in .NET

Here is an example to see how event sourcing would work.

First, let's create a new base event class:

public abstract class Event
{
    public abstract Guid StreamId { get; }

    public DateTime CreatedAtUtc { get; set; }
}

Also create some events that will inherit from this class:

public class StudentCreated : Event
{
    public required Guid StudentId { get; init; }

    public required string FullName { get; init; }

    public required string Email { get; init; }

    public required DateTime DateOfBirth { get; init; }

    public override Guid StreamId => StudentId;
}

public class StudentUpdated : Event
{
    public required Guid StudentId { get; init; }

    public required string FullName { get; init; }

    public required string Email { get; init; }

    public override Guid StreamId => StudentId;
}

public class StudentEnrolled : Event
{
    public required Guid StudentId { get; init; }

    public required string CourseName { get; set; }

    public override Guid StreamId => StudentId;
}

public class StudentUnEnrolled : Event
{
    public required Guid StudentId { get; init; }

    public required string CourseName { get; set; }

    public override Guid StreamId => StudentId;
}

The streamId is mapped to the unique identifier of the base entity.

We have 4 events that can create, update, enroll and un-enroll a student. To save these events create a StudentDatabase and the Student class and explain what is going on.

public class StudentDatabase
{
    private readonly Dictionary<Guid, SortedList<DateTime, Event>> _studentEvents = new();
    private readonly Dictionary<Guid, Student> _students = new();

    public void Append(Event @event)
    {
        var stream = _studentEvents!.GetValueOrDefault(@event.StreamId, null);

        if (stream == null)
        {
            stream = new SortedList<DateTime, Event>();
            _studentEvents[@event.StreamId] = stream;
        }

        @event.CreatedAtUtc = DateTime.UtcNow;
        _studentEvents[@event.StreamId].Add(@event.CreatedAtUtc, @event);

        _students[@event.StreamId] = GetStudent(@event.StreamId)!;
    }

    public Student? GetStudent(Guid studentId)
    {
        if (!_studentEvents.ContainsKey(studentId))
        {
            return null;
        }

        var student = new Student();
        foreach (var @event in _studentEvents[studentId].Values)
        {
            student.Apply(@event);
        }

        return student;
    }

    public Student? GetStudentView(Guid studentId)
    {
        return _students!.GetValueOrDefault(studentId, null);
    }
}

Now, the Student class

public class Student
{
    public Guid Id { get; set; }

    public string FullName { get; set; }

    public string Email { get; set; }

    public List<string> EnrolledCourses { get; set; } = new();

    public DateTime DateOfBirth { get; set; }

    private void Apply(StudentCreated studentCreated)
    {
        Id = studentCreated.StudentId;
        FullName = studentCreated.FullName;
        Email = studentCreated.Email;
        DateOfBirth = studentCreated.DateOfBirth;
    }

    private void Apply(StudentUpdated updated)
    {
        FullName = updated.FullName;
        Email = updated.Email;
    }

    private void Apply(StudentEnrolled enrolled)
    {
        if (!EnrolledCourses.Contains(enrolled.CourseName))
        {
            EnrolledCourses.Add(enrolled.CourseName);
        }
    }

    private void Apply(StudentUnEnrolled unEnrolled)
    {
        if (EnrolledCourses.Contains(unEnrolled.CourseName))
        {
            EnrolledCourses.Remove(unEnrolled.CourseName);
        }
    }

    public void Apply(Event @event)
    {
        switch (@event)
        {
            case StudentCreated studentCreated:
                Apply(studentCreated);
                break;
            case StudentUpdated studentUpdated:
                Apply(studentUpdated);
                break;
            case StudentEnrolled studentEnrolled:
                Apply(studentEnrolled);
                break;
            case StudentUnEnrolled studentUnEnrolled:
                Apply(studentUnEnrolled);
                break;
        }
    }
}

The Student Database not only has the events stored for every student but also the latest view of a student, so we don't have to calculate it each time. If our application crashes, when we restart it, we can get the latest view of a student by applying all the events in order!

To see this all in action, you can write something like this:

var studentDatabase = new StudentDatabase();

var studentId = Guid.Parse("410efa39-917b-45d4-83ff-f9a618d760a3");

var studentCreated = new StudentCreated
{
    StudentId = studentId,
    Email = "nick@dometrain.com",
    FullName = "Nick Chapsas",
    DateOfBirth = new DateTime(1993, 1, 1)
};

studentDatabase.Append(studentCreated);

var studentEnrolled = new StudentEnrolled
{
    StudentId = studentId,
    CourseName = "From Zero to Hero: REST APIs in .NET"
};
studentDatabase.Append(studentEnrolled);

var studentUpdated = new StudentUpdated
{
    StudentId = studentId,
    Email = "nickchapsas@dometrain.com",
    FullName = "Nick Chapsas"
};
studentDatabase.Append(studentUpdated);

var student = studentDatabase.GetStudent(studentId);

var studentFromView = studentDatabase.GetStudentView(studentId);

The GetStudentView method would retrieve the student from the cache.

The GetStudent method would create a new student and apply all of the events on the student to get the latest version.

After all these, let's end it by talking about the pros and cons.

Id	StreamId	EventType	EventData	CreatedAtUtc
1	410efa39-917b-45d4-83ff-f9a618d760a3	StudentCreated	`{ "StudentId":"410efa39-917b-45d4-83ff-f9a618d760a3", "Email":"nick@dometrain.com", "FullName":"Nick Chapsas", "DateOfBirth":"1993-01-01T00:00:00Z" }`	2023-10-01 10:00:00
2	410efa39-917b-45d4-83ff-f9a618d760a3	StudentEnrolled	`{ "StudentId":"410efa39-917b-45d4-83ff-f9a618d760a3", "CourseName":"From Zero to Hero: REST APIs in .NET" }`	2023-10-01 10:05:00
3	410efa39-917b-45d4-83ff-f9a618d760a3	StudentUpdated	`{ "StudentId":"410efa39-917b-45d4-83ff-f9a618d760a3", "Email":"nickchapsas@dometrain.com", "FullName":"Nick Chapsas" }	2023-10-01 10:10:00

For this example, we used the StudentId as StreamId.

Pros and Cons of Event Sourcing

Pros

Auditability: Full history of changes.
Scalability: Especially when combined with CQRS.
Flexibility: Ability to evolve the system over time.

Cons

Complexity: Steeper learning curve.
Event Versioning: Managing changes to event schemas.
Tooling: Requires more sophisticated infrastructure.

Conclusion

Event sourcing might seem a bit scary at first, but by shifting our perspective from simply storing the current state to recording every change as an event, we gain a rich history of how our system evolves over time. This approach doesn't just help with debugging or auditing; it fundamentally changes how we think about data and its lifecycle.

Imagine being able to replay every action that led your application to its current state, just like rewinding a movie to understand how the plot unfolds. This level of insight can be invaluable, especially when dealing with complex systems or when you need to comply with strict regulatory requirements.