Sourcing: Part 1 – Exploring Event Sourcing in Raku

I’ve been studying a very interesting approach to building systems called Event Sourcing. In event sourcing, every change to an application’s state is captured as an immutable event. Instead of storing only the current state, you persist a sequence of events that record all the actions that have taken place. This approach not only allows for an accurate reconstruction of the system state by replaying events but also provides a robust audit log and improves debugging and analysis capabilities. For my studies, I started writing a framework to easily implement event sourcing in Raku. It’s called Sourcing. Sourcing has a few key concepts. In this first part, we’ll discuss three of them: Events Event Store Projections Events Events are the core building blocks in an event-sourced system. Each event represents a fact that has occurred in the past, such as an order being created or a payment being initiated. Here are some important points about events: Immutability: Once an event is recorded, it should never change. This immutability guarantees that the historical record remains accurate and reliable. Past Facts: Events capture facts that have already happened, meaning they are a definitive record of the state transitions within your system. Audit Trail: By recording every event, you create a full history of all actions, making it easier to trace issues or understand how a certain state was reached. Event Store The Event Store is a specialized storage mechanism for events. It acts as a single source of truth for all the state changes in your system by providing these features: Append-Only Log: Events are stored in an append-only manner, ensuring that once an event is recorded, it is never modified. Order Preservation: The sequence of events is maintained, which is critical for reconstructing the state by replaying the events in the order they occurred. Durability and Auditability: The event store keeps a permanent record of every event, which can be used for debugging, analytics, or even recreating the state at any given point in time. Projections Projections are responsible for transforming the raw stream of events into a read model that is easier to query and work with. Many projections can/should use the same set of events. They serve several important functions: State Aggregation: Projections listen to events from the Event Store and update a view or model that represents the current state of the system. Multiple Views: You can create multiple projections for the same set of events, each tailored to different client needs. For example, one projection might track the status of a food order while another aggregates sales data. Decoupling: By separating the write model (events) from the read model (projections), you can optimize the performance and scalability of your application. The following code shows how a projection, DeliveryStatus, is implemented in our framework. Each projection must define an apply method for every event it wants to handle. When an event is emitted, the projection’s corresponding apply method is called to update its state. Food Delivery Example To demonstrate the framework, let’s build a simple Food Delivery system. This system uses a few types of events to represent different stages in the order lifecycle, an example of one of the events class wold be: use Sourcing; use UUID::V4; unit event Sourcing::FoodDelivery::Event::OrderCreated; has Str $.order-id = uuid-v4; has Str $.delivery-code = (1 ..^ 100)>> .fmt("%02d") .pick; has UInt $.user-id is required; has UInt $.restaurant-id is required; has Str %.item{Str}; Event description OrderCreated Represents when a new order is created. PaymentInitiated Indicates that the payment process has started. PaymentConfirmed Signals that the payment was successfully completed. PaymentFailed Represents a failure or error during the payment process. OrderAccepted Indicates that the restaurant has accepted the order and begun preparation. DeliveryStarted Marks the moment when the deliverer collects the order. DeliveryCompleted Represents the event when the deliverer confirms that the delivery is complete. OrderDelivered Marks the final stage when the order is successfully delivered. Projection: DeliveryStatus Projections consume events to maintain a read model. In this example, the DeliveryStatus projection tracks the status of an order and stores the delivery code required for the deliverer. When a projection object is created, it is registered with a manager (to be described in a future post) and remains in memory until it is unloaded (a feature still in development). Projections don't need to use all events and events don't need to be used by projections. On this example, I'm defining some Payment events that are not being used by the projection. That's completely ok. if in the future, a new projection

Mar 2, 2025 - 18:47

Sourcing: Part 1 – Exploring Event Sourcing in Raku

I’ve been studying a very interesting approach to
building systems called Event Sourcing. In event sourcing,
every change to an application’s state is captured as an
immutable event. Instead of storing only the current state,
you persist a sequence of events that record all the actions
that have taken place. This approach not only allows for an
accurate reconstruction of the system state by replaying
events but also provides a robust audit log and improves
debugging and analysis capabilities.

For my studies, I started writing a framework to easily
implement event sourcing in Raku. It’s called Sourcing.

Sourcing has a few key concepts. In this first part, we’ll discuss three of them:

Events
Event Store
Projections

Events

Events are the core building blocks in an event-sourced system.
Each event represents a fact that has occurred in the past,
such as an order being created or a payment being initiated.
Here are some important points about events:

Immutability:

Once an event is recorded, it should never change.
This immutability guarantees that the historical
record remains accurate and reliable.

Past Facts:

Events capture facts that have already happened,
meaning they are a definitive record of the state
transitions within your system.

Audit Trail:

By recording every event, you create a full history of all actions,
making it easier to trace issues or understand
how a certain state was reached.

Event Store

The Event Store is a specialized storage mechanism for events.
It acts as a single source of truth for all the state changes
in your system by providing these features:

Append-Only Log:

Events are stored in an append-only manner,
ensuring that once an event is recorded,
it is never modified.

Order Preservation:

The sequence of events is maintained,
which is critical for reconstructing the state by replaying
the events in the order they occurred.

Durability and Auditability:

The event store keeps a permanent record of every event,
which can be used for debugging, analytics,
or even recreating the state at any given point in time.

Projections

Projections are responsible for transforming the raw stream
of events into a read model that is easier to query and work with.
Many projections can/should use the same set of events.
They serve several important functions:

State Aggregation:

Projections listen to events from the Event Store
and update a view or model that represents the current state of the system.

Multiple Views:

You can create multiple projections for the same set of events,
each tailored to different client needs. For example,
one projection might track the status of a food order
while another aggregates sales data.

Decoupling:

By separating the write model (events) from the read model (projections),
you can optimize the performance and scalability of your application.

The following code shows how a projection, DeliveryStatus,
is implemented in our framework.
Each projection must define an apply method for every event it wants to handle.
When an event is emitted, the projection’s corresponding
apply method is called to update its state.

Food Delivery Example

To demonstrate the framework, let’s build a simple Food Delivery system.
This system uses a few types of events to represent different
stages in the order lifecycle, an example of one of the events class
wold be:

use Sourcing;
use UUID::V4;

unit event Sourcing::FoodDelivery::Event::OrderCreated;

has Str  $.order-id      = uuid-v4;
has Str  $.delivery-code = (1 ..^ 100)>> .fmt("%02d") .pick;
has UInt $.user-id       is required;
has UInt $.restaurant-id is required;
has Str  %.item{Str};

Event	description
OrderCreated	Represents when a new order is created.
PaymentInitiated	Indicates that the payment process has started.
PaymentConfirmed	Signals that the payment was successfully completed.
PaymentFailed	Represents a failure or error during the payment process.
OrderAccepted	Indicates that the restaurant has accepted the order and begun preparation.
DeliveryStarted	Marks the moment when the deliverer collects the order.
DeliveryCompleted	Represents the event when the deliverer confirms that the delivery is complete.
OrderDelivered	Marks the final stage when the order is successfully delivered.

Projection: DeliveryStatus

Projections consume events to maintain a read model. In this example,
the DeliveryStatus projection tracks the status of an order and stores
the delivery code required for the deliverer.

When a projection object is created, it is registered with a manager
(to be described in a future post) and remains in memory until it is
unloaded (a feature still in development).

Projections don't need to use all events and events don't need to be
used by projections. On this example, I'm defining some Payment events
that are not being used by the projection. That's completely ok. if
in the future, a new projection is created, it will consume those events
(if defined to) and generate new objects using those historical data.

use Sourcing;
use OrderCreated;
use OrderAccepted;
use DeliveryStarted;
use DeliveryCompleted;
use OrderDelivered;

unit projection Sourcing::FoodDelivery::DeliveryStatus;

has Str      $.order-id    is aggregation-id;
has Str      $.status       = "";
has DateTime $.last-status .= now;
has Str      $.delivery-code;

method summary is query{ :sync } {
    %(
        :$!order-id,
        :$!status,
        :$!last-status,
        :$!delivery-code,
    )
}

multi method apply(OrderCreated $_) {
    $!status        = "placed";
    $!last-status   = .timestamp;
    $!delivery-code = .delivery-code;
}

multi method apply(OrderAccepted $_) {
    $!status        = "preparing";
    $!last-status   = .timestamp;
}

multi method apply(DeliveryStarted $_) {
    $!status      = "collected";
    $!last-status = .timestamp;
}

multi method apply(DeliveryCompleted $_) {
    $!status      = "delivered";
    $!last-status = .timestamp;
}

multi method apply(OrderDelivered $_) {
    $!status      = "done";
    $!last-status = .timestamp;
}

The apply Method

Every projection needs an apply method candidate for each
event it intends to handle. When an event is emitted,
the projection automatically calls the corresponding apply
method to update its state based on the event’s data.

The is query Trait

Methods marked with the is query trait are exposed to clients.
When marked with the :sync parameter, these methods first process
all pending events, ensuring that the projection’s state is
up-to-date before returning the result.

Usage Example

Note: Although events should ideally be emitted via dedicated commands,
for this example we emit them directly.

...

my Sourcing::Client $s          = Sourcing::Client.new;
$s.register-class: "DeliveryStatus";
my RedEventStore::Client $store = RedEventStore::Client.new, 
my \DeliveryStatus              = $s.get-class-client("DeliveryStatus");

sub emit-event(Sourcing::Event $event, :$order) {
    say "{ gray $event.timestamp.hh-mm-ss }: emitting the event: { yellow $event.^shortname }";

    $store.add-event: $event;
    given $order.summary {
        say "Current state: STATUS: { green . }; CODE: { green . }"
    }
    sleep 1
}

sub MAIN(Str :$order-id = uuid-v4) {
    say "Using order-id: { red $order-id }";

    my $order           = DeliveryStatus.new: :$order-id;
    my $user-id         = ^10 .pick;
    my $restaurant-id   = ^10 .pick;
    my $prepare-minutes = 10;
    my $deliverer-id    = ^10 .pick;

    emit-event :$order, OrderCreated.new:      :$order-id, :$user-id, :$restaurant-id;
    emit-event :$order, PaymentInitiated.new:  :$order-id, :payment-data;
    emit-event :$order, PaymentConfirmed.new:  :$order-id;
    emit-event :$order, OrderAccepted.new:     :$order-id, :$restaurant-id, :$prepare-minutes;
    emit-event :$order, DeliveryStarted.new:   :$order-id, :$deliverer-id;
    emit-event :$order, DeliveryCompleted.new: :$order-id, :delivery-code("10");
    emit-event :$order, OrderDelivered.new:    :$order-id;
}

Running this, it will print out something like this:

Projection: UserOrders

Another simple projection we could use here would be a projection
to show all orders of a user. It would be simply consume a single
type of event.
Different from the first projection that aggregates the events
using the order-id, the UserOrders uses the user-id as aggregation-id
to group all event "from" the same user together.
(I plan to, at some point, make it possible to map event attributes
with different named to a aggregation-id on projections).

use Sourcing;
use OrderCreated;

unit projection Sourcing::FoodDelivery::UserOrders;

has Str $.user-id is aggregation-id;
has Str @.orders;

method users-orders is query {
    @!orders
}

multi method apply(OrderCreated $_) {
    @!orders.push: .order-id
}

Conclusion

In this post, we explored the fundamentals of event sourcing
and how to implement a basic framework in Raku using the Sourcing library.
We covered the core concepts of events, the event store, and projections,
and demonstrated these ideas with a practical Food Delivery example.
Event sourcing provides a robust foundation for building systems that are scalable,
auditable, and easier to debug. In future posts, we’ll delve deeper into advanced
topics such as command handling, event replay, and integrating with
production-grade data stores. Stay tuned for more insights into building
reliable and resilient systems with event sourcing!