Building An Event-Driven Orchestration Engine

Let’s consider a typical “flight+hotel booking” process with the following steps:

  • Process payment

  • Book flight

  • Book hotel

  • Notify customer

Each of those steps is processed by a different service and may (and will) fail for technical (network issues, system failures…) or business (fraud detection, inventory not available anymore…) reasons. As often, the main difficulties in an actual implementation would come from those issues and the needs to respond gracefully to them (retry, reimbursement, notifications, booking cancellations..).

Centralized Orchestration

A first implementation that comes to mind is to have a coordinator that calls each system and ensure that each step is orchestrated according to business needs. For example a simple controller sequentially requesting each service through HTTP calls can do the trick. But It’s a brittle implementation as any issue into only one service will propagate to the entire process.

Also when you start having a lot of services and processes, it becomes an issue to keep track of who is using a particular service, making it difficult to update. Also each orchestrator needs to know each service to be able to connect directly (which servers? Which APIs?).

Event-Driven Choreography

The current practice is to implement an event-driven architecture. Each service is connected to a message bus (hello Kafka!), subscribes to some messages and publishes others. For example, the payment service will subscribe to the “CheckoutTravelCart” command message and produce a new “PaymentProcessed” event. This latter will be catch by the flight booking service to trigger a “FlightBooked” event, catch by the hotel booking service to trigger the “HotelBooked” event, and so on.

This approach is seen as more decoupled as you do not have synchronous interactions between services. If a particular service is down, the process will pause and will resume as soon as the broken service is fixed. Nevertheless, this architecture is far from being a silver bullet as the definition of your business processes is actually distributed through events subscriptions and publishing, making it hard to update and really hard to have a clear understanding of the business situation without adding a dedicated service that will actually record and monitor each event.

Event-Driven Orchestration

Actually, you can extend the event-driven architecture by adding an orchestrator service. This service will be in charge of triggering commands based on the history of past events. This orchestrator maintains a durable state for each business process, making it easy both to implement sophisticated processes and to have a source of truth for each process state. Moreover, each service is even more decoupled as it does not need anymore to be aware of other service’s events. They can behave as simple asynchronous services receiving their own commands and publishing their own events only.

The main downside of this architecture is that it’s actually a lot of work to implement — you need: