Microservice architecture has already become a widespread approach to IT infrastructure planning. Therefore, we decided to use it when building a custom document generation solution. In this article, you will learn how we solved the problem of microservice orchestration and why MassTransit makes a really good choice in this case.
If you decide to apply the microservice approach in your IT project, you need to choose an appropriate communication method between the microservices. If the microservices in the app need to work independently from one another, the best option to choose is a queue. Here, there are two possible approaches.
Microservice choreography
Even using a queue, you can make microservices dependent upon one another. After completing a task, a microservice has to send a message on what is to be done next, e.g., send an event to the queue of another microservice. If the first microservice does not know about the existence of the second one, at least it should know what event type it should publish. So, each microservice generates events informing the next one what happens next.
This approach, where a process is managed in a distributed way by many microservices, is called “choreography.” If you want to change the process, e.g., add a new microservice, you need to modify the existing ones. Therefore, choreography will be a good choice in the case of projects that have a limited number of microservices or where tasks are not executed by microservices in a fixed order.
Microservice orchestration
However, in projects where the tasks performed by microservices are connected in a logical sequence or there are numerous microservices, you should opt for orchestration. In this approach, microservices generate events stating that they have finished a task, but do not decide what happens next.
Instead, we create a separate microservice that is a “state machine.” The remaining microservices only deal with their own tasks. They are consumers of events (commands) raised by the state machine, which orders them to perform specific actions.
Once such an action is completed, a microservice sends a message that the action has been done, instead of sending a command of what to do next. The message about task completion from the microservice is consumed by the state machine, which decides what the next action to be taken is.
MassTransit as an abstraction layer and microservice orchestrator
In our project of building a custom document generation solution with an architecture based on microservices, we chose the microservices orchestration approach. To put this in place, we decided to use MassTransit, an open-source library for .NET. It works as an abstraction layer for queues and supports solutions available on the market, like RabbitMQ and Azure Service Bus.
Thanks to this support, you can choose a tool for the queue at a later stage of the project. You can also change it easily, as this change does not require many changes in the code.
Apart from an abstraction layer for a queue, MassTransit has a built-in state machine called Automatonymous. This is a library where process flow, correlation, events, and request handling are defined by C# syntax. It also allows for the persistence of a state. As in the case of the queue, here too you can choose any data container you want, such as Redis or Azure Cosmos DB. Alternatively, you can use Entity Framework as an abstraction layer, allowing you to integrate with any database for which there is a provider for this library.
Using MassTransit in a microservice-based application—a use case
Let’s analyze a practical case where MassTransit can be used: a system to generate documents based on microservice architecture. Documents are created based on provided data and then archived. After archiving a document, we need to inform users what ID it has, so that they can easily find it if such a need arises in the future.
Let’s assume that we have already prepared a web app allowing users to fill in the data necessary to generate a document, a microservice for document generation, and a microservice to archive documents with a database connected.
We want to build a microservice with a state machine that will orchestrate the entire process. The document creation process will be as follows:
- A web app sends an event (DocumentSubmittedEvent) informing a user that data needed to generate a document has been filled in.
- The web app should not send an event-command to generate the document. This should be handled by an orchestrator. Thanks to this approach, before a document is generated, we will be able perform other actions on the data, if such a need arises. These actions could be: sending an email notification, sending data for review, etc.
- A state machine receives the event and based on it defines the next event: an event-command to generate a document (GenerateDocumentEvent). It also changes its state to “DocumentGenerating.”
- A document generation microservice receives the document generation event, performs the requested action (i.e., generates the document), and sends an event that the document has been generated (DocumentGeneratedEvent).
- The state machine receives the event and based on it determines the next step: an event-command to archive the document. (ArchiveDocumentEvent). It also sets itself to the state DocumentArchiving.
- After receiving the event-command, an archiving microservice archives the document and sends the event DocumentGeneratedEvent.
- The state machine that is in the DocumentArchiving state receives DocumentArchivedEvent and finishes the process.
Fig. 1 The document generation process—overview
Configure a MassTransit state machine for microservice orchestration
In order to implement the above process, you need to do the following.
Define the state
The state class needs to inherit from the class SagaStateMachineInstance, have a correlative identifier, a field to store a current state, and, if we plan to use Redis for persistence, a property with the version.
Apart from these properties, we can define any data set that will be stored with the state all the time. This data set will be also available every time we get the event that is received by the process.
public class DocGenState : SagaStateMachineInstance, ISagaVersion
{
public Guid CorrelationId { get; set; }
public string CurrentState { get; set; }
public int Version { get; set; }
}
Define a state machine process
A state machine definition is a class that inherits from MassTransitStateMachine. It needs to have properties defining states in which the machine can be set, and events to which it will respond or which it will publish.
In a constructor, we define correlative identifiers of all events. In our case, this will be always the same value: the property ID of the message.
Next, we need to define an initial event that will result in the creation of a new process instance. The “TransitionTo” method allows you to define the state to which the process should be set after receiving the event. The “Publish” method allows you to publish a new event in a queue. In our case, this will be a document generation request.
The subsequent calls of the “During” method define the events to which the machine should respond in a given state. It also defines states to which it should transit and which event it should publish.
The last call of the “During” method defines the transition to the final state called by the Finalize() method.
public class DocGenStateMachine : MassTransitStateMachine<DocGenState>
{
public State DocumentGenerating { get; private set; }
public State DocumentArchiving { get; private set; }
public Event<DocumentSubmittedEvent> DocumentSubmittedEvent { get; private set; }
public Event<DocumentGeneratedEvent> DocumentGeneratedEvent { get; private set; }
public Event<DocumentArchivedEvent> DocumentArchivedEvent { get; private set; }
public DocGenStateMachine()
{
Event(() => DocumentSubmittedEvent, x => x.CorrelateById(x => x.Message.Id));
Event(() => DocumentGeneratedEvent, x => x.CorrelateById(x => x.Message.Id));
Event(() => DocumentArchivedEvent, x => x.CorrelateById(x => x.Message.Id));
InstanceState(x => x.CurrentState);
Initially(
When(DocumentSubmittedEvent)
.TransitionTo(DocumentGenerating)
.Publish(context => new GenerateDocumentEvent(context.Message.Id,
context.Message.Data))
);
During(DocumentGenerating,
When(DocumentGeneratedEvent)
.TransitionTo(DocumentSigning)
.Publish(context => new ArchiveDocumentEvent (context.Message.Id, context.Message.Document))
);
During(DocumentArchiving,
When(DocumentArchivedEvent)
.Finalize()
);
}
}
Registering a state machine on HostBuilder
In order to run the state machine in the background of the service, you just need to call the AddSagaStateMachine method in the configuration manager of HostBuilder, indicating in the generic parameters the definitions of the process and the state. Additionally, you can immediately define the persistence method. In our case, we chose Redis.
There is also the option to use InMemory persistence. This means that after restarting a microservice with the state machine, you will lose all the data and all the ongoing processes will need to be restarted. You can choose this option if the current process state for a given case is not critical and you can lose it and restart the process. Still, such a solution is mainly suitable for test environments.
config.AddSagaStateMachine<DocGenStateMachine, DocGenState>()
.RedisRepository(r =>
{
r.DatabaseConfiguration(configurationString);
r.KeyPrefix = "saga";
});
Wrap-up
Using MassTransit as an abstraction layer to communicate with a queue and to implement an orchestrator in the document generation solution based on microservice architecture proved to be a good choice. It allowed us to easily build a state machine that centrally manages the communication between microservices, informs us about the workflow, and makes microservices truly independent from each other.
Of course, you can do much more with MassTransit and a state machine than was described in this article. You can find more details in the official MassTransit documentation.
