Micro Services - synchronize data

 Data Consistency Across Microservices

In a distributed microservices architecture, ensuring data consistency is crucial. When a failure occurs in one step, other transactions must either be canceled or rolled back to maintain consistency. 

 

Traditional Approach - Traditionally, all updates are made as a single transaction, ensuring that either all operations succeed or fail together. This approach is feasible in monolithic architectures where a single database can be used to update all records simultaneously. However, this is not possible with microservices, which typically use distributed databases, each maintaining its own data. 

 

ACID Properties

Transactions in microservices should adhere to ACID properties: 

Atomicity: All or nothing when committing data changes.

Consistency: Data transitions from one valid state to another.

Isolation: Transactions run in isolation and are not affected by concurrency.

Durability: Committed data changes are durable and persist in the database.

 

2-Phase Commit Pattern

The 2-Phase Commit pattern focuses on data consistency rather than availability. It introduces a transaction manager (TxnMgr) to ensure data consistency. The process involves: 

Event Publication: The UI/BFF publishes an event to the bus.

Transaction Manager: TxnMgr subscribes to the event and creates a record in its database, detailing the components involved in the transaction.

Prepare Event: TxnMgr publishes a "prepare" event, which other microservices subscribe to and prepare for the task.

Vote Collection: Microservices publish their vote (success/fail) after preparation. TxnMgr records these votes.

Commit/Rollback: If all votes are successful, TxnMgr publishes a "commit" event, and microservices complete the task. If any service fails, TxnMgr publishes a "rollback" event, and microservices roll back their preparation steps.

Pros: 

    Guaranteed atomicity across services.

    ACID compliant.

Cons: 

    Performance bottleneck due to waiting for TxnMgr responses.

    Single point of failure with TxnMgr.

    Complexity in handling retries and failures.

 

Saga Pattern

The Saga pattern focuses on atomicity and is more popular than the 2-Phase Commit pattern. It uses a Saga Execution Coordinator (SEC) and saga logs to manage transactions. The process involves: 

Saga Entries: Incoming transactions are converted into saga entries, each representing a service task with a corresponding compensation request (rollback request).

Status Recording: SEC records the status of each microservice. If all responses are complete, the transaction is complete.

Communication: Requests can be sent to individual services together or one-by-one using HTTP or events.

 

Pros: 

    Ensures data integrity.

    Fault tolerance with compensation requests.

    Scalable and loosely coupled.

Cons: 

    Implementation complexity.

    Performance overhead due to asynchronous communication and compensating actions.

Eventual Consistency Pattern

The Eventual Consistency pattern prioritizes availability over strict ACID properties. Asynchronous messages are published, and microservices process them independently. Data may be temporarily out of sync, but it will eventually become consistent.  

Pros: 

    High availability.

    Simple implementation.

Cons: 

    Temporary data inconsistency.

    Requires careful handling to ensure eventual consistency.