Using sagas to maintain data consistency in a microservice architecture by Chris Richardson

Thanks for sharing

This does mot work where ot maaters. It will eventially work where nobody cares.

Really enjoyed the talk, but imagine if they these talks used a richer language like F#, Rust, Scala, etc that have enums and traits instead of all the class based fluff ❤

surprisingly interesting talk and could really relate. great learning reference

Great talk! One question though: If you use a DB table to buffer the messages to be sent to the message broker, haven't you just moved the problem of exactly-once-delivery? What if the mechanism that dispatches the messages from your table to the broker crashes/rolls back?

how can I get that PPT ??

It's been really informative demonstration about how the Saga works entirely, when it comes to publish a message to the message broker (queue) in order to make them transactional , we can design them to get ACK back then consider it as done thus give it a go

for peuple watching in 2023. don’t use saga!
use a database that support multi/shard transaction!

how awesome would it be to get an architectural/code review from Chris Richardson?

Excellent talk. Learned a lot. Thanks

Why not use TC/C ?

I'm using gateway api as a middleman for my microservices

Very interested presentation thnx,
i want to give my idea concerning the saga code source demo, the speaker send a command reserveCredit. i think it's will be better to publish the orderCreated Event. sending Commands to a remote services create dependancies between each others instead of throwing events and let's interested subscribers consume event and translate them to a commands ( onOrderCreated --> translate OrderCreated to ReserveCreditCommand)

excellent, I have looking for something like this for days. really thanks to Chris Richardson

For the last part, "Transactional messaging", why can't we utilize out of the box solutions like Kafka instead of using database commit log tailing?

why are we trying to do this ? to make systems less complex , to have a smaller focused development teams ? to reduce time to production ? I feel we are making systems more complex , we are adding many more smaller databases , we are re-inventing the transaction mechanism such a waste of time , please do a video where you really need to have microservices and where not, its become a fashion to bounce the terms microservices , REST, K8S

This is dope 😍

Choreography looks much better to me and more loosely coupled

Chris the way you are referring to implicit orchestration through events, I am confused then what is choreography?

Hi, i understand that 2PC is out of the scope between two microservices, but inside a single microservices you can use it. For example when you have to write a record on DB and send a message to a broker to start another micorservice.

MassTransit to the rescue 2021

The problem here is that in today's world of commerce, most times the credit service and payment service(s) are totally and separate from the order service and customer service and not controlled by the organization domain. So this kind of orchestration would not be required. Typically what happens is the front end layer calls the third party service to handle the customer credit and/or payment and once that is done there is data sent back to the front end. That data is then used as input to the order service and there is absolutely no direct dependency between the two at all. Even if there is a payment service within the domain it still would operate independently of the order service with the expectation that the order would not be submitted until the payment is successful.

A better example of this kind of pattern is a modern web based dashboard that a user logs into where they have various kinds of sub applications that they can drill down into and do various things. All of those sup applications would be the prototypical microservices that are independent and have their own separate stacks for whatever functionality that they perform. And the UI layer is doing most of the orchestration in terms of authentication and messaging between them.

Great talk. Really useful

the money transfer case, I think this kind of function should be designed as in one microservice and done inside a single DB transaction. If there is no option to do so, then probably an intermediate cash pool (fake, intermediate account) is needed

Had to set it to 1.5 speed make it more consumable

pro tips 1.25x sounds normal.

I readed alot about Saga design pattern and I developed two years a system than implement Saga design pattern and I still learned new things from this video. Thank you.

Microservices isn't an architecture. Decoupling is. Microservices is nothing new. Decoupling has been around forever.
(This is nothing related to the video, just felt like saying it.).

If there's an eventual consistency (achieved using the messaging system like queue or pub-sub), what's the need for a saga (after all transactions are for consistency)?

10mins and still didn't found saga.

Very interesting talk!

One question I had near the end of the talk — it seems like using the DB’s change log as the mechanism to publish to the message broken has a large drawback: it tightly couples the internal DB scheme to the API a service exposes to the Saga (which defeats one purpose of microservices, since you want to avoid leaking your DB).

I didn’t quite follow why having each individual service coordinate a 2PC between its own DB and a message broker was problematic. This seems much less bad than using distributed TXNs to implement the entire Saga, since the 2PC is an implementation detail of each service. The only single point of failure is the message broker, and I think this is true regardless of your approach?

Unfortunately the approach presented here is not perfect when comes to scalability: Order.state == PENDING is basically a database write lock in disguise.

great talk, got a lot of useful information

Excellent, it took last 15mints to understand 1hour repeat and see. Its 4 years before, now i want to see what are latest changes or implications update in saga, any suggestions?? Please replay to comment

These are very old and good known problems of any distributed system. Main point is when you go from monolith to microservoces you go from single instance to distributed with all of its problems and complications.

You could have a Saga Service that handles all sagas?

I stopped watching when he says use database as a temporary message queue. WTF

Thanks buddy :D

It's better to understand the current

Thank you very much

This is a real talk on real use cases on real problems

So much repetition of the phrase "microservice architecture". Just say microservices! XD

Great talk, like that you explained very important part as processing data and sending event atomically. Other resources usually do not explain this important part. But I want to mention one moment, as usually messaging systems guarantee “at least once” delivery, we have to think about idempotency. And if we implement it, we don’t need complicated atomic save and send mechanism anymore, we now can save data and then try to send message to the next queue and ack/commit current queue. For example If we have server failure after saving to DB and before sending to next queue and acking current, then the other instance will get and process current message (here idempotency come into play as we already saved data to DB and another instance will do the same processing cycle with the message - save to db and send to next queue)

Old wine in new bottle. I have been doing this design pattern of State Machine Orchestrator in my SOA EDA , with compensating transactions to rollback what services had done till the point of failure. The failure event is either broadcast to all the services or to the Orchestrator so it can co-ordinate the cancellation. As I listened to the talk it was like going through my design document.
Message table to message broker has a drawback when the load is high. As it is DB spefic to pull data out to push on message broker it introduces a bottleneck in the DB. Ideal for this is using a distrbuted cache DB or like he mentioned the replication and use the replicated data to message broker.

Excellent talk. It touched all the touchpoints

Newbies to Micorservices without much experience may think this is something rocketscience, but it as been there for long in a different avatar.