A Practical Introduction to Integration Patterns

3K

A practical introduction always requires a very practical little game to start with. Everybody knows Facebook, the biggest social network platform ever built. Billion of users, million of pages views everyday. But in the end, what does Facebook ‘look like’?

As if you were to open the hood of your car, how would you imagine Facebook ‘behind the scenes’?

I) Monolithical vs Microservices

When I make a request to Facebook, say to see the crazy photos my friends took during their vacation, first it would go through a ‘central’ server (taking care of the rendering for example), be transferred to a service layer, until the database where the photos are stored. The rendering server would craft the HTML for me to see, and I would be able to contemplate their latest photos.

However, this is not how Facebook is working, Facebook is designed to be loosely coupled using a microservices architecture.

As you can see in this schema, all the differents functionalities of Facebook are separated one from another. You have the photo service that only handles requests related to the photos, or the comment service that stores all the comments that users write on posts or images.

Facebook might even want to communicate with third parties applicationssuch as Instagram. For example, you might want to share your latest photos on Instagram directly, or retrieve a funny video you found on Instagram to share on Facebook.

When Facebook is communicating directly with “its services”, it is called a service oriented architecture. However, when Facebook makes a call to a third party API, such as Instagram, we call that a business-to-business architecture..Those two architectures are facing the same challenge : the need of an underlying integration process that reduce the complexity of having systems that need communicate with each other, yet they don’t speak the same language.

Back to the schema:

Compared to the previous schema, we can clearly notice the system heterogeneity. Databases may be built on MySQL or MongoDB, services are not guaranteed to be built on REST, as they could be built on SOAP or even RNI. As frameworks are evolving fast, they even could be GraphQL in the near future. As a consequence, to preserve the scalability and the expansion potential of our infrastructure, we need :

An architectural component that is going to abstract the communication between components that were not built to work together.

II) Entreprise Service Bus

Shortened ESB, the entreprise service bus is an architectural pattern/specification that abstracts the communication between components. This little schema might help you understand this concept a bit better.

On the left side, you have multiple endpoints communicating with each other using either REST APIs, SOAP or GraphQL APIs. As you can see on the left, building the systems and having them working together can be a bit of a challenge. Every single combination would have its own way to transform and pass data along the architecture. This architecture would lead to a lot of redundancy in the systems. Moreover, what happens in one of my Java services go down for ten minutes? Does my React application completely stops? Wouldn’t that make it a single point of failure for my applications?

The architecture on the right is a much cleaner to solve this issue as the entire logic of our data processing flow is centralized in the ESB.

The ESB will take care of implementing transformers (see EIP patterns) that will handle once and for all every single data transformation that our systems might need.

But, as every single piece of information is going through the ESB, doesn’t that make it a single point of failure also?

ESB are heavily coupled with messaging middlewares that, if implemented correctly, will be able to ensure that no data is lost along the way. Entreprise Service Buses are heavily coupled with asynchronous communication, meaning that counterparties don’t have to be active at the same time to be able to communicate, thus ensuring data reliability even in case of networks interruptions.

III) When to use?

Implementing resilient and robust architectural patterns is not a goal in itself, it is a way for you to ensure that data resiliency is maintained along the data flow. As a consequence, entreprise service buses are not meant to be used anytime, anywhere, in any situation.

Do I need it?

In small monolithical applications, that have a very standard workflow, entreprise service buses integration would increase the application complexity and the development time. On the other side, for large applications, that rely heavily on other applications to work, it is a perfect architectural component to use.

Is my system exposed to network failures?

If your system is communicating over the network with other applications or third parties applications, you may want to add an ESB to ensure that no messages are lost along the way. In this case, ESB are very crucial as one message lost can have tremendous impacts on the state of your processes and on the application itself.

Is my application the small part of something bigger?

This criteria applies for people working for very large companies. Often, those large companies divide the work in different services and they obviously cannot force every single developer in every single team to use specific patterns to code.

If we take the example of Google, the Google Maps team might code everything in Go, while the Google AI Team will code all its stack in Python. In this case, as frameworks and systems are inconsistent with each other, an ESB is a perfect block to add to your stack. Coupled with smart routing patterns, you can develop a workflow that will enable complex applications to work with each other without any difficulty.

So.. are you going to add an ESB to your stack tomorrow? If so, leave me a comment below.

Are you passionate about software architecture and software engineering? Make sure to check out my other publications on the subject.

Leave a comment and share it if you liked the article!