Life of MI : How an ESB went on to serve Microservices?
A good Integration solution facilitates users and organizations to create an ecosystem which provides a connected experience between various heterogeneous systems. It should ensure that everything is working together. Apart from connecting, it should also add value through different functionalities based on the business logic. It should provide the flexibility to adjust the business logic as the goals change with time. The ecosystem should have the capability to scale up and down and have the capability to be future proof.
“You must take life the way it comes at you and make the best of it.”
― Yann Martel, Life of Pi
What is an ESB?
An Enterprise Service Bus (ESB)¹ is fundamentally an architecture. It is a set of rules and principles for integrating numerous applications together over a bus-like infrastructure. ESB products enable users to build this type of architecture, but vary in the way that they do it and the capabilities that they offer. The core concept of the ESB architecture is that you integrate different applications by putting a communication bus between them and then enable each application to talk to the bus. This decouples systems from each other, allowing them to communicate without dependency on or knowledge of other systems on the bus.
From the above explanation you will get an understanding of what an ESB architecture actually means. There have been many open source ESB products in the past. With time, the concept of an ESB is losing its value and microservice architecture is becoming the norm. Solutions are now developed in a loosely coupled manner with high maintainability. Also services have a requirement to be independently deployed. As a result ESBs have a need to evolve so that they can provide the connectivity, aligning with the microservices architecture.
What is WSO2 Micro Integrator?
WSO2 Micro Integrator (MI) was introduced with the intention of running a lightweight ESB inside containers, so that the ESB concept can be used with the Microservice architecture. There is a need to bridge the gap between the ESB and microservices and WSO2 Micro Integrator is a suitable candidate to do that.
WSO2 Enterprise Integrator 6 is one of the famous open source Integration platforms that contains a ESB, Message broker and an analytics profile within itself. It is well suited for running in a centralized environment and the ESB that is inside the Enterprise Integrator is well suited to fit in a monolithic architecture. (You deploy it on a server and all the services will reside inside it.) On the other hand, Micro Integrator is built on top of the ESB product and provides the cloud native capabilities along with the ESB. It is the main Integration runtime of WSO2 Enterprise Integrator 7 series.
The lower startup time, the reduced product size and single configuration file helps the product to be run on the container based environments. Services now can be decoupled, and they can be run on independent Micro Integrator instances. Starting and stopping of instance happens instantaneously which helps administrators to swap newer versions of the services with a smaller downtime. You can stop and start the services that require maintenance, and the other services can run on peacefully in other Micro Integrators instances without any interruption. It also has first class support for RabbitMQ in case you want to scale up your deployment with the help of a Message Broker.
The product is totally configuration driven, and integration logic is written purely in XML based configuration files. This facilitates users who do not have any programming knowledge to write Integration logic. There is also a dedicated open source Integration studio which helps users to write the Integration logic with the help of Graphical representations. Developers can easily write logic to connect with different applications with simple drag and drop. (I will talk about it more as you read along…)
Evolution of Micro Integrator
As you can see above, Micro Integrator was introduced as a flavour of Enterprise Integrator which was targeting to run on containers. Initially it was shipped with Enterprise Integrator as a profile. With time and demand, it was introduced as a separate product. It also got embedded to the Integration studio so that developers can try out their solutions inside the Studio itself. Micro Integrator then became the Integration runtime of Enterprise Integrator 7.0. Micro Integrator 1.2.0 (It is more suitable to call it as the Integration runtime now) is an improved version which can run on both Container based and centralized environments.
“Life will defend itself no matter how small it is.”
― Yann Martel, Life of Pi
Should you move your deployment to Microservices architecture?
microservices are a magic pixie dust
Many enterprise applications are large, complex monoliths that are developed by large teams that struggle to keep up with the needs of the business. Consequently, adopting the microservice architecture is an appealing option. As you might expect, migrating to microservices requires an enterprise to tackle numerous technology related challenges. But enterprises also encounter obstacles that have less to do with technology and more to do with strategy, process, and organization.
There are many arguments for and against microservices. It does not suit everyone. These slides from Chris Richardson would help you to understand the challenges you might face when moving from monolith to microservices. Some of them, I have highlighted below.
- Development teams might not have any coordination among themselves.
- Moving to Microservices without basic Software development techniques is troublesome.
- Some services are inseparable which might lead to service duplication.
- Inadequate support from the High level management of the organization.
Based on the organization’s goals, it might not be appropriate for everyone to make such a move. Hence we cannot eliminate the fact that the Integration is still mandatory for monolith/centralized architectures.
With WSO2 Micro Integrator, the users are not limited towards microservices architecture. If they are comfortable with monolith architecture, they can continue to do so. With Micro Integrator 1.2.0 (Enterprise Integrator 7.1.0), all the capabilities that are needed for a centralized deployment are bought back.
- Coordination among services for High availability
- Hot deployment of services without server restart
- Improved user store for managing users
When Micro Integrator was originally introduced, the goal was that the server will be immutable after server startup since adding services while the container is running does not make sense. But from 1.2.0, it is not mandatory for the server to be immutable.
How the developer experience has changed from Enterprise Integrator 6 to 7?
- As I explained above, Micro Integrator is the main Integration runtime that comes under the Enterprise Integrator 7 umbrella. The major difference anyone would notice between the ESB profile of EI 6 and MI is that the management console has been completely removed. The reason for this is that previously, the management console was tightly coupled with the Integration runtime, and you need individual consoles to monitor and manage each runtime. With Micro Integrator, it is not the case. There is an independent Monitoring dashboard and CLI tool which you can use to monitor multiple MI instances. The dashboard and the CLI was written from the scratch and the experience is totally different compared with EI 6.
- Apart from the monitoring purposes, in EI 6 you can create Integration solutions in the Management console itself. With Micro Integration, development of Integration solutions is totally moved to Integration studio and the Studio provides first class support for the development, debug, testing and packaging of the artifacts.
- One other significant difference you might feel is handling the server configuration. Earlier there were several xml files that the server administrator should maintain in order to run the server. With Micro Integrator , there is only one configuration file and it is called as
deployment.toml - The SOAP based admin services that were available in the 6xx series have been rewritten with REST based APIs and it is now called the Management APIs.
- Micro Integrator contains a File based registry instead of a Database registry. It helps you to mount volumes to containers and maintain a single registry in a container environment.
- Features like coordination and secret management have been significantly improved to cater the needs of Monolith and Microservice architecture.
- Users are required to think that the Integration runtime is totally decoupled from the monitoring and development environment. This kind of approach suits both Monolithic and Microservices architecture.
Getting started with Micro Integrator
An excellent way to get started with Micro Integrator is the Integration Studio. It has a dedicated Getting started page in which you can try out the Samples and Templates. There is a template guide for each sample as well. It gives clear instructions on how each sample should be run. The studio also has a builtin Micro Integrator instance in which you can test and try out the solution. The studio can be used to design, develop, debug and test the Integration solutions flawlessly. It also has a Docker and Kubernetes Project system which you can use to create the Docker images of MI with the Integration artifacts you create.
With Micro Integrator and Integration Studio, a developer can develop Integration solutions with no programming knowledge at all. Integration Studio is a great example of a Low Code Development Platform. Creating applications in a Graphical manner is what enables everyone in this world to write an application on their own. Multiple vendors are now concentrating on developing developer tools which can create a No code or a Low code development environment. Recently introduced Amazon Honeycode is a classic example of how different vendors are trying out this approach and Low Code development is still a popular choice among users.
As a summary, the Micro Integrator (Integration runtime) of Enterprise Integrator is a solution to bridge the gap between an ESB and microservices. With the help of Integration studio, writing Integration logic has become simple. People can choose to run it in their monolith architecture or they can go on to use it as a means to implement Microservices architecture. It is totally up to the developers. They can choose their own story.
I've told you two stories about Micro Integrator... So which story do you prefer???Resources
