Skip to main content

The rise of Agile Integration, Integration is not DEAD nor LAME!

I wanted to blog about this for a very long time, but because of work and being too lazy to turn on my laptop on the weekends, now I finally have a couple of hours to sit down and start blogging.

"Integration is DEAD."
"No, no, let's talk about microservice!!"
"What is that again? Integration, you mean SOA? It's lame... ".

These makes the majority of the percentage when I talk about Integration. In fact, if you happen to be in any the software conference, you see the rooms full when it's about "container" and "microservice." I understand the nature of a developer, always seeking for the latest and greatest technology out there. BUT!! In my opinion, I don't think Integration is going AWAY if you containerize your application. Neither does it will DISAPPEAR in a microservice architecture. In fact, Integration just gets more complicated and tricky when you try to break an application into smaller pieces and have them running all over the place in the distributed environment!

It was hard to deal with when we have to orchestrate, choreograph and compose all the coarse-grained services together, and imaging letting tinier, smaller and more refined services talks to each other. Yeah, trying to get an oversized heavy-weight strong bull to co-ordinate and communicate is one thing. I don't think to have to conduct thousands of small cute duckling, (I know, they are now smaller to handle one by one, light and easy to replace) is going be any easier. :-)

Maybe the word Integration was abusively used by too many vendors (Us too ;-D ),  and it is too often associated with ESB, which IMHO should be one of the Integration strategies. ESB can be overly complicated with a wrapped around RPCs.   Integration is about connecting, composing between services, neglecting integration when implementing microservice architecture (MSA) is going to be disastrous. I am sure a lot of smart people out there designing the MSA has already taken this into account. But they probably did not realize that because Integration is already deeply ingrained into their minds, even in microservice we still want loose coupling. Again, in my opinion, the wall between application development and integration are getting thinner and thinner, by that I mean you do integration in microservice way and apply integration in your microservice. So, yes, integration needs to change, adapt and adjust accordingly but it never goes away:
  • Current deployment method
    • As now services are updated more frequent than ever before, we need to quickly respond to the change, in the integration layer. We can package it into the container and standardize the deployment method across the company. 
  • Speed and automated delivery
    • A quicker respond to developer, continuous integration and delivery, faster time to markets can be achieved by adopting the DevOps trend. Integration should also be part of this cycle. Not OUTSIDE! 
  • Flexible for changes
    • Making sure the message can be route to the correct service, allowing automatic discovery of services, with load balance capability that shares the load no matter it's synchronized or event base call. Providing and compose with flexible contracts like APIs between service can also minimize the impact on the changes. 
  • More robust towards failure
    • In the integration, it's all about the isolation of failure, and handles the errors, increase service resiliency. Also the ability for it to adopt version changes, and service lifecycle management is also one of the important factors in Integration.
Some concepts of doing integration in MSA comparing with monolithic application are poles apart. For example, continue from the bull and duckling theme. When you try to get a couple stubborn, massive bulls together and adjust position takes a lot of energy, and often you will need to be creative and smart to improve according to their position. Therefore the pipelines have to be smart and do many manipulations. Whereas the ducklings are easy to move around and cheap to replace. If one got sick, naughty or does not have the right color you need, simply toss it and get a newly hatched egg. (Alright, I don't mean any animal cruelty here, it's just a metaphor. ). The pipe that brings these microservice can be dumb or simple. Because each duckling is cheap to create. 

In my eye, microservice should never go freely create on wherever and whenever it should be at least logically bound. I call this application domain. By circling these microservice, we will be able to see a much clear picture of inbound microservice integration and outbound microservice integration. 

The inbound microservices integration will be a lot more flexible in protocols, data formats because of the possibility to connect to various datasources and SaaS application. Since they are bounded within the same application domain, it's likely the data models are shared between these microservices. They tend to be more coupled with each other (although service autonomy is where we are trying to get. They are probably handled by the same person.) There will be a couple of microservices responsible for composing some of the other microservices together too. The logic inside should be simple and straight forward. And each of the microservice should easily take care of the protocol and data extract itself. 

On the other hand, the outbound microservices integration should be strictly loosely coupled with inbound microservice. Preferably access commonly managed endpoints from each application domain, yes, I mean APIs. It is mostly device or client bound. As it often required to put together a more coarse-grain service like the old ESB days for the client application or from a process. 
Both inbound and outbound integration can be synchronous and asynchronous, all depends on the need of your system. So don't limit yourself to just one option. I strongly recommend the Enterprise Integration Pattern Book, as I said, the integration problem is still there. The patterns are extremely useful in this manner. 

Agile Integration is Red Hat's opinionated way of doing the integration. And most importantly it's backed up by Fuse (Apache Camel) and OpenShift. 

It talks about 3 main pillars, which resonate my integration believes in MSA world.
  • Distributed Integration 
    • Lightweight, support distributed deployment
    • Pattern Based Integration
    • Reusable Connectors
  • Containers
    • Cloud native solutions
    • Lean artifacts, individually deployable
    • Container based scaling and high availability
  • API
    • Well defined, re-usable, and well managed end-points 
    • Ecosystem leverage


So! I hope my murmur on this blog post will at least makes you think "Integration is NOT LAME". And start thinking about it when next time you create your next microservice.

Comments

Jian Feng said…
Great post!
They are very helpful thoughts.
nick middleton said…
Once at Webroot.com/safe, downloading Webroot antivirus program is easy and quick. Webroot.com/safe
naina sh said…

Learn how to transfer your Norton security to a new computer, Formatted or ... If you do not have an account, click Create account, and then complete the sign-up process. In the Norton Setup window, click Download Norton.

Norton Setup & installation Enter your product key online. Download & Install Norton Antivirus Product Visit setup website www.norton.com/setup to register your code.
aol mail said…
How to reset a aol password. Open your aolmail.com in your web browser. Click Sign in. Click Forgot your password? Enter the email address you used to create the account. You'll need access to this email account to reset your password. Check your email inbox, and click the link in the email you received to reset your password.
www.aolmails.xyz/aol-mail-help
aol mail
aol mail help
pritam mishra said…

We have a three-part technology to defend against chargebacks. Using our technology frees you up to concentrate on your business and keeps up your merchant account so that you can grow your business.
chargeback management



adobe support number – Adobe is an American multinational software company which is well known for its creation of multimedia and creativity software products.
adobe support number

www.mcafee.com/activate - Activate Your McAfee Retail card by simply visiting our website mcafee.com/activate and Get Started with McAfee Security.
www.mcafee.com

Norton Setup said…
Insert the Microsoft Office media disc into the DVD drive. Click "Start" followed by "Computer." Double-click the disc drive if Windows fails to launch setup automatically. Enter your product key when prompted and click "Continue."
www.office.com/setup

Popular posts from this blog

Red Hat Fuse - Announcing Fuse 7 Tech preview 3 release.

Red Hat Fuse 7.0 technical preview three is out today! On the pathway to become one of the best cloud-native integration platform, Fuse gives developer freedom to choose how they want to develop the integration solution, where they want to deploy it and capabilities to address new integration personas that do not have development experience.
By supporting the three major runtime, developer is free to work on the runtime of their choice.By supporting standalone and cloud deployment, it simplifies the complexity to distinguish between these environments, allowing application to deploy freely among the environment of your choice. All levels of developers are welcome, you can either dive deep into creating customize complex integration logic, or using the new low code platform to quickly build a simple integration. In this Tech Preview release you get it all.
Fuse StandaloneSpring-boot for microserviceKaraf 4 for OSGi loverJBoss EAP for JavaEE developersFuse on OpenShiftPlugins for easy co…

JBoss EAP 6 - 效能調校 (一) DataSource 的 Connection Pool

效能沒有什麼Best Practice, 反正能調整的就那些。 通常,一個程式的效能大概有70-80% 都跟程式怎麼寫的其實比較有關係。

最近我最疼愛的小貓Puji 因為膀胱結石開刀的時候過世了,心情很差請原諒我的口氣沒有很好,也沒有心情寫部落格。

Puji R.I.P.

=======================正文=======================

這個題目很多人叫我寫,可是這題目好大,這分明就是整死我咩~
所以我會分幾段慢慢寫。

JBoss 的 SubsystemDatasource WebWeb Service EJB Hibernate JMSJCAJVM 調校OS (作業系統)

先來看一下 DataSource Subsystem, DataSource 的部分主要是針對Connection Pool 做調校。

通常,程式都會需要跟資料庫界接,電腦在本機,尤其是在記憶體的運算很快,但是一旦要外部的資源連接,就是會非常的耗資源。所以現在的應用程式伺服器都會有個Pool 放一些先連接好的 資料庫connection,當程式有需要的時候就可以馬上提供,而不用花那些多餘的資源去連接資料庫。

這就是為什麼要針對Connection Pool 去做調校。

以下會討論到的參數,都是跟效能比較有關係,Datasource 還有很多參數,像是檢核connection 是否正確的,我都不會提到。如果你追求的是非常快速的效能,那我建議你一個檢核都不要加。當然,這樣就會為伺服器上面執行的程式帶來風險。這就是你要在效能與正確,安全性上面的取捨了。 (套句我朋友說的話,不可能又要馬兒好,又要馬兒不吃草的..)

最重要的調校參數就是 Connection 的 Pool 數量。(也就是那個Pool 裡面要放幾條的connection.) 這個參數是每一個應用程式都不一樣的。

min-pool-size 

Connection Pool 最少會存留的connection 數量

max-pool-size 

Connection Pool 最多可以開啓的 connection 數量

prefill

事先將connection pool 裡面建立好min-pool-size 的connection.

我的建議是觀察一下平常程式要用到的量設定為 min-pool-size 。
加上…

JBoss Fuse - Fuse workshop 101 - Part One

On my way to Hong Kong for a day of workshop on JBoss Fuse, and as I go through my Slide deck, I cannot find any decent easy workshop for beginners. Therefore I decide make a workshop that is easy for Camel first timer to get their hands dirty.

The first of part of the workshop is an introduction to Camel, it first goes through what is exactly inside JBoss Fuse.

For part one of the workshop, it takes your through the very basic of Camel, one of the very important component inside JBoss Fuse.
Every Camel need to have a runtime container to run in, inside camel we call it a CAMEL CONTEXT.  Inside every Camel context, you can define lots of camel route and registry, don't worry about what those are, we will explain later.


So inside out blueprint xml, you will see a tag called camelContext.



Next up is camel route, they are a chain of command or process defined by you, as a developer.
Inside the camel route, there are consumer endpoints that listens to the incoming messages, producers …