Skip to main content

Red Hat JBoss Fuse - Load Balance Quartz Cron Jobs

In the beginning of the year I posted a post on how to do cron jobs with Apache Camel in JBoss Fuse, the reason I am doing it is because sometimes we need to off-load the integration jobs and schedule them to run in less time during busy hours. Cron Jobs allows you to run the integration application at the certain period of time you set it to run, it will be mostly in the night time. We use the Quartz2 component in camel to achieve this.

Just to recap a little on the scenario I had. Basically it starts up two instance that runs the same cron jobs, the job simply writes a file to an FTP server. The job is implemented by Quartz2 and setting it to run twice every minute,both instances are connected to a database (In my code I used H2 for easy installation, but there is another option to use PostgreSQL database, see my previous blog). The reason we want to connect them to database is to make sure these two instance will share the loading of the the job. This is very useful if you have a long running job, and can divide them into different parts and share the loading among distributed resources.
Don't forget to setup the database beforehand, setting the database table schema, download the Quartz library, you will find the scripts to setup tables are under docs/dbTables, find the sql file of the database you are using. Run the sql file to create all the tables needed in your database.

And let's go back to the camel project itself. The setting of the database are in the properties.

<bean id="quartz2" class="org.apache.camel.component.quartz2.QuartzComponent">
    <property name="propertiesFile" value="org/blogdemo/demojob/"/>


Since we are going to deploy this with fabric, we will place the properties file under main/fabric

Inside contains detail about the shared database such as driver, ID , password and also the setup detail like, number of threads, driverDelegateClass used and most importantly if this setting is clustered.

# Main Quartz configuration
org.quartz.scheduler.skipUpdateCheck = true
org.quartz.scheduler.instanceName = DatabaseClusteredScheduler
org.quartz.scheduler.instanceId = AUTO
org.quartz.scheduler.jobFactory.class = org.quartz.simpl.SimpleJobFactory
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.dataSource = quartzDataSource
org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.isClustered = true
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 10
org.quartz.jobStore.clusterCheckinInterval = 20000

# JobStore: JDBC jobStoreTX
org.quartz.dataSource.quartzDataSource.driver = org.h2.Driver
org.quartz.dataSource.quartzDataSource.URL = jdbc:h2:file:~/h2/cronjob;AUTO_SERVER=TRUE
org.quartz.dataSource.quartzDataSource.user = sa
org.quartz.dataSource.quartzDataSource.password = 

org.quartz.dataSource.quartzDataSource.maxConnections = 5

The camel route is very simple and easy, 

<route id="quartzRoute">
        <from uri="quartz2://myGroup/myTimer?cron=0/45+*+*+*+*+?"/>
        <setHeader headerName="CamelFileName">
            <simple>MyJob-${date:now:yyyyMMdd HH:mm:ss}.txt</simple>
            <simple>${date:now:yyyyMMdd HH:mm:ss} Hello This is the content from Quartz2 - {{instancename}}</simple>
        <to uri="sftp://demo@localhost?password=ZAQ!2wsx"/>
Basically, it is kicked of by the cron job, on every 0 and 45 second, and then creates a file under name MyJob-YYYYMMDDHHMMSS.txt with content Hello This is the content from Quartz2 in the file. 

And to deploy the application to JBoss Fuse, I need to create a fragment bundle for connection pooling, here we use c3p0 and the JDBC driver. And then deploy both onto the same instance. 

Deploy another container with the same job profile and fragment bundle. 

Add another profile,  that prints the content received in FTP server. 

If you encounter any problem when deploying, it's probably OSGi not loading the bundles correctly, you can simply restart the container or make sure the following bundles are activated in the container. (Check by using osgi:list
[ 157] [Active     ] [            ] [       ] [   80] Apache ServiceMix :: Bundles :: c3p0 (, Fragments: 148
[ 167] [Active     ] [            ] [       ] [   80] H2 Database Engine (1.4.181)
[ 172] [Active     ] [Created     ] [       ] [   80] Quartz Two Demo (1.0.0.SNAPSHOT)

Take a look at the log running quartzprint profile, you will find the jobs are clustered and loads are shared between two nodes. 

And detail of the job can also be found in console of each container.

You can find the Demo code and detail instructions here


Pravin Deshmukh said…
This comment has been removed by the author.

Popular posts from this blog

JBoss EAP 6 - 效能調校 (一) DataSource 的 Connection Pool

效能沒有什麼Best Practice, 反正能調整的就那些。 通常,一個程式的效能大概有70-80% 都跟程式怎麼寫的其實比較有關係。

最近我最疼愛的小貓Puji 因為膀胱結石開刀的時候過世了,心情很差請原諒我的口氣沒有很好,也沒有心情寫部落格。

Puji R.I.P.



JBoss 的 SubsystemDatasource WebWeb Service EJB Hibernate JMSJCAJVM 調校OS (作業系統)

先來看一下 DataSource Subsystem, DataSource 的部分主要是針對Connection Pool 做調校。

通常,程式都會需要跟資料庫界接,電腦在本機,尤其是在記憶體的運算很快,但是一旦要外部的資源連接,就是會非常的耗資源。所以現在的應用程式伺服器都會有個Pool 放一些先連接好的 資料庫connection,當程式有需要的時候就可以馬上提供,而不用花那些多餘的資源去連接資料庫。

這就是為什麼要針對Connection Pool 去做調校。

以下會討論到的參數,都是跟效能比較有關係,Datasource 還有很多參數,像是檢核connection 是否正確的,我都不會提到。如果你追求的是非常快速的效能,那我建議你一個檢核都不要加。當然,這樣就會為伺服器上面執行的程式帶來風險。這就是你要在效能與正確,安全性上面的取捨了。 (套句我朋友說的話,不可能又要馬兒好,又要馬兒不吃草的..)

最重要的調校參數就是 Connection 的 Pool 數量。(也就是那個Pool 裡面要放幾條的connection.) 這個參數是每一個應用程式都不一樣的。


Connection Pool 最少會存留的connection 數量


Connection Pool 最多可以開啓的 connection 數量


事先將connection pool 裡面建立好min-pool-size 的connection.

我的建議是觀察一下平常程式要用到的量設定為 min-pool-size 。

Red Hat JBoss Fuse/A-MQ - Fuse and A-MQ Version 6.3 GA is released!

Fuse and A-MQ 6.3 GA has just went out. Maybe, you would think this is just only a minor version release why should I care? Hold your thoughts on that! Because they have done a lot of improvements and also added many new features into this release.

Besides various bug fixes and making sure Fuse Fabric is much more stable. There are two major change in this version update:

New Tooling in JBoss Developer Studio (JBDS) 9.1 GA. Newer Apache Camel version – Camel v2.17. I was really impressed by the work put in to make developing Camel application much simpler. First is the installation of tooling itself. Now it has a all-in-one installer so you don't need to worry about which plugins you need to check. See the videos below to see the new "Getting Started" of Fuse 6.3.

And If you notice from the above video, the presentation of camel route in JBDS has also updated. It fixed some of the miss representation of logic and making it easier to read.

Old Camel Route
New Camel Route
On …

Red Hat JBoss Fuse - Getting Started with Fuse Integration Service 2.0 Tech preview

I just realized that I did not do a getting started for Fuse Integration Service 2.0 Tech preview before I did the pipeline demo, thanks for those of you who reminded me! :)

To get started with FIS 2.0, for people who has just getting to know the technology, here is how I interpret it. Basically, it's divide into two aspect,

1. Integration development, FIS uses Apache Camel as the core technology that creates, orchestrate, compose microservices into a super lightweight thin integration layer, and become the API provider and service orchestrator through exposing RESTful or messaging service endpoints. And you can choose to either package and run it with Spring-Boot or Karaf.

2. Application Deployment and Management, FIS takes advantages of OpenShift platform, and allows you to separately deploy the micro-integration service among distributed environment, at the same time takes care of the failover, high availability, load balancing and service lookup problem for you.

So, now we know …