JBoss Fuse - Connecting to Twitter and the Aggregator pattern

If you think connecting to Facebook is easy with Camel, try connecting to Twitter! It's every easier!
Before connecting to Twitter, make sure you have register a twitter developer account, and go to https://apps.twitter.com/. Create a new Twitter App.

There are a few mandatory parameter for authentication if you want to play with Twitter, they are :

• consumerKey - The consumer key
• consumerSecret - The consumer secret 
• accessToken - The access token
• accessTokenSecret - The access token secret

Also Twitter has Consumer  as it handle mostly polling functionality, such as search from content, streaming tweets from twitter etc.

• timeline/home - read or polls tweet from user home
• search - search specific keywords,
• streaming/filter - get streaming of data from twitter.

Whereas Producer handles tweeting and calling of twitter API, they might also include searching and timeline polling too.

• directmessage - gets the message 
• search - search for keywords
• timeline/user - polling from users

For more detail please visit the product documentation

Note, currently Twitter API is in version V1.1, using the JBoss Fuse 6.1 should work without a problem. since it's base on Twitter4j V3.x

To play with Camel-Twitter connector.
First, we are going to create a twitter app(make sure you have already register an developer account for Twitter), and then go to https://apps.twitter.com/ 

Provide details of the app, then you will come to the application console, click on (modify app permissions) so we can read and publish into our account.

Click on Generate My Access Token and Token secret to get the authentication keys under Your Access Token.

Save both customer and access token and token secret somewhere handy, because we are going to need them later. Make sure you have installed JBDS and the integration plugin.
Go to JBDS, create a new blueprint Fuse project, enter Group ID and artifact ID.

Open pom.xml and add camel-facebook dependency.

Drag the endpoints from palette to canvas,
Endpoint with uri :

twitter://timeline/home?type=polling&delay=10&consumerKey={{consumerKey}}&consumerSecret={{consumerSecret}}&accessToken={{accessToken}}&accessTokenSecret={{accessTokenSecret}}

and log: ${body.source}

Add the properties file containing twitter authentication settings, the 4 must have parameter mentioned above, and fill them in by mapping the tokens you get from creating the Twitter App.

• consumerKey - The consumer key
• consumerSecret - The consumer secret 
• accessToken - The access token
• accessTokenSecret - The access token secret

Map the properties file into Camel Context by using "propertyPlaceholder"

<propertyPlaceholder location="classpath:twitter.properties" id="twitter"/>

So you can start test the first part of the Twitter Camel Example. Related videos can be found here:


Now, going back to our demo. In part 2, we are going to poll tweets every 10 secs from your personal timeline, and analysis the source device of the tweets, and make a summary of how many time each device was use and tweets the result back to Twitter!

From part one, we have already polling tweets from twitter, now to further process the content, add a content switch to the route, that will add 4 different header content to the header bane "devicetype" and call another route for further process.

    
        
        
        
            
                ${body.source} == "Web Client"
                
                    Web
                
            
            
                ${body.source} == "iPhone"
                
                    iPhone
                
            
            
                ${body.source} == "Android"
                
                    Android
                
            
            
                
                    Others
                
            
        
        
    

Notice we are using "seda" to pass the route, seda. The seda: component provides asynchronous SEDA behavior, so that messages are exchanged on a BlockingQueue and consumers are invoked in a separate thread from the producer. We do that because the twitter connector creates one route exchange per returned object, instead of returning a list of tweets, and in our example, the order of process does not matter.

With all the different tweets coming in separately, we now want to summarize the total device, that's when the EIP Aggregator becomes very handy. The Aggregator from the EIP patterns allows you to combine a number of messages together into a single message. In this component, we can implement our own aggregation strategy, and choose different ways when to stop the aggregation, like aggregate every 10 seconds, aggregate every 5 messages, or after certain sizes or even end in some predicted condition.  (For more details, please read the product document). In our demo, we are aggregating every 10 seconds.

So create a java bean that holds the device data (Web, iPhone, Android and Others)

package org.blogdemo.twitterdemo;

public class SourceCounter {
 
 
 private int web = 0;
 private int android = 0;
 private int iphone = 0;
 private int others = 0;

 public void addType(String type){
  if("Web".equals(type))
   web ++;
  if("Android".equals(type))
   android++;
  if("iPhone".equals(type))
   iphone++;
  if("Others".equals(type))
   others++;
 }
 
 public void addWeb() {
  web++;
 }
 
 public void addAndroid() {
  android++;
 }
 
 public void addIphone() {
  iphone++;
 }
 
 public void addOthers() {
  others++;
 }
 
 
 
 
 
 public int getWeb() {
  return web;
 }
 
 public void setWeb(int web) {
  this.web = web;
 }
 public int getAndroid() {
  return android;
 }

 
 public void setAndroid(int android) {
  this.android = android;
 }
 public int getIphone() {
  return iphone;
 }
 public void setIphone(int iphone) {
  this.iphone = iphone;
 }

 
 public int getOthers() {
  return others;
 }

 public void setOthers(int others) {
  this.others = others;
 }
 
 

 
 public String toString() {
  return "web:["+web+"] iphone:["+iphone+"] android:["+android+"] others:["+others+"]";
 }
 
}

And create our own aggregation strategy by adding number to the java bean, base on the "devicetype" header.

package org.blogdemo.twitterdemo;

import org.apache.camel.Exchange;
import org.apache.camel.processor.aggregate.AggregationStrategy;

public class SourceAggrateStrategy implements AggregationStrategy {

 
 
 @Override
 public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
  SourceCounter counter = null;
  
  if (oldExchange == null) {
   counter = new SourceCounter();
        }else{      
         counter = oldExchange.getIn().getBody(SourceCounter.class);
        }
  
  String newType = newExchange.getIn().getHeader("deviceType").toString(); 
  counter.addType(newType);
 
  
  
  newExchange.getIn().setBody(counter);
     //newExchange.getOut().setBody(counter);
     
  return newExchange;
 }
 
 
}

Add the strategy into Camel context in the blueprint.xml,

<bean id="sourceAggrateStrategy" class="org.blogdemo.twitterdemo.SourceAggrateStrategy" />

Setup the aggregator,

And then at the end, we are exporting the result back to Twitter, using the twitter connector again.

    
        
        
            
                true
            
            
                Summarizing the device tweeting every 10 secs from my twitter using Camel, ${body}
            
            
            
        
    

Here is the part two video, it takes you through step by step of building the demo.


You can also find the code and the one click ready in my Github account too!
https://github.com/weimeilin79/twitterdemo

Following the instruction in Github, installing the application onto JBoss Fuse, you can also see the run time detail too!

Thanks!