First Scala book in Swedish is out! Congrats guys!

On a personal note I'd like to congratulate my professional acquaintance Örjan Lundberg and his fellows Olle Kullberg and Viktor Klang for finalizing the Scala introductory book "Ett första steg i Scala". It's to my knowledge the first Swedish book to explain the concept of the Scala language.

You can find it at: http://www.studentlitteratur.se/o.o.i.s?id=2474&artnr=33847-01&csid=66&mp=4918

Congratulations guys! Hope I get the chance to read it soon!

Camel goes visual

So it seems that the Camel guys over at FuseSource has finally released the preview of the Camel development IDE. Basically an eclipse plugin that helps you author the spring xml files for your routes visually. I haven't tried it yet, but its an interesting tool which hopefully can help attract integrators into this nice framework.

There's already a short introduction up on their site, so head on to http://fusesource.com/ and have a look.

According to FuseSource they're also working on a runtime web version which you could use to inspect and author your flows directly against an executing ServiceMix/Karaf JVM.

So with Spring Integration having visual IDE support in their latest release of the STS tooling and Camel joining now it will be interesting to see what the Mule guys will bring to the table when it comes to IDE's. A sneak peak last year showed something similar (http://blogs.mulesoft.org/new-mule-ide-coming/) and you can also have a look at Saddle IDE which seems to target Mule developers as well (http://www.saddle-integration.org/).

Seems like all of them are going the XML-authoring way where your IDE generates XML configs in the background.

2011 already looks like a promising year for integration developers!

Eclipse XML autocompletion patch for Mule users

Earlier I wrote about my problems finding a good XML editor which works with Mule configuration files, as the one included in Eclipse WTP does not support XSD schema substitution groups.

That time I simply gave up and registered for a trial of the Oxygen XML eclipse plugin editor since that was the only one that I found that could give me autocompletion while modeling flows.
Well, it seems that the MuleSoft guys has figured out a way to patch this issue with Eclipse, which is very good news for the myriad of Mule users out there. This patch may also be worth downloading even if you're not intending to use Mule and regularly deal with schema substitutions in eclipse.

I have not yet tested the patch since I've been quite busy lately on my free time, but I'm sure it will be a welcome fix for many integrators.

Bye bye Oxygen, been nice to know you!

Soap through JMS support for WCF

One of the classic problems you face when dealing with webservices is how to assure a guaranteed delivery of your messages. Since http by its definition cannot guarantee that your messages are never lost a lot of initiatives has tried to solve this problem.

Currently this is typically handled in one of the following ways:

WS-Transaction
Not primary used as a means of guaranteed delivery, WS-Transaction still in some sense tries to solve the same problem with a consistency angle. I'll not dwell into the architectural reasons for WS-Transaction but simply note that it introduces a tight coupling between the service consumer and provider and is not preferable in i.e. a B2B solution. It is my firm belief that http/xml should not be used as a transaction medium since it was just not defined with that in mind in the first place. It also has performance implications as well as introduces complex call-chains and possible deadlocks.

WS-Reliable Messaging
Typically your provider and consumer would store outbound messages in some kind of message store and delete these first when they've been acknowledged by the other part through a conversation mechanism. Messaging platforms are often used as the persistent store, but regular databases could also be included, either as part of the messaging provider or by the WS stack.

Retry mechanisms and idempotent provider services
Clients simply retry the calls until a successful invocation is acheived. This puts a requirement on the provider to be idempotent either through the nature of the service, or by utilizing some framework for blocking resent requests based on a uniqueness in the message.

Soap through JMS
Most, if not all Java WS runtimes now support sending soap messages through a JMS provider. Soap/JMS is not yet a standard, but several providers have in fact agreed on a set of JMS properties that maps to a soap world, so it is actually quite interoperable.

This works fine for most Java implementations, but the .NET world has (to my knowledge) been left out on this option for a while.
Well, today I noticed an interesting thing in the Websphere MQ 7 documentation which apparently has support for hooking in WCF services/consumers to the Soap/JMS space. IBM has implemented transport interceptors that could be configured on WCF components. These are bundled with the MQ7 client and as far as I can see does not change either the MQ setup or your .NET code.

http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/topic/com.ibm.mq.csqzav.doc/un12000_.htm

To me this is very good news as that makes Soap through JMS a valid alternative to the quite complicated stacks above even when you're looking for Java-WCF interoperability and guaranteed deliveries, given that IBM WMQ is already a part of your infrastructure.

Once again messaging comes to the rescue! Either as the actual transport layer or as a building block for enabling WS-Reliable Messaging.

Now I just have to bribe some .NET programmer to hook up with for a lab since my .NET skills are, lets say.. nonexistent at the moment

Software Development Thoughts: Logging Do's and Dont's

What every java programmer must know...
Software Development Thoughts: Logging Do's and Dont's

Comparing OSS offerings, like comparing apples to oranges

Rounding up the OSS integration frameworks described earlier is not easy and by no means a task of selecting an overall "winner". I've only scratched the surface on them, and most of the experience comes from testing simple samples, reading forums and blogs and in some cases implementing small solutions. There are also a bunch of frameworks that I haven't had time to test yet, so I'll focus on Mule, WSO2 and Apache Camel. Note that these are my personal reflections and experiences, so if you disagree completely, feel free to post so and call me a moron (maybe you're right :-)

Mule ESB

Marketed as the "most popular" open source ESB this was a good alternative to start with.

Mule is quoted as a stand-alone ESB and offers a wealth of connectivity options. The Mulesoft company behind it are very commited to the product and posts regular updates to the documentation as well as blogs.

However, the configurations are all designed by editing a quite cumbersome XML files which might offput some users wishing for less "bracket coding". Editing these files is only more constrained by the amount of schemas that needs to be included, i.e. each transport has its own schema and restrictions.

It supports a stand-alone runtime, hot deployments and extra functions such as registries and management consoles if you opt for the Enterprise license.

Mule was not created with EIP in mind from the startup, but updates in the most recent version made it easier to design flows through the "flow" construct even though I've already found some bugs while using it.

WSO2 ESB

Out of the three, wso2 is the only one that requires sort of an application server to run in the form of the Carbon server.

However, the management console is outstanding with its included support for statistics and registry. You also have the option to graphically design your integrations which is always good for beginners.

There are three things that bother me with WSO2 at the moment though:

- Big focus on the WS-stack, which is not surprising given the WSO2 team background. If you're after a webservice framework, this product is perhaps the most promising of the three, but as far as I see it might not be perfectly suited for other integration scenarios.

- Synapse. WSO2 is built upon Apache Synapse, a project that doesn't have as much community activity as the other two alternatives. Only the future will tell if it will grow in the same pace as the other ones, but I'm at least a bit concerned about its future.

- The need for an application server

This might be a political issue, but I'd think it will be more difficult to sell in a new application server to your management than a simple API runtime. Other might see this as a strength, but when you're already sitting on a stack of servers you'll undoubtley will get the question why you'll need another one.

Apache Camel

The most lightweight alternative of the three I've tested and probably the most flexible as well.

Since Camel is built as an implementation of the EIP patterns it fits right in to creating integration flows, which is the logical way of implementing integration.

With Camel you get alternatives wherever you go. Do you wish to develop your flows through XML? Or maybe java? Or perhaps Scala or Groovy? Would you like to execute it in a spring environemnt? Or as a routing component inside the ServiceMix OSGi runtime? Or inside a web application? Options everywhere... Which is acutally a good thing! Hopefully your team and organisation can settle with one and develop best practices using the one you prefer. Otherwise you could end up with a mess of implementations, but this holds true for all types of development environments, so its not a problem per se.

The only downside I could possibly think of when using camel is that since it's so easy to use you might start putting integration logic all over your projects instead of at a central ICC team of integrators and designers. Integration is a very complex area and should be handled by dedicated staff with end-to-end focus in mind. Camel with it flexibility and pragmatic approach might lead you to solving integration tasks as part of application projects, but as you can see I'm only nitpicking for a possible future problem here.

I especially like the fact that they've started to experiment with Scala as a a first-class citizen as I personally see a great future in Scala. Having this support from the start gives me hope that this component will have a long future even after Java has been declared as "dead" or legacy.

As noted these are just my first impressions of the frameworks and I might be proven wrong or have missunderstood some parts of them, so take these words with a grain of salt.

There are also a lot of alternatives that I haven't been able to try out extensively yet, for example JBoss ESB, ServiceMix and Spring Integration 2.0 which looks promising. So many toys, so little time...

See you soon!

Climbing the Camel

I remember my first ride with a live camel during one of our family vacations half a lifetime ago. It was a quite challenging task just to get up on the beast, and even hairier when this huge animal started to move. Rocketing back and forth it eventually managed to instill some sense of trust as it started to swaggle on with me sitting high atop on its back. After the first terror waned, the experience was actually simply a blast!

Well, recently I had the chance to climb up to the camel's back again, but now in another context. This camel was maybe even more scarier as I had never seen something quite like it and I was first startled by its complexity. But again, once it started moving I understood the pure beauty of the beast and felt safe and secure again.

I'm of course talking about Apache Camel, the lightweight OSS framework for designing and executing integration flows. It takes a pragmatic aproach to integration and has a very clean and efficient way of defining your integration logic between separate endpoints.

Camel is all about lightweight integration, and even though it does not qualify as a complete ESB since it misses some important parts such as manageability and hot deployments of your solutions, it certainly has some features which puts it right into the Enterprise integration space.

First of all one has to understand that camel is not an integration product per se, but rather an API for defining integration patterns. It does not execute by itself but should preferably be bundled together with some existing hosting environment i.e. JBI, Spring applications, your messaging engine, an application server or what not.

Setting up a Camel project is easy, since you could start out from Maven archetypes found on the Camel website. These will help you download all dependencies and create your Spring context. Camel does not force you to use either Maven or Spring, but much of the samples and tutorials on the net (and the documentation) assumes that you're already an experienced Maven/Spring user. After passing these first hurdles (which really had mostly to do with setting Maven up correctly) I was able to try out my first flows.

You have several options for choosing how to define your integration logic. The two major ones are through either Spring xml files similar to i.e. Mule ESB or a fluid Java API. There are also options for Scala and Groovy if that's your fashion. No matter which, the flow executes the same way as the design languages are only a means to model your integrations.

This separation is quite brilliant as it allows you to select the technology you are most proficient with, and the runtime will adapt. It also means that cross-functions such as automatic documentation is supported no matter which language you choose.

Our first flow

So, enough babbling. Lets have a look at an example:

public class ExampleRouteBuilder extends RouteBuilder {

/**

* A main() so we can easily run these routing rules in our IDE

*/

public static void main(String... args) throws Exception {

Main.main(args);

}

/**

* Lets configure the Camel routing rules using Java code...

*/

public void configure() {

from("jms:queue:purchaseOrder?concurrentConsumers=5")

.to("log:fromJms")

.split().xpath("//lineItem")

.choice()

.when(body().contains("Pen"))

.to("jms:queue:Pencils")

.when(body().contains("Paper"))

.to("jms:queue:Papers")

.otherwise()

.to("file:target/messages/invalid");

}

}

So, what is going on here?

Well, the first thing you'll notice is that we're extending the RouteBuilder abstract class and overide its configure() method. RouteBuilders are classes that configure your integration flows, or "routes" in camel language. The route itself starts with the from() method call and is described by chaining methods together.

Regular code completion assist then shows you which types of "Processors" you can apply to the route. A processor is basically some part that does something with a message exchange in a pipe and filter architecture. Apache comes bundled with processors for most of the EIP patterns such as the split defined in the example above, but you can of course add extra processors by just implementing the org.apache.camel.Processor interface or by sending the message to a spring bean method.

As you can see we've managed to include at least 4 EIP patterns in just a couple of lines (tecnically just one line of java, but you get the idea...) of code, namely the competing consumers, wiretap (the log endpoint), splitter and content-based router patterns. Now that's impressive!

The same flow could as stated above be modeled using XML in a Spring-like fashion by just including the Camel namespace and using content-assist in your favourite XML editor with the exact same outcome, so it's up to you to decide which method that suits you best.

Transports

Camel has the concept of adressing endpoints as URI's with the endpoint type first followed by the address and optional parameters on a query format. In our example we're instructing the route to use 5 concurrent threads to get the messages from the purchaseOrder jms queue.

The API relies on third party providers to implement the endpoint factories called components, so for our jms provider we could bind the jms transport to a component realised by i.e. Websphere MQ. This could either be performed by declaring the component through code, or as below as a spring bean in your context configuration.

<bean id="jms" class="org.apache.camel.component.jms.JmsComponent">

<property name="connectionFactory">

<bean class="com.ibm.mq.jms.MQXAQueueConnectionFactory">

<property name="hostName" value="localhost"/>

<property name="queueManager" value="QM_ESB"/>

</bean>

</property>

</bean>

This snippet declares that the MQXAQueueConnectionFactory should be the queueconnectionfactory for our jms endpoints. Just make sure that the application user has access rights to the queue manager by including the user either in the mqm group or some other group with connect/read access.

You could also define your endpoints as beans in the Spring XML file and just reference them by name, making the actual endpoints changeable depending on i.e. your different test environments setup.

So you could define your endpoints in a Camel context xml as:

<camel:endpoint id="PurchaseOrders.IN" uri="jms:queue:purchaseOrder?concurrentusers=5"/>

.. and then just reference this endpoint in your route as:

from("PurchaseOrders.IN")

This gives you a very good flexibility for choosing the actual endpoint location and even transport outside the actual flow.

Other cool features that Camel brings are for example

- Automatic type conversion:

Note how we could just use an Xpath expression even though we did not know the exact inbound object format? Camel automatically type converts between a bunch of different standard formats so you dont need to know whether the data is delivered as i.e. a bytearray, string, jaxb objects, xml stream etc.
If you don't find support for your particular format you can always write your own converters and easily plug these in to the runtime.

- A bunch of transport components out of the box:

We only saw two very common components in the example, but have a look at the http://camel.apache.org/components.html and you'll find that even the most obscure transport options are available.

- Bean support for i.e. defining a Spring bean as a service.

So, this was just a quick intro on the capabilities of Camel. I advise you to take a look at the number of articles written about it and get a feeling for this awesome framework at http://camel.apache.org/articles.html

'til next time,

Over and out!