tisdag 14 april 2015

Ok, ok I get it... So this Node thingy IS actually quite cool...

I consider myself a polyglot developer who typically uses the best tools for the particular task at hand. Nevertheless when it comes to actually building production ready applications my main language has historically been within the JVM, more times than not relying on the always trusty Java ecosystem with it's abundance of libraries and frameworks.

Today I really got one of those OMGOD moments when you realize just how easy alternative technologies makes your life comparing to what you usually have to keep up with.

At my current customer project we unfortunately have to deal with a remote external SOAP service, which in my and most developers opinion is a protocol born out of hell and is the perfect example of an over engineered solution to something that should be pretty simple.

After dealing with a botched release due to threading problems and bugs within the SOAP framework we used I intended to switch to CXF, which is probably the most popular JAX-WS implementation in Java land.

Many hours later I had a working client after tearing my hair out fighting questions such as:

  • How do we authenticate using WS-Security
  • Which JAR's to include in the Maven build? (CXF has a lot of modules)
  • How to make the JAXB code generation behave well and produce the types of classes we need
  • How to intercept and log relevant request / responses over the wire
  • How to cope with threading, JAX-WS is not thread safe while CXF as an implementation claims to be
  • How to configure CXF to use our logging framework of choice
  • etc...

You get the idea, a lot of ceremony just to create something that could send structured data over the wire and parse responses into something that we can work with.

Just out of curiosity I went ahead and started investigating how you would create a soap client in Node.js land as I have previously built a couple of lightweight internal Node apps for testing, service bridging etc. My first thought was that the cool JS kids would probably never even poke a stick at something like SOAP, so I did not expect to find any good NPM modules for it.

Turns out that even JS developers aren't immune to the protocol madness of SOAP and someone actually went ahead and wrote a very clean and simple client and service module. After reading the example (https://github.com/vpulim/node-soap) and about 5 minutes of experimenting I was up and running. This was what it took to create the same thing I fought for hours with CXF.
npm install soap

  var soap = require('soap');
  var url = 'gateway.wsdl'; //local file, could also be a remote http url
  var args = {};

  soap.createClient(url, function(err, client) {
    //WS-Security UserNameToken out of the box
    client.setSecurity(new soap.WSSecurity('myUser', 'myPassword'));
    //GetProducts is an operation in the wsdl
    client.GetProducts(args, function(err, result) {
      if (!err) {
        //Do something valuable with the response, for now just log.
        //Each XML element in the response would become a corresponding field on the result object
        console.log(JSON.stringify(result, null, 2));
      else {

So, I guess Java and CXF lost this one. And for Node... well it's something you probably should consider learning ASAP. Even if you're a grumpy old-school JVM developer like yours truly.

tisdag 12 november 2013

Love is in the details. Or when I really fell in love with Dropwizard

So Dropwizard seems to be the rage around Java / Scala developers today as an extremely clean and minimalistic framework for DTRS ("Doing The Right Stuff") when it comes to serving up thick clients with REST-ful data. If you haven't checked it out yet, I urge you to.
It is really one of those few frameworks that looks so shiny and sexy that you immediately start thinking about in which of your projects you could introduce it. The last time I was this enthustiastic about a new library was when I first laid my eyes on Camel, which shares the same minimalistic mindset.

Now, a lot has been said about it's excellent choice of best-of-breeds libraries, the YAML configuration, server-less environment and the ability to quickly assemble healtchecks. But the tiny one thing that made me realize that the folks at Yammer really put in a lot of thought into the package was not as profound as the above points. It was a really small feature that simply tells you that these guys has been in the trenches and know what they're doing. Now, which of all features could that be?

Error logging...

As I started to play around with a small hello world hobby project and got my first 500 error back at me it said something like:

"There was an error processing your request. It has been logged (ID 2e7c06aac7bcf2aa)."

Looking into the server logs I see a stacktrace with the same ID appended:

 ERROR [2013-11-12 11:18:23,651] com.yammer.dropwizard.jersey.LoggingExceptionMapper: Error handling a request: 2e7c06aac7bcf2aa  
 ! java.lang.RuntimeException: My extremely PITA error  
 ! at se.com.eyc.employeecatalog.EmployeeCatalogue.allEmployees(EmployeeCatalogue.java:38) ~[dropwizard.jar:na]  
 ! at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_45]  
 ! at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_45]  
 ! at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_45]  
 ! at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_45]  
 ! at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[jersey-server-1.17.1.jar:1.17.1]  
 ! at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) ~[jersey-server-1.17.1.jar:1.17.1] ! at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[jersey-server-1.17.1.jar:1.17.1]  
 ! at com.yammer.dropwizard.jersey.OptionalResourceMethodDispatchAdapter$OptionalRequestDispatcher.dispatch(OptionalResourceMethodDispatchAdapter.java:37) ~[dropwizard-core-0.6.2.jar:na]  
 ! at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) ~[jersey-server-1.17.1.jar:1.17.1]  
 ! at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100) ~[jersey-server-1.17.1.jar:1.17.1]  
 ! at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.17.1.jar:1.17.1]  

Now this is great handing out of the box for three reasons:
  • Your customer / client system has an ID they can refer to which you can find with a simple grep in the logs
  • The default behaviour in DW is to NOT hand your stacktrace to your clients (unlike say a plain Tomcat among others). I can not believe that you still today sometimes see customer facing sites with no configured error page. That actually happened to me just a couple of days ago when a pretty big web shop happily informed me that they had not tweaked their connection pool for the MySQL database in the backend. This is just sloppy and DW simply does the right thing for you out of the box.
  • The stacktrace actually contains the artifact id and version of the dependencies that was involved in the stack. This is pure genious as it helps tremendesly i.e. when you submit bugs since everyone can easily see which versions of what you were using.
So, just a small example why I think DW rocks and proves that the guys who built it knows what they're doing. It seems to be a stable foundation to build your next web app or service on. I sincerely hope that it can stay out of the typical OS feature creep and stay simple and lean.

Til next time!

torsdag 6 december 2012

Using Groovy and Camel for your scripting purposes

This topic has been displayed at other blogs, but it's such a great concept and served me very well quite recently so I think it warrants a revisit.

As a JVM developer (isn't that what we ought to call us nowaday when Scala/Groovy/Clojure is all the hype?) you're often sidelined by your scripting wiz fellows when it comes to writing quick "one-off" scripts for testing purposes and what-not.

I've personally gone as far as learning a bit of Python for this purpose, but I tend to miss my favorite java libraries and the ecosystem, although I must give the deepest respect to Python which is a really nice and powerful language. I must also confess that I've yet to find the time needed to become really proficient with it.

On our current project we had a requirement for a one-way http multicast solution in one of our test environments, as the backend system we were mediating requests to had more test environments than the rest of the chain. In EIP patterns you'd call this a fanout, multicast proxy or something similar.

Well... I thought this would be a really good fit for Camel, so I started setting up a standard Maven project. After a while it became quite obvious that while the Camel route itself was extremely small and simple, the project consisted of more Maven ceremony than actual code.

As this was an application for a test environment I was really not that interested in having a full build process for it. I'd rather just have something up and running in as few lines as possible.

Well, groovy scripts + grape to the rescue...

As any of you who've dealt with Groovy programming know, it is a great language for creating standalone scripts which are compiled and executed at runtime. Although Scala is my personal #1 choice when it comes to the JVM languages and offers scripting support as well, Groovy has one up in the Grape ecosystem which I've not yet seen in the Scala world.

So after moving the essential part of my application (the route) to it's own groovy script and adding a few Grape annotations to download the external Camel dependencies the solution was basically finished and executable by running "groovy Multicaster.groovy". Mind you, this is an extremely simple route and the dependencies will be downloaded at runtime which would be totally forbidden in a production environment. But for this purpose it fit the bill perfectly.

 import org.apache.camel.*  
 import org.apache.camel.impl.*  
 import org.apache.camel.builder.*  
 def camelContext = new DefaultCamelContext()  
 camelContext.addRoutes(new RouteBuilder() {  
   def void configure() {  
       .convertBodyTo(byte[].class) //To read multiple times, default is InputStream which is only readable once  
       .removeHeader(Exchange.HTTP_PATH) //So as not to forward URI to next call...  
     //Using SEDA here as it implies async handoff  
         //bridgeEndpoint is used to indicate that http headers (SOAPAction etc) should be copied  
         'http://testserver1:6060/myService?bridgeEndpoint=true&throwExceptionOnFailure=false', //consumer1  
         'http://testserver2:6060/myService?bridgeEndpoint=true&throwExceptionOnFailure=false', //consumer2  
         'http://testserver3:6060/myService?bridgeEndpoint=true&throwExceptionOnFailure=false', //etc...  
   final String STATIC_RESPONSE = '''\  
             <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ipl="http://www.acme.se/ipl">  
 addShutdownHook{ camelContext.stop() }  
 synchronized(this){ this.wait() }  

The combination of Camel in a scripting environment is extremely powerful. Just imagine the possibilities for statistic gathering utilities etc.. Now off to some more Groovying..

fredag 1 juni 2012

Free integration breakfast seminar in Stockholm, Oslo and Copenhagen

I'd just like to inform you that we at Redpill Linpro will be hosting a series of breakfast seminars about the FuseSource line of products, meaning Camel, CXF, ActiveMQ and ServiceMix.

The seminars will give you a technical quickstart to using Camel and ServiceMix in your projects and is intended for developers and architects. So there will be no fluffy slides or sales pitches but rather a more "down to the core" type of seminar with lots of coding and examples of why Camel is such an awesome integration framework to work with.

If you are interested and either in the Stockholm, Oslo or Copenhagen areas, just register at the specific event. The seminar is of course free. Hope to see you there!

Stockholm (12/6)
Oslo (13/6)
Copenhagen (14/6)

Until next time!


I'd like to update you with the links to the presentations as well as the source code. Many thanks to all of you who attended. Especially to Claus Ibsen (the Camel project lead) who took the time to show up in the audience at the Copenhagen session.

Slides at slideshare:

Code at github:

torsdag 8 mars 2012

Bridging between JMS and RabbitMQ (AMQP) using Spring Integration

An old customer recently asked me if I had a solution for how to integrate between their existing JMS infrastructure on Websphere MQ with RabbitMQ.

Although I know that RabbitMQ has the shovel plugin which can bridge between Rabbit instances I've yet not found a good plugin for JMS <-> AMQP forwarding.
The first thing that came to my mind was to utilize a Spring Integration mediation as SI has excellent support for both JMS and Rabbit.

Curious as I am I started a PoC and this is the result. It takes messages of a JMS queue and forwards to an AMQP exchange that is bound to a queue the consumer application is supposed to listen to. I used an external HornetQ instance in JBoss 6.1 as the JMS Provider, but I am 100% secure that the same setup would work for Websphere MQ as they both implement JMS.

Be aware that I've done no performance tweaking or QoS setup yet as this is just a proof-of-concept. For a real setup you'd probably have to think about delivery guarantees versus performance and etc...

The code will be available at a GitHub repository near you soon..

SpringContext in XML:

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/integration"
xsi:schemaLocation="http://www.springframework.org/schema/integration/amqp http://www.springframework.org/schema/integration/amqp/spring-integration-amqp-2.1.xsd
http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration-2.0.xsd
http://www.springframework.org/schema/integration/jms http://www.springframework.org/schema/integration/jms/spring-integration-jms-2.1.xsd
http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.0.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

<beans:bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
<beans:property name="environment">
<beans:prop key="java.naming.factory.initial">org.jnp.interfaces.NamingContextFactory</beans:prop>
<beans:prop key="java.naming.provider.url">jnp://localhost:1099</beans:prop>
<beans:prop key="java.naming.factory.url.pkgs">org.jnp.interfaces:org.jboss.naming</beans:prop>

<beans:bean id="jmsQueueConnectionFactory"
<beans:property name="jndiTemplate">
<beans:ref bean="jndiTemplate"/>
<beans:property name="jndiName">

<!-- Channels and adapters for SI -->
<int-jms:message-driven-channel-adapter connection-factory="jmsQueueConnectionFactory" destination-name="myJmsQueue" channel="rabbitChannel"/>
<channel id="rabbitChannel"/>
<int-amqp:outbound-channel-adapter channel="rabbitChannel" exchange-name="fromJmsExchange" amqp-template="rabbitTemplate"/>

<!-- Connectivity to Rabbit -->
<rabbit:template id="rabbitTemplate" connection-factory="cf"/>
<rabbit:connection-factory id="cf" host="localhost"/>

<!-- Rabbit entities, to be created at context startup -->
<rabbit:admin connection-factory="cf"/>
<rabbit:queue name="fromJMS"/>
<rabbit:direct-exchange name="fromJmsExchange">
<rabbit:binding queue="fromJMS"/>

Maven POM:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">









torsdag 1 december 2011

Gotcha when using camel-servlet

In my current project I'm using Apache Camel for the integration, and I bumped into a significant "oups"-moment while using the camel-servlet component.

I found that while I had multiple applications (WAR's) deployed I often started the wrong routes, even though I was hitting the correct endpoint url.

Say i.e. that you have two application, A and B. Both of these has the same endpoint addresses, i.e. servlet:/myservice

The logical thing would be that you could reach application A through:

and B through:

Now imagine my surprise when requests for the A application would up in the route for B. It took me a while to figure this out, but here's the reason for this...

When you expose your routes through servlet's you are using the CamelHttpTransportServlet class. An example web.xml could be something like this:

<!-- Camel servlet -->

<!-- Camel servlet mapping -->

Now, if you name your servlet the same for both applications you're actually only publishing one common servlet class. The camel servlet component does not take your context-root into account, which means that you hit the same endpoint irregardless whether you go to /A/myservice or /B/myservice. The one that gets hit is the one that was last deployed (or started).

Ok, easily fixed I thought... Just rename the servlet-name to something more unique for each application...
After that my routes where no longer reachable and I was just left with 404's each time I tried to hit the endpoints.
After some trial and error as well as googling I found out that if you rename your servlet from the default "CamelServlet" you *have* to specify the servlet name in your endpoint uri as well.
That is, the endpoint in application A would be i.e.:


and B:


So, the lesson is:

- Always rename the camel servlet name, you never know which other applications will be deployed together on the same server. Your endpoints might clash with these and you'll have a mess figuring out why.

- Always include the servletName in your endpoint uri.

These tips should definitely be added to the documentation page to help others avoid making the same mistake.

'til next time!

torsdag 1 september 2011

Implementing your own "SOA" registry, the maven way

Saw a very interesting article on InfoQ by Ben Wilcock where he described a dead simple SOA R&R using a very pragmatic approach with Maven combined with a simple HTML app.

As an integration consultant I've worked with registries ranging from overly complex offerings (IBM Websphere Registry and Repository) to time-consuming simple wikis, but I have to say that his solution is arguably the most elegant yet.

The best part is that you both get the visibility as your schemas and contracts are available to all your stakeholders, while keeping your co-developers happy by just pointing them to a Maven project to include in their dependencies.

I'm definately hijacking this concept to my current project, and I find it very interesting that this solution has been under my nose all this time without reflecting on it.