Saturday, 27 February 2010

neo4j on OSGi

I need to have a deeper look at neo4j, which I found out about via InfoQ today, but initially I thought I'd see how OSGi friendly it is.

At first look I was quite excited - the neo4j jars are actually bundles. Yay! :)

So I dropped the contents of the neo4j lib folder in to my Apache FileInstall folder to see what happened.

org.apache.felix.log.LogException: org.osgi.framework.BundleException: The bundle could not be resolved. Reason: Missing Constraint: Import-Package: org.neo4j.index; version="1.0.0"

And was probably due to:

org.apache.felix.log.LogException: org.osgi.framework.BundleException: The bundle could not be resolved. Reason: Missing Constraint: Import-Package: org.apache.lucene.analysis; version="0.0.0"

Aha! A quick look at the provided Lucene jar reveals that it is not a bundle. Boo! :(

Easy enough to sort out though, create a quick bnd wrapper! Or so I thought. The following wraps the Lucene jar creating a bundle which exports all the packages from the Jar and imports all detected dependencies:

-classpath: libs/wrapped/lucene-core-2.9.1.jar
Export-Package: *;version=${Bundle-Version}
Bundle-SymbolicName: lucene
Bundle-Version: 2.9.1

To use the above, put it in a file called lucene.bnd and run the bnd tool on it. However, that wasn't the end of the problem...

org.apache.felix.log.LogException: org.osgi.framework.BundleException: The bundle could not be resolved. Reason: Missing Constraint: Import-Package: sun.misc; version="0.0.0"

sun.misc ... WTF? sub.misc is an optional package used by PersonalJava and a few other things that I doubt many people care about these days. Unfortunately bnd detects this package usage when it creates the bundle and makes an import for it. This can be overcome by creating an import specifically for that package and marking it optional:

Import-Package: sun.misc;resolution:=optional,\

The * at the end is quite important, as that tells bnd to make sure it imports all other detected dependencies, as normal.

After that the bundles installed and activated without any problems. Sweet. So I will probably come back and look at neo4j in more detail after I get some sleep.

However, this is the second Apache project that I have looked at this week which has not made it's JAR files bundles by default, the other being Batik (from the XML Graphics project). It isn't a difficult thing to, so why don't they all do it?

Thursday, 25 February 2010

My thoughts on "Alex Buckley on Jigsaw: Modularity for the JDK"

These are the key points I took away from this talk at JAX London:
- Jigsaw is a modularity solution for breaking up the JDK to improve start up times, memory usage, etc
- It is totally backward compatible, e.g. your Java 6 apps should work on it
- if you don't choose a subset of the JDK, you get it all
- The classpath, as it works now, will live forever, i.e. to support those that don't want to use modules
- People can use Jigsaw if they want
- Jigsaw is nothing to do with JSR 294
- There's a bunch of additional optimisations that could be done in addition to modularisation
- Packages will be split across modules, but that's not as important as in a dynamic module system and is basically impossible anyway because of the way the JDK hangs together
- Class loading will work as it does now and all classes from the JDK modules get loaded in to the same class loader
- There was an implication that it would be possible to distribute a cut down version of the JDK which included just the modules you wanted, but that requires clarification.

I don't see Jigsaw having any negative affect on OSGi. In fact the smaller Java runtime that would be the result of this work only enhances the OSGi value proposition. Creating enterprise applications with OSGi is already lightweight compared to using J2EE and [insert any bloated app server here], so having less stuff to bootstrap can only improve that further.

The most significant impression I took away, was that most of Alex's points (e.g. multi-dimensional versioning??) were motivated his grappling with modularising the JDK. This seems to be reflected in his previous responses on the JSR-294 experts mailing list and is disappointing because the JDK we love so dearly has some fundamental flaws so modularity solutions driven by these flaws are potentially inherently flawed also.

I also found it funny that he blamed the poor architecture of the JDK on the fact it was started in 1995 ... so only 23 years after the term modularity was first coined. But I suppose the crazy number of bidirectional dependancies is understandable and as Neil Bartlett says is also a bi-product of it getting so big; a warning to other up-coming languages, e.g. Scala

Thursday, 11 February 2010

In Bed with Vaadin and OSGi

In this post I will show how to create a modular rich internet application that will run on any browser using Java, Vaadin and OSGi. I'll make all sources and binaries available for download at the end of the post.


I've been working with Flex and combining it with OSGi (on the server) at Arum for the last couple of years and while Flash, especially using Flex as a development tool, is a fantastic platform with great penetration, I have found that I still yearn for solutions which require no plugins. In the last few weeks this has become even more important to me with the announcement of the (Flashless) iPad and embarking on a personal project that needs to run on every browser, not just a browser with a Flash Player plugin installed.

Before starting at Arum I had investigated creating modular RIAs using GWT and OSGi. GWT lets you write your UI code as Java then as part of another build step spits out HTML and JavaScript that works across browsers. Most importantly for me was the fact that converting the Java to HTML/JS was a build step, and not done at runtime. As a result, it is impossible to create modular applications with GWT, though that might change with GWT 2.0. However, even if they add GWT modularity, it will still require a build step to convert Java to HTML/JS.

Having spent a great deal more effort looking for a solution to this problem (how to create a plugin free, cross browser RIA using Java and OSGi) I finally discovered Vaadin.

Vaadin is a suite of HTML UI components that have already been built using GWT, then are composed in to a running application using Java... at runtime.

A "Hello World" Vaadin application looks like this:

Vaadin and OSGi

I was particularly interested to learn that someone had considered using Vaadin with OSGi, though I personally do not like the concept of wrapping it in a J2EE style web application (WAR), nesting the OSGi container inside:

It is understandable that people would assume that if a developer that wants to create an RIA with Java they will want to deploy it as a WAR. However, I generally don't use application servers anymore. They tend to be bloated and have a large footprint soaking up much of a VM's heap space just to get started, and as a result tend to have a fairly long startup cycle (from several seconds to over a minute in some cases). Using an OSGi container *as* an application server gives you a much more lightweight application that doesn't require as much memory and starts up almost instantly and out of everything in the J2EE spec, the only thing I regularly come back to is servlets. Everything else seems to be an exercise in telling developers how to build applications with the view of scaling up hardware - obviously a winning approach for Sun in the old days, but not so useful for people wishing to get away with minimum footprint these days (for example by creating a cloud architecture which uses the minimum resources required and scaling up as demand dictates, not the software architecture).

Right out of the box (from at least version 6.2.2 onwards anyway) the Vaadin jar file is already configured as an OSGi bundle which means I can write other bundles which use those APIs.

My next activity was to make Vaadin not just OSGi friendly, but OSGi's "best friend forever". To do this I wanted to achieve the follow things:
- Allow bundles to register Vaadin Applications (whiteboard pattern - it's easier to register services than look them up) and automatically create a http resource for them.
- Allow bundles to register static resources (themes, widgetsets, etc)
- Make no changes to the core Vaadin jar file that you can download from their site

The result is three classes which are built in to a separate bundle that I currently call com.vaadin.osgi.

- StaticResources runs as an OSGi component and serves the themes and widgetsets directly from the core Vaadin bundle.

To add your own theme or widget set create a fragment which contains your theme/widgetset files and export those as packages. The Fragment-Host should be set to the Vaadin core bundle. The fragment containing your theme/widgetset resources will be added to the core Vaadin bundle dynamically.

Of course static resources should really be deployed separately to a web server that proxies servlet requests on to the container, so I wouldn't recommend this for use in a production environment.

- VaadinOSGiApplicationManager also runs as an OSGi component and looks for OSGi component factories with their component.factory property set to

The component factory name (specified as is used as the alias to register an instance of VaadinOSGiServlet which then uses the component factory to create an instance of Application. For example, if was set to myapp, the servlet would be registered at http://localhost:8080/myapp

- VaadinOSGiServlet creates instances of applications that have been registered with the container via a component factory.

This is what is registered by the VaadinOSGiApplicationManager.

Creating an OSGi Vaadin Hello World Application

To get the hello world application shown near the beginning of this post up and running we need to create a bundle which registers the application as a component factory. There is no additional Java code required for this step and by far the easiest way to do this is by describing the bundle with a bnd file.

To create the bundle simply run bnd. I used the following command on my system:

java -jar ~/Downloads/bnd-0.0.384.jar helloworld.bnd

More about bnd and how to get it can be found here:

The result is a bundle that uses declarative services to register your application as a component factory. This is the cool part - if you want to access other OSGi services just create binding methods for them in your application and update the bnd file. Declarative services then injects your application with the services automatically when an instance of the application is created.

Vaadin has this great concept of 'production mode'. When an application is not in production mode, you can append ?debug to your application's URL in order to see a debug console that shows you all kind of useful information such as the traffic between the browser and server, how long requests take, the JSON that has been passed backwards and forward and also gives you a useful 'restart application' button which forces the creation of a new application instance. Obviously, you will want your application to be in production mode at some point so to do this you need to pass "productionMode=true" as a property to the VaadinOSGiApplicationManager and because it is running as a component you can do this using configuration admin. With Apache Felix Fileinstall this is as easy as creating a file called com.vaadin.osgi.VaadinOSGiApplicationManager.cfg and specifying productionMode=true as a line in the file.

Every application will no doubt want it's own theme, right? To do that, again we're going to use bnd to create a bundle which simply adds our theme to the resources in the core Vaadin bundle, a using a useful and under-used feature of OSGi known as Fragments.

Create a folder called etc/theme under which create your theme assets. For example, etc/theme/VAADIN/themes/helloworld/styles.css. The following bnd file would then create a fragment.

Install the fragment and update the core Vaadin bundle and your theme is now available (delivered via the StaticResources component). With the theme installed the hello world application now looks like this:

Again, using bnd to create my bundle:

java -jar ~/Downloads/bnd-0.0.384.jar helloworld.theme.bnd

And that is that; a themed hello world application that is primed for dynamic modularity.


[Edit 10th September 2010]

Since writing this I've published an add-on to the Vaadin Directory that contains the bundles you need to run your apps as OSGi apps and a simple example. You can find that here:


Lastly, I have to thank Neil Bartlett for mentoring me through some of the OSGi concepts I've used here. Thanks Neil! You can find Neil's website full of useful OSGi information here: