Automating Eclipse Source Bundle Generation using Maven

Introduction:

In my previous blog post, I explained about how to convert third party and in-house API libraries to OSGi bundles. Once you convert your in-house API libraries to OSGi bundles, you will not be able to browse/debug your API source code, while working on GUI code. "Attach source" will not be an option anymore in Eclipse, since the dependency is now a plugin-dependency. To be able to view/debug the source code of plugin dependency, you need to have the corresponding source bundle in target platform.

What is Eclipse Source Bundle?

Eclipse source bundle is a normal source jar and is also a valid OSGi bundle. It has OSGi related information in it's manifest. Below is a sample manifest for a source bundle:

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: cruise
Build-Jdk: 1.6.0_10
Bundle-ManifestVersion: 2
Bundle-Name: Utility
Bundle-SymbolicName: com.mycompany.utility.source
Bundle-Vendor: MyCompanyName
Bundle-Version: 1.0.0.SNAPSHOT
Eclipse-SourceBundle: com.mycompany.utility;version="1.0.0.SNAPSHOT";roots:="."

Here the last line (Eclipse-SourceBundle) is the one which identifies this bundle as a source bundle. This says that this bundle is source bundle of "com.mycompany.utility" bundle whose version is "1.0.0.SNAPSHOT". Note that bundle name and version has to match exactly.

roots directive says which folder within the source bundle has source code. By default it is "." and it is true for most cases. If you have a special case, you can change it.

Automation:

Automation of source bundles can be done using maven build helper and maven source plugins. In your project's POM or parent POM, add the following block in build section:

<!-- Build helper maven plugin sets the parsedVersion.osgiVersion property -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.5</version>
<executions>
<execution>
<id>set-osgi-version</id>
<phase>verify</phase>
<goals>
<goal>parse-version</goal>
</goals>
</execution>
</executions>
</plugin>

<!-- source maven plugin creates the source bundle and adds manifest -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>2.1.1</version>
<executions>
<execution>
<id>attach-sources</id>
<phase>verify</phase>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
<configuration>
<archive>
<manifestEntries>
<Bundle-ManifestVersion>2</Bundle-ManifestVersion>
<Bundle-Name>${name}</Bundle-Name>
<Bundle-SymbolicName>${groupId}.${artifactId}.source</Bundle-SymbolicName>
<Bundle-Vendor>${organization.name}</Bundle-Vendor>
<Bundle-Version>${parsedVersion.osgiVersion}</Bundle-Version>
<Eclipse-SourceBundle>${groupId}.${artifactId};version="${parsedVersion.osgiVersion}";roots:="."</Eclipse-SourceBundle>
</manifestEntries>
</archive>
</configuration>
</plugin>

Here, maven source plugin takes care of creating the source jar and adding appropriate information to it's manifest so that it can be a valid OSGi source bundle. One small problem is, maven adds "-" before qualifier, whereas OSGi adds "." before qualifier. If you use BND to turn your jar into bundle, the bundle version would be 1.0.0.SNAPSHOT, whereas maven version is 1.0.0-SNAPSHOT. This is where, maven build helper plugin helps above. Make sure to use the latest version (1.5) of this plugin, which has parse-version goal. This goal sets the parsedVersion.osgiVersion property, which has the version string, but has the "-" replaced by ".".

Automate copying source bundles to Eclipse target platform:

Now, that source bundles are automatically created and installed/deployed to your maven repo, next step is to make sure they are automatically copied to Eclipse target platform, so that Eclipse can recognize it. Only then, you can view/debug the source code from within Eclipse. In my previous post, I explained about how to automate copying dependency bundles to target platform. Look at "Automate Copying Bundles to Eclipse target platform" section in that post. Similarly, you should be able to copy the dependency source bundles to target platform.

<properties>
<pluginsDir>./plugins</pluginsDir>
</properties>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.1</version>
<executions>
<!-- This execution downloads and copies the dependency bundles -->
<execution>
<id>copy-dependencies</id>
<phase>process-resources</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${pluginsDir}</outputDirectory>
</configuration>
</execution>
<!-- This execution downloads and copies the dependency source bundles -->
<execution>
<id>copy-dependency-sources</id>
<phase>process-resources</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<classifier>sources</classifier>
<failOnMissingClassifierArtifact>false</failOnMissingClassifierArtifact>
<outputDirectory>${pluginsDir}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>

Maven dependency plugin takes care of downloading/copying the dependencies and it's corresponding source bundles. Here, "sources" classifier takes care of the source bundle. Now, whenever you do "mvn process-resources", this would pull in latest OSGi bundle dependencies and also it's source bundles from maven repo and copy to plugins folder in target platform.

Conclusion:

We saw what an Eclipse source bundle is and how to automatically create it, whenever the bundle is built. We also saw how to download and copy the source bundles from maven repo to your local Eclipse target platform, using maven plugins.

Eclipse ClassLoading and Buddy Policy

Eclipse Class Loading:

One of the recent things we did on our project is to convert all our Eclipse plugin dependencies from jars on classpath to OSGi bundles. You can see why we had to do this and how we did this in my previous blog post. As you might know, in Eclipse, each plugin (or bundle) has it's own classloader. When a plugin needs to load a class, it will look in java.* package, or it it's own bundle, or from one of the import packages or from one of the required bundles or from one of the plugin's fragments or from packages imported using DynamicImport-Package. The search for class is done in the same order as specified here. If none of these work, then Eclipse will not be able to find the class.

You could think this is good enough and yes, this works for most cases. But, in some cases where third party libraries need access to your plugin/bundle, this would fail. This is because only your bundle has dependency to third party library and not the other way. So, this sounds like a circular reference right? This is where buddy policy comes to rescue. Before looking at buddy policy, let me say that you should not use buddy policy unless needed. This is because:

1. Your code may not follow good design and often you should be able to refactor your code to avoid this.
2. Buddy policy is Eclipse specific and OSGi does not know anything about this.
3. Also, since this is not normal OSGi, BND bundle generator, does not support this and you have to manually customize your bundle manifest, by adding the buddy policy entry.

Buddy policy and it's use-cases:

Let me explain buddy policy using some of the valid use-cases (which I've encountered), where buddy policy comes into rescue:

1. Using ORM frameworks like iBatis/Hibernate - Your database bundle would depend on iBatis/Hibernate and the other way is not true. This works in normal java projects since all jars share the same classpath and so iBatis/Hibernate can load your classes at runtime. But, in Eclipse/OSGi world, since iBatis/Hibernate does not know about your classes (since they are in a different bundle, which it cannot see) it cannot load any of your classes at run-time. So, the database bundle should have: Eclipse-RegisterBuddy: org.apache.ibatis.sqlmap in it's manifest and iBatis bundle should have Eclipse-BuddyPolicy:Registered in it's manifest. Here, iBatis bundle says that I'm registered for buddy policy and database bundle says I'm a buddy of iBatis. So, whenever iBatis cannot find a class or resource, before throwing ClassNotFoundException, bundle class loader will see if the iBatis bundle is registered for buddy policy. If yes, it will check who are the buddies of iBatis. If it finds any buddies, it will try to load the class from it's buddies. Note that buddy-policy search is not recursive i.e if a buddy does not have a requested class, it's buddies won't be searched recursively. In our case, since the database bundle is a buddy of iBatis, the classes are loaded from the database bundle. Similar case holds true for hibernate also.
2. Using commons configuration - Say if your bundle uses commons configuration to load a properties file (for ex like, Configuration config = config = new PropertiesConfiguration("myproperties.properties");), your bundle would depend on Commons configuration, but the reverse is not true. But, commons configuration bundle needs access to myproperties.properties, which lives in your bundle. So, your bundle should have: Eclipse-RegisterBuddy: org.apache.commons.configuration in it's manifest and commons configuration bundle should have Eclipse-BuddyPolicy:Registered in it's manifest. This way commons configuration can access the properties file which resides in your bundle.
3. Using XStream - Say if your bundle uses XStream to serialize the objects, your bundle would depend on XStream, but reverse is not true. But, XStream needs access to the classes in your bundle during deserialization process. So, your bundle should have: Eclipse-RegisterBuddy: com.thoughtworks.xstream in it's manifest and XStream bundle should have Eclipse-BuddyPolicy:Registered in it's manifest. This way XStream can instantiate the classes which resides in your bundle.

Buddy Policy Options:

Note that in the above cases, Eclipse-BuddyPolicy:Registered has been used. As explained above, "Registered" buddy policy only consults all the registered buddies to load the requested class. There are other buddy policy options, but I did not have to use them till now. "Registered" works for most cases and is very performant also, since the bundle has to search only it's buddies to load the requested class. Anyway, here are the other buddy policy options for reference:

• dependent - All bundles which depends on this bundle will be searched to load the class. This option would also work in all of the above cases, but would be slightly less performant since the search had to be done in potentially more bundles.
• global - Searches all the available packages which are exported. This also would work in our use cases above, but will be potentially even more less performant than dependent.
  
• app, ext, boot - These are the other three options available and they use the app, ext and boot classloaders respectively to load the requested class.