Thursday, October 30, 2008

An embedded jar classloader in under 100 lines

To me one of the most annoying things about deploying a standalone java application is packaging. You can't just jar your code and dependencies into a single jar that's ready to run without jumping through hoops. All I want to be able to do is have jars live inside jars and have it all just work. As it turns out this is not an uncommon request, but it seems to be one that is also often dismissed, with few viable solutions. How hard can it be?

As it turns out, its not very hard at all, albeit with a few limitations. In just a couple of hours and using under one hundred lines of code I was able to write a classloader that would allow me to create my single jar deployment. The logic is fairly simple, walk the jar entries in the jar the app was loaded from and recursively extract any embeded jars and add them to the classpath. This sacrifices some temporary storage space at runtime, but it's an acceptable trade off in my eyes.

This solution was fine, but I wanted to be able to launch the app with the typical 'java -jar myapp.jar' convenience instead of overriding the system classloader on the command line. This requires a way to bootstrap the main application using this new classloader. Another fifty odd lines later and I have a bootstrapper that lets the magic happen with the addition of a single manifest entry.

Here is what the classloader looks like:


public class JarClassLoader extends URLClassLoader {

private static void close(Closeable closeable) {
if (closeable != null) {
try {
closeable.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}

private static boolean isJar(String fileName) {
return fileName != null && fileName.toLowerCase().endsWith(".jar");
}

private static File jarEntryAsFile(JarFile jarFile, JarEntry jarEntry) throws IOException {
InputStream input = null;
OutputStream output = null;
try {
String name = jarEntry.getName().replace('/', '_');
int i = name.lastIndexOf(".");
String extension = i > -1 ? name.substring(i) : "";
File file = File.createTempFile(name.substring(0, name.length() - extension.length()) + ".", extension);
file.deleteOnExit();
input = jarFile.getInputStream(jarEntry);
output = new FileOutputStream(file);
int readCount;
byte[] buffer = new byte[4096];
while ((readCount = input.read(buffer)) != -1) {
output.write(buffer, 0, readCount);
}
return file;
} finally {
close(input);
close(output);
}
}

public JarClassLoader(URL[] urls, ClassLoader parent) {
super(urls, parent);
try {
ProtectionDomain protectionDomain = getClass().getProtectionDomain();
CodeSource codeSource = protectionDomain.getCodeSource();
URL rootJarUrl = codeSource.getLocation();
String rootJarName = rootJarUrl.getFile();
if (isJar(rootJarName)) {
addJarResource(new File(rootJarUrl.getPath()));
}
} catch (IOException e) {
e.printStackTrace();
}
}

private void addJarResource(File file) throws IOException {
JarFile jarFile = new JarFile(file);
addURL(file.toURL());
Enumeration<JarEntry> jarEntries = jarFile.entries();
while (jarEntries.hasMoreElements()) {
JarEntry jarEntry = jarEntries.nextElement();
if (!jarEntry.isDirectory() && isJar(jarEntry.getName())) {
addJarResource(jarEntryAsFile(jarFile, jarEntry));
}
}
}

@Override
protected synchronized Class<?> loadClass(String name, boolean resolve)
throws ClassNotFoundException {
try {
Class<?> clazz = findLoadedClass(name);
if (clazz == null) {
clazz = findClass(name);
if (resolve)
resolveClass(clazz);
}
return clazz;
} catch (ClassNotFoundException e) {
return super.loadClass(name, resolve);
}
}
}

And with the help of some modified code lifted from a sun tutorial on jar classloading we have a bootstrapper that looks like this:

public class JarRunner {
private static final String BUNDLE_MAIN_CLASS = "Bundle-MainClass";

public static void main(String[] args) {
try {
invokeClass(getMainClassName(), args);
} catch (Throwable e) {
e.printStackTrace();
}
}

public static String getMainClassName() throws IOException {
ProtectionDomain protectionDomain = JarRunner.class.getProtectionDomain();
CodeSource codeSource = protectionDomain.getCodeSource();
URL rootUrl = codeSource.getLocation();
if (rootUrl.getFile().toLowerCase().endsWith(".jar")) {
URL rootJarUrl = new URL("jar", "", rootUrl + "!/");
JarURLConnection uc = (JarURLConnection) rootJarUrl.openConnection();
Attributes attr = uc.getMainAttributes();
return attr != null ? attr.getValue(BUNDLE_MAIN_CLASS) : null;
}
throw new IllegalStateException("JarRunner is not inside a jar file");
}

public static void invokeClass(String name, String[] args) throws ClassNotFoundException, NoSuchMethodException, InvocationTargetException {
final String mainMethodName = "main";
JarClassLoader loader = new JarClassLoader(((URLClassLoader) ClassLoader.getSystemClassLoader()).getURLs(), Thread.currentThread()
.getContextClassLoader());
Class clazz = loader.loadClass(name);
Method method = clazz.getMethod(mainMethodName, new Class[] { args.getClass() });
method.setAccessible(true);
int mods = method.getModifiers();
if (method.getReturnType() != void.class || !Modifier.isStatic(mods) || !Modifier.isPublic(mods)) {
throw new NoSuchMethodException(mainMethodName);
}
try {
method.invoke(null, new Object[] { args });
} catch (IllegalAccessException e) {
// This should not happen, as we have
// disabled access checks
}
}
}


Now all that is necessary to build an embedded jar application is to add these two classes to the application's jar bundle and add a couple of manifest entries that look like this:

Main-Class: JarRunner
Bundle-MainClass: your.app.MainClass


Simple as that, well sort of. There are a couple of limitations with this approach, specifically if you use things like JSR-232 scripting or certain JDK XML based functionality that expects the system classloader to have visibility of the required classes it won't be able to find them.

An alternative approach I could have possibly taken is to avoid using a custom classloader altogether and use the same recursive jar extraction logic from the JarClassLoader in the JarRunner and have it add the extracted jar paths to the system classloader using the URLClassLoader.addURL() method.

11 comments:

jdotsoft said...

Alternative is to use http://www.jdotsoft.com/JarClassLoader.php
It is very simple to use.

Q said...

MarkG, I hadn't seen that solution before, but looking at the code it appears to contain a lot of redundant code that could have otherwise been avoided had it subclassed URLClassLoader.

eludev said...

Works fine out of the box, thank you.
I prepared a Java 1.4 version (unfortunately, this is the version present on most Windows systems).
Here is this version :
for JarClassLoader :
package qdolan.loader;
import java.io.*;
import java.net.*;
import java.util.jar.*;
import java.util.*;

public class JarClassLoader extends URLClassLoader {

private static boolean isJar(String fileName) {
return fileName != null && fileName.toLowerCase().endsWith(".jar");
}

private static File jarEntryAsFile(JarFile jarFile, JarEntry jarEntry) throws IOException {
InputStream input = null;
OutputStream output = null;
try {
String name = jarEntry.getName().replace('/', '_');
int i = name.lastIndexOf(".");
String extension = i > -1 ? name.substring(i) : "";
File file = File.createTempFile(name.substring(0, name.length() - extension.length()) + ".", extension);
file.deleteOnExit();
input = jarFile.getInputStream(jarEntry);
output = new FileOutputStream(file);
int readCount;
byte[] buffer = new byte[4096];
while ((readCount = input.read(buffer)) != -1) {
output.write(buffer, 0, readCount);
}
input.close();
output.close();
return file;
}
finally {}
}

public JarClassLoader(URL[] urls, ClassLoader parent) {
super(urls, parent);
try {
URLClassLoader ucl = (URLClassLoader)parent;
java.net.URL url = ucl.getURLs()[0];
String rootJarName = url.getFile();
if (isJar(rootJarName)) {
addJarResource(new File(url.getPath()));
}
} catch (IOException e) {
e.printStackTrace();
}
}

private void addJarResource(File file) throws IOException {
JarFile jarFile = new JarFile(file);
addURL(file.toURL());
Enumeration jarEntries = jarFile.entries();
while (jarEntries.hasMoreElements()) {
JarEntry jarEntry = (JarEntry)jarEntries.nextElement();
if (!jarEntry.isDirectory() && isJar(jarEntry.getName())) {
addJarResource(jarEntryAsFile(jarFile, jarEntry));
}
}
}

protected Class loadClass(String name, boolean resolve) throws ClassNotFoundException {
try {
Class clazz = findLoadedClass(name);
if (clazz == null) {
clazz = findClass(name);
if (resolve) resolveClass(clazz);
}
return clazz;
} catch (ClassNotFoundException e) {
return super.loadClass(name, resolve);
}
}
}
For JarRunner :
package qdolan.loader;
import java.io.*;
import java.lang.reflect.*;
import java.net.*;
import java.util.jar.*;

public class JarRunner {
private static final String BUNDLE_MAIN_CLASS = "Bundle-MainClass";

public static void main(String[] args) {
try {
invokeClass(getMainClassName(), args);
} catch (Throwable e) {
e.printStackTrace();
}
}

public static String getMainClassName() throws IOException {
URLClassLoader ucl = null;
try {
ucl = (URLClassLoader)Class.forName("qdolan.loader.JarRunner").getClassLoader();
}
catch(ClassNotFoundException cnfe) {
}
java.net.URL rootUrl = ucl.getURLs()[0];
if (rootUrl.getFile().toLowerCase().endsWith(".jar")) {
URL rootJarUrl = new URL("jar", "", rootUrl + "!/");
JarURLConnection uc = (JarURLConnection) rootJarUrl.openConnection();
Attributes attr = uc.getMainAttributes();
return attr != null ? attr.getValue(BUNDLE_MAIN_CLASS) : null;
}
throw new IllegalStateException("JarRunner is not inside a jar file");
}

public static void invokeClass(String name, String[] args)
throws ClassNotFoundException, NoSuchMethodException, InvocationTargetException {
final String mainMethodName = "main";
JarClassLoader loader = new JarClassLoader(((URLClassLoader) ClassLoader.getSystemClassLoader()).getURLs(),
Thread.currentThread().getContextClassLoader());
Class clazz = loader.loadClass(name);
Method method = clazz.getMethod(mainMethodName, new Class[] { args.getClass() });
method.setAccessible(true);
int mods = method.getModifiers();
if (method.getReturnType() != void.class || !Modifier.isStatic(mods) || !Modifier.isPublic(mods)) {
throw new NoSuchMethodException(mainMethodName);
}
try {
method.invoke(null, new Object[] { args });
} catch (IllegalAccessException e) {
// This should not happen, as we have
// disabled access checks
}
}
}
And the manifest lines for ant (tags removed because of stupid restrictions on tags) :

name="Main-Class" value="qdolan.loader.JarRunner"/
name="Bundle-MainClass" value="web.ftp.FtpSave"

Lance N. said...

What is the license for this code? I would like to use it in some Apache projects.

Q said...

You may use this code under an Apache license.

Lance N. said...

Thank you. Where in the world are you? I wasn't expecting a reply so quickly at my 2am.

Q said...

Australia

Anonymous said...

This code is similar to the one that I created, however, the file system is a bottle neck and slows down large deployable jars.

Check out the following eclipse source for creating a dynamic self contained jar loader that does not require copying the jar to a temp file: http://javasourcecode.org/html/open-source/eclipse/eclipse-3.5.2/org/eclipse/jdt/internal/jarinjarloader/

Q said...

Anonymous, I actually use your eclipse internal solution in one of my projects already and had planned to use the same technique for an old project that didn't work well with my previous approach. And yes, I agree, for very large projects, or when running multiple instances using the filesystem becomes a problem.

Anonymous said...

Apache Licence Requires Attribution. Can you please provide some guidance on who we should be atttributing?

Q said...

Anonymous,

You can attribute this code to Quinton Dolan (qdolan at gmail).