Using Java as Native Linux Apps - Calling C, Daemonization, Packaging, CLI (Brian McCallister)
This is a summary of the excellent JavaZone 2012 talk Going Native (vimeo) by Brian McCallister. Content: Using native libraries in Java and packaging them with Java apps, daemonization, trully executable JARs, powerful CLI, creating manpages, packaging natively as deb/rpm.
JNR and JNA use the functions dlopen, dlsym to dynamically load a shared library and symbols (i.e. functions) in it.
The JNA code belows shows how to load the library MagickWand (/usr/lib/libMagickWand.so) and call some methods mapped to native functions on it, e.g. MagickWandGenesis.
See Getting Started with JNA for more examples, this time based on JNA.
Leveldbjni uses JNI to access the native leveldb library and Hawt-JNI to generate JNI code from annotations (instead of using the more user-friendly JNA or JNR) and to include it inside its JAR and to access it.
The auto-loading of the packaged native library is done by the Hawt-JNI call
- it does the same thing as
You might want to check out the example Hawt-JNI maven project and perhaps alos How to Add HawtJNI to an Existing Maven Build. Notice you don't need to use Hawt to build you library, you can use is as well to only package it in (goal package-jar) and to load it.
This is how a maven library with baked-in native libs looks like:
Isn't that cool?
You can use Brian's Maven plugin really-executable-jars to create such an executable JAR for you.
End result:
Check out Brian's project Dwarf (link at the bottom of this post) for a practical example using the Maven plugin.
And the use of it:
That's all, folks! Go and watch the Going Native video!
1. Using Native Libs in Java
Calling Native Libs
Calling native libraries such as C ones was hard and ugly with JNI but is very simple and nice with JNA (GPL) and JNR (Apache/LGPL)JNR and JNA use the functions dlopen, dlsym to dynamically load a shared library and symbols (i.e. functions) in it.
The JNA code belows shows how to load the library MagickWand (/usr/lib/libMagickWand.so) and call some methods mapped to native functions on it, e.g. MagickWandGenesis.
// 1. The interfaceto use a dynamically loaded native lib:
import jnr.ffi.Pointer;
public interface MagickWand {
public void MagickWandGenesis();
/* MagickWand* */ Pointer NewMagickWand();
int MagickReadImage(Pointer wand, String path);
}
// 2. Using it:
int MagickFalse = 0;
final MagickWand w =
Library.loadLibrary("MagickWand", MagickWand.class);
w.MagickWandGenesis();
Pointer magick_wand = w.NewMagickWand();
int rs = w.MagickReadImage(magick_wand, "bunny.jpg");
if (rs == MagickFalse) {
fail("bad exit code");
}
See Getting Started with JNA for more examples, this time based on JNA.
Packaging Native Libs with Your Java App
Ok, so we can call a native library but how we get to it? One way to ensure the native lib is on the target system is to package it with your java application. Hawt-JNI is a library and Maven plugin that (among others) makes it possible to build a native library, attach it to you application and load it at runtime.Leveldbjni uses JNI to access the native leveldb library and Hawt-JNI to generate JNI code from annotations (instead of using the more user-friendly JNA or JNR) and to include it inside its JAR and to access it.
The auto-loading of the packaged native library is done by the Hawt-JNI call
private static final Library LIBRARY = new Library("simple", Simple.class);
static {
LIBRARY.load();
}
- it does the same thing as
System.loadLibrary("simple") but also extracts the library from the JAR and loads if from there (copied from the doc).You might want to check out the example Hawt-JNI maven project and perhaps alos How to Add HawtJNI to an Existing Maven Build. Notice you don't need to use Hawt to build you library, you can use is as well to only package it in (goal package-jar) and to load it.
This is how a maven library with baked-in native libs looks like:
$ unzip -l leveldbjni-all-1.2.jar | grep libleveldbjni
655754 02-27-12 12:44 META-INF/native/linux32/libleveldbjni.so
707423 02-27-12 12:44 META-INF/native/linux64/libleveldbjni.so
446052 02-27-12 12:44 META-INF/native/osx/libleveldbjni.jnilib
2. Daemonization
Daemonization of Java apps wasn't easy. You'd often use commons-daemon or create a wrapper script leveraging e.g. Ubuntu's start-stop-daemon. Daemonization is difficult because you need to do lot of stuff that is difficult from the JVM: fork, setsid so you parent doesn't own you, dup to protect your files, fork again, …. Fork in Java is risky (you'd miss all system threads such as gc; risk of incosistencies), the rest can be faked via posix - but you can basically use posix_spawn instead (via jnr-ffi). Read on!Gressil: The Daemonization Library (~25min in the talk)
Gressil is a Java library created by Brian for simple and nice daemonization of Java apps (it basically spawns a new JVM via jnr-ffi, handles pid file, stdin/stderr/out etc.)
public static void main(String[] args) throws IOException
{
new Daemon().withMainArgs(args)
.withPidFile(new File("/tmp/chatty.pid"))
.withStdout(new File("/tmp/chatty.out"))
.withExtraMainArgs("hello", "world,")
.withExtraJvmArgs(remoteDebugOnPort(5005))
.daemonize();
...
}
Isn't that cool?
3. Using Java As a Proper Linux Program (27min)
JAR is just a ZIP file and zip allows for arbitrary crap at the beginning (until a magic number) => you can prepend e.g. a shell script to expand the jar (beware: you need the empty line after the exec otherwise jar gets confused):
$ head -4 ./target/dwarf # this is actually a .jar
#!/bin/sh
exec java -jar "$0" "$@"
$
You can use Brian's Maven plugin really-executable-jars to create such an executable JAR for you.
End result:
# Instead of executing this:
$ java -jar ./dwarf.jar --waffles=yes
# You can execute this:
$ dwarf --waffles=yes
Check out Brian's project Dwarf (link at the bottom of this post) for a practical example using the Maven plugin.
4. Better CLI With Airline (~31 min)
Airline is a Java library for creating better command-line interfaces. It has also support for commands (such as git's git add, git log) that are implemented by individual classes. It even generates bash autocompletion.
public static void main(String[] args)
{
CliBuilder builder = Cli.buildCli("git", Runnable.class)
.withDescription("the stupid content tracker")
.withDefaultCommand(Help.class)
.withCommands(Help.class, Add.class);
builder.withGroup("remote")
.withDescription("Manage set of tracked repositories")
.withDefaultCommand(RemoteShow.class)
.withCommands(RemoteShow.class, RemoteAdd.class);
Cli gitParser = builder.build();
gitParser.parse(args).run();
}
// And the add command implementation:
@Command(name="add", description="Add file contents to the index")
public static class Add implements Runnable
{
@Option(type=OptionType.GLOBAL, name="-v", description="Verbose mode")
public boolean verbose;
@Arguments(description = "Patterns of files to be added")
public List patterns;
@Option(name="-i", description="Add modified contents interactively.")
public boolean interactive;
public void run()
{
System.out.println(getClass().getSimpleName());
}
}
And the use of it:
$ git help
usage: git [-v] <command> []
The most commonly used git commands are:
add Add file contents to the index
help Display help information
remote Manage set of tracked repositories
See 'git help <command>' for more information on a specific command.
5. Creating Manpages
The Man pages native format is terrible but fortunately you can use Markdown and the command-line utility ronn to turn it into a manpage. (It basically turns md into html and then via some black magic incantantions into grof.)6. Packaging (37min)
You have an executable, a man page, perhaps some scripts and now you want to install them on the target system. The best way is to use its native package manager.- .deb: mvn plugin jdeb (deb ~ tar)
- .rpm: mvn plugin rpm-maven-plugin, needs locally installed rpmbuild
- mac: a mess, use e.g. homebrew manually
Example Project: Dwarf
Brian used many of this techniques in his open source project Dwarf so check it out. It uses f.ex jdeb, manpage generation from Markdown via ronn (manually), executable jar.That's all, folks! Go and watch the Going Native video!
Related
Brian's posts- Java Daemonization with posix_spawn(2), also introducing Gressil
- POSIX from Java - proper access to POSIX and libc system calls, jnr-posix
- Making Really Executable Jars