Wednesday, 29 October 2008

Java Native Access

Every Java programmer knows about JNI, but what about JNA?

I'm pretty impressed with this. The documentation seems pretty comprehensive and the examples are well constructed. The software seems very stable. Obviously you can kill the VM if you dereference invalid pointers etc., but after a short period of familiarisation, I found that I could knock together a Java API for a subset of an existing C/C++ DLL (kernel32), using just the published API documentation to guide me, in very little time. I wrote a little test program that gets the compressed and uncompressed size of arbitrary files, and prints proper error messages for files it cannot open.

I make no claim that the example code is particularly elegant, but it shows what you can do. One great advantage is that you don't need to install a separate C/C++ development environment if a DLL already exists that gives you the needed native functionality.

Note that JNI is built for raw speed - some applications use it for example to perform graphics manipulations thousands of times a second. Using JNA, because it uses inspection and all kinds of clever tricks, you need to budget about ten times as much execution time per transition from Java to native machine code and back again. For many applications, that's irrelevant because native code is only called infrequently (up to a few hundred times a second).

7 comments:

Immo Hüneke said...

Here is the code. Sorry about the formatting; just paste into Eclipse and use SHIFT-CTRL-F!

package com.zuhlke.jna.examples;

import com.sun.jna.ptr.IntByReference;
import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.NativeLong;
import com.sun.jna.Pointer;
import com.sun.jna.Structure;
import com.sun.jna.examples.win32.W32API.HANDLE;
import com.sun.jna.ptr.NativeLongByReference;
import com.sun.jna.win32.StdCallLibrary;

/** Simple example of native library declaration and usage. */
public class Kernel32Example {

public interface Kernel32Library extends StdCallLibrary {
public static final int FORMAT_MESSAGE_FROM_SYSTEM = 0x1000;
public static final int FORMAT_MESSAGE_ALLOCATE_BUFFER = 0x100;
public static final int FILE_SHARE_READ = 1;
public static final int FILE_SHARE_WRITE = 2;
public static final int CREATE_NEW = 1;
public static final int CREATE_ALWAYS = 2;
public static final int OPEN_EXISTING = 3;
public static final int OPEN_ALWAYS = 4;
public static final int TRUNCATE_EXISTING = 5;
public static final Pointer INVALID_HANDLE_VALUE = new IntByReference(-1).getPointer().getPointer(0);

Kernel32Library INSTANCE = (Kernel32Library) Native.loadLibrary("kernel32",
Kernel32Library.class);
// Optional: wraps every call to the native library in a
// synchronized block, limiting native calls to one at a time
Kernel32Library SYNC_INSTANCE = (Kernel32Library)
Native.synchronizedLibrary(INSTANCE);

// DWORD WINAPI GetLastError(void);
public int GetLastError();

// DWORD WINAPI FormatMessage(
// __in DWORD dwFlags,
// __in_opt LPCVOID lpSource,
// __in DWORD dwMessageId,
// __in DWORD dwLanguageId,
// __out LPTSTR lpBuffer,
// __in DWORD nSize,
// __in_opt va_list *Arguments
// );
public int FormatMessageA(int dwFlags, Pointer lpSource,
int dwMessageId, int dwLanguageId,
String[] lpBuffer, int nSize,
Pointer arguments);

// HANDLE WINAPI CreateFile(
// __in LPCTSTR lpFileName,
// __in DWORD dwDesiredAccess,
// __in DWORD dwShareMode,
// __in_opt LPSECURITY_ATTRIBUTES lpSecurityAttributes,
// __in DWORD dwCreationDisposition,
// __in DWORD dwFlagsAndAttributes,
// __in_opt HANDLE hTemplateFile
// );
public HANDLE CreateFileA(String lpFileName, int dwDesiredAccess,
int dwShareMode, Pointer lpSecurityAttributes,
int dwCreationDisposition, int dwFlagsAndAttributes,
HANDLE hTemplateFile);

// DWORD WINAPI GetFileSize(
// __in HANDLE hFile,
// __out_opt LPDWORD lpFileSizeHigh
// );
public int GetFileSize(HANDLE hFile, NativeLongByReference lpFileSizeHigh);

// DWORD WINAPI GetCompressedFileSize(
// __in LPCTSTR lpFileName,
// __out_opt LPDWORD lpFileSizeHigh
// );
public int GetCompressedFileSizeA(String lpFileName, NativeLongByReference lpFileSizeHigh);

}

public static void main(String[] args) {
if(args.length < 1) {
System.err.println("Usage: Kernel32Example <pathname>\n");
System.exit(1);
}
String fileName = args[0];
System.out.println("Checking size of file " + fileName + "\n");
HANDLE nullHandle = new HANDLE();
HANDLE fh = Kernel32Library.INSTANCE.CreateFileA(fileName, 0,
Kernel32Library.FILE_SHARE_READ + Kernel32Library.FILE_SHARE_WRITE, Pointer.NULL,
Kernel32Library.OPEN_EXISTING, 0,
nullHandle);
if (Kernel32Library.INVALID_HANDLE_VALUE.equals(fh.getPointer())) {
analyseWinError();
}
NativeLongByReference lpFileSizeHigh = new NativeLongByReference(new NativeLong());
int fileSize = Kernel32Library.INSTANCE.GetFileSize(fh, lpFileSizeHigh);
if (fileSize < 0) {
analyseWinError();
}
NativeLongByReference lpComprFileSizeHigh = new NativeLongByReference(new NativeLong());
int comprFileSize = Kernel32Library.INSTANCE.GetCompressedFileSizeA(fileName, lpComprFileSizeHigh);
if (comprFileSize < 0) {
analyseWinError();
}
System.out.print("Size: " + String.valueOf(fileSize));
System.out.println(", Compressed Size: " + String.valueOf(comprFileSize));
}

private static void analyseWinError() {
int errorCode = Kernel32Library.INSTANCE.GetLastError();
String[] lpBuffer = new String[1];
int msgLen = Kernel32Library.INSTANCE.FormatMessageA(
Kernel32Library.FORMAT_MESSAGE_ALLOCATE_BUFFER + Kernel32Library.FORMAT_MESSAGE_FROM_SYSTEM,
Pointer.NULL,
errorCode, 0,
lpBuffer, 80,
Pointer.NULL);
System.out.flush();
System.err.println("Error code " + String.valueOf(errorCode));
if (msgLen > 0) {
System.err.println(lpBuffer[0]);
}
System.err.flush();
}
}

Immo Hüneke said...

I was worried about one particular aspect of JNA and posted the following query on the users mailing list. As you can see from the reply, it's very much a case of caveat emptor. If you know what you're doing (and more importantly, what your called DLL is doing), it is perfectly safe.

When using JNI, there is a well known method of crashing the JVM by changing the value of the FPU control word within the native code and failing to restore it. Whenever the VM executes the so-called “Safepoint Blob” to switch threads, it has to pop all the floating-point registers into a thread context area and does not expect the FPU to raise an exception when it underflows. Some native libraries need to unmask floating-point exceptions, so to honour the JNI contract they should restore the CW register afterwards (a relatively slow operation, which is why the JNI implementation doesn’t do this).

I can’t find any documentation on this issue where JNA is concerned. Does JNA safeguard the FPUCW each time execution passes between Java and native code, or does the native library have to conform to a contract similar to JNI? Are there any documented instances of VMs crashing sporadically with error messages along the following lines?

* Java HotSpot(TM) Client VM warning: Floating point control word changed by native JNI code.
* An unexpected error has been detected by HotSpot Virtual Machine: EXCEPTION_ACCESS_VIOLATION
* EXCEPTION_FLT_DIVIDE_BY_ZERO
* EXCEPTION_FLT_STACK_CHECK

More background: Sun's bug database numbers 6346124 and 5105765

Timothy Wall replied:

JNA doesn't do anything to preserve the control word state. No one's reported errors, but that would be dependent on whether anyone is using a library which changes the control word state (same as JNI).

Having an optional save/restore option on a per-call basis in JNA might be preferable to the global JVM workarounds, but none exists at the moment.

Carlos L. said...

Thanks for yor comment, but your code does not work across processes.

Running the code in 2 java machines, it will not operate.
The reason (apparent) is that your code generates a new instance of kernel32, rather than calling the windows allocated kernel32.dll.

I wrote a DLL in c++ that forwards the calls to windows.

Here is the code, if someone likes to use it:

-----------------------------------

c++ dll
-----------------------------------

BOOL APIENTRY DllMain( HANDLE hModule,
DWORD ul_reason_for_call,
LPVOID lpReserved
)
{
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
case DLL_PROCESS_DETACH:
break;
}
return TRUE;
}




extern "C" __declspec(dllexport)int EcuOpenMutexA(unsigned long permissions, int initialOwner, LPSTR mutexName, int*errorCode)
{
HANDLE handle;
handle = OpenMutexA(permissions, FALSE, mutexName);
*errorCode = GetLastError();
return (unsigned long)handle;
}


extern "C" __declspec(dllexport)int EcuCreateMutexA(LPSECURITY_ATTRIBUTES permissions, int initialOwner, LPSTR mutexName, int*errorCode)
{
HANDLE handle;
handle = CreateMutexA(permissions, FALSE, mutexName);
*errorCode = GetLastError();
return (int)handle;

}

extern "C" __declspec(dllexport)int EcuWaitForSingleObject(int handle, int elapsed, int*errorCode)
{
int retCode;
retCode = WaitForSingleObject((HANDLE)handle, elapsed);
*errorCode = GetLastError();
return retCode;
}

extern "C" __declspec(dllexport)int EcuReleaseMutex(int handle, int*errorCode)
{
int retCode;
retCode = ReleaseMutex((HANDLE)handle);
*errorCode = GetLastError();
return retCode;
}

extern "C" __declspec(dllexport)int EcuCloseHandle(int handle, int*errorCode)
{
int retCode;
retCode = CloseHandle((HANDLE)handle);
*errorCode = GetLastError();
return retCode;
}


// This is the constructor of a class that has been exported.
// see EcuMutex.h for the class definition
CEcuMutex::CEcuMutex()
{
return;
}
-----------------------------
EcuMutex.h
-----------------------------
#ifdef ECUMUTEX_EXPORTS
#define ECUMUTEX_API __declspec(dllexport)
#else
#define ECUMUTEX_API __declspec(dllimport)
#endif

// This class is exported from the EcuMutex.dll
class ECUMUTEX_API CEcuMutex {
public:
CEcuMutex(void);
// TODO: add your methods here.
};
--------------------------
java
--------------------------

private interface IEcuMutex32 extends StdCallLibrary
{
int EcuCreateMutexA(Pointer security, int initialOwnership, byte[] name, IntByReference errorCode);
int EcuOpenMutexA (int privileges, int initialOwnership, byte[] name, IntByReference errorCode);
int EcuReleaseMutex(int handle, IntByReference errorCode);

public static final int INIFITE = -1;
public static final int SYNCHRONIZE = 0x100000;
public static final int WAIT_ABANDONED = 0x80;
public static final int WAIT_OBJECT_0 = 0x0;
public static final int WAIT_TIMEOUT = 0x102;

int EcuWaitForSingleObject(int handle, int timeout, IntByReference errorCode);
int EcuCloseHandle(int handle, IntByReference errorCode);

public static final int ERROR_FILE_NOT_FOUND = 2;
public static final int ERROR_ACCESS_DENIED = 5;
public static final int ERROR_ALREADY_EXISTS = 183;
public static final int FORMAT_MESSAGE_FROM_SYSTEM = 0x1000;
public static final int FORMAT_MESSAGE_ALLOCATE_BUFFER = 0x100;
public static final int FORMAT_MESSAGE_IGNORE_INSERTS = 0x200;
public static final int FILE_SHARE_READ = 1;
public static final int FILE_SHARE_WRITE = 2;
public static final int CREATE_NEW = 1;
public static final int CREATE_ALWAYS = 2;
public static final int OPEN_EXISTING = 3;
public static final int OPEN_ALWAYS = 4;
public static final int TRUNCATE_EXISTING = 5;
public static final Pointer INVALID_HANDLE_VALUE = new IntByReference(-1).getPointer().getPointer(0);


}

private static void init()
{
if(onceDone)
{
return;
}
onceDone = true;
ecuMutex32 = unSyncEcuMutex32 = (IEcuMutex32) Native.loadLibrary("EcuMutex", IEcuMutex32.class);
if(ecuMutex32 == null)
{
throw new Error("Imposible instanciar la clase Kernel32");
}
// Optional: wraps every call to the native library in a
// synchronized block, limiting native calls to one at a time
// Aquí eso es innecesario, ya que las rutinas que utilizan ésto son sincronizadas.
// --- ecuMutex32 = (IEcuMutex32) Native.synchronizedLibrary(unSyncEcuMutex32);

if(ecuMutex32 == null)
{
throw new Error("Imposible establecer instancia sincronizada de (IKernel32)kernel32");
}
}

Immo Hüneke said...

I bow to your superior knowledge! Thanks Carlos.

Carlos L. said...

such a quick reply, I am surprised.
Isn't the internet weird!

Thanks for sharing your experience, Y have used more than one of your posts.

Carlos Lauterbach said...

Hi,
I found, later, that the problem occurs when using ECLIPSE.
The suggested intemediate DLL makes kernel32 work in all environments.

kimberly said...
This comment has been removed by a blog administrator.