File Handles in the JVM

I recently spent three or four fruitless hours (spread across several days) trying to get a bunch of Fit tests to run reliably. The setup fixture was used to clean out the folders in which the program under test was intended to find its input data and write its output data. Subsequent fixtures created the input files, executed the program under test, and analysed the output files.

So far so good, but as the number of tests gradually increased we started to get failures apparently at random. I suspected that the setup fixture was not deleting all the files out of the input and output folders, so increased the rigour of this method by invoking the FileUtils deleteDirectory method and throwing a runtime exception if it failed even after five retries.

It turned out that this was frequently failing because some process was keeping file handles open on files within the test folder hierarchy. Under Windows, this prevents the containing folder from being deleted or moved. We suspected virus checkers, search indexers and the like, but it turned out to be the Fit test harness itself. Even though we had taken great care to flush and close every file stream used by our Fit fixtures, the VM was not always releasing the corresponding file handles and so the next invocation of the setup fixture failed.

The answer turned out to be surprisingly simple: invoke the garbage collector and finalizer before retrying the call to deleteDirectory. Now it works every time!

Code sample:

...
import static org.apache.commons.io.FileUtils.deleteDirectory;
...
    public static void zapDirectory(String path) {
        ...
        File directory = new File(path);
        for (int i = RETRY_COUNT; directory.exists(); i--) {
            try {
                deleteDirectory(directory);
            } catch(IOException e) {
                if (i <= 0) {
                    throw new RuntimeException("Cannot delete folder \"
                        + path + "\" because: " + e.getMessage());
                }
                try {
                    System.gc();
                    System.runFinalization();
                    Thread.sleep(SLEEP_INTERVAL);
                } catch(InterruptedException e1) {
                    // don't care if interrupted
                }
            }
            if (directory.exists()) {
                throw new RuntimeException("Cannot delete folder \"
                    + path + "\" for an unknown reason");
            }
        }
    }