Buggy MemoryMapping in the JDK

I just ran into a problem using memory-mapped byte buffers on Java 5. The basic use case is read from a socket, write to a file, and then map the resulting file into memory to perform digest calculations, etc. across the entire file. It works the first time, but any subsequent attempt to rewrite the file will fail with the message The requested operation cannot be performed on a file with a user-mapped section open.

After some searching, I found that this is a relatively common problem, and it is a facet of the way that memory mapping actually works on the underlying OS. It seems you can have speed, or safety, but not both. The issue is with the current impossibility of producing a reliable and platform-independent unmap command. An example of unmap() at the C level can be seen here.

In fairness to Sun, it’s probably not fair to blame them for the existance of this problem – it seems to be currently intractable. The end of their evaluation note in the Bug Parade entry for this bug reads:

We at Sun have given this problem a lot of thought, both during the original
development of NIO and in the time since. We have yet to come up with a way to
implement an unmap() method that’s safe, efficient, and plausibly portable
across operating systems. We’ve explored several other alternatives aside from
the two described above, but all of them were even more problematic. We’d be
thrilled if someone could come up with a workable solution, so we’ll leave this
bug open in the hope that it will attract attention from someone more clever
than we are.

Just for reference, the code that creates the buffer is shown below:


 FileChannel fc = new FileInputStream(requestedFile).getChannel();
       int sz = (int)fc.size();
       
       MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, sz);
       
       byte buffer[] new byte[sz];
       bb.get(buffer, 0, sz);

ClassCastExceptions and Hibernate mappings

I just had an issue where Hibernate was throwing ClassCastExceptions when trying to persist an entity to the database. I tracked the problem down a specific property – a byte[]. I had just switched my entity mappings from a hbm.xml file to an annotations-based approach. The solution was fairly straightforward, as it turned out – just specify the mapping type explicitly as “binary”. It seems that Hibernate may be attempting to map it as a blob type, which doesn’t map transparently to primitive byte arrays (yet – I see there is a PrimitiveByteArrayBlobType in the Hibernate Annotations API). Meanwhile, I just declare the mapping like so:




  @Column
  @Type(type="binary")
  public byte[] getRawData() {
    return rawData;
  }

Eclipse 3.1 M6 and Generics

The Eclipse compiler seems to choke on the following code, which is some generics-related issue. Consider the following:

private Set<ActionState> stateHistory = new TreeSet<ActionState>();

Now, I know a TreeSet will be stored in natural ascending order, but just consider the case where you might try to do this:

@Transient
public ActionState getCurrentState() {
if (stateHistory == null || stateHistory.size() == 0) {
return null;
}
return Collections.max(stateHistory);
}

Eclipse chokes with the error:


Bound mismatch: The generic method max(Collection< ? extends T>) of type Collections is not
applicable for the arguments (Collection< ? extends ActionState>) since the type ActionState is not
a valid substitute for the bounded parameter >

This is confusing, as ActionState implements Comparable.

If I change the offending line to read:

return ((TreeSet<ActionState>)stateHistory).last();

It works fine. It looks like it may be a bug in Eclipse’s compiler, as colleagues using IntelliJ IDEA have no such problems.

Log4j XML Configuration

For some reason, I can’t seem to retain a key piece of information – how to configure the logging level per-package in the log4j XML configuration. I’m putting this snippet here to cater for the likely event that I’ll forget it again. It’s all to do with the category element, as it happens.

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">

<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">

  <appender name="ConsoleAppender" class="org.apache.log4j.ConsoleAppender">
    <layout class="org.apache.log4j.SimpleLayout"/>
  </appender>

  <category name="org.apache">
        <priority value="warn"/>
  </category>

  <root>
    <priority value="debug" />
    <appender-ref ref="ConsoleAppender"/>
  </root>

</log4j:configuration>

Spring, Hibernate and LazyInitializationExceptions

I’ve recently started using Spring for its comprehensive support for Hibernate 3. It makes writing DAOs a snap. There are a couple of wrinkles, however, one is the fact that lazy loading in Hibernate 3 is the default, combined with the fact that Spring will close the Hibernate Session automatically, giving you a LazyInitializationException when you try to access mapped fields from your persistent objects. The solution to this is the “Open Session In View” pattern, which Spring provide a ready-made filter for. Shown below is an extract from my PersistentTestCase, superclass, which all my DAO tests extend, and voila, no more Lazy Exceptions.



protected void setUp() throws ClassNotFoundException, SQLException {
      ClassPathXmlApplicationContext appContext = new ClassPathXmlApplicationContext("application_context.xml");
      factory = (BeanFactoryappContext;
      
      sessionFactory = (SessionFactory)factory.getBean("test.sessionFactory");
      Session s = sessionFactory.openSession();
      TransactionSynchronizationManager.bindResource(sessionFactory, new SessionHolder(s));  
  }
  
  protected void tearDown() {
     SessionHolder sessionHolder = (SessionHolderTransactionSynchronizationManager.unbindResource(sessionFactory);
     SessionFactoryUtils.closeSessionIfNecessary(sessionHolder.getSession(), sessionFactory)
  }

Some Subversion Annoyances

A few things really bug me about Subversion. Overall, I like it, and will continue to use it, but there have been a few casualties along the way from the CVS-to-Subversion migration:

  • CVSGraph doesn/t work anymore. I know TSVN has a built-in revision viewer, but a) the graphs aren’t as nice, and b) have you ever tried it on a tree with a large number of revisions? Good luck!
  • The Eclipse CVS plugin. Now, I know that you can download SubClipse, but it’s extra effort, and it’s not as mature, or stable, as the built-in CVS equivalent. Plus, you need to download extra dependencies to get it to work.
  • Those nice StatCVS reports won’t work any more, either. Which I found out when a maven:site command bombed out 75% of the way through.

I know these are relatively small issues, and I’m sure they will be resolved in time, but they are annoying.

Ant Subversion Task, Part 2

I have an updated version of the Ant Subversion task that supports commits. Note to self: look at the DAV protocol spec to find out more about what is actually going on under the hood. The one snag I hit is that you won’t always have a valid Repository instance (e.g. when committing), as the SVN root is read from the administrative files, so you need to handle the credential passing explicitly for this case. Here’s a sample build.xml showing what works now:

<?xml version=”1.0″?>

<project name=”svn” default=”main” basedir=”.”>

<property name=”foo” value=”bar”/>

<target name=”main”>
<taskdef name=”svn” classname=”uk.co.researchkitchen.javasvn.ant.SvnTask” />
<delete dir=”c:/temp/sample” failonerror=”false”/>

<svn command=”checkout” svnRoot=”http://localhost/svn/sample/trunk” dest=”c:/temp/sample” revision=”10″ verbose=”true”/>

<svn command=”update” dest=”c:/temp/sample” verbose=”true”/>

<svn command=”commit” dest=”c:/temp/sample” verbose=”true” message=”hello ${foo}”/>

</target>
</project>

The code is here: SvnTask.java

I must say, I’m really impressed with the JavaSVN library – it looks pretty powerful.

Ant Subversion Task

One thing that Ant lacks is a built-in Subversion task. I am using Subversion now for almost everything (work + Open Source stuff), and find the lack of a built-in task a bit of a pain. With this in mind, I decided to make a first cut at writing an Ant task for Subversion based on the JavaSVN library. So far, it seems to be coming along well. For a sample build.xml:

<target name=”main”>
<taskdef name=”svn”
classname=”uk.co.researchkitchen.javasvn.ant.SvnTask” />
<delete dir=”c:/temp/sample” failonerror=”false”/>
<svn command=”checkout” svnRoot=”http://localhost/svn/sample/trunk” dest=”c:/temp/sample” revision=”10″/>
</target>

It will produce:

Buildfile: C:\sandbox\javasvn-src-0.8.7.2\svn-ant.xml
main:
[delete] Deleting directory C:\temp\sample
[svn] Checking out /svn/sample/trunk to C:\temp\sample
[svn] Checked out /svn/sample/trunk [revision 10]
BUILD SUCCESSFUL
Total time: 9 seconds

The code is here, if you’re interested (or have any suggestions). I’m not sure about the Reflection-based method delegation mechanism, I may replace this. This was built with Java 5.0, but using an IDE (Eclipse 3.0) that doesnt support Tiger semantics fully yet.

package uk.co.researchkitchen.javasvn.ant;

import java.io.File;
import java.lang.reflect.Method;
import java.util.HashMap;
import java.util.Iterator;

import org.apache.tools.ant.BuildException;
import org.apache.tools.ant.Task;
import org.tmatesoft.svn.core.ISVNWorkspace;
import org.tmatesoft.svn.core.SVNWorkspaceManager;
import org.tmatesoft.svn.core.internal.io.dav.DAVRepositoryFactory;
import org.tmatesoft.svn.core.internal.ws.fs.FSEntryFactory;
import org.tmatesoft.svn.core.io.SVNException;
import org.tmatesoft.svn.core.io.SVNRepository;
import org.tmatesoft.svn.core.io.SVNRepositoryFactory;
import org.tmatesoft.svn.core.io.SVNRepositoryLocation;
import org.tmatesoft.svn.core.io.SVNSimpleCredentialsProvider;

/**
 * 
 * @author rwinston
 *
 * Ant task that encapsulates some simple Subversion functionality (e.g. checkout) in pure Java, using the
 * JavaSVN libraries from http://tmate.org/svn/
 * 
 * &lt;svn commmand="checkout" username="foo" password="bar" svnRoot="http://localhost/svn/sample/trunk" 
 * dest="/usr/sandbox/sample" revision="1" /&gt;
 * 
 */
public class SvnTask extends Task {

  // The URL of the project we are checking out (e.g. http://localhost/svn/sample/trunk)
  private String svnRoot = null;

  // The destination directory for a checkout
  private File dest = null;

  // The revision to check out
  private long revision = ISVNWorkspace.HEAD;

  // The default command to execute
  private static final String DEFAULT_COMMAND = "checkout";

  // The SVN command to execute
  private String command = null;

  // Credentials
  private String username = null;

  private String password = null;

  boolean usingCredentials = false;

  // The SVN repository location
  private SVNRepositoryLocation location = null;

  // The repository representation
  private SVNRepository repository = null;

  /**
   * 
   */
  public SvnTask() {
    setTaskName("svn");
  }

  /**
   * @return Returns the password.
   */
  public String getPassword() {
    return password;
  }

  /**
   * @param password The password to set.
   */
  public void setPassword(String password) {
    this.password = password;
  }

  /**
   * @return Returns the username.
   */
  public String getUsername() {
    return username;
  }

  /**
   * @param username The username to set.
   */
  public void setUsername(String username) {
    this.username = username;
  }

  /**
   * @return Returns the command.
   */
  public String getCommand() {
    return command;
  }

  /**
   * @param command The command to set.
   */
  public void setCommand(String command) {
    this.command = command;
  }

  public long getRevision() {
    return revision;
  }

  public void setRevision(long revision) {
    this.revision = revision;
  }

  /**
   * @return Returns the dest.
   */
  public File getDest() {
    return dest;
  }

  /**
   * @param dest The dest to set.
   */
  public void setDest(File dest) {
    this.dest = dest;
  }

  /**
   * @return Returns the svnRoot.
   */
  public String getSvnRoot() {
    return svnRoot;
  }

  /**
   * @param svnRoot The svnRoot to set.
   */
  public void setSvnRoot(String svnRoot) {
    this.svnRoot = svnRoot;
  }

  /**
   * Set up and execute the relevant SVN command 
   *
   */
  public void execute() throws BuildException {

    if (command == null || command.equals("")) {
      command = DEFAULT_COMMAND;
    }

    if (username != null && password == null) {
      log("Password cannot be null if username is defined!");
      return;
    } else {
      usingCredentials = true;
    }

    try {
      // Initialize the DAV and local FS factories
      DAVRepositoryFactory.setup();
      FSEntryFactory.setup();

      location = SVNRepositoryLocation.parseURL(svnRoot);
      repository = SVNRepositoryFactory.create(location);

      if (usingCredentials)
        repository.setCredentialsProvider(new SVNSimpleCredentialsProvider(username, password));

      if (dest == null) {
        dest = getProject().getBaseDir();
      }

      if (!dest.exists()) {
        dest.mkdirs();
      }

      // Execute the relevant SVN command  
      dispatchCommand(command);

    } catch (SVNException e) {
      throw new BuildException(e, getLocation());
    }
  }

  /**
   * Dispatch to a named method based on a command name (or abbreviation)
   * @param command
   * @throws BuildException
   */
  private void dispatchCommand(String command) throws BuildException {

    final HashMap commands = new HashMap();

    // Map of commands and short names ( See org.tmatesoft.svn.cli.SVNCommand for details)
    commands.put(new String[] { "status", "st", "stat" }, "doStatus");
    commands.put(new String[] { "import" }, "doImport");
    commands.put(new String[] { "checkout", "co" },  "doCheckout");
    commands.put(new String[] { "add" }, "doAdd");
    commands.put(new String[] { "commit", "ci" }, "doCommit");
    commands.put(new String[] { "update", "up" }, "doUpdate");
    commands.put(new String[] { "delete", "rm", "remove", "del" }, "doDelete");
    commands.put(new String[] { "move", "mv", "rename", "ren" }, "doMove");
    commands.put(new String[] { "copy", "cp" }, "doCopy");
    commands.put(new String[] { "revert" }, "doRevert");
    commands.put(new String[] { "mkdir" }, "doMkdir");
    commands.put(new String[] { "propset", "pset", "ps" }, "doPropSet");
    commands.put(new String[] { "propget", "pget", "pg" }, "doPropGet");
    commands.put(new String[] { "proplist", "plist", "pl" },   "doPropList");
    commands.put(new String[] { "info" }, "doInfo");
    commands.put(new String[] { "resolved" }, "doResolved");
    commands.put(new String[] { "cat" }, "doCat");
    commands.put(new String[] { "ls" }, "doLs");
    commands.put(new String[] { "log" }, "doLog");
    commands.put(new String[] { "switch", "sw" }, "doSwitch");

    String methName = null;

    // Search the command map
        for (Iterator keys = commands.keySet().iterator(); keys.hasNext();) {
            String[] names = (String[]) keys.next();
            for (int i = 0; i < names.length; i++) {
                if (command.equals(names[i])) {
                    methName = (String) commands.get(names);
                    break;
                }
            }
            if (methName != null) {
                break;
            }
        }
        if (methName == null) {
            log("Command name " + command + " not recognized");
            throw new BuildException();
        }

        // Now locate and execute the appropriate method
        Method[] methods = this.getClass().getMethods();
        Method theMethod = null;

        for(int mIndex = 0; mIndex < methods.length; ++mIndex) {
          if(methods[mIndex].getName().equals(methName)) {
            theMethod = methods[mIndex];
            break;
          }
        }

        if(theMethod == null) {
          log("Cannot find a method called " + methName + " for SVN command " + command);
          throw new BuildException();
        }

        try {
      theMethod.invoke(this, null);
    }  catch (Exception e) {
      throw new BuildException(e, getLocation());
    }

  }


  /**
   * Execute a svn checkout
   * @throws SVNException
   *
   */
  public void doCheckout() throws SVNException {
    ISVNWorkspace workspace = SVNWorkspaceManager.createWorkspace("file", dest.getAbsolutePath());

    /*
     * Execute this when e.g. verbose mode is set 
     workspace.addWorkspaceListener(new SVNWorkspaceAdapter() {
      public void updated(String updatedPath, int contentsStatus, int propertiesStatus, long rev) {
        if ("".equals(updatedPath)) {
          return;
        }
        log("A  " + updatedPath);
      }
    });
    
    */

    log("Checking out " + location.getPath() + " to " + dest.getAbsolutePath());
    revision = workspace.checkout(location, revision, false, true);
    log("Checked out " + location.getPath() + " [revision " + revision + "]");

  }
}

XSLTC and JDK 5.0

If you’ve started using JDK 5.0 and are doing XSL transformations via Ant (e.g. for things like JUnit or Checkstyle reports), you may have problems. I had problems that manifested themselves like this:

cruisecontrol-2.2.1\work\checkout\sample\trunk\build.xml:239: javax.xml.transform.TransformerException: java.lang.RuntimeException: Unrecognized XSLTC extension 'org.apache.xalan.xslt.extensions.Redirect:write'

Which is of course because Java 5 ships with the XSLTC compiler. The easy fix is to change any instances of
xmlns:redirect="org.apache.xalan.xslt.extensions.Redirect
to
xmlns:redirect="http://xml.apache.org/xalan/redirect"