Firefox/CruiseControl Extension

I’ve been having some issues with my installation of Firefox for a while now (notably Javascript-based hyperlinks not working properly), so I decided to download the latest trunk version of Deer Park (Firefox 1.1), which is in alpha at the moment.

It’s early days for Deer Park, but from what I’ve seen so far, it will be excellent. It’s noticeably faster at starting up than Firefox is, for starters. And it now has built-in SVG support, which is a very useful addition – hopefully we can start publishing UML diagrams directly from our modeling tools as SVG straight onto a Wiki someday soon. For now, though, the SVG support still has a few flaws – it can’t seem to handle compressed SVG files, and it still only supports a subset of SVG 1.1. The JavaScript engine looks like it has been beefed up considerably as well. One of the most noticeable enhancements is the addition of native XML handling, via ECMAScript for XML. Oh, and I’ve also been using the Outlook 2003 Silver theme, which is very cool.

One small plugin I found recently (and which I had been idly wondering if one existed a couple of days beforehand) is the CruiseControl Firefox extension. This is a nifty little extension that queries a CruiseControl installation and displays the build status of your projects in the Firefox status bar. Quite neat – and a very small plugin (only about 5K, including images). The only problem is that it (like most other CC-related applications that I have seen) is just a simple screen-scraper that retrieves the build results page and parses the HTML, extracting the status and last build time of each project in the status list. We have customized our CruiseControl installation so much that it can’t find what it is looking for in the returned page anymore. So I decided to take a quick look at the plugin itself and try to fix it. It turned out to be much easier than I thought.

Writing a Firefox extension is actually very simple, and the CruiseControl extension is the perfect starting point if you want to learn how an extension is put together. I just downloaded the extension, unzipped the .XPI (which is just a zip file), and created a 10-line Ant build script to build the extension. The extension’s logic is contained in one single JavaScript file, and the quirky Mozilla XPCOM model is easy enough once you get used to it. For instance, here is how to log to the JavaScript console:

function ccLog(aMessage) {
if (isCcDebug()) {
var consoleService = Components.classes[';1'].getService(Components.interfaces.nsIConsoleService);
consoleService.logStringMessage("ccMonitor: "+aMessage);

Notice the naming syntax for the Mozilla interfaces.

This begs the obvious question – why doesnt CC have a Web Services API where you can query this kind of information programmatically? Firefox itself has a stable and mature Web Services API, and it would make writing these kind of applications a piece of cake.

Javadoc bug in JDK 5.0

Came across a surprising bug in JDK 5.0. today. “Surprising” because it’s so easy to reproduce, and the use case would seem to be pretty common. If you run javadoc over a codebase that uses custom annotations, and the annotation classes themselves are not in the runtime classpath, javadoc will throw ClassCastExceptions and consequently fail to generate the indexed contents correctly. In my case, the annotation classes are the Hibernate annotations, and javadoc complains about not being able to find the EJB3 javax.persistence classes in the classpath, along with the Hibernate classes. This is normally no problem, but if your annotation classes themselves are unavailable in the classpath, then javadoc will throw lots of exceptions.

The actual output looks like:

[javadoc] Standard Doclet version 1.5.0_03
[javadoc] Building tree for all the packages and classes…
[javadoc] java.lang.ClassCastException:
[javadoc] at
[javadoc] at

The culprit seems to be the annotationType() method from AnnotationDescImpl. If we look at that, we get:

* Returns the annotation type of this annotation.
public AnnotationTypeDoc annotationType() {
ClassSymbol atsym = (ClassSymbol)annotation.type.tsym;
return (AnnotationTypeDoc)env.getClassDoc(atsym);

The offending line is (AnnotationTypeDoc)env.getClassDoc(atsym). The getClassDoc() method looks like:

* Return the ClassDoc (or a subtype) of this class symbol.
ClassDocImpl getClassDoc(ClassSymbol clazz) {
ClassDocImpl result = classMap.get(clazz);
if (result != null) return result;
if (isAnnotationType(clazz)) {
result = new AnnotationTypeDocImpl(this, clazz);
} else {
result = new ClassDocImpl(this, clazz);
classMap.put(clazz, result);
return result;

I have a feeling that the error may be due to Annotation types not being processed correctly when they are not in the classpath, and consequently not being initialized as ClassDoc instances. The easy fix/workaround is to pass the runtime classpath to the javadoc task.

Strangely enough, a very similar bug was posted and fixed for 1.4.

Visual C# Very Slick

I must confess, I’ve always been a fan of Microsoft’s IDEs for developers. One thing you have to give the Redmondians credit for is that they realized at a very early stage that healthy developer support was the key to maintaining interest and support of their products in the marketplace. To that end, they released more APIs, SDKs, and IDEs than you could shake a large stick at, and some of them were even quite good. The Visual Studio family of tools, for instance, was a mixed bag, but it had a couple of gems. Alongside the tragic Visual Interdev, the utterly terrible Visual SourceSafe, and the vomit-inducing Visual Basic, there were some slick and useful applications. One of these was ( and is ) Visual C++, which easily wiped the floor with any other C++ IDE, even in Borland’s heyday. This product has matured and improved with age – I recently gave a quick test drive to the Visual C++ 2005 beta whilst fixing a bug in some legacy code, and I was impressed.

Another very nice addition to the Visual Studio family was unfortunately short-lived. Way back when, MS introduced a product called Visual J++. This was their first (and last) step into the Java IDE market. The language was Java, but being Microsoft, it wasn’t 100% “pure” Java – it had some custom extensions that made it very useful in a Windows-only environment. Chief among these was J/Direct, which basically was a mechanism that inserted proprietary bytecode into generated .class files, relying on the extensible bytecode mechanism written into the JVM specification by Sun. This turned out to be really powerful, and a godsend for us (at the time) Windows programmers. It allowed you to declare and invoke native function calls in a similar manner to how VB did, and all the parameter and datatype marshalling was done under the hood. The entire Windows API was at your beck and call, and all for very little effort. A set of wrapper classes for common chunks of the Win32 API was supplied, and was called WFC (Windows Foundation Classes). I remember giving a presentation at a Java conference in San Jose back in 1998 on this stuff, and the Java guys who managed to get over their disgust at me presenting on a MS-specific topic were amazed at what it could do. There were two other great attributes to this package as well – it had a very nice event model (called delegation, which you can find in Visual C#), and a drag-and-drop visual form editor, good examples of which were relatively rare in the Java world at the time. So you got the elegance of the Java syntax, with the ease of GUI construction that at the time only existed in VB.

Unfortunately, it couldn’t last. MS and Sun fell out, and took their mutual dislike to the courts. Visual J++ gradually become sidelined, and then an MS technology evangelist told me at a conference that J++ was quietly being dropped, and a replacement was being mooted (at the time, it was codenamed “Cool”).

Which brings us back to 2005, and Visual C#. What you get when you work with Visual C# is the direct descendent of Visual J++. The same guy (Anders Hejlsberg) has lead the development stream for both J++ and C#, and he has fused a lot of the original ideas of J++ with many (many) more directly from Java. And it’s really, really good. I’m actually going to try and earmark a piece of work that I know will be Win desktop-specific and I’m going to do it using Visual C# 2005. It took me no time at all to knock up a desktop client for CruiseControl, and I was rather pleased with the result.

Of course, there are limitations – portability being the obvious one. But I do think it’s wise to have as many tools at your disposal as you have individual problem areas to attack (also a famous theory in economics). And I think Visual C# will be able to solve quite a few problems for me in the future.

Blob Hell

I was happy to see that the JDBC 4.0 spec will contain support for improved BLOB and CLOB handling. Blob handling is still one of the most awkward areas of cross-database implementation work. It would be nice to have a consistent interface for blob creation, update, and streaming across databases. Ostensibly, that’s what we have in JDBC 3 . However, in practise, things are not so simple.

One of the fundamental issues is how the RDBMS handles LOB data. To perform an INSERT or UPDATE of blob data, the RDBMS hands the client back a pointer or “locator” to the blob data. The manner in which they do this varies across implementations and vendors. This means in practise that you have different limitations depending on which DBMS you are using. Oracle has always been a culprit (until version 10) with its infamous 4k limit on streaming data, which has resulted in hundreds of applications and frameworks (Hibernate among them) having to write Oracle-specific JDBC-level code to work around this issue.

We are currently working on a system where we have written a server component which accepts standard FTP connections from clients. The clients send large binary files via FTP, which are streamed and stored into the database within a single transaction. Various consistency checks are performed along the way, and if any of the checks fail, an error is returned to the client and the transaction is rolled back. This means that if the client receives a successful return, they can be sure that the data is stored and ready to be processed.

The main problem has been with trying to successfully stream the blob data in a clean and portable way. Our production database is Oracle, but it’s nice to be able to test on say, MySQL or HSQL. If you want to go straight to JDBC, you can use Oracle’s empty_blob() function, grab a blob locator from that, and then use the oracle.sql.BLOB type’s getBinaryOutputStream() to get an OutputStream to write to. You can then chunk the data from a socket directly to the database server. However, achieving this at a higher level is altogether more challenging. Thankfully, Spring has some streaming LOB support, which is what we have based our current solution on.

Debugging CVSGraph

Continuing on from my last post about fixing some issues related to MySQL and Python on Solaris, I came across another issue this morning which also necessitated digging out a copy of good ole gdb. The basic issue was that CVSGraph segfaulted every time I attempted to generate a revision graph. This was reproducible every time, no matter what the input. Turning the verbosity to the maximum allowed level did not produce anything useful.

First thing I did was download a copy of gdb, and did a configure/make/make install. I initially thought that if I could get cvsgraph to produce a core dump on exit, I would be able to examine it within gdb and get some clues about the cause. However, I couldn’t get CVSGraph to automatically core dump, even after setting core policy using coreadm, as shown here.
UPDATE: I found the missing piece of the puzzle – the maximum core dump size had not been set via ulimit. Setting this enabled automatic core dumping.

The next step was to actually load cvsgraph into gdb and run a test session inside the debugger. After a couple of runs, I had isolated the problem to a specific routine. I set a breakpoint and ran through the test case again:

First, I set up the command line arguments:

(gdb) set args -c /usr/local/viewcvs-1.0-dev/cvsgraph.conf -r /usr/cvsroot cobra/build.xml,v

Then set a breakpoint at the relevant location:

(gdb) break cvsgraph.c:1092
Breakpoint 1 at 0x1311c: file cvsgraph.c, line 1092.

Then kick off the target program:

(gdb) run
Starting program: /root/cvsgraph-1.5.1/cvsgraph -c /usr/local/viewcvs-1.0-dev/cvsgraph.conf -r /usr/cvsroot cobra/build.xml,v

Once gdb hits the breakpoint, it stops and waits for instructions:

Breakpoint 1, expand_string (s=0x50d51 “d”, rcs=0x54450, r=0x545b8, rev=0x50770, prev=0x0, tag=0x0) at cvsgraph.c:1092
1092 t = mktime(&tm);

I manually stepped forward a couple of times until I hit the problem:

(gdb) n
1094 if(env)
1095 setenv(“TZ”, env, 1);

Program received signal SIGSEGV, Segmentation fault.
0xff3a0510 in memcpy () from /usr/platform/SUNW,Sun-Fire-V240/lib/

Now we can do a stack backtrace to see where we were at the time:

(gdb) bt
#0 0xff3a0510 in memcpy () from /usr/platform/SUNW,Sun-Fire-V240/lib/
#1 0x0001ed6c in setenv (name=0x1f4e0 “TZ”, value=0xffbfffac “GB”, replace=1) at ../../../libiberty/setenv.c:156
#2 0x00013140 in expand_string (s=0x50d51 “d”, rcs=0x54450, r=0x545b8, rev=0x50770, prev=0x0, tag=0x0) at cvsgraph.c:1095
#3 0x00016a28 in make_layout (rcs=0x54450) at cvsgraph.c:2937
#4 0x00019ecc in main (argc=327696, argv=0x50010) at cvsgraph.c:3879

So know we have isolated the problem down to setenv(), which is implemented in the GNU libiberty adapter library. I exited gdb and wrote a simple test case based on what CVSGraph was doing at the point in question, and found that the problem can be easily reproduced by calling putenv() to create an environment variable, and then immediately calling setenv() to reset the value. This may be due to a bug in the libiberty putenv implementation.

In reality, we dont really need the call to putenv() here – its actually redundant, as setenv() will allocate space for the new variable if necessary. So I simply commented out the offending line, remade CVSGraph, and now we have the (very useful IMHO) graphical branching display from ViewCVS.

Python/MySQL installation problems

I am currently in the process of setting up a continuous integration server, which involves installing packages like CVS, Ant, CruiseControl, Tomcat, Apache, MoinMoin, ViewCVS, and MySQL, and basically glueing them all together. It’s a relatively long-winded process (especially on Solaris, as it won’t necessarily have a lot of the GNU prerequisites installed). However, a lot of it is boilerplate and individual packages can be installed relatively quickly.

During the course of installing the various packages and prerequisites involved, you find that from time to time you need to make manual adjustments to makefiles, switch libraries or library versions, or alter the parameters passed to the C compiler, for instance. This is always where I am grateful that I have a pretty good knowledge of C-based development on Unix (rusty as it is), as it comes in very handy when installing lots of open-source onto a Unix box. However, one problem almost had me stumped.

Whilst trying to install the CVS query functionality that comes with ViewCVS, I needed to install MySQL-Python as a prerequisite. This is ostensibly a simple two-step procedure, python build and (as root) python install. However, it failed immediately, citing incorrect compiler flags. I located the point in where it sets those parameters (which it gets from mysql_config), and set them to more sensible parameters for Solaris. This compiled, but failed on the link stage with ld: warning: file /usr/local/mysql/lib/libmysqlclient.a(libmysql.o): wrong ELF class: ELFCLASS64. I tracked this down to a 64-bit MySQL version installed on a 32-bit platform. After uninstalling and reinstalling the correct version of MySQL, I attempted another build of MySQL-Python. This time, it compiled and linked fine , produced a shared library, and was duly installed.

Next, I tried to run a simple Python test script to verify that it was OK. This failed with:

File “/usr/local/lib/python2.3/site-packages/MySQLdb/”,
line 27, in ?
import _mysql
ImportError: /usr/local/bin/python: fatal: relocation error:
file /usr/local/lib/python2.3/site-packages/ symbol
mysql_errno: referenced symbol not found

This was altogether tougher to diagnose. I first thought that mysql_errno might be defined differently for the reentrant versus the non-reentrant MySQL client libraries, so I replaced "-lmysqlclient_r" with "-lmysqlclient" on the linker command line. This didn’t make any difference. I also explicitly added the MySQL library path into the LD_LIBRARY_PATH variable, which also made no difference.

Eventually, I found the answer (at least for my version of Solaris) – pass the shared lib flag to the linker explicitly – for some reason the installer command line doesn’t do this correctly. I found this out by running gcc manually, initially replicating the flags used by the Python installer.

The full gcc command line then looked like this:

gcc -I /usr/local/mysql/include/ -I /usr/local/include/python2.3/ -Wl,-G -fPIC -Wall _mysql.c -o `mysql_config --libs_r`

When the shared lib had built, I copied it to the appropriate spot in the Python build directory, and then just ran install to produce a working MySQL-Python client lib.

An easy way to fix this is to modify Under the line that reads:

extra_compile_args = config(“cflags”)

Add the following lines: