BDD for JSP in Eclipse with Groovy Webtest

Pak Wing and I needed to put together a rapid prototype of a document generator that delivers web pages using custom templates together with various back-end sources of information. The first thing we did, of course, was to sketch out a test plan to identify the capabilities of our system. A simple first test was to see that the correct page title would be generated in response to a GET with varying parameters in the query string, including an invalid case.

The next question was how to execute the tests. My previous experience with Canoo Webtest had been favourable, so I was keen on this approach - but not on the traditional XML notation that Webtest uses. Since it has become feasible to use Groovy as the test scripting language, we decided to use this as it can support the behaviour-driven approach (where test cases are expressed in the "given, when, then" form), is a more compact notation than XML and easier for our stakeholders to read.

However, Canoo's manuals are a little light on detail about how to get all this to work in an Eclipse environment, so I'm recording my findings here as I go along.

Software to Install
Packages:

• Java SE JDK
• Eclipse
• Groovy
• Canoo Webtest
• An application server - in our case, we wanted to use JSP, so Tomcat version 7 was the obvious choice. As we're developing under Windows 7, we chose the Windows Service installer version (but note that the service should not be started - Eclipse will run the server when you're ready to deploy the application)

Eclipse plug-ins:

• Eclipse Web Developer Tools
• Eclipse XML Editors and Tools
• Groovy-Eclipse Feature (optionally plus sources)
• Web Page Editor (optional)
• Don't forget integration with your favourite Software Configuration Management system

I hope that was everything - please drop me a comment if you find you're missing some vital component that I've forgotten to mention.

Starting your Project
After a number of experiments, we found that the most straightforward approach was to create a new Dynamic Web Project and to convert it to a Groovy project afterwards, using the Groovy entry in the context menu. You can also convert to a Maven project subsequently, if that is your preferred build system and you have the M2E (Eclipse Maven Project) plugin installed.

I like Maven's organisation of source folders into src/main and src/test. Put your Java files (e.g. servlets) in src/main/java and test scripts in src/test/groovy. HTML and JSP files go into src/main/resources. Set up the Java Build Path appropriately.

It is standard practice to name the package for a JUnit test script identical to the package of the class under test. We followed this practice also for Groovy test scripts. A Groovy Webtest script is in fact a thinly disguised Ant build script, but you run it as a JUnit test (while avoiding all the tedium of writing JUnit test cases in Java). The Ant libraries are supplied with Webtest.

Running Your First Test
Here's my version of the simple test script shown on the Canoo home page. Note the addition of firewall configuration:



If you paste this into your own Eclipse project, you will immediately be informed that the import "com.canoo.webtest.WebtestCase" cannot be resolved. To fix this, open the Java Build Path configuration, click on the Libraries tab and add a new User Library. I called it WEBTEST_LIBS. To define this User Library, click "Add JARs", navigate to the lib folder under your Webtest installation directory, highlight all jar files you find there and click OK.

Obviously you should amend the firewall configuration in the above constructor function TestGroovyWebTest() as appropriate.

To run the test script, simply right-click the Groovy file, select "Run as..." and click "JUnit Test". You should get both the standard JUnit output of a green bar (hooray!) and a Webtest monitor window that shows the test in progress, followed by a result screen in your web browser.

Create and Deploy a Dummy Application
For the next stage, we wanted the Groovy test script to actually interact with the application we were developing.

First configure your server. In Eclipse, go to the View menu, select "Show View", expand the "Server" section and select "Servers". If this is not available, you probably have not installed the Eclipse Web Developer Tools feature correctly.

Right-click in the Servers panel and select "New". Add the server (you should find it in a pull-down list). Configure the correct run-time environment for the application server you have installed, and name it (probably "localhost"). Click "Finish" (don't configure any apps at this stage).

Create a very simple index.jsp page under src/main/resources:



Now you can right-click the JSP file, select "Run as..." and click "Run on server". The output ought to appear in a new edit panel.

You can now add a test case to invoke your new app and check that it has produced the right page:



Gotchas
There are a few problems to look out for:

• If you collaborate with someone else on your project, and their Eclipse and other configurations do not match yours precisely, you may find that some of the build path configuration has to be changed after they import your project. To minimise this, make sure that for example you configure "workspace default" as the JRE for the project.
• In one case, we found that Webtests could not be run more than once. The reason was that each run produces a report under the same folder into which the Java and Groovy compilers place class files. For some reason, with certain installations of the Tomcat server, the synchronisation mechanism meant that the test reports were being deployed along with the application, which meant that Webtest could not delete them before running the next test. In these cases, we found that the only remedy was to restart Eclipse and then start Tomcat again - too tedious in the long run. It might be less hassle just to run the Groovy test script outside Eclipse, but we have not tried this.

Perils of using an elderly FitLibrary

I learned about a really useful tool today while trying to hunt down a bug in a set of Fit automated acceptance tests.

Last September, my colleague Darren Bishop had set up the automated acceptance test infrastructure for a Maven/Java project I subsequently joined. He used version 20080812 of the Maven dependency (org.fitnesse.fitlibrary), probably because it was the latest available in the Maven Central repository. Thus the stage was set for my troubles last night and this morning!

While working on a small enhancement with a developer in the customer's product support unit, I noticed that one of the Fit tests was producing a blank report and not being counted in the summary totals. A comparison with other spreadsheets that did run properly revealed no significant differences.

Back at base, it quickly became apparent that I had to single-step through the program to find out where it was going wrong. But I could find no source jar for this version of fitlibrary - the latest dependency version available with source was from 2006. What to do?

After a futile attempt to desk-walkthrough the code of FolderRunner and SpreadsheetRunner, Keith Braithwaite suggested decompiling the class files. JD-Eclipse to the rescue! I installed it in my Eclipse and found it so insanely useful that I will never want to be without it again. Basically it creates beautifully formatted source code for you on demand (obviously without the programmer's original comments and documentation). This means that you can set breakpoints and single-step through library code even if you have no source attachment for your library. Installation was really easy. After that, if you single-step into a library method, the "source" is opened automatically. You can also open a .class file directly in the package explorer by browsing the libraries on your project's build path.

There's just one slight gotcha: the line numbers in the decompiled code rarely match the original, so if the class file contains line number annotations your instruction pointer and breakpoints may appear to be a few lines away from the location reported in the debugger's thread view. Ignore the cursor and read off the current line number from the comment at the start of each line. You can turn off the display of metadata and line numbers in the JD preferences, and I have done so - it needs a restart of Eclipse to take effect, but setting breakpoints is much easier afterwards.

So, what was the solution to the mystery?

In the FitLibrary class SpreadsheetRunner, the method collectTables() used an Iterator to return the rows of the test script one at a time and convert them into arrays of values that the runner could subsequently invoke to execute the test:

HSSFRow row = (HSSFRow)it.next();
HSSFCell[] cells = getCells(row);
String[] borderedCellValues = getBorderedCellValues(cells, workbook);

Unfortunately, getCells() dimensioned an array of HSSFCell objects using the value returned by a row:

private HSSFCell[] getCells(HSSFRow row) {
int maxCell = row.getLastCellNum();
HSSFCell[] cells = new HSSFCell[maxCell];
...
return cells;
}

When the row was "empty", getLastCellNum() returned -1, which caused the next line to throw a NegativeArraySizeException. This exception was caught by FolderRunner.runFile(), which attempted to report the problem to the standard output, but this didn't appear in the Maven test log.

I have not found out what makes POI decide that a row is "empty". There was only a single spreadsheet row in the entire suite of tests that was interpreted in this way. I achieved a workaround by pasting a copy of another blank row of cells onto this row, but the danger is that anyone could inadvertently reintroduce the error.

Lessons:

• You can still use the Java debugger if you don't have the source. JD-Eclipse worked very well for me, even though it is only at version 0.1.3. (I wonder if it will decompile classes written in other languages such as JRuby?)
  


• It's time someone made a proper Maven dependency, with sources and javadoc, of the latest Fit, Fitnesse and FitLibrary versions - the current version of FitLibrary, dated April 5th, 2011 in fact solves the exact problem I encountered!
  

Ubuntu Desktop 9.10

For a project recently begun, I needed a Linux virtual machine and I downloaded the excellent Chrysaor VMWare appliance pre-built with Ubuntu Desktop 9.10 (karmic koala) and the VMWare Tools (which actually work as specified - you can cut/paste text and even entire files between your host OS and the VM).

NB I was only able to download this using a direct internet connection, because our company firewall blocks BitTorrent. There don't appear to be any FTP or HTTP mirrors of these VMWare images.

There were just a couple of problems with this virtual machine image, for which I have discovered workarounds.

1. Timezone persistently defaults to EST, no matter what I do in the time/date control panel. It turns out that certain system files need editing by hand (change to reflect your actual location and language):
   
   • sudo rm /etc/localtime
   • sudo ln -s /usr/share/zoneinfo/Europe/London /etc/localtime
   • sudo vi /etc/timezone -- set it to Europe/London
   • sudo vi /etc/profile -- add the following line at the end:
     export TZ=/etc/localtime
   • sudo vi /etc/default/locale -- set it as follows:
     LANG="en_GB.UTF-8"
2. Login crash: for a while I thought I had broken the system by installing some upgrade or enabling shared folders. The symptom was that when I logged in, the progress bar would be displayed for a few seconds but instead of displaying the desktop, Ubuntu would restart the windowing system and display the login prompt again. I eventually discovered by trial and error that this was due to starting up in full-screen on my second monitor. Provided I always login while the VM is running in a non-maximised window, it seems very reliable. If you're suffering similar problems, look in your vmware.log file for messages similar to the following:
   

   VMXVmdbGuestLaunchMenuCB: buf is null.
   VMXVmdbGuestLaunchMenuCB: failed: ret = VMDB failure
   GuestRpc: Channel 3 reinitialized.

I recommend that the first thing you do is create a new user ID for yourself with admin rights, and log in under that user to do everything else.

I upgraded Mercurial to 1.4.1 and TortoiseHg to 0.9.2 by downloading the following Debian packages and installing them manually using dpkg -i:

1. mercurial_1.4.1-1_i386.deb and mercurial-common_1.4.1-1_all.deb from ftp://mirror.switch.ch/mirror/ubuntu/pool/universe/m/mercurial/
   
2. tortoisehg_0.9.2-1_all.deb and tortoisehg-nautilus_0.9.2-1_all.deb from ftp://mirror.switch.ch/mirror/ubuntu/pool/universe/t/tortoisehg/

Note that Tortoise doesn't work unless you install the python-iniparse package. Version 0.3.1-1 works with Tortoise 0.9.2-1.

I was able to get Eclipse (Galileo) as a Ubuntu package and installed the excellent HgEclipse plugin from http://hge.javaforge.com/hgeclipse using the Eclipse "Install New Software" option.

My Favourite Keyboard Shortcuts

That was the title of a session I proposed for today's first-ever Software Craftsmanship Conference, at the BBC's attractive Media Centre in London's White City. To my surprise, the session was accepted. It worked quite well despite attracting only about half a dozen participants. I certainly learned a good deal from it!

The output of the session has been posted on the conference Wiki. However, you'll have to be logged in to see it, as the wiki is private to conference participants. I don't know if Jason Gorman plans to change this now the conference is over.

The top tips (as a result of a quick spot vote at the end) were:

The three-step refactoring tango: Jason's recipe for method extraction where one or more parameters are themselves method calls. In Eclipse, it goes ALT-SHIFT-L (extract local variable), ALT-SHIFT-M (extract method), then ALT-SHIFT-I (inline local variable as method parameter).

Expand/shrink current selection: In Eclipse, ALT-SHIFT-up/down arrow or in IDEA, CTRL-w/W. With the cursor on a given word, the word is selected. Repeated applications of the same keyboard combo selects progressively larger portions of the source file (up to and including the entire file). Using the opposite key combination reduces the size of the selection all the way back to 0.

Test-Driven Web Development on Spring and Hibernate

Adam Shimali and I will be piloting our TDD course at SPA2009 on the Sunday afternoon. Three two-hour samples of a full two-day course for free - and you get a 10% discount off the cost of the full course for yourself or a colleague.

The course is very hands-on. If you bring a laptop (Windows or MacOS) we can provide you with the complete Eclipse development environment installation on a CD or memory stick - you'll be good to go in about five minutes. It will help if you've done a bit of Java programming before, but even this is not essential as you'll be paired with someone experienced.

There's still time to book for SPA2009, but don't delay!

Ramaze - first impressions continued

My earlier posting was getting too long, so I'm continuing in this new one. Meanwhile, the tutorial's author has kindly left a comment on the original posting, indicating that the tutorial is due to be updated pretty soon. I look forward to that!

Chapter 11 of the tutorial can be used pretty much as-is, but I found that after an error, the redirection specified by the error method would be overwritten by the helper aspect. After some research, I discovered that the latest version of Ramaze supports a redirection status operator, redirected?. Using this, the helper aspect can be written very simply:

helper :aspect
after( :create, :delete, :open, :close ) {
  redirect Rs() unless redirected?
}

Chapter 12 of the tutorial needs no changes, except that you don't have to add the flash section to the page template because we did it earlier.

Now I wanted to convert the application to something more suited to enterprise deployment (and also more suited to shared hosting deployment - typically such environments provide an Apache server and MySQL database). So the first thing to do was to move from YAML to SQL - I chose MySQL.

If you haven't yet installed MySQL, do so before the next step. Take care to ensure that your PATH environment variable contains the MySQL binary as well as its libraries (hold down the Windows key and press PAUSE to bring up the system properties, then choose Environment Variables in the Advanced tab). For example, on my system the System PATH begins:

C:\ruby\bin;
C:\Program Files\MySQL\MySQL Server 5.0\bin;
C:\Program Files\MySQL\MySQL Server 5.0\lib\opt;
C:\Program Files\MySQL\MySQL Server 5.0\lib\debug;
...

Add a new database for the application and create a user account named "ramaze" with password "TodoList" for both localhost and remote-host access:

mysql -u root -p
******
CREATE DATABASE IF NOT EXISTS todolist_db;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE,
  DROP, RELOAD, PROCESS, FILE, REFERENCES, INDEX,
  ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
  LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW,
  CREATE ROUTINE, ALTER ROUTINE ON *.*
  TO 'ramaze'@'localhost'
  IDENTIFIED BY 'TodoList' WITH GRANT OPTION;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE,
  DROP, RELOAD, PROCESS, FILE, REFERENCES, INDEX,
  ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES,
  LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW,
  CREATE ROUTINE, ALTER ROUTINE ON *.*
  TO 'ramaze'@'%'
  IDENTIFIED BY 'TodoList' WITH GRANT OPTION;
quit

I decided to use Sequel as my database access layer. You probably also have to install the gems mysql and sequel before the code will work - I am not sure about the mysql gem, as it was one of the things I installed before I eventually discovered the need to set up the PATH correctly.

gem install sequel
gem install mysql

I tried choosing the 'ruby' option, but this was unable to generate the native code on my machine. So I uninstalled it and tried again, choosing the 'mswin32' option, which worked fine.

Now take a safety copy of your model file, todolist.rb, and start modifying it to use MySQL instead of YAML. Replace the following lines:

require 'ramaze/store/default'
TodoList = Ramaze::Store::Default.new('todolist.yaml')

with the following lines:

require 'rubygems'
require 'sequel'

DB = Sequel.mysql('todolist_db',
  :user => 'ramaze',
  :password => 'TodoList',
  :host => 'localhost')

class TodoList < Sequel::Model(:tasks)
  set_schema do
    primary_key :id
    varchar :title, unique => true, :null => false
    boolean :done
  end
end

To begin with, I tried using the title as the primary key. But I soon found that not only was it necessary to define the title field first and then separately to name it as the primary key, the user-supplied title was not always suitable as a key value due to the presence of shell metacharacters and so on. So I decided to go with the Sequel flow and use a system-generated ID as the primary key.

Next I found that the model didn't support the method original(), which the original simple model supports. So I supplied one myself (subsequently found to be unnecessary after refactoring main.rb):

  # Copy all records into a list
  def self.original
    tasks = []
    self.dataset.each {|r| tasks.push [r[:title], {:done => r[:done]}]}
    return tasks
  end

More important was to add some initialisation code after the class was defined:

unless TodoList.table_exists?
  DB.transaction do
    puts "Creating table 'tasks'\n"
    TodoList.create_table
  end
end

Now try running the application. It seems to work, but nothing gets stored in the database. At this point we have to bite the bullet and refactor the main module to support a more relational view of the underlying data model.

In the index() method, we'll start using the ID of each task to identify it instead of the title. Instead of requesting an array of key-value pairs from the original() method, we get an array of task objects from the dataset() method. That of course implies that we have to extract the title and the ID from the task object and use them as appropriate:

  def index
    @title = ["To-Do List"]
    @tasks = []
    TodoList.dataset.each do |task|
      id = task[:id]
      title = task[:title]
      if task[:done]
        status = 'done'
        toggle = A('Open Task', :href => Rs(:open, id))
      else
        status = 'not done'
        toggle = A('Close Task', :href => Rs(:close, id))
      end
      delete = A('Delete', :href => Rs(:delete, id))
      @tasks << [title, status, toggle, delete]
    end
    @tasks.sort!
  end

The methods open(), close(), task_status() and delete() have to change because they all now take an id as parameter:

  def delete id
    unless TodoList.delete id
      failed "Cannot delete task no.: #{id}"
    end
  end

  def open id
    task_status id, false
  end
  
  def close id
    task_status id, true
  end

  def task_status id, status
    unless task = TodoList[id]
      failed "No such task no.: #{id}"
      redirect_referer
    end
    task[:done] = status
    TodoList[id] = task
  end

I decided to override the []= method of the model, so that tasks would actually be written to the database. It was an easy step from there to create a new row in the tasks table whenever the ID parameter to this method was absent or nil. So the create() method becomes:

  def create
    if title = request['title']
      title.strip!
      if title.empty?
        failed("Please enter a title")
        redirect '/new'
      end
      if TodoList.find(:title => title)
        failed("Task '#{title}' already exists")
      else
        TodoList[nil] = {:title => title, :done => false}
      end
    end
  end

Note the check for duplicates, which is easy to do now that we can look up titles in the dataset.

Turning to the file todolist.rb, here are the methods I had to add to allow tasks to be added and deleted in the database:

  def self.delete(id)
    puts "Attempting to delete '#{id}'\n"
    DB.transaction do
      if task = TodoList.find(:id => id)
        task.destroy()
      else
        puts "Not found\n"
        return false
      end
    end
  end

  # Assignment should update the underlying database
  def self.[]=(id, values)
    DB.transaction do
      if (id == nil || !(task = TodoList.find(:id => id)))
        task = TodoList.new
      end
      task.title = values[:title]
      task.done = values[:done]
      task.save
    end
  end

That's pretty much it. But before I stopped, I wanted to prettify the user interface a bit. I didn't like the fact that the column widths tended to change whenever the sole "not done" item was added to or deleted from the list, and in any case I preferred clickable icons to text links. So I designed some icons:

not done

done

delete

Feel free to copy the icon images. To let the Mongrel server access these, they have to be placed in the folder "public" of your project. Then I redesigned the index() method and the corresponding index.html file very slightly to use these (first declaring some constants):

  DELETE_ICON  = '<img src="delete_sml.gif">'
  NOTDONE_ICON = '<img src="notdone_sml.gif">'
  DONE_ICON    = '<img src="done_sml.gif">'
  
  # the index action is called automatically when no other action is specified
  def index
    @title = ["To-Do List"]
    @tasks = []
    TodoList.dataset.each do |task|
      id = task[:id]
      title = task[:title]
      if task[:done]
        toggle = A(DONE_ICON, :href => Rs(:open, id))
      else
        toggle = A(NOTDONE_ICON, :href => Rs(:close, id))
      end
      delete = A(DELETE_ICON, :href => Rs(:delete, id))
      @tasks << [title, toggle, delete]
    end
    @tasks.sort!
  end

    <p><a href="/new">New Task</a></p>
    <?r if @tasks.empty? ?>
      <p>No Tasks</p>
    <?r else ?>
      <table>
  <?r @tasks.each do |title, toggle, delete| ?>
    <tr>
      <td class="title" > #{title}  </td>
      <td class="toggle"> #{toggle} </td>
      <td class="delete"> #{delete} </td>
    </tr>
  <?r end ?>
      </table>
    <?r end ?>

The full source code is attached in the comments to this post.

Ramaze - first impressions

I've spent the past couple of days evaluating Ramaze. I started with the tutorial To-Do-List example and tried to move beyond that to something resembling a real web application backed with a proper database (MySQL). In general, I feel that this framework is more intuitive to use and therefore gets results faster than e.g. Ruby on Rails, if you're having to learn both from scratch. I also found it very easy to refactor the code, because it encourages a fairly aspect-oriented style and all changes are loaded immediately without having to re-start the server. Last but not least, the designers of Ramaze have considered enterprise-level deployment requirements from the very start, so your Ramaze application can run on anything from a shared virtual host rented for a few dollars per year to an enterprise server farm or on-demand compute-cloud configurations.

Ramaze also leverages the power of Ruby in very elegant ways. For example, here's a description of a basic content-management system based on Ramaze in just a single page of code.

I'm going to walk you through the to-do list tutorial, which dates back to 31st January 2008. Ramaze has evolved a fair bit since then and I have had to find out what the changes are to adapt the tutorial to the current version. Hopefully my observations will be incorporated in a future version of the tutorial. All of my work was done on Windows XP (blame my employer, not me :-) so the installation instructions etc. are related to that.

First of all, if you don't yet have Ruby and Ramaze installed, here's how you do it:

• Grab the latest Ruby installer
• Install with default options
• Open a command shell and make sure that your Ruby bin directory is on the PATH
• gem install ramaze

You may find yourself being asked to approve the installation of various dependencies. Just agree to each one. If gem offers you options (ruby, java, mswin32 etc.), choose "ruby" or "mswin32" (assuming you're on a Windows 32-bit machine).

My next big problem came with Eclipse. I wanted to use Eclipse as my IDE for Ruby and Ramaze. Trying to use the Help -> Software Updates feature to install Ruby support, I discovered by much trial and error that it was possible to do so, provided you configure the update site http://update.aptana.com/update/rails/3.2/ and select "Uncategorized -> Ruby Development Tools" and nothing else. Using this plug-in, you get a language-sensitive editor and the ability to launch Ruby applications, not much else, but it's a whole lot better than using just a text editor. The first time you try to open the Ruby perspective, the Hierarchy pane displays an error message ("Could not create the view: org/eclipse/search/internal/ui/text/ResourceTransferDragAdapter"). Just close it and use the Navigator pane instead. I'd be interested to know if there is a better Ruby plug-in for Eclipse.

While I was about it, I also installed the Web Page Editor from one of the Eclipse update sites. This came in useful when I was working on template pages later.

Then I started working my way through the tutorial. The first discrepancy came in chapter 3 - the View. Ramaze has evolved so that you supply just partial pages under the view folder. So the view/index.xhtml file in the tutorial should contain only the part of the page that will be supplied to the main page template in the variable @content. It looks like this:

  <ul>
   <?r
     TodoList.each do |title, value|
     status = value[:done] ? 'done' : 'not done'
   ?>
     <li>#{title}: #{status}</li>
   <?r end ?>
 </ul>

Of course, at this stage you won't have assigned a value to @title, so your page will be rendered without a page title or first heading.

Also on this page, ignore the instruction to "run ramaze" from the command line. That won't work under Windows. Instead, Eclipse comes to the rescue. Just right-click the start.rb file in the Navigator pane and select "Run As... -> Ruby Application". When you want to terminate the server, click the red stop button in the console pane. Subsequently you can re-run the server by selecting start.rb from the pull-down menu under Eclipse's green Run button.

In chapter 4 of the tutorial, again please omit the HTML scaffolding and just write

  <?r if @tasks.empty? ?>
   No Tasks
 <?r else ?>
   <ul>
     <?r @tasks.each do |title, status| ?>
       <li>#{title}: #{status}</li>
     <?r end ?>
   </ul>
 <?r end ?>

Now add the following line at the very top of the index method:

  @title = ["To-Do List"]

The standard page template inserts this value both in the page title and the top-level heading. You may have noticed that the MainController contains the line

  layout '/page'

This indicates that the file view/page.xhtml will be used as the template, not src/element/page.rb as described in the tutorial.

In chapter 5 of the tutorial, obviously you should not add the level-one heading as shown. Just put the link at the very top of the file view/index.xhtml.

Next, the new file view/new.xhtml is also much simplified:

  <a href="/">Back to TodoList</a>
 <form method="POST" action="create">
   Task: <input type="text" name="title" /><br />
   <input type="submit" />
 </form>

Before creating an action for create, as described in the tutorial, you will also need an action for new. This is the only way that I have found to generate a title for the new.xhtml page. Insert the following in main.rb:

def new
@title = ["Create a new To-Do List item"]
# See view/new.xhtml
end

Chapter 6 of the tutorial remains pretty much unchanged. Just remember not to delete the @title initialisation at the very top of the index method.

In chapter 7 of the tutorial, I found that I was able to add and delete tasks to my heart's content until I tried deleting a task such as "Lunch with Bob?". Ramaze has a very simple mapping scheme from URL path to action argument, which stops parsing the path_info as soon as it encounters a "?" or ";". I spent a considerable amount of time trying to think of ways to circumvent this and pass the offending characters as url-encoded escape sequences, but in the end it was all to no avail. The lesson is that you would be well advised to never use user-supplied values as keys into your data. Instead, treat the user-supplied string as a data field and generate your own system ID for each item. By the way, this lesson applies equally well in other frameworks. Following my completion of the tutorial I will show how I refactored the code to achieve greater robustness. For now, just avoid using metacharacters in your task titles!

The solution to the problem of stuck tasks was of course to edit the YAML database file to delete the offending characters.

Chapter 8 of the tutorial seems to me now to be more or less redundant. All I had to do was to modify the file view/page.xhtml slightly to incorporate the error message from the flash object. Here's the resulting body section:

   <h1>#{@title}</h1>
  <div id="error">
    <p>#{flash[:error]}</p>
  </div>
  <div id="content">
    #@content
  </div>

Note that there is no need for any logic to decide whether an error message is present. If it's there, it gets rendered as part of the page. Otherwise the renderer omits that element.

The prettification step in chapter 9 of the tutorial still applies as it stands, except that the stylesheet entries are inserted into the file view/page.xhtml instead of the Page element. I also added a style for the error message, where present. The resulting HTML header now contains the following entries:

   <style type="text/css">
    body      { margin:      2em; font-family:Verdana }
    #content  { margin-left: 2em;               }
    #error    { margin-left: 2em; color:red;    }
  </style>
  <style type="text/css">
    table     { width:       80%;               }
    tr        { background:  #efe; width:100%;  }
    tr:hover  { background:  #dfd;              }
    td.title  { font-weight: bold; width: 60%;  }
    td.status { margin:      1em;               }
    a         { color:       #3a3;              }
  </style>

I had real fun in chapter 10 of the tutorial. Firstly, I had to install the mongrel adapter:

gem install mongrel

For both mongrel itself and its fastthread dependency, I chose the "ruby" option.
When I tried to start Ramaze again, it complained that some other process was already listening on port 80. After some detective work with process explorer, I discovered that this was a little peer-to-peer application called kservice from Kontiki Inc. I had probably unwittingly installed it with some Internet media player. After following the uninstall instructions kindly blogged by "Geoff" I was finally able to start my server on port 80.

(to be continued)

Maven multi-module builds and Eclipse PDE

I have not blogged about Maven and Eclipse for a while, but I have learned a couple of important new lessons recently.

1. Even though OSGI gives you the capability of supporting multiple versions of the same bundle simultaneously, the same isn't true of Eclipse features that contain those bundles (a.k.a. Eclipse plug-ins). So I discovered that when feature A depends on feature B and C, where B depends on feature D version 1.1.0 and C depends on feature D version 1.1.2, you will get two versions of feature D in your Eclipse target platform, resulting in a cryptic error message from the PDE build of the form "unable to find feature: D". What it means is that B wants version 1.1.0 of D, but this has been masked by the presence of version 1.1.2. Our likely approach to this will be to avoid putting version dependencies (at least, versions of included features) into feature.xml files at all. There is no need, after all: the precise versions of everything needed should be specified in the accompanying POM, which assembles the RCP target platform for the build.

2. If you're using multi-module builds, there is no point in using goals such as "test" or "package". Unless you at least install the artifact created during the build of each module into your local Maven repository, the next module along will still get the previous version as a dependency, and the build will fail.

3. If you're using multi-module builds, don't run the install or deploy goal in the same invocation as the site or site-deploy goal. Maven tries to generate the site documentation for the top level before it has built the subordinate levels, so the dependencies required to create the reports are not present and the build will fail. We now routinely run "mvn clean install" or "mvn clean deploy" followed by "mvn site" or "mvn site-deploy" respectively.

Eclipse and Maven - a quick update

I have not posted anything about the progress of my researches into Eclipse and Maven integration for a while. That's because for a long time, we didn't get significantly further along in our research. But just recently we've started making significant progress again.

A very important lesson was that the pde-maven-plugin is really not worth persevering with - at least in our situation. We decided that we wanted to build every Eclipse plug-in and every Eclipse feature using its own POM, so that we could for example generate code coverage reports for each component individually and also configuration-manage them at that level of granularity. It turned out to be not all that difficult to invoke the PDE headless build for these circumstances from Maven using about two dozen lines of Ant script.

After nine months of Eclipse 3.3, the pde-maven-plugin has still not been officially updated to handle the changes from Eclipse 3.2 (we had to use the Alpha 2 snapshot version downloaded from Codehaus) and so we're concerned that we might have similar upward compatibility problems when Eclipse 3.4 comes out in a few months.

When it's all been finalised, I hope to publish a longer explanation of how our build system works here. Don't hold your breath though; at the moment we still have to solve a few issues - for example, we need to figure out how to produce the source plug-ins alongside the executable ones.

Taking the pain out of Eclipse RCP builds and exports

It is well worth reading the manual, as the old saying goes - but sometimes the manual can be misleading, especially if the software being documented has evolved since it was written.

Eclipse Help

If you're using Eclipse 3.3, the advice in the 2006 edition of The Book about enabling Help in your Eclipse RCP applications is no longer quite accurate. Section 13.2 ("Getting the Help Plug-ins") tells you to install lots of things, while my experimentation has shown that all you need are the following plug-ins in your target platform and any launch configuration, plus their dependencies:

• org.eclipse.help.ui
• org.eclipse.help.webapp

You will also find this information in the built-in help provided with Eclipse RCP Developer under Platform Plug-in Developer Guide -> User assistance support -> Help -> Configuration/setup -> Rich Client Platform (RCP) help. On discovering it, I felt a little like Arthur Dent finding the planning application for the bypass through his house (look for the telling phrase "Beware of the Leopard")! Interestingly, the on-line version of the same help page disagrees - listing a lot of superfluous plug-ins - which confused me for quite a while.

The dependencies, all of which you can copy from the plugins folder of your Eclipse installation to your target platform, are as follows:

• org.eclipse.help.base
• org.apache.jasper
• org.apache.lucene
• org.apache.lucene.analysis
• org.eclipse.equinox.http.jetty
• org.eclipse.equinox.http.registry
• org.eclipse.equinox.http.servlet
• org.eclipse.equinox.jsp.jasper
• org.eclipse.equinox.jsp.jasper.registry
• org.eclipse.osgi.services
• javax.servlet
• javax.servlet.jsp
• org.mortbay.jetty
• org.apache.commons.logging
• org.apache.commons.el

Also, to use Eclipse Help successfully under Windows (and that includes your own help for exported / launched applications), you have to have your Internet Explorer preferences set up to use no proxy server when accessing localhost / 127.0.0.1. One way to do this in Internet Explorer's Tools -> Internet Options -> Connections -> LAN Settings is to turn off all three main check boxes. You could also try setting the "auto detect settings" checkbox or the manual proxy server configuration (provided you also check the "bypass proxy server for local addresses" checkbox).

Editing and Debugging

For some things, Eclipse needs source plug-ins to be present in the target platform location, not just in the installation location.I have not yet established all the circumstances under which this may be the case, but I have found that if you are trying to use the special-purpose editor to work on a plugin.xml file, unless the source plug-ins are present in the target platform location, when you right-click on an extension point in the Extensions tab you won't see the correct extension types in the context menu (just "Generic").

To support the most common types of UI extension, such as menu entries, copy the source plug-ins for org.eclipse.platform and org.eclipse.rcp from your Eclipse plugins folder into the target platform (hint: they are folders!). It is not even necessary to select them when specifying the target platform in Eclipse Preferences, but does no harm.

Source organisation

It turns out that the line of least resistance is to separate your plug-ins and features into two groups:

• All the stuff you have access to in source form directly from its repository
• Everything else - basically the Eclipse and third-party stuff

The second category forms your Eclipse RCP target platform, which should be a minimal Eclipse RCP installation plus all the features and plug-ins and their dependencies that you need in your application, plus source plug-ins where available. I created a target platform initially by downloading the RCP Runtime Platform plus Delta Pack (version 3.3.1.1, which matches the version of Eclipse RCP Developer I was using), but then deleted a lot of things that were not relevant to my intended deployment architecture before adding the plug-ins and features I really needed (see above).

For the first category, I recommend setting up your source folder structure to match the output in the build directory.

• product root
• features
• a.b.c.d.feature
• etc.
  
• plugins
• a.b.c.d
• a.b.c.d.product
• a.b.c.d.update-site
  
• etc.

It helps if you append "feature", "product" or "update-site" as the last component of the folder name, as appropriate, where a folder contains something other than a plug-in. Once each of these has been turned into an Eclipse project in your workspace, they will simply appear as a linear alphabetically ordered list in the package explorer, so this helps you to navigate.

Ignore the features supplied with the Eclipse SDK, RCP Runtime and Delta Pack. Instead, define your own features to include just the plug-ins on which your application depends. It helps to factorise this into a half-dozen features that collect Eclipse plug-ins for various capabilities - e.g. the feature com.example.myproject.help.feature could be used to list all the plug-ins shown above that are needed to provide application help. Then if you want to include help with an application, all it has to do is list this feature in its application feature.xml as an included feature (of course you still have to write the help texts and create a help plug-in to contain them!).

Version numbering

In product.xml and feature.xml files, it is a good idea to set the version numbers of included plug-ins and features to 0.0.0 (or a range, where applicable) to ensure that the latest version is always used during the build - the exported feature.xml files will end up containing the resolved version numbers.

More on the Maven POM for PDE headless builds

It turns out that you can override almost any property in the build.properties file using an equivalent plugin configuration item within the buildProperties section.

Moreover, by observing certain conventions the job is made much easier. One of these is that the PDE build takes place within a "build directory" that contains subdirectories named "features" and "plugins", and that the product's project directory (where pom.xml resides) is one of the subdirectories of "plugins". Therefore, in the Maven POM, it would be logical to define buildDirectory as "../..". Here's a possible approach:

  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>pde-maven-plugin</artifactId>
        <version>1.0-alpha-2-SNAPSHOT</version>
        <extensions>true</extensions>
        <!-- Custom lifecycle configuration -->
        <configuration>
          <eclipseInstall>${env.M2_HOME}/../eclipse</eclipseInstall>
          <pdeProductFilename>prototyp.product</pdeProductFilename>
          <pdeBuildVersion>3.3.2.R331_v20071019</pdeBuildVersion>
          <buildProperties>
             <base>${maven.work.dir}</base>
             <baseLocation>${maven.work.dir}/eclipse</baseLocation>
             <buildDirectory>${basedir}/../..</buildDirectory>
          </buildProperties>
        </configuration>
        <!-- Also bind to mvn clean -->
        <executions>
          <execution>
            <id>clean-pde</id>
            <phase>clean</phase>
            <goals>
              <goal>clean</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

Please note: we use a convention that the Maven installation is alongside the Eclipse IDE installation, hence the definition of eclipseInstall above. We also define the maven.work.dir property in the settings.xml file - this is where all the stuff goes that Maven needs, e.g. the local repository and the Eclipse PDE target platform.

Currently I am working on a Maven plugin that will assemble the Eclipse PDE target platform automatically from project dependencies. I'll document this here once it's working.

Further background information about Maven PDE builds

I was asked in a comment to publish the settings.xml file for my Maven installation, because the correspondent was not able to build the example project successfully.

Our project makes use of a local Maven proxy server to mirror the combined contents of the Maven Central repository, various snapshot repositories and the project's own built artifacts. In the Maven conf directory, we have therefore installed the following settings.xml file, which reflects this arrangement:

<settings>
  <servers>
    <server>
      <id>project-repo</id>
      <username>...</username>
      <password>...</password>
    </server>
  </servers>
  <profiles>
    <profile>
      <id>main-profile</id>
      <properties>
        <!-- settings.localRepository is set by the USER settings.xml -->
        <!-- Maven working directory -->
        <maven.work.dir>${settings.localRepository}/..</maven.work.dir>
        <!-- Clover working directory: clover license is expected in ${clover.work.dir}/license -->
        <clover.work.dir>${maven.work.dir}/clover-work</clover.work.dir>
        <!-- Additional plugin provider path for Eclipse target platform -->
        <plugin.target.dir>${maven.work.dir}/plugin_target_dir</plugin.target.dir>
        <!-- project sites will be deployed to ${site.dir} with mvn site:deploy -->
        <site.dir>${maven.work.dir}/maven_site</site.dir>
      </properties>
      <repositories>
        <repository>
          <id>central</id>
          <name>Internal Mirror of Central Repositories</name>
          <url>http://....:9999/repository/</url>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <id>central</id>
          <name>Internal Mirror of Central Plugins Repository</name>
          <url>http://....:9999/repository/</url>
        </pluginRepository>
      </pluginRepositories>  
    </profile>
  </profiles>
  <activeProfiles>
    <activeProfile>main-profile</activeProfile> 
  </activeProfiles>
</settings>

In the .m2 directory under the user's home directory, we install a minimal settings.xml, which simply indicates where to find the local Maven repo on your hard disk. However, I have customised my copy so that I can do builds when not connected to the company network but with access to the Internet. This shows you where you can obtain things like the PDE Maven plugin snapshot.

<settings>
  <localRepository>C:\data\...\maven_repo</localRepository>
  <profiles>
    <profile>
      <id>publicRepos</id>
      <activation>
        <property>
     <name>searchPublicRepos</name>
     <value>true</value>
   </property>
      </activation>
      <repositories>
        <repository>
          <id>codehausSnapshots</id>
          <name>Codehaus Snapshots</name>
          <releases>
            <enabled>false</enabled>
            <updatePolicy>always</updatePolicy>
            <checksumPolicy>warn</checksumPolicy>
          </releases>
          <snapshots>
            <enabled>true</enabled>
            <updatePolicy>never</updatePolicy>
            <checksumPolicy>fail</checksumPolicy>
          </snapshots>
          <url>http://snapshots.maven.codehaus.org/maven2</url>
          <layout>default</layout>
        </repository>
        <repository>
          <id>mavenCentral</id>
          <name>Maven Central</name>
          <releases>
            <enabled>true</enabled>
            <updatePolicy>daily</updatePolicy>
            <checksumPolicy>warn</checksumPolicy>
          </releases>
          <snapshots>
            <enabled>false</enabled>
            <updatePolicy>never</updatePolicy>
            <checksumPolicy>warn</checksumPolicy>
          </snapshots>
          <url>http://repo1.maven.org/maven2/</url>
          <layout>default</layout>
   </repository>
        <repository>
          <id>mavenSnapshots</id>
          <name>Maven Snapshots</name>
          <releases>
            <enabled>false</enabled>
            <updatePolicy>never</updatePolicy>
            <checksumPolicy>warn</checksumPolicy>
          </releases>
          <snapshots>
            <enabled>true</enabled>
            <updatePolicy>daily</updatePolicy>
            <checksumPolicy>warn</checksumPolicy>
          </snapshots>
          <url>http://people.apache.org/repo/m2-snapshot-repository/</url>
          <layout>default</layout>
   </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <id>codehausSnapshots</id>
          <name>Codehaus Snapshots</name>
          <releases>
            <enabled>false</enabled>
            <updatePolicy>always</updatePolicy>
            <checksumPolicy>warn</checksumPolicy>
          </releases>
          <snapshots>
            <enabled>true</enabled>
            <updatePolicy>never</updatePolicy>
            <checksumPolicy>fail</checksumPolicy>
          </snapshots>
          <url>http://snapshots.maven.codehaus.org/maven2</url>
          <layout>default</layout>
        </pluginRepository>
        <pluginRepository>
          <id>mavenCentral</id>
          <name>Maven Central</name>
          <releases>
            <enabled>true</enabled>
            <updatePolicy>daily</updatePolicy>
            <checksumPolicy>warn</checksumPolicy>
          </releases>
          <snapshots>
            <enabled>false</enabled>
            <updatePolicy>never</updatePolicy>
            <checksumPolicy>warn</checksumPolicy>
          </snapshots>
          <url>http://repo1.maven.org/maven2/</url>
          <layout>default</layout>
   </pluginRepository>
        <pluginRepository>
          <id>mavenSnapshots</id>
          <name>Maven Snapshots</name>
          <releases>
            <enabled>false</enabled>
            <updatePolicy>never</updatePolicy>
            <checksumPolicy>warn</checksumPolicy>
          </releases>
          <snapshots>
            <enabled>true</enabled>
            <updatePolicy>daily</updatePolicy>
            <checksumPolicy>warn</checksumPolicy>
          </snapshots>
          <url>http://people.apache.org/repo/m2-snapshot-repository/</url>
          <layout>default</layout>
   </pluginRepository>
      </pluginRepositories>
    </profile>
  </profiles>
</settings>

As I mentioned in passing in my original post, you also have to make sure that you (a) correct the version number of two dependencies in the pde-maven-plugin's POM, and (b) upload all the Eclipse plugins that the pde-maven-plugin needs to your local repository or to your project repository, using the eclipse:to-maven goal.

I have actually set up a Maven project to do upload Eclipse plugins repeatably and reliably in case we need to clean up and rebuild the project repository (e.g. after an upgrade of Eclipse). Here's the POM:

<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>....</groupId>
  <artifactId>devclipse-to-maven</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>pom</packaging>
  <name>Eclipse plugins to Maven Artifacts converter</name>
  <url>....</url>
  <description>....</description>

  <!-- ******************************************************************
    Inherit from project grandparent pom
    ****************************************************************** -->
  <parent>
    <groupId>....</groupId>
    <artifactId>configuration</artifactId>
    <version>1.8</version>
  </parent>

  <!-- ******************************************************************
    Environment Information
    ****************************************************************** -->
  <scm>
    <connection>
      scm:svn:http://..../devclipse_to_maven/
    </connection>
    <developerConnection>
      scm:svn:http://..../devclipse_to_maven/
    </developerConnection>
    <tag>HEAD</tag>
    <url>
      http://..../devclipse_to_maven/
    </url>
  </scm>

  <!-- ******************************************************************
    Build configuration
    ****************************************************************** -->
  <properties>
    <eclipse.source.dir>${env.M2_HOME}/../eclipse</eclipse.source.dir>
    <eclipse.ext.dir>${env.M2_HOME}/../ext/eclipse</eclipse.ext.dir>
    <eclipse.usr.dir>${env.M2_HOME}/../usr/eclipse</eclipse.usr.dir>
    <eclipse.target.dir>${project.build.directory}/eclipse</eclipse.target.dir>
  </properties>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>1.1</version>
        <executions>
          <execution>
            <id>copy-plugins-and-features</id>
            <phase>process-resources</phase>
            <goals>
              <goal>run</goal>
            </goals>
            <configuration>
              <tasks description="Grab the features and plugins from
               the Eclipse SDK that have been found empirically to
               be required as part of the target platform. We will
               upload them to the Maven repository in the next phase.">
                <mkdir dir="${eclipse.target.dir}/plugins"/>
                <copy todir="${eclipse.target.dir}/plugins">
                  <fileset dir="${eclipse.source.dir}/plugins">
                    <include name="com.ibm.icu_*.jar"/>
                    <include name="org.eclipse.ant.core_*.jar"/>
                    <include name="org.eclipse.compare_*.jar"/>
                    <include name="org.eclipse.core.commands_*.jar"/>
                    <include name="org.eclipse.core.contenttype_*.jar"/>
                    <include name="org.eclipse.core.databinding_*.jar"/>
                    <include name="org.eclipse.core.expressions_*.jar"/>
                    <include name="org.eclipse.core.filebuffers_*.jar"/>
                    <include name="org.eclipse.core.filesystem_*.jar"/>
                    <include name="org.eclipse.core.jobs_*.jar"/>
                    <include name="org.eclipse.core.net_*.jar"/>
                    <include name="org.eclipse.core.resources_*.jar"/>
                    <include name="org.eclipse.core.runtime.compatibility.auth_*.jar"/>
                    <include name="org.eclipse.core.runtime_*.jar"/>
                    <include name="org.eclipse.core.variables_*.jar"/>
                    <include name="org.eclipse.debug.core_*.jar"/>
                    <include name="org.eclipse.equinox.app_*.jar"/>
                    <include name="org.eclipse.equinox.common_*.jar"/>
                    <include name="org.eclipse.equinox.preferences_*.jar"/>
                    <include name="org.eclipse.equinox.registry_*.jar"/>
                    <include name="org.eclipse.help_*.jar"/>
                    <include name="org.eclipse.jdt.core_*.jar"/>
                    <include name="org.eclipse.jdt.debug_*/**/*"/>
                    <include name="org.eclipse.jdt.launching_*.jar"/>
                    <include name="org.eclipse.jface.databinding_*.jar"/>
                    <include name="org.eclipse.jface.text_*.jar"/>
                    <include name="org.eclipse.jface_*.jar"/>
                    <include name="org.eclipse.osgi_*.jar"/>
                    <include name="org.eclipse.rcp.source.win32.win32.x86_*/**/*"/>
                    <include name="org.eclipse.swt.win32.win32.x86_*.jar"/>
                    <include name="org.eclipse.swt_*.jar"/>
                    <include name="org.eclipse.team.core_*.jar"/>
                    <include name="org.eclipse.team.ui_*.jar"/>
                    <include name="org.eclipse.text_*.jar"/>
                    <include name="org.eclipse.ui_*.jar"/>
                    <include name="org.eclipse.ui.console_*.jar"/>
                    <include name="org.eclipse.ui.editors_*.jar"/>
                    <include name="org.eclipse.ui.forms_*.jar"/>
                    <include name="org.eclipse.ui.ide_*.jar"/>
                    <include name="org.eclipse.ui.navigator_*.jar"/>
                    <include name="org.eclipse.ui.navigator.resources_*.jar"/>
                    <include name="org.eclipse.ui.views_*.jar"/>
                    <include name="org.eclipse.ui.views.properties.tabbed_*.jar"/>
                    <include name="org.eclipse.ui.workbench_*.jar"/>
                    <include name="org.eclipse.ui.workbench.texteditor_*.jar"/>
                    <include name="org.eclipse.update.configurator_*.jar"/>
                    <include name="org.eclipse.update.core_*.jar"/>
                    <include name="org.eclipse.update.ui_*.jar"/>
                  </fileset>
                </copy>
                <mkdir dir="${eclipse.target.dir}/features"/>
                <copy todir="${eclipse.target.dir}/features">
                  <fileset dir="${eclipse.source.dir}/features">
                    <include name="none_*"/>
                  </fileset>
                </copy>
                <copy todir="${eclipse.target.dir}/plugins"
                  failonerror="false">
                  <fileset dir="${eclipse.ext.dir}/plugins">
                    <include name="org.polarion.team.svn.client.javahl.win32_*/**/*"/>
                    <include name="org.polarion.team.svn*.jar"/>
                    <include name="org.sf.easyexplore_*.jar"/>
                  </fileset>
                </copy>
                <copy todir="${eclipse.target.dir}/features"
                  failonerror="false">
                  <fileset dir="${eclipse.ext.dir}/features">
                    <include name="none_*"/>
                  </fileset>
                </copy>
                <copy todir="${eclipse.target.dir}/plugins"
                  failonerror="false">
                  <fileset dir="${eclipse.usr.dir}/plugins">
                    <include name="none_*"/>
                  </fileset>
                </copy>
                <copy todir="${eclipse.usr.dir}/features"
                  failonerror="false">
                  <fileset dir="${eclipse.ext.dir}/features">
                    <include name="none_*"/>
                  </fileset>
                </copy>
              </tasks>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-eclipse-plugin</artifactId>
        <version>2.4</version>
        <executions>
          <execution>
            <id>installation</id>
            <phase>install</phase>
            <configuration>
              <eclipseDir>${eclipse.target.dir}</eclipseDir>
            </configuration>
            <goals>
              <goal>to-maven</goal>
            </goals>
          </execution>
          <execution>
            <id>deployment</id>
            <phase>deploy</phase>
            <configuration>
              <eclipseDir>${eclipse.target.dir}</eclipseDir>
              <deployTo>${distributionManagement.repository.id}::default::${distributionManagement.repository.url}</deployTo>
            </configuration>
            <goals>
              <goal>to-maven</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

  <!-- ******************************************************************
    Project Information
    ****************************************************************** -->
  <organization>
    <name>....</name>
    <url>http://..../</url>
  </organization>

  <developers>
    <developer>
      <id>ihu</id>
      <name>Immo H&uuml;neke</name>
      <email>ihu@zuhlke.com</email>
      <url>http://aspsp.blogspot.com/</url>
      <organization>Z&uuml;hlke Engineering</organization>
      <organizationUrl>http://www.zuhlke.com/</organizationUrl>
      <roles>
        <role>architect</role>
        <role>developer</role>
      </roles>
      <timezone>0</timezone>
      <properties>
        <picUrl>http://www.spaconference.org/cgi-bin/wiki.pl/?mugimmohuneke.jpg</picUrl>
      </properties>
    </developer>
  </developers>

  <inceptionYear>2008</inceptionYear>

</project>

Hope this helps. If you still have problems, please e-mail me your Maven output - I may be able to guess where it is going wrong.

Where's the system bundle jar file? (cont.)

Well, you probably won't be surprised to learn that the result is no different using the latest maven-eclipse-plugin version 1.5-SNAPSHOT, dated 30th November 2007. I had to create an entire Maven project to invoke the to-maven goal on the maven-eclipse-plugin, just so that I could specify the version I wanted to run. But this should be quite a useful asset in the long run anyway: I can now type

mvn install

or

mvn deploy

to invoke the corresponding actions. At the moment, however, there is no way to get beyond any plugin whose MANIFEST.MF specifies system.bundle as a required bundle.

The OSGI Bundle Repository Browser confirms that system.bundle does in fact refer to the org.eclipse.osgi jar file. I therefore decided to supply the missing artifact, by using file:deploy-file to create it from org.eclipse.osgi...jar. I added this to the POM of my eclipse-to-maven project and linked it to the initialize phase.

It still doesn't work. For some reason, because the Require-Bundle instruction in MANIFEST.MF does not specify a version number, the maven-eclipse-plugin cannot find the artifact I created. It tells me that it was unable to resolve the version range for version [0,).

So as an experiment, I altered the MANIFEST.MF of the Xerces plugin to explicitly require the actual version number of the org.eclipse.osgi bundle - and hey presto, this was resolved successfully. But we really can't go through all manifests of all Eclipse plugins every time we want to run this plugin and update the system.bundle requirement!

This must be a bug in the maven-eclipse-plugin (caused by a documented incompatibility between Eclipse 3.2 and 3.3, which has led a number of bundle committers to add "Require-bundle: system.bundle" to their manifests), so I have submitted a bug report about it.

In fact, on closer investigation it transpired that there are currently only two Eclipse plugins that fall into this category: org.eclipse.xerces and org.eclipse.xml.resolver. In fact it wasn't even necessary to create a system.bundle artifact - I just altered the two manifests temporarily to express a dependency on the exact version of the org.eclipse.osgi plugin supplied with the Eclipse installation in question. With this modification, the eclipse:to-maven goal was able to run successfully to completion.

On to the next stage! But first, there is a weekend to take care of...

Where's the system bundle jar file?

Quick update to my last posting: a number of Eclipse plugins have recently sprouted this RequireBundle attribute thanks to a published incompatibility between Eclipse 3.2 and 3.3 in the area of Boot Delegation.

It looks as if the System Bundle is either mapped to org.eclipse.osgi or doesn't really exist as a separate jar file in its own right at all. Maybe there's a newer version of the Maven Eclipse plugin that understands this?

Creating Maven artifacts from Eclipse plugins

Along the long and tortuous road of creating Maven builds for Eclipse RCP applications, one of the first steps is to take the Eclipse plugins that make up the target platform and convert them into Maven artifacts. This ensures that the Maven POM is able to refer to them by its holy triad of group ID, artifact ID and version.

In the Maven Eclipse Plugin, a goal eclipse:to-maven is provided that is meant to satisfy exactly this need. Given an Eclipse installation directory, it trawls through all the plugins it can find and installs them in your local Maven repository, even generating POMs that correctly show interdependencies between plugins. If you optionally specify a deployTo parameter, it will even deploy the resulting artifacts to a remote repository for everyone on your project to use.

When I tried this for my Eclipse RCP installation, however, it failed. There were two versions of the plugin org.apache.xerces in the plugins folder, and Maven could not resolve which of them to turn into a Maven artifact. This might not have been a problem but for the fact that the version numbers of both were the same (2.0.8) apart from the timestamp - one was a folder, the other a self-contained jar file.

The same problem has also been encountered by others - e.g. when trying to use the slightly older but equivalent eclipse:make-artifacts goal.

Having checked in the Eclipse Plugin View that there was nothing dependent on it, I removed the older of the two Xerces plugins. The to-maven goal was now able to run - but the build still didn't complete (see below).

I consider this to be a bug in the Eclipse RCP distribution version 3.3.1.1. Despite a lengthy discussion in the bug tracker five years ago, nothing appears to have been done. I have accordingly lodged a new bug report and we'll see whether this gets fixed in a future release.

The new problem now is the generated POM for org.apache.xerces. For some reason the Maven eclipse plugin is unable to determine the version number of some dependency (the error message doesn't make a lot of sense):

[INFO] Unable to resolve version range for dependency
Dependency {groupId=system, artifactId=bundle,
version=[0,), type=jar} in project org.apache.xerces

Inspection of the MANIFEST.MF in the Xerces plugin jar file reveals:

Bundle-Version: 2.8.0.v200705301630
Require-Bundle: system.bundle,org.apache.xml.resolver;
visibility:=reexport

So that is why Maven can't be sure what version is wanted. Luckily, this particular artifact is already in the Maven central repository, so to short-circuit all this I removed the Xerces plugin jar temporarily as well and re-ran the Maven command.

To my surprise, the build failed this time with the following message:

[INFO] Unable to resolve version range for dependency
Dependency {groupId=org.apache, artifactId=xerces,
version=[2.8.0,3.0.0), type=jar}
in project org.eclipse.wst.xml.core

This makes no sense, since version 2.8.0 of org.apache.xerces clearly does exist in the central repository. Maybe Maven doesn't consider 2.8.0-v200705301630 to be in the above range?

So I tried another tack: restore the older Xerces plugin to the Eclipse plugins folder. This time the Xerces plugin is successfully installed in the local repository, and the plugin org.eclipse.wst.xml.core likewise. Instead, the build fails a bit later with:

[INFO] Unable to resolve version range for dependency
Dependency {groupId=system, artifactId=bundle,
version=[0,), type=jar}
in project org.apache.xml.resolver

This is exactly the problem that the Xerces plugin failed on in the first instance. So I suspect that I'm going to have to find out how to create a Maven artifact called system.bundle. Watch this space.

Build Eclipse RCP products using Maven 2 - how hard can it be?

I am working as part of a project team developing a suite of applications that use Eclipse RCP as their platform. We are also using CruiseControl for continuous integration and testing. We currently build Eclipse RCP products, features and fragments using Eclipse PDE, partly because of its OSGI support. We would like to move to using Maven 2 in order to match the way we build all our Eclipse plugins and Java libraries. This will make it straightforward to generate comprehensive documentation on the Web, stay in control of dependencies, link into our bug tracking and progress tracking system and so on.

A parallel initiative is my attempt to set up the Eclipse development environment for each team member so that Maven builds and Eclipse builds work seamlessly together on the workstation. Initially I thought that this would require the use of the Maven 2 Integration for Eclipse plugin. However, it turned out that even this is not nearly as closely integrated into Eclipse as Ralf Quebbemann's equivalent Maven 2 Integration is integrated into IntelliJ IDEA, for example. We found it easiest to configure various Maven goals as "external tools" in Eclipse's Run menu and to use the Maven eclipse plugin to generate or update the Eclipse classpath and project files (mvn eclipse:eclipse).

[Footnote: mvn eclipse:configure-workspace works during installation, but only if the configured Maven repositories are online, which rather defeated my objective of creating a standalone developer kit installer. I found that this goal simply inserts a definition of the M2_REPO classpath variable into one of Eclipse's metadata files, so I made the installer do that directly instead. It now runs much faster too.]

Despite the existence of the book Better Builds with Maven, a common complaint about Maven 2 is that it is not sufficiently well documented for heavy-duty use. The book was last updated in August 2007, since when two further releases of Maven have come out. The book doesn't mention Eclipse RCP or PDE. In trying to marry the use of Maven with Eclipse, we are very much having to hoe our own furrow.

I'm using this blog to record the approaches we take and how we fare. Hopefully after a few weeks (not too many!) it will be possible to summarise all the findings into a "how to" for future reference.

My current understanding is that the most promising approach would be using a combination of the maven-eclipse-plugin (with the PDE property set to true) and the pde-maven-plugin. These allow us to create the Eclipse project and classpath files slightly differently in order to permit "headless" PDE builds to take place. As I understand it, the PDE Maven plugin generates an Ant build file that can then be used to run the headless PDE build. There are a few wrinkles; for example we may have to obtain a more recent snapshot of the pde-maven-plugin because the last official release doesn't appear to work with Europa.

One project which does appear to have been successful in building RCP products using Maven is jLibrary, an open source document management / content management system. Martin Perez and Ryszard Petrowski have documented their build approach for jLibrary 1.2. I intend to study this and learn from it. More anon!