Doomed to oblivion: a very promising cure for cancer

Read about research by Professors Justyna Leja and Magnus Essand into a virus that can destroy pancreatic cancer cells without noticeable side effects. Then share their frustration that no company or government is prepared contribute the ridiculously small sum of £2M needed to fund clinical trials. Without these, the treatment will never be licensed for use in humans.

At the time of writing, the campaign to crowdsource the necessary funding has been running for almost two weeks, with a further four to go - but less than 10% of the total has been raised so far. This needs a massive push now! Please alert your friends.

Moving Mail to a New Laptop

My wife was getting fed up with her eMac running slower and slower, plus incompatibility between her MS Office X and the MS Office 2010 in use at her company, so for her birthday she got a Windows 7 laptop. Until the weekend just gone, however, she didn't make much use of it because it was beyond her ability to move her mail account from Entourage to Outlook and to move all her files across. The address book was a particular problem.

Eventually we found ways to achieve almost all these migrations. The document files were the easiest part - local area network connection, turn on Windows sharing on the Mac, just drag the files over (we had to retry a few times because the connection reset while copying long files).

To migrate the mail messages, we decided to use Thunderbird instead of Outlook (this PC doesn't connect to a corporate Exchange server). Drag each folder from Entourage to a Finder folder or the desktop. This exports the messages in the form of a .mbox file. Rename to remove the extension and copy them to Thunderbird's Mail folder in the current profile.

The mail addresses turned out to be rather a tougher proposition. You can export each address card to a .vcf file, but Thunderbird found no useful information in them. It dawned on me later that this might be a line-ending issue - we could have run mac2unix and then unix2dos before importing. But by then we had solved the problem in a different way.

Entourage allows you to drag address cards to the Mac's native address book application. This was synchronized with Plaxo using my wife's account. Plaxo supports export of the address book in LDIF format, which Thunderbird was able to import without problems. If you can't use this route, you can also export the entire Mac address book in a single operation to .vcf (or possibly even to LDIF). I suggest adjusting the line endings to Windows conventions before attempting to import to Thunderbird.

Then it turned out that the vast majority of e-mail addresses that she needed were not in either address book. Entourage just maintains a cache of recently used addresses and we could find no way to export all of these into any sort of useful file format. To the rescue came Adam Haeder with a simple shell script to extract plausible e-mail addresses from mbox files. I was able to adapt it for use under Cygwin and enhance it a bit to cope with non-US addresses and escaped newlines and spaces. The resulting tab-delimited file was suitable for import to Thunderbird.

 #!/bin/bash  
   
 # This script will parse an mbox file, displaying all of the From: email addresses, removing ones that  
 # are from postmaster, mail admins, etc  
   
 FILE=$1  
   
 if [ ! -r $FILE ]; then  
  if [ -r /var/spool/mail/$FILE ]; then  
  FILE="/var/spool/mail/$FILE"  
  else  
  echo "Sorry! Neither $FILE nor /var/spool/mail/$FILE exists, or I can't read them"  
  exit  
  fi  
 fi  
   
 # Using cat to read the input means you can run this over many mbox files simultaneously  
 # dos2unix and mac2unix standardise line endings to LF only  
 # First sed script joins lines that have been split with a "=" at line end  
 # grep isolates lines starting with "From:" and egrep -vi rejects all non-human addresses  
 # Next grep discards any lines that contain no email address (@)  
 # The next sed script turns =20 into spaces and discards trailing spaces and "From:"  
 # The next converts [mailto:x@y.z] to <x@y.z> form  
 # Any bare SMTP addresses are converted to "x@y.z <x@y.z>"  
 # Any SMTP addresses with no friendly name but in "<>" delimiters are converted similarly  
 # Finally the "<>" delimiters are removed and replaced with a tab separator  
 # Result is sorted uniquely (ignore case & leading blanks) and converted to DOS line endings  
 cat $FILE | dos2unix.exe | mac2unix.exe |\  
 sed '/=$/{N;s/=\n//}' |\  
 grep "^From:" | egrep -vi \  
 "(postoffice|\  
 postman|\  
 administrator|\  
 bounce|\  
 MAILER-DAEMON|\  
 postmaster|\  
 Mail Administrator|\  
 Auto-reply|\  
 out of office|\  
 Mail Delivery System|\  
 Email Engine|\  
 Mail Delivery Subsystem|\  
 Mail.Administrator|\  
 non.deliverable)" |\  
 grep '@' |\  
 sed 's/=20/ /g;s/\s*$//;s/^From:\s*//' |\  
 sed 's/\[mailto:\(.*\)\]/<\1>/g;s/"//g' |\  
 sed '/^[^<]*$/s/^\(.*\)$/\1 <\1>/' |\  
 sed 's/^<\(.*\)>$/\1 <\1>/' |\  
 sed 's/^\(.*\S\)\s*<\(.*\)>/\1\t\2/' |\  
 sort -ubf | unix2dos.exe  

BDD for JSP in Eclipse with Groovy Webtest

Pak Wing and I needed to put together a rapid prototype of a document generator that delivers web pages using custom templates together with various back-end sources of information. The first thing we did, of course, was to sketch out a test plan to identify the capabilities of our system. A simple first test was to see that the correct page title would be generated in response to a GET with varying parameters in the query string, including an invalid case.

The next question was how to execute the tests. My previous experience with Canoo Webtest had been favourable, so I was keen on this approach - but not on the traditional XML notation that Webtest uses. Since it has become feasible to use Groovy as the test scripting language, we decided to use this as it can support the behaviour-driven approach (where test cases are expressed in the "given, when, then" form), is a more compact notation than XML and easier for our stakeholders to read.

However, Canoo's manuals are a little light on detail about how to get all this to work in an Eclipse environment, so I'm recording my findings here as I go along.

Software to Install
Packages:

• Java SE JDK
• Eclipse
• Groovy
• Canoo Webtest
• An application server - in our case, we wanted to use JSP, so Tomcat version 7 was the obvious choice. As we're developing under Windows 7, we chose the Windows Service installer version (but note that the service should not be started - Eclipse will run the server when you're ready to deploy the application)

Eclipse plug-ins:

• Eclipse Web Developer Tools
• Eclipse XML Editors and Tools
• Groovy-Eclipse Feature (optionally plus sources)
• Web Page Editor (optional)
• Don't forget integration with your favourite Software Configuration Management system

I hope that was everything - please drop me a comment if you find you're missing some vital component that I've forgotten to mention.

Starting your Project
After a number of experiments, we found that the most straightforward approach was to create a new Dynamic Web Project and to convert it to a Groovy project afterwards, using the Groovy entry in the context menu. You can also convert to a Maven project subsequently, if that is your preferred build system and you have the M2E (Eclipse Maven Project) plugin installed.

I like Maven's organisation of source folders into src/main and src/test. Put your Java files (e.g. servlets) in src/main/java and test scripts in src/test/groovy. HTML and JSP files go into src/main/resources. Set up the Java Build Path appropriately.

It is standard practice to name the package for a JUnit test script identical to the package of the class under test. We followed this practice also for Groovy test scripts. A Groovy Webtest script is in fact a thinly disguised Ant build script, but you run it as a JUnit test (while avoiding all the tedium of writing JUnit test cases in Java). The Ant libraries are supplied with Webtest.

Running Your First Test
Here's my version of the simple test script shown on the Canoo home page. Note the addition of firewall configuration:



If you paste this into your own Eclipse project, you will immediately be informed that the import "com.canoo.webtest.WebtestCase" cannot be resolved. To fix this, open the Java Build Path configuration, click on the Libraries tab and add a new User Library. I called it WEBTEST_LIBS. To define this User Library, click "Add JARs", navigate to the lib folder under your Webtest installation directory, highlight all jar files you find there and click OK.

Obviously you should amend the firewall configuration in the above constructor function TestGroovyWebTest() as appropriate.

To run the test script, simply right-click the Groovy file, select "Run as..." and click "JUnit Test". You should get both the standard JUnit output of a green bar (hooray!) and a Webtest monitor window that shows the test in progress, followed by a result screen in your web browser.

Create and Deploy a Dummy Application
For the next stage, we wanted the Groovy test script to actually interact with the application we were developing.

First configure your server. In Eclipse, go to the View menu, select "Show View", expand the "Server" section and select "Servers". If this is not available, you probably have not installed the Eclipse Web Developer Tools feature correctly.

Right-click in the Servers panel and select "New". Add the server (you should find it in a pull-down list). Configure the correct run-time environment for the application server you have installed, and name it (probably "localhost"). Click "Finish" (don't configure any apps at this stage).

Create a very simple index.jsp page under src/main/resources:



Now you can right-click the JSP file, select "Run as..." and click "Run on server". The output ought to appear in a new edit panel.

You can now add a test case to invoke your new app and check that it has produced the right page:



Gotchas
There are a few problems to look out for:

• If you collaborate with someone else on your project, and their Eclipse and other configurations do not match yours precisely, you may find that some of the build path configuration has to be changed after they import your project. To minimise this, make sure that for example you configure "workspace default" as the JRE for the project.
• In one case, we found that Webtests could not be run more than once. The reason was that each run produces a report under the same folder into which the Java and Groovy compilers place class files. For some reason, with certain installations of the Tomcat server, the synchronisation mechanism meant that the test reports were being deployed along with the application, which meant that Webtest could not delete them before running the next test. In these cases, we found that the only remedy was to restart Eclipse and then start Tomcat again - too tedious in the long run. It might be less hassle just to run the Groovy test script outside Eclipse, but we have not tried this.

Feed the world

Respect to my daughter Ruth for deciding to give up meat and dairy products for Lent. Her reasons are explained in her blog and she promises to post a series of mouthwatering recipes there.

Ubuntu Linux desktop 11.10 proxy settings solved

Watch out for system proxy settings in Ubuntu Desktop 11.10 (at least in its LXDE guise, which is what I am running in a VM). The control panel "Network" has disappeared and even if you install it using apt or Synaptic you'll find that it doesn't have any proxy options. gconf does, but seems to have no effect. Answer is to edit /usr/share/applications/chromium-browser.desktop and append

--proxy-pac-url="http://example.com/"

to the Exec line.