Tuesday, 17 November 2015

Practical GUI testing using SikuliX

Rejoice with me - after much banging of head against various walls, I have finally succeeded in building a GUI test out of Java, jUnit, Gradle and SikuliX 1.1.0. It starts a new game in the Mac OS Chess application, enters the opening move using drag/drop, and quits the application (answering correctly in the "save" dialog).

The secret was to give up on the idea of using the Python test script fragments created in the SikuliX IDE directly. Rather, I now translate these scripts line by line into Java; e.g. from

Each short Python script is converted to a simple method on an object representing the application or individual screen under test. These methods can then be invoked as fixtures from any test framework, e.g. jUnit or Fit. The resulting test suite is built with Gradle and can be run as a standalone application to test the target application end-to-end.

One of the tricky bits was to set the image search path correctly so that the images embedded in the jar file can be located by the SikuliX API, whether you're running in the IDE or standalone. By storing a dummy SikuliX script within the src/main/resources folder (e.g. images.sikuli) you can use the SikuliX IDE directly to capture and fine-tune images for use in your tests.

Depending on the CI environment of the project, another tricky part may be to provide both the application under test and the test suite with a graphical pseudo display on which to run, but there is quite a lot of advice in the SikuliX documentation as well as on Stack Overflow about that.

Next steps:

  • Test this approach under Windows and Linux too
  • Enable the use of FIT in place of jUnit (more appropriate for whole-system tests)
  • Make use of the optional Tesseract library to read back values from the screen
  • Build and run test suite in various CI environments


Immo Hüneke said...

Update: I have integrated SikuliX with Fit - by forgetting about using Python and going directly to the underlying SikuliX Java API instead. Contact me if you want to know more.

Charan said...

Dear Immo Hüneke,

Could you please share the build.gradle entries for SikuliX ?