ReadMore

Me and my friend are currently working on a term paper about Google Fusion Tables. It’s about it’s possibilities and potential. Our goal is to create a cross-platform mobile application using Sencha Touch, therefore we need to write a lot of JavaScript code for the UI and to access the Google Fusion Tables. Fortunately Google provides an inofficial JSONP API, so we get native JavaScript access without another server roundtrip (there is a good blog post explaining the usage with jQuery).

The next question is: if we write tons of code in JavaScript, how are we going to unit test this code. We already used QUnit in a former project and it’s fairly easy to use, so we decided to use that. This all works very well, you get a nice web page where all your tests run in the browser and you get nice output, everything’s fine.

But, wait! What about our build? We use Jenkins CI, which is configured to build our project every time we push new code to GitHub. Wouldn’t it be nice if our tests run when we build, so we get immediate feedback and never ever forget to run them and fix our code?

Yes, it definitely would be!

Now we have several problems

• How do I run JavaScript without a browser?
• Does Jenkins “speak” JavaScript?
• Is there a solution for ant? Do I need another tool? Another server?
• How do I get my test results as XML, so Jenkins can handle them?

I checked serveral solutions to my this problems: From node.js to Rhino (there is even an ant based unit testing framework using Rhino) to TestSwarm. They were all too limited or too bloated. I couldn’t believe this is such a big deal to run QUnit JavaScript code, get the results and publish them in Jenkins.

This is when I found out about PhantomJS, a nice and handy headless WebKit browser, that runs JavaScript code. Exactly what I need. As we are using Amazon EC2 for this project, I tried to install PhantomJS on a basic Amazon AMI. I just couldn’t get it to work, when I asked in Twitter if someone succeeded to setup PhantomJS. Philipp Küng pointed out (thanks again!) that it is much easier to setup with Ubuntu. Well then, launch a new Instance on EC2, choose the latest Ubuntu AMI, and after some simple commands you’re good to go:

sudo add-apt-repository ppa:jerome-etienne/neoip
sudo apt-get update
sudo apt-get install phantomjs

After that I had to setup Xvfb (virutal framebuffer), because EC2 instances are headless. Generally see the PhantomJS Wiki for further information.

Now I have a PhantomJS instance, the published QUnit tests and Jenkins. To be able to retrieve the data from PhantomJS I created a simple PHP script, which runs on the new instance and just executes a shell command (idea from this blog post).

<?php
/* Display 99 is configured on the server using Xvfb */
$value = shell_exec("DISPLAY=:99 phantomjs run_qunit.js http://gft.rdmr.ch/test/js/index.html junit-xml");
echo $value;
?>

With this URL I was able to create a simple ant target, which fetches the latest results from the tests:

<target name="test-js" depends="deploy">
    <get src="${test.url}" dest="${result.dir}/${result.js.file}"/>
</target>

Where ${test.url} referes to my PHP script, and ${result.dir} is configured as the location of JUnit reports in Jenkins (see screenshot):

Now the last missing piece is to get proper output for Jenkins. I adapted the QUnit test runner from the PhantomJS examples accordingly (called run_qunit.js, see above, here is my version). I added a new command line parameter type, which determines the type of output the runner generates. With the value junit-xml, the output can directly be used by Jenkins, this is the reason for the junit-xml parameter in the PHP script above.

The generated XML looks like that:

<?xml version="1.0"?>
<!--
 Tests completed in 2294 milliseconds.
36 tests of 36 passed, 0 failed. 
-->
<testsuite name="QUnit - JavaScript Tests" timestamp="2012-03-08T00:11:27Z" tests="36" failures="0" time="2.294">
<testcase name="PaintSwitzerland" classname="GoogleFusionTable">
</testcase>
<testcase name="Construtor" classname="GftLib">
</testcase>
<testcase name="Constants" classname="GftLib">
</testcase>
<testcase name="doGet" classname="GftLib">
</testcase>
<testcase name="doPost" classname="GftLib">
</testcase>
<testcase name="doGetJSONP" classname="GftLib">
</testcase>
<testcase name="doPostJSONP" classname="GftLib">
</testcase>
<testcase name="ExecSelect" classname="GftLib">
</testcase>
</testsuite>

And in Jenkins like that:

Overview of tests:

Failed test:

Seit letzter Woche habe ich SimpleTest im Einsatz und bin davon ziemlich begeistert. Das Framework bietet alles was man von einem konventionellen Unit-Testframework her kennt, wie diverse Assertion-Methoden oder Gruppierung von Testfällen zu Testsuiten. Daneben gibt es aber noch eine ganze Reihe von coolen Erweiterungen:

• Es gibt einige vordefinierte Test-Reports, welche den Erfolg (oder Misserfolg) der Tests anzeigen. Wenn einem diese nicht genügend kann man diese (dank OO) ganz einfach erweitern. So habe ich meinen “persönlichen” Testreporter geschrieben, welcher mir neben den Details zu den Fehlern auch die Details zu den erfolgreichen Tests beschreibt (dazu muss man nur die Methode paintPass() überschreiben), das Resultat ist in obigem Bild ersichtlich.
• Man kann auch sogenannte Web-Tests machen, d.h. man kann das Browserverhalten simulieren und so beispielweise automatisiert ein Formular ausfüllen und abschicken lassen, anschliessend dann das Ergebnis prüfen.
• Das erzeugen von Mock-Objekten ist kinderleicht, das Framework bietet eine einfache Methode um eine “leere” Hülle eines Objekts zu generieren (z.B. einer Datenbankverbindung)

Das Tutorial führt einem durch die ersten Schritte, die API gibt den vollständigen Überblick. Durch die verschiedenen Reporter kann das gleiche Skript auch gebraucht werden um in einem automatisierten Build zu laufen, so wie man es z.B. von JUnit kennt.

Ich habe momentan alle meine Testfiles in einem Verzeichnis und habe mir deshalb ein simples Skript geschrieben, welche alle Tests ausführt:

[Pastie-Link]

Um jedes einzelne File jedoch immer noch separat ausführen zu können, include ich autorun.php nur genau dann, wenn ich das File direkt anspreche:

if (count(get_included_files()) == 1)
{
    require_once("includes/simpletest/autorun.php");
}

Ich kann das Framework also nur empfehlen!

I really like unit testing, and nowadays there is an helping framework for almost every environment and/or programming language. For PL/SQL this is utPLSQL, which consists of a bunch of packages that you need to put on your database and then you should be good to test your could. You should.

I just spent some hours debugging my test code, so I want to share this with you, maybe I can save someone out there some time.

My setting

• Schema A: utPLSQL package
• Schema B: package XY (code to test), package UT_XY (test code)

I have two schemas A and B. In schema A utPLSQL is installed, and in B is the code to test. It is recommended to have your code to test and your test code in the same schema.

The following script is used to run the tests:

 begin
   for c in (select object_name,owner
               from all_objects
              where object_name like 'UT\_%' escape '\'
                and object_type ='PACKAGE')
   loop
     begin
     utPLSQL.test(
       package_in => substr(c.object_name,4),
       prefix_in => 'UT_',
       owner_in => c.owner
     );
   exception
     -- in case of a faulty unittest continue with the others.
     when others then
       null;
   end;
   end loop;
 end;
 /

But I kept getting the error:

 Program named "XY" does not exist.

So I thought that somehow the schema A does not “see” the packages in the schema B and created public synonyms. Unfortunately this didn’t help. Then I tried to move the “UT_” package to schema A, which worked, but now I got an error that the code to test and the test code are in different schemas.

There must be something obvious I’m missing!

And there it was: A had not “execute” grant on the packages in B.

  grant execute on b.xy to A;

Now everything just worked fine.