KDevelop unit-tests run regularly

September 23, 2008 at 11:52 pm | Posted in KDE, KDevelop | 7 Comments

So we’re living in the year 2008 and test-driven development has reached open source communities since some time now. KDE4, especially kdelibs comes with quite a bunch of unit-tests (though still to few I guess, at least for some parts) which gives us a kind of “we’re safe” feeling.

Unfortunately that feeling is treacherous. Because writing unit-tests is only 33% of the whole thing, the other 66% are running them regurlarly and publishing the results of running. A million unit-tests don’t help anybody if they’re not run and thus nobody catches the introduced regressions early on. Ideally running unit-tests would be a pre-commit-thing, but unfortunately due to the size of our code-base thats not feasible. Also you’d need to run all of them, because a little change in kdecore might trigger a bug in kdeui which you wouldn’t see if you ran only kdecore unit-tests.

So far kdelibs unit-tests aren’t run on a regular basis – at least not to my knowledge and the results are also not published. Its not quite nice, but building kdelibs every night is a pretty expensive thing. Apart from that some of the automated tests even need an X11 server running.

For KDevPlatform and KDevelop I was able to “solve” the problem. The unit-tests are now being run every night on my machine and the summary of that run is posted to the kdevelop-devel list. Unfortunately we already have 3 failing tests in kdevplatform, not the best start. But we’ll get around to that.

PS: Using a KDE4 release build didn’t improve performance – SCNR🙂

7 Comments

  1. If you use git you can add a githook to automatically run the autotest that goes with the class you modified before commit. You can check out the hooks I wrote for Arora here:
    http://github.com/Arora/arora/tree/master/git_hooks

    And to make autotests I wrote an autotest generator:
    http://benjamin-meyer.blogspot.com/2007/11/auto-test-stub-generator.html

  2. This is a joke, right? Do you seriously want to tell me the debug build is exactly as fast as a release build? Sorry, but thats just silly.

  3. @Andre: To be more precise: Building qt-copy with -release instead of -debug doesn’t make a difference. Building kdelibs with RelWithDebInfo instead of Debug indeed produced a nice speedup in khtml, but nowhere else.

  4. Pre-commit hooks are not a special git feature – about any old SCM system has them. It’s just a matter of policy which ones to use/add.

  5. @Andres Pre-commit hooks on the local system are new. hooks on the server are old, but never used to QA. WIth it on the local system you can run intensive tests without fear of taking down the server or causing very long commit delays.

  6. @apaku: Use Release and export CFLAGS=-g instead of using RelWithDebInfo, or maybe try export CFLAGS=”-DNDEBUG -DQT_NO_DEBUG” with RelWithDebInfo, that way the debug output will be omitted and that speeds up KDE significantly.

  7. Let me know how well the regressions are being fixed in your project as I’ve been running unit tests every night and the results seem to be ignored by the developers.

    See http://www.koffice.org/testresults/status.html


Sorry, the comment form is closed at this time.

Blog at WordPress.com.
Entries and comments feeds.

%d bloggers like this: