apaku

Parser Work

I’ve made quite some progress the last days in the parser work for KDevelop4’s python support. That means the plugin itself doesn’t compile at all anymore 🙂

Thats quite ok, given that I’ve exchanged the parser generator (switched to the kdev-pg-qt) and that created a new AST from hand based on the Python 2.6 language reference. That new AST is closer to the actual structure of the code, whereas the generated AST is closer to how the language can be parsed. I’ve already started a AST builder that creates the new AST structure from the existing AST and also integrated that into the parser driver.

The result is sthat now the parser and its structures can be discarded as soon as the new AST structure is built. That should reduce memory overhead quite a bit as all the strings for tokens that are not used in the AST (like keywords, operators and such) are not held onto.

One hard small stumbling block was that kdev-pg and kdev-pg-qt didn’t initialize the token members in the generated AST. This results in them being initialized to 0. Obviously thats a proper token index in the token stream and you get a valid token and thus token text for that index. Unfortunately this means that whenever you have a rule like this: ( name=IDENTIFIER | 0 ) you need an extra boolean to store wether the member name actually was parsed or not.

The solution: Let kdev-pg-qt create proper initialization code. First attempt, done late last night (2-4 am) was obviously stupid and didn’t even get close to it. The right way was of course creating a new Generate-class that uses the visitor pattern to let the default visitor do the hard work of walking the kdev-pg-internal model and only hook into the variable declarations. In that part you have access to the member variable information and can print something like (*yynode)->name = -1;. This Generator put right after creating a new AST node solved my problem. Thats my 3rd feature contribution to kdev-pg(-qt), YAY 🙂

To close this post just a short note regarding the bashing going on on the dot: KDE4.0 is for enthusiasts, developers and eventually power users. Not for average joe who’s accustomed to kde3’s desktop and hesitant to change his habits. And quite frankly I don’t see that much of a difference between plasma and kde3 desktop from the users pov, its a taskbar with applets (or plasmoids, whatever), a desktop where you can put things (things being not only icons but also cool widgets) and a menu. And Aaron, before you get passionate again ;), I know the underlying technology is completely different and the abilities of plasma are way beyond what kicker and Co could ever deliver – but it really doesn’t look all that different at first glance.

KDevelop-PG Qt

The last two days I’ve spent hacking on KDevelop-PG, the parser generator originally written by Roberto Raggi and used for forthcoming QMake, Python and Ruby support in KDevelop4.

There are basically two changes I’ve done, change from STL like class and method names to Qt-ish and change the container classes and similar things to use Qt classes – both for the kdevelop-pg code and the code it generates for your gramar. Most important reasoning for this is maintainability by those people that currently use KDevelop-PG, which is Alexander Dymo and myself. I guess its not that big a problem for Alexander, but I often have a hard time really understanding how STL works – mostly due to me not yet spending any time for an in-depth review.

As introducing Qt as a dependecy for kdevelop-pg as well as the generated parsers is a step not every possible user of kdevelop-pg wants to take we’ve essentially forked the parser generator, creatind KDevelop-PG-Qt. So anybody who wants to have a good LL(1) parser generator (with static and also kind of dynamic lookahead for k>1) without the heavy Qt library can still use kdevelop-pg.

The work was tedious, but not too hard, especially with KDevelop3 project wide search and search+replace 🙂 I’ve also already changed the python parser to the new KDevelop-PG-Qt, though the rest of its language support will have to wait until the hand-crafted Ast is done.

My next todo item for KDevelop-PG-Qt is making it bootstrap itself, possibly including a hand-written lexer to be able to read unicode input properly – just in case anybody wants that 🙂

Vcs and Projectmanagement

So I merged the reworked Subversion plugin back into kdevelop’s trunk last week. The immediate consequences are

• less GUI, in particular no way to get a log between specific revisions, or a diff
• the context stuff from the new history widget is broken, due to different design ideas between VcsJob and KJob

However code-wise its a lot cleaner, IMHO. Much less low-level care-taking of memory pools, copying of char* and what not. On the other hand rapidsvn’s C++ library proved to be a bit incomplete, though it was relatively easy to extend the existing client.

On the brighter side, the new models and widgets in the plugin are already general enough to be moved to the vcs library so that any kdevelop4 plugin can easily ask a projects vcs support to fetch a log and have it displayed in a dialog. This will make it possible to have a generalized context menu in the project manager, so you’d have a “Vcs” submenu, instead of “Cvs”, “Subversion”, “Git”, “…” depending on which item you look at.

Other great news: KDevelop4 can now build, install and clean CMake and QMake projects. First I had checkboxes in front of each folder and target item in the project manager. But after about a week of seeing this I think this simply doesn’t work, its not obvious that the checkbox allows to select the items that “Build” should build. So I took up the suggestion from the kdevelop-devel list and added a detail view to the project manager again. For now its just a buildset view, you can add/remove items from the project manager to it and then have kdevelop build all items in it. Its also already hideable and has a preliminary switcher, so later other detail views can be added.

Currently I’m thinking about adding one that lists the non-project files in a folder that is selected in the projectmanagement view. There’s no way yet to add items to a project (the managers don’t implement that yet), so something like this is really needed when we want to start working with KDevelop4 (we == the kdev hackers).

Last but not least: I hope Hamish doesn’t shoot me for leaking this, but KDevelop4 got a new GUI mode. Or to be precise, it got its old one revamped. Yes thats right, KDevelop4 will have an IDEAl mode, that means toolbuttons and splitters instead of Qt dockwidgets, but it also means auto-hiding toolviews as we had in 3.3 and earlier. It already has a killer feature from Eclipse: Maximize, yes Sir thats right. KDevelop4 can maximize any of its toolviews with a simple click or a simple keyboard shortcut. I haven’t yet tested all features of the new Ui, but I already know it rocks. Thanks Hamish.

Ok, as I’ve written a lot today here are two screenshots from the current state of kdevelop4.

KDevelop4 IDEAl mode, the projectmanager is “docked”, the problem reporter is auto-hide (click the image for the fullsize version).

KDevelop4 history view, similar to what you get in a toolview in Eclipse (click the image for the fullsize version).

Taking up speed

KDevelop4 has lost quite some speed in development after the SoC this year, mainly due to the limited time of the active developers. Luckily this currently changes, Hamish as re-introduced the class browser and problem reporter and is currently working on the user interface 🙂

His re-engagement in KDevelop hacking infected me as well, I dropped my private pet project and dived back into the project support. The result is that KDevelop4 can now, build, install, clean, prune (distclean) and configure Make, CMake and QMake projects (uhm, well CMake doesn’t prune yet as CMake’s Makefiles don’t produce an according makefile target). I think thats enough to use KDevelop4 for basic hacking among us KDevelop hackers, once the rest of the “showstoppers” are in place. IIRC (details on the list I think) this includes a bit more code navigation, better ui and of course vcs support.

The latter already works – AFAIK, but is not well integrated yet. I’ll try to bring my rework of the plugin design into a state where I can merge it back intro trunk this week. Then it’ll take only a couple of days to add generic version control stuff to the context menus.