I’m pleased to say that André Klapper and Karsten Bräckelmann are new Openismus employees.  They will be sharing bugmaster duty for maemo.org, with André taking over full time from September. They worked as a team before doing similar work for Ximian’s Evolution, and they are well known for their bug herding for GNOME, so I have great confidence in them.

They will ensure a healthy ongoing flow of information for Maemo, between internal developers, external developers and users. Along with Dave Neary concentrating on documentation, and Niels Breet working on infrastructure, this is part of a big push to make Maemo’s community work as well as the most successful open source projects. I think you’ll hear lots more about that.

PostgreSQL is an open source database. I guess that they like databases and understand when they can be useful.

But PostgreSQL has no bug tracker. Patches must be sent to mailing lists. Bugs are sent to a mailing list, apparently via a web-form, and some bugs are listed on a page, apparently written by a person.

Want to see what bugs are open and what’s happening to them? Can’t. Want to see what’s the current status of your patch? Can’t.

Glom has been through a few refactorings. I’m sure that some of its functions are never called by anything, and I’d like to remove those to clean up the code a little.

Is there some tool that can tell me what’s not used in a C/C++ application? gcov can apparently tell me what code is used at runtime during specific tests, but running the application shouldn’t be necessary. I guess that some combination of nm and strip could give me a list, but that’s not ideal.

I’ve recently been using Anjuta 2. It’s far better than Anjuta 1, and less annoying than Eclipse’s arbitrarily eccentric UI, though Eclipse is obviously more feature-packed. Here is an example of how Anjuta makes a very commonly-used feature simple and explicit, while Eclipse hides the feature and leaves you in doubt. Nobody will make money selling Anjuta training courses.

Using an existing code base with Anjuta:

• Select the File/”New Project From Existing Sources” menu item.
• In the wizard, click Next through the title page.
• In the wizard, enter the project title and browse to the path to your existing code and click Next.
• In the wizard, click Apply on the confirmation page.

Using an existing code base with Eclipse:

• Select the File/New/Project… menu item.
• Choose C++/”Standard Make C++ Project”
• In the New Project wizard, enter a project name. Uncheck “Use default location” and browse for the path to your existing code.
• In the New Project wizard, ignore the (very complex) “Referenced C/C++ Projects” page and click Finish.
• Wonder if it adds or changes files in your existing directory. Actually, the “Standard Make C++ Project” doesn’t, but others do. Presumably it always does if you did not uncheck “Use default location”.
• (There’s also a way to create a new Eclipse project by checking out from svn or your favorite source control system, if you have an appropriate Eclipse plugin. I find that annoying as a way to create a project for an existing code base.

Over the last couple of days I added a new “layout item” to Glom’s detail view. It works like the existing Related Records portal, but presents the related records via a calendar. It uses Mathias Hasselmann’s recent gtk_calendar_set_detail_func() addition in GTK+’s svn trunk.

Obviously this isn’t a perfect example. Any school administrator would be annoyed at having to add new lessons every week even if they are the same as the week before. Schedules are notoriously difficult to handle in generic database systems, but maybe there’s a neat way to even make this example useful by adding just one or two new features.

Sorry to various people who I will miss, but I won’t be at LinuxTag in Berlin this year. I would have liked to be there.

Instead we will be in Edinburgh/North-Berwick for 2 weeks introducing Liam to his English-speaking Aunt and Grandfather. It was the only time that we could all be in one place.

While experimenting with Glom versus a big musicbrainz database, valgrind’s memcheck helped me to find lots of small memory leaks in Glom, libgda, and libepc, and one huge GValue leak in libgda. Viewing a table of 600,000 records initially took 2 minutes (or 2 hours in Glom 1.6) and 1000Mb of memory. It now takes 1 minute and 250Mb of memory, mostly all during the execution of the SQL query.

That’s still awful, but it looks like libgda-4.0 will do this in about 1 second with 35Mb of memory, no more than when using the raw libpq API. That seems acceptable and makes me think that Glom is going to be fine with large tables. I don’t know what’s wrong with libgda-3.0. Maybe it’s not really using a database cursor when asked.

That had worried me for a while so I’m glad to know it will be OK. But I’ll wait for libgda-4.0’s API to settle down before switching Glom to it, to avoid disrupting the other developers working on Glom.

I also think that the 250Mb is being leaked every time I switch from Details to List view in Glom, but valgrind’s leak check seems to hit an endless loop, or is otherwise overloaded by this challenge, even when using less rows, so I’m not sure what the cause might be. If the leak is still there with libgda-4.0 then it should be easier to investigate.

There are still some other smaller leaks reported by valgrind, but I don’t believe most of them. For instance, it reports some leaks in std::string constructors even when the std::strings are temporary instances, not created by new.

By the way, here is my valgrind command for leak detection:

G_SLICE=always-malloc G_DEBUG=gc-friendly GLIBCPP_FORCE_NEW=1 GLIBCXX_FORCE_NEW=1 valgrind –tool=memcheck –leak-check=full –leak-resolution=high –num-callers=30 –suppressions=/somewhere/valgrind-python.supp –suppressions=/somewhere/gtk.suppression yourapp &> valgrind_output.txt

I found some of those environment variables, and Johan’s GTK+ valgrind suppressions file on the live.gnome.org valgrind page. And here is the official Python valgrind suppressions file.

For the past week or so I’ve been improving how Glom shows big tables in its list view. I created a local copy of the musicbrainz database for testing. For instance, the “artist” table has almost 600,000 rows. Glom 1.6 is awful at this, but Glom in svn trunk is getting better. There are several problems which I’ll mention in different posts.

By the way, I made it slightly easier to use Glom on an existing database, but it will still act oddly on non-Glom field types, and it needs you to edit an XML file. I’m now ready to add a use-an-existing-database feature to the GUI and make it handle other field types, and maybe other database constraints, if anybody wants to pay Openismus to do that.

GtkTreeView calls gtk_tree_model_iter_next() on every row

The first problem is that GtkTreeView always calls gtk_tree_model_iter_next() on every row in your model, even if it has millions of rows. Even if you have a custom GtkTreeModel and even if you are using fixed-height mode. Surprisingly that happens quite quickly, but it’s really not sensible.

And this makes it even less possible to attach extra data via pointers in the user_data fields in your GtkTreeIter, because you can’t free that data until the model is destroyed, because you have no other way of knowing when nobody else could be using that GtkTreeIter. GtkTreeIter is not reference-counted - it’s copied by value. Even tinymail (otherwise very efficient) must have a huge array in memory so that the GtkTreeIter’s index into that array stays valid.

So GtkTreeView just isn’t made for millions of records. I don’t want to implement a new GtkTreeView, but if someone does then I hope we have something in GTK+ that we can all use. A generic Iterator type in Glib (Philip Van Hoof’s page) might be a small start on that.

As was mentioned on the maemo-developers mailing list, Openismus is looking for a new full-time employee to act as the bugmaster for maemo.org, to help make it a more responsive, and less one-sided, developer community. Some obvious candidates have already received emails from me, but I’ll mention the job here in case there is any one I haven’t thought of.

A bugmaster keeps a project’s database of bug reports under control, helps the developers to know which bugs are most important, keeps users informed, and worries about bugs that block contributors. A bugmaster usually defines some bug reporting and bug triaging processes to keep things running smoothly. Anyone who has led a bugsquad such as GNOME’s is of course ideal. There’s lots of information online about this kind of role.

So an ability to organize people, information and tasks (and make them almost self-organizing) is most important. There’s a certain amount of cat-herding involved, but you get to deal with some cool cats. Familiarity with databases (SQL), and server maintenance is generally helpful.

Ideally you live in Germany. If we find a great candidate in the rest of the EU then we’ll figure something out. You’ll be working from home. This is a rare opportunity to be rewarded for doing meaningful work with effective people.

If you are interested, please email me to tell me about yourself and what you think the job will involve.

We didn’t do a new major version of Glom recently, though we usually follow the GNOME schedule, because some of the new features are not quite ready, such as the drag-and-drop layout, and the new printout designer.

But this does mean that we have a chance to get a bug-fixed version of Glom’s stable version into Ubuntu’s stable version, for the first time. I’ve had some success with that, thanks to Chris Brotherton. It can still take a long time (10 days last time, though it felt like longer), and the longer it takes the more chance there is that a change in some other package can be a reason to wait. But I have a feeling that things are getting easier.

Of course, the more usable Glom is in Ubuntu, the more bug reports we can get. Many thanks to Jani Monoses and Pavel Mlčoch for keeping me busy, particularly for discovering the problems with changed behaviour in PostgreSQL 8.3, which we discovered late because PostgreSQL 8.3 only just started working for some people recently. These bugfixes mean yet more tarball releases for Ubuntu to package, and there’s always the risk that they will stop taking new versions as they really hit their freeze. Hopefully, they’ll manage to do just one more update because Jani found some more critical bugs.

After Ubuntu Hardy is released then our upstream bug fixes will probably never get into Ubuntu Hardy unless we can claim that they are security problems, as is still the problem with Glom in Ubuntu Gutsy. At FOSSCamp in Prague in May I’m apparently meant to be leading some discussion about how to get stable updates into stable distros. I have some ideas about who a distro (and its users) should trust, and about what is currently just providing a false sense of security at the cost of user experience. I don’t know what to expect from FOSSCamp so it might not have any effect. I didn’t see the point in flying to the last FOSSCamp in Boston for one day just for some seemingly aimless meeting and talking, but Prague is not far away by train, so it’s worth a look.