Brussels and Prague

General, ,

I’ve been travelling more than usual in May and I’m not finished.

I spent a couple of days in Brussels at Thomas Vander Stichele‘s place, getting a crash course in the Flumotion streaming media server because I’ll be writing Flumotion‘s user documentation.

I’m writing quite a lot of documentation these days, and enjoying it as a holiday from writing code. It’s stress free in comparison. I’d like to make Openismus known for creating documentation along with our development and QA work. There’s always a need for it.

After Brussels I spent a few days with Liam’s grandparents in Karlsruhe before heading to Prague for FOSSCamp. Being away from him for so long was not easy.

Things I read on the train to Prague:

  • The GStreamer Application Development Manual: This is a truly useful document. I now feel like I understand how to use gstreamer. That has allowed me to make sense of the surprisingly mature gstreamer C++ bindings and should help my understanding of Flumotion. I now want to read the GStreamer Plugin Writer’s Guide too.
  • The GNOME Documentation Style Guide: This was better than expected, written by true strunkandwhiteistas who show real experience of writing technical documentation. I didn’t learn much new but was glad to see the advice presented for others. Many points are repeated, but that’s maybe to allow each audience to get the whole message without reading the other sections. Still, the text is sometimes long winded, suggesting a rogue extra author.

FOSSCamp was not quite the dull unstructured talk-fest that I feared, just because of the quality of the attendees, each of whom had something fascinating to explain. It was indeed mostly just talk, with little chance of any resulting action, but it was at least interesting talk.

I quickly introduced Glom to a small group of people who seemed positive and led a larger discussion about updates of stable upstream releases in stable versions of distros, mostly focusing on Ubuntu because only the Ubuntu people seemed to have opinions. Maybe the other distros’ processes are not so easily influenced. I think we already have a result, which I can hopefully mention soon.

I stayed an extra day to meet André Klapper, who is attending the Ubuntu Developer Summit, so we could talk about his bugmaster work for Openismus. We fight entropy. I attended the Ubuntu Mobile sessions in the morning before taking the train home to Munich, but it was impenetrable to anyone not already involved. But that’s UDS – it’s for people already working on stuff.

Both FOSSCamp and the overlap with UDS allowed me to meet many of my favourite people and I am guilty of enjoying their company when I should have been meeting more new people instead.

On Saturday we fly to Scotland (North Berwick) for two weeks so Liam can meet his other grandfather and aunt. I’ve tried to plan the pain out of flying with a five-month old baby for the first time, but it’s sure to be a challenge.

André Klapper and Karsten Bräckelmann joining Openismus

General, ,

I’m pleased to say that André Klapper and Karsten Bräckelmann are new Openismus employees.  They will be sharing bugmaster duty for maemo.org, with André taking over full time from September. They worked as a team before doing similar work for Ximian’s Evolution, and they are well known for their bug herding for GNOME, so I have great confidence in them.

They will ensure a healthy ongoing flow of information for Maemo, between internal developers, external developers and users. Along with Dave Neary concentrating on documentation, and Niels Breet working on infrastructure, this is part of a big push to make Maemo’s community work as well as the most successful open source projects. I think you’ll hear lots more about that.

PostgreSQL has no Bugzilla

General

PostgreSQL is an open source database. I guess that they like databases and understand when they can be useful.

But PostgreSQL has no bug tracker. Patches must be sent to mailing lists. Bugs are sent to a mailing list, apparently via a web-form, and some bugs are listed on a page, apparently written by a person.

Want to see what bugs are open and what’s happening to them? Can’t. Want to see what’s the current status of your patch? Can’t.

Discovering Unused Code?

General,

Glom has been through a few refactorings. I’m sure that some of its functions are never called by anything, and I’d like to remove those to clean up the code a little.

Is there some tool that can tell me what’s not used in a C/C++ application? gcov can apparently tell me what code is used at runtime during specific tests, but running the application shouldn’t be necessary. I guess that some combination of nm and strip could give me a list, but that’s not ideal.

Anjuta versus Eclipse: Using an existing code base

General

I’ve recently been using Anjuta 2. It’s far better than Anjuta 1, and less annoying than Eclipse’s arbitrarily eccentric UI, though Eclipse is obviously more feature-packed. Here is an example of how Anjuta makes a very commonly-used feature simple and explicit, while Eclipse hides the feature and leaves you in doubt. Nobody will make money selling Anjuta training courses.

Using an existing code base with Anjuta:

  • Select the File/”New Project From Existing Sources” menu item.
  • In the wizard, click Next through the title page.
  • In the wizard, enter the project title and browse to the path to your existing code and click Next.
  • In the wizard, click Apply on the confirmation page.

Using an existing code base with Eclipse:

  • Select the File/New/Project… menu item.
  • Choose C++/”Standard Make C++ Project”
  • In the New Project wizard, enter a project name. Uncheck “Use default location” and browse for the path to your existing code.
  • In the New Project wizard, ignore the (very complex) “Referenced C/C++ Projects” page and click Finish.
  • Wonder if it adds or changes files in your existing directory. Actually, the “Standard Make C++ Project” doesn’t, but others do. Presumably it always does if you did not uncheck “Use default location”.
  • (There’s also a way to create a new Eclipse project by checking out from svn or your favorite source control system, if you have an appropriate Eclipse plugin. I find that annoying as a way to create a project for an existing code base.

Glom: Calendar Portal

General,

Over the last couple of days I added a new “layout item” to Glom’s detail view. It works like the existing Related Records portal, but presents the related records via a calendar. It uses Mathias Hasselmann’s recent gtk_calendar_set_detail_func() addition in GTK+’s svn trunk.

Obviously this isn’t a perfect example. Any school administrator would be annoyed at having to add new lessons every week even if they are the same as the week before. Schedules are notoriously difficult to handle in generic database systems, but maybe there’s a neat way to even make this example useful by adding just one or two new features.

Glom\'s Lesson Planner Example

Big Tables in Glom: Leaks and libgda

General, ,

While experimenting with Glom versus a big musicbrainz database, valgrind’s memcheck helped me to find lots of small memory leaks in Glom, libgda, and libepc, and one huge GValue leak in libgda. Viewing a table of 600,000 records initially took 2 minutes (or 2 hours in Glom 1.6) and 1000Mb of memory. It now takes 1 minute and 250Mb of memory, mostly all during the execution of the SQL query.

That’s still awful, but it looks like libgda-4.0 will do this in about 1 second with 35Mb of memory, no more than when using the raw libpq API. That seems acceptable and makes me think that Glom is going to be fine with large tables. I don’t know what’s wrong with libgda-3.0. Maybe it’s not really using a database cursor when asked.

That had worried me for a while so I’m glad to know it will be OK. But I’ll wait for libgda-4.0’s API to settle down before switching Glom to it, to avoid disrupting the other developers working on Glom.

I also think that the 250Mb is being leaked every time I switch from Details to List view in Glom, but valgrind’s leak check seems to hit an endless loop, or is otherwise overloaded by this challenge, even when using less rows, so I’m not sure what the cause might be. If the leak is still there with libgda-4.0 then it should be easier to investigate.

There are still some other smaller leaks reported by valgrind, but I don’t believe most of them. For instance, it reports some leaks in std::string constructors even when the std::strings are temporary instances, not created by new.

By the way, here is my valgrind command for leak detection:

G_SLICE=always-malloc G_DEBUG=gc-friendly GLIBCPP_FORCE_NEW=1 GLIBCXX_FORCE_NEW=1 valgrind –tool=memcheck –leak-check=full –leak-resolution=high –num-callers=30 –suppressions=/somewhere/valgrind-python.supp –suppressions=/somewhere/gtk.suppression yourapp &> valgrind_output.txt

I found some of those environment variables, and Johan’s GTK+ valgrind suppressions file on the live.gnome.org valgrind page. And here is the official Python valgrind suppressions file.

Big Tables in Glom: GtkTreeView

General, ,

For the past week or so I’ve been improving how Glom shows big tables in its list view. I created a local copy of the musicbrainz database for testing. For instance, the “artist” table has almost 600,000 rows. Glom 1.6 is awful at this, but Glom in svn trunk is getting better. There are several problems which I’ll mention in different posts.

By the way, I made it slightly easier to use Glom on an existing database, but it will still act oddly on non-Glom field types, and it needs you to edit an XML file. I’m now ready to add a use-an-existing-database feature to the GUI and make it handle other field types, and maybe other database constraints, if anybody wants to pay Openismus to do that.

GtkTreeView calls gtk_tree_model_iter_next() on every row

The first problem is that GtkTreeView always calls gtk_tree_model_iter_next() on every row in your model, even if it has millions of rows. Even if you have a custom GtkTreeModel and even if you are using fixed-height mode. Surprisingly that happens quite quickly, but it’s really not sensible.

And this makes it even less possible to attach extra data via pointers in the user_data fields in your GtkTreeIter, because you can’t free that data until the model is destroyed, because you have no other way of knowing when nobody else could be using that GtkTreeIter. GtkTreeIter is not reference-counted – it’s copied by value. Even tinymail (otherwise very efficient) must have a huge array in memory so that the GtkTreeIter’s index into that array stays valid.

So GtkTreeView just isn’t made for millions of records. I don’t want to implement a new GtkTreeView, but if someone does then I hope we have something in GTK+ that we can all use. A generic Iterator type in Glib (Philip Van Hoof’s page) might be a small start on that.

Openismus: Looking for a Bugmaster

General, , ,

As was mentioned on the maemo-developers mailing list, Openismus is looking for a new full-time employee to act as the bugmaster for maemo.org, to help make it a more responsive, and less one-sided, developer community. Some obvious candidates have already received emails from me, but I’ll mention the job here in case there is any one I haven’t thought of.

A bugmaster keeps a project’s database of bug reports under control, helps the developers to know which bugs are most important, keeps users informed, and worries about bugs that block contributors. A bugmaster usually defines some bug reporting and bug triaging processes to keep things running smoothly. Anyone who has led a bugsquad such as GNOME’s is of course ideal. There’s lots of information online about this kind of role.

So an ability to organize people, information and tasks (and make them almost self-organizing) is most important. There’s a certain amount of cat-herding involved, but you get to deal with some cool cats. Familiarity with databases (SQL), and server maintenance is generally helpful.

Ideally you live in Germany. If we find a great candidate in the rest of the EU then we’ll figure something out. You’ll be working from home. This is a rare opportunity to be rewarded for doing meaningful work with effective people.

If you are interested, please email me to tell me about yourself and what you think the job will involve.