Tag Archives: Openismus

André Klapper and Karsten Bräckelmann joining Openismus

General, ,

I’m pleased to say that André Klapper and Karsten Bräckelmann are new Openismus employees.  They will be sharing bugmaster duty for maemo.org, with André taking over full time from September. They worked as a team before doing similar work for Ximian’s Evolution, and they are well known for their bug herding for GNOME, so I have great confidence in them.

They will ensure a healthy ongoing flow of information for Maemo, between internal developers, external developers and users. Along with Dave Neary concentrating on documentation, and Niels Breet working on infrastructure, this is part of a big push to make Maemo’s community work as well as the most successful open source projects. I think you’ll hear lots more about that.

Big Tables in Glom: Leaks and libgda

General, ,

While experimenting with Glom versus a big musicbrainz database, valgrind’s memcheck helped me to find lots of small memory leaks in Glom, libgda, and libepc, and one huge GValue leak in libgda. Viewing a table of 600,000 records initially took 2 minutes (or 2 hours in Glom 1.6) and 1000Mb of memory. It now takes 1 minute and 250Mb of memory, mostly all during the execution of the SQL query.

That’s still awful, but it looks like libgda-4.0 will do this in about 1 second with 35Mb of memory, no more than when using the raw libpq API. That seems acceptable and makes me think that Glom is going to be fine with large tables. I don’t know what’s wrong with libgda-3.0. Maybe it’s not really using a database cursor when asked.

That had worried me for a while so I’m glad to know it will be OK. But I’ll wait for libgda-4.0’s API to settle down before switching Glom to it, to avoid disrupting the other developers working on Glom.

I also think that the 250Mb is being leaked every time I switch from Details to List view in Glom, but valgrind’s leak check seems to hit an endless loop, or is otherwise overloaded by this challenge, even when using less rows, so I’m not sure what the cause might be. If the leak is still there with libgda-4.0 then it should be easier to investigate.

There are still some other smaller leaks reported by valgrind, but I don’t believe most of them. For instance, it reports some leaks in std::string constructors even when the std::strings are temporary instances, not created by new.

By the way, here is my valgrind command for leak detection:

G_SLICE=always-malloc G_DEBUG=gc-friendly GLIBCPP_FORCE_NEW=1 GLIBCXX_FORCE_NEW=1 valgrind –tool=memcheck –leak-check=full –leak-resolution=high –num-callers=30 –suppressions=/somewhere/valgrind-python.supp –suppressions=/somewhere/gtk.suppression yourapp &> valgrind_output.txt

I found some of those environment variables, and Johan’s GTK+ valgrind suppressions file on the live.gnome.org valgrind page. And here is the official Python valgrind suppressions file.

Big Tables in Glom: GtkTreeView

General, ,

For the past week or so I’ve been improving how Glom shows big tables in its list view. I created a local copy of the musicbrainz database for testing. For instance, the “artist” table has almost 600,000 rows. Glom 1.6 is awful at this, but Glom in svn trunk is getting better. There are several problems which I’ll mention in different posts.

By the way, I made it slightly easier to use Glom on an existing database, but it will still act oddly on non-Glom field types, and it needs you to edit an XML file. I’m now ready to add a use-an-existing-database feature to the GUI and make it handle other field types, and maybe other database constraints, if anybody wants to pay Openismus to do that.

GtkTreeView calls gtk_tree_model_iter_next() on every row

The first problem is that GtkTreeView always calls gtk_tree_model_iter_next() on every row in your model, even if it has millions of rows. Even if you have a custom GtkTreeModel and even if you are using fixed-height mode. Surprisingly that happens quite quickly, but it’s really not sensible.

And this makes it even less possible to attach extra data via pointers in the user_data fields in your GtkTreeIter, because you can’t free that data until the model is destroyed, because you have no other way of knowing when nobody else could be using that GtkTreeIter. GtkTreeIter is not reference-counted – it’s copied by value. Even tinymail (otherwise very efficient) must have a huge array in memory so that the GtkTreeIter’s index into that array stays valid.

So GtkTreeView just isn’t made for millions of records. I don’t want to implement a new GtkTreeView, but if someone does then I hope we have something in GTK+ that we can all use. A generic Iterator type in Glib (Philip Van Hoof’s page) might be a small start on that.

Openismus: Looking for a Bugmaster

General, , ,

As was mentioned on the maemo-developers mailing list, Openismus is looking for a new full-time employee to act as the bugmaster for maemo.org, to help make it a more responsive, and less one-sided, developer community. Some obvious candidates have already received emails from me, but I’ll mention the job here in case there is any one I haven’t thought of.

A bugmaster keeps a project’s database of bug reports under control, helps the developers to know which bugs are most important, keeps users informed, and worries about bugs that block contributors. A bugmaster usually defines some bug reporting and bug triaging processes to keep things running smoothly. Anyone who has led a bugsquad such as GNOME’s is of course ideal. There’s lots of information online about this kind of role.

So an ability to organize people, information and tasks (and make them almost self-organizing) is most important. There’s a certain amount of cat-herding involved, but you get to deal with some cool cats. Familiarity with databases (SQL), and server maintenance is generally helpful.

Ideally you live in Germany. If we find a great candidate in the rest of the EU then we’ll figure something out. You’ll be working from home. This is a rare opportunity to be rewarded for doing meaningful work with effective people.

If you are interested, please email me to tell me about yourself and what you think the job will involve.

Glom 1.6 bug fixes

General, , ,

We didn’t do a new major version of Glom recently, though we usually follow the GNOME schedule, because some of the new features are not quite ready, such as the drag-and-drop layout, and the new printout designer.

But this does mean that we have a chance to get a bug-fixed version of Glom’s stable version into Ubuntu’s stable version, for the first time. I’ve had some success with that, thanks to Chris Brotherton. It can still take a long time (10 days last time, though it felt like longer), and the longer it takes the more chance there is that a change in some other package can be a reason to wait. But I have a feeling that things are getting easier.

Of course, the more usable Glom is in Ubuntu, the more bug reports we can get. Many thanks to Jani Monoses and Pavel Mlčoch for keeping me busy, particularly for discovering the problems with changed behaviour in PostgreSQL 8.3, which we discovered late because PostgreSQL 8.3 only just started working for some people recently. These bugfixes mean yet more tarball releases for Ubuntu to package, and there’s always the risk that they will stop taking new versions as they really hit their freeze. Hopefully, they’ll manage to do just one more update because Jani found some more critical bugs.

After Ubuntu Hardy is released then our upstream bug fixes will probably never get into Ubuntu Hardy unless we can claim that they are security problems, as is still the problem with Glom in Ubuntu Gutsy. At FOSSCamp in Prague in May I’m apparently meant to be leading some discussion about how to get stable updates into stable distros. I have some ideas about who a distro (and its users) should trust, and about what is currently just providing a false sense of security at the cost of user experience. I don’t know what to expect from FOSSCamp so it might not have any effect. I didn’t see the point in flying to the last FOSSCamp in Boston for one day just for some seemingly aimless meeting and talking, but Prague is not far away by train, so it’s worth a look.

Just-Works IR Remote Controls on Ubuntu

General, ,

gnome-lirc-properties GUI is now in Ubuntu Hardy, after much effort.

We had a variety of remote controls for testing, so we could at least guarantee that it worked for a certain list. Here are our results. The ones that worked perfectly were recognized by gnome-lirc-properties’ auto-detection button and their keys were immediately recognized in the preview area.

  • Streamzap PC Remote Control (IR receiver and remote control): Worked perfectly.
  • SnapStream Firefly PC Remote (IR receiver and remote control): Worked perfectly.
  • Sound Blaster Remote Control Upgrade – White Box (Has its own IR receiver): Worked perfectly.
  • Pinnacle Remote Kit (IR receiver and control): lirc has no driver for this, and no patch has been submitted to lirc by anyone that can confirm that it works.
  • Xbox 360 Universal Media Remote. (Just a remote, though the XBox has a built-in IR receiver): This didn’t work with any of the other IR receivers, but it presumably works with an actual XBox running Ubuntu. We’d love to hear from people running Ubuntu Hardy on an XBox. Does the auto-detection in our GUI work?

We also tried two other replacement remote controls that have no IR receiver of their own. They work with the other IR receivers – you can use the key learning feature in our GUI to configure them.:

  • Sony RM-V202 4-Device Universal Remote Commander.
  • One For All “Universal Replacement Remote” URC4110.

IMG_3153

So I’m adding one of these to my Ubuntu Hardware page, because it just works.

Two Months and Smiling

General, , , ,

Liam is just over two months old now. He’s a little more aware of the world, though not really interacting with it much yet. He started to smile properly a week ago, and every day he makes slightly different vowel sounds. After two months of sleeping, eating and crying, the first wide smile makes a big emotional impact.

He’s now nearly 5 Kg, almost double his birth weight, and my back knows it.

liam_weighing_cropped_small.jpg

Sigi and I take turns so we can get other things done. I have the midnight to 4am shift, when I give him the bottle, and 10am to 12pm. I generally get work done between 12:00 noon and 18:00, and sometimes in that midnight to 4am shift. He’s quite an easy baby, I think, but we are very lucky that there are two of us who can give him lots of time. It would be nice if he didn’t need to sleep in my arms quite so much.

I have decided to avoid taking on much new work before June, so we can keep this daily routine for a while more. I’ll also look for a small office in the neighbourhood – The Glockenbachviertel in Munich, in case anyone has a suggestion. But there is work to do so I’m still trying to hire new people.

Clutter Tutorial done for now

General, , ,

Now that Clutter 0.6 is out, I found the time to finish the planned sections for the Clutter tutorial that I mentioned in December, for Openismus. Here is the html version and a PDF, though of course the PDF doesn’t have the nice links into the API reference documentation and the links to the actual files of the examples.

The guys at OpenedHand haven’t had time to review this yet, so be aware that there could be significant errors. But I think it’s mostly accurate, useful, and unique, though the appendices and full example are obviously lacking. I’ll try to keep it up to date and correct as I get feedback.

Having done all this investigation into Clutter, I have formed some opinions:

  • Clutter is a nice way to move semi-3D things around on the screen to create “innovative” user interfaces. Its abstractions for actors and behaviours really give you something to work with compared to using OpenGL directly. But it’s still hard to create natural-feeling responsive applications and I don’t know exactly what’s lacking. It feels like there is a disconnect between the act of coding and the very dynamic UIs that the code creates. I sometimes feel I’d like to just put actors on a rail, twist that rail about, connect some actors together with struts or springs, start them moving, let the user push and pull them around within constraints, and trigger extra behaviour when they reach certain positions, or reach each other.
  • Clutter is now a very basic API. You will need to implement a lot of simple functionality to create applications that act as today’s users expect. The Tidy “reference-implementation” example toolkit thing is not a shared library – it’s meant as an inspiration for you to do all this yourself, and it is itself not very complete. I don’t think that is a wise strategy, though rampant bad reimplementation is the standard in today’s mobile/embedded development projects.
  • Neither you or Clutter should want to reimplement the huge amount of UI logic that is, for instance, in GTK+. Without reusing all that work, Clutter applications will generally have inconsistent or inadequate text editing, scrolling, focus, keyboard accessibility, right-to-left language support, etc. Applications will share little common API so it won’t be easy to throw new developers at your project. Somehow we need a way to use some GTK+ widgets in Clutter without them looking and feeling like they are visitors from an extra dimension.

These problems could probably be solved by lots of hard work by smart people. I don’t know what the near-future strategy for Clutter is, so I don’t know if the current state is intentional or if anything particular is planned. It’s certainly getting better all the time.

gnome-lirc-properties: Using PolicyKit to get sudo access

General, , ,

Can’t get PolicyKit’s ObtainAuthorization() to work.

gnome-lirc-properties, which I just mentioned needs to edit /etc/lirc/lircd.conf, which requires sudo/root access. Various APIs exist to get temporary sudo/root access but everyone now seems to agree that the new PolicyKit system is the way to go. I’d like to link to a website for it. All the system administration control panels in Ubuntu Hardy Heron seem to use PolicyKit already.

But I can’t get the thing to work, maybe because I’m using Python, and I haven’t found any help so far. I have defined the PolicyKit mechanism, but I can’t get even a simple test of the ObtainAuthorization() method to do anything interesting. This python code does not return anything other than None from ObtainAuthorization() and none of the callbacks are called:

#!/usr/bin/python

import pygtk
pygtk.require('2.0')
import gtk

import dbus
import os

class TestWindow:
    def on_button_clicked(self, widget, data=None):

	#Call the D-Bus method to request PolicyKit authorization:

        session_bus = dbus.SessionBus()
        policykit = session_bus.get_object('org.freedesktop.PolicyKit.AuthenticationAgent', '/', "org.gnome.PolicyKit.AuthorizationManager.SingleInstance")
        if(policykit == None):
           print("Error: Could not get PolicyKit D-Bus Interface\n")

        gdkwindow = self.window.window
        xid = gdkwindow.xid

        print "Calling ObtainAuthorization..."

        #This complains that no ObtainAuthorization(ssi) exists:
        #granted = policykit.ObtainAuthorization("test_action_id", xid, os.getpid())

        #TODO: Neither of the async callbacks are called, and how could the return value be useful if it is async?
        # Note: Other code (such as gnome-panel) seems to use ShowDialog instead of ObtainAuthorization, though
        # ShowDialog is apparently deprecated (I don't know when), but that also has no effect.
        granted = policykit.ObtainAuthorization("test_action_id", xid, os.getpid(), reply_handler=self.__handleAuthReply, error_handler=self.__handleAuthError)

        print "...Finished."

        print "granted=", granted

    def __handleAuthReply(self, granted):
        print "handleAuthReply: granted\n"

    def __handleAuthError(self, exception):
        print "handleAuthError: not granted: %s\n" % exception

    def on_delete_event(self, widget, event, data=None):
        # Close the window:
        return False

    def on_destroy(self, widget, data=None):
        gtk.main_quit()

    def show(self):
       self.window.show()

    def __init__(self):

        self.window = gtk.Window(gtk.WINDOW_TOPLEVEL)
        self.window.connect("delete_event", self.on_delete_event)
        self.window.connect("destroy", self.on_destroy)

        self.button = gtk.Button("Obtain Authorization")
        self.button.connect("clicked", self.on_button_clicked, None)
        self.window.add(self.button)
        self.button.show()

window = TestWindow()
window.show()
gtk.main()

gnome-lirc-properties has a real mechanism and tries to use ObtainAuthorization() for real, if someone wants to look at it properly. Here is the gnome-lirc-properties svn, and a tarball (it doesn’t do much yet).

How to use PolicyKit from an application

In return for asking the lazy web, I’ll try to describe how I think it should work, based mostly on David Zeuthen’s description. There are many opportunities to get something wrong:

  • You install a D-Bus service (a PolicyKit “mechanism”) that does the thing that needs sudo access. PolicyKit will control access to this mechanism. For instance, the Clock applet’s mechanism has a SetTimeZone method. gnome-lirc-properties uses the Python D-Bus bindings to implement its mechanism. I know that our WriteConfigFile() method is far too generic, but we will improve it when we get this working.
  • Update:: And this mechanism’s implementation should ask PolicyKit if it has authorization, and return an error if it doesn’t. Thanks mclasen. I’ll try to find out how to do this. Update: You can do this via the PolicyKit (on the session bus) object’s IsProcessAuthorized() method, now used in the gnome-lirc-properties PolicyKit mechanism.
  • As with other D-Bus services, you install a .service file, so that D-Bus activation can start it on demand. PolicyKit mechanisms use the System bus rather than the Session bus, so you install it in $(datadir)/dbus-1/system-services (/usr/share/dbus-1/system-services on Ubuntu). Here is the .service file for gnome-lirc-properties.
  • As with other D-Bus services, you install a .conf file, saying who can use the service. This is installed in $(sysconfdir)/dbus-1/system.d (/etc/dbus-1/system.d/ on Ubuntu). This is the .conf file for gnome-lirc-properties
  • A .policy file tells PolicyKit who should be allowed to use the mechanism. You install this in $(datadir)/PolicyKit/policy (/usr/share/PolicyKit on Ubuntu). Here is the .policy file for gnome-lirc-properties. I believe this allows system administrators to configure access to these features via these .policy files, and I believe that is the whole point of this D-Bus “mechanism” abstraction.
  • Applications that want to use this mechanism call the ObtainAuthorization() method on the “org.freedesktop.PolicyKit.AuthenticationAgent” D-Bus service, passing the action ID that is mentioned in the .policy file. If necessary, PolicyKit asks for a password (or whatever) for authentication, and presumably then gives the application the appropriate rights. The dialog uses (translatable) text from the .policy file, so it’s very specific to your application. Many applications still seem to be using the ShowDialog() method instead, but that has been deprecated.