For the past week or so I’ve been improving how Glom shows big tables in its list view. I created a local copy of the musicbrainz database for testing. For instance, the “artist” table has almost 600,000 rows. Glom 1.6 is awful at this, but Glom in svn trunk is getting better. There are several problems which I’ll mention in different posts.
By the way, I made it slightly easier to use Glom on an existing database, but it will still act oddly on non-Glom field types, and it needs you to edit an XML file. I’m now ready to add a use-an-existing-database feature to the GUI and make it handle other field types, and maybe other database constraints, if anybody wants to pay Openismus to do that.
GtkTreeView calls gtk_tree_model_iter_next() on every row
The first problem is that GtkTreeView always calls gtk_tree_model_iter_next() on every row in your model, even if it has millions of rows. Even if you have a custom GtkTreeModel and even if you are using fixed-height mode. Surprisingly that happens quite quickly, but it’s really not sensible.
And this makes it even less possible to attach extra data via pointers in the user_data fields in your GtkTreeIter, because you can’t free that data until the model is destroyed, because you have no other way of knowing when nobody else could be using that GtkTreeIter. GtkTreeIter is not reference-counted – it’s copied by value. Even tinymail (otherwise very efficient) must have a huge array in memory so that the GtkTreeIter’s index into that array stays valid.
So GtkTreeView just isn’t made for millions of records. I don’t want to implement a new GtkTreeView, but if someone does then I hope we have something in GTK+ that we can all use. A generic Iterator type in Glib (Philip Van Hoof’s page) might be a small start on that.