Category Archives: General

Ubuntu Mountain View Summit

Canonical has generously offered to pay my way to the Ubuntu MountainView developers summit in November, to bring some extra points of view from the GNOME project.

I’ll arrive on Thursday evening, so I can spend Friday taking a look at San Francisco. I need to find a relatively cheap place to stay for 2 nights. This will be my second free junket of the year, though the last time I spent most of my tourist day in the hotel with a nasty cold. The California sun should make that unlikely this time. If anybody would like to meet to eat vegetarian on Friday night (3rd November), just comment or email me.

I have a strange adolescent star-truck enthusiasm about visiting the west coast for the first time – and only my second time in the states. Back in my early pre-internet twenties, I wanted nothing more than to go work in Silcon Valley, because it seemed like the center of the software world. I probably would have done it if the visa restrictions hadn’t required a university degree. Now I feel that southern California’s urban sprawl is probably as characterless and disfunctional as the U.K.’s Thames Valley where I lived at the time, and I’m content to enjoy the quality of life in Europe, working with the world’s best software people via the internet. But I suspect that visiting the Googleplex will still feel special.

Overselling Portland

I have nothing against promotion, but overselling your product becomes a mistake pretty quickly. Statements such as “For the first time, ISVs are able to port their applications to Linux regardless of desktop environment.” will just ruin OSDL’s reputation and kill OSDL’s opportunity to help those ISVs. This isn’t a one-off mistake either.

Let’s make it clear:

  • a) Portland 1.0 is a set of command-line tools to help applications to integrate with the Linux desktop. That is useful.
  • b) When ISVs think of porting their applications to Linux, they think first of GUI toolkits, file handling, networking.

a) is not b). We have lots of great b). For instance, ISVs who have suffered with MFC or win32 love GTK+ (or gtkmm, etc) or Qt. They are a bit annoyed that they can’t really use win32 or MFC on Linux, but they aren’t surprised and they are pleased that the alternative is a good one.

But if you tell them that Portland 1.0 is the alternative platform they are looking for, they will very quickly conclude:
1. “Erm, command-line? So, Linux hasn’t done much in the last 10 years. Bye.”
2. “Platform? No it’s not. How stupid do you think I am? How stupid are you? Bob, don’t forward me any more calls from OSDL.”

And Portland is never meant to be a desktop-agnostic GNOME/KDE-combining GUI toolkit. If you let people think that then it’s your mistake not theirs.

OSDL (and the Portland project) are doing great stuff, making a better Linux desktop, and creating a marketplace for ISVs. Don’t ruin it with this nonsense.

Glom 1.1/1.2

Glom logoI’ve been working on a Glom 1.1/1.2 branch recently, trying to quickly add some features in time to release a stable version for the Ubuntu Edgy release. The Glom 1.1.6 announcement has a list of what’s new. My favourite new features are:

Add Related Table

This is a time-saver that lets you quickly add a table and a relationship that uses it. So, for instance, if you have

Album
Nebraska
Asbury Park
Closing Time

you can quickly add an ID field, and a related table, so you have:

Album Artist ID
Nebraska 1
Asbury Park 1
Closing Time 2

with

Artist ID Artist Name
1 Bruce Springsteen
2 Tom Waits

In future, I’d like it to take existing data from existing fields and put that into the related record, so that in one step we could get those two tables from this:

Album Artist Name
Nebraska Bruce Springsteen
Asbury Park Bruce Springsteen
Closing Time Tom Waits

Use of the pygda API

Via the records.connection object you can now use the whole pygda API (The libgda Python API) from your button scripts and calculated fields, so there is no limit to what you can do with the underlying database. Be careful with this great power. More details on the Glom API page.

This feature was easy to implement thanks to the wonderful pygobject_new() function from pygtk, which is a lot like gtkmm’s Glib::wrap().

Debian Repository Analyzer for license compliance

Over the last few weeks I’ve been using spare hours to create a utility to help companies discover the licenses of their software and to help them decide what licenses they should use for software that uses open source dependencies, and what they should do with their sources and modifications. Assuming that they are using a debian apt dependency system.

It’s a python script that uses the python-apt API to discover all the packages in a debian repository, and then looks for licenses in the source tarball or debian diff. It then tries to figure out what license texts are actually the same, using Python’s SequenceMatcher, because many license files contain short but irrelevant bits of unique text at the start. That can take an hour or so.

Then it creates a .glom file that you can open in Glom (>= 1.1.3). That creates a database and fills it, then lets you explore the data. At this point a human can give the licenses names such as “GPL”, “LGPL”, etc, which will then show up against all the packages that had that license text. And then you can see all the licenses of each package’s dependencies.

Here is version 0.1.4, under the GPL, as I believe is required by the python-apt license. Update: Here is a screenshot.

For now, I had to hardcode the base URL of the repository. I haven’t actually tested it completely with the example repository that’s in the sources.list, but it does work with the (secret) repository for which it was written.

Tips on using python-apt

Michael Vogt has been very helpful with my questions about python-apt, but he’s not to blame for my python coding. Remember, I’m a C++ coder. Python-apt is very useful, but the API is currently rather obscure and confused by the presence of two similar APIs alongside each other. Michael is working on making it sane.

I was also frustrated by the general lack of documentation of Python APIs. It’s often very difficult to know what type of object is likely to be returned by a method and to know what methods an object supports.

So here are some of the things that I learnt from Michael Vogt, so that they don’t just sit in my Inbox where nobody else can see them.

Using a local sources.list and cache instead of your system’s

You need to set a bunch of config variables. Here’s my full list so far:

apt_pkg.Config.Set("Dir::Etc::sourcelist", "./sources.list")
apt_pkg.Config.Set("Dir::Cache::archives", "./tmp_apt_archives")
apt_pkg.Config.Set("Dir::State", "./tmp_apt_varlibapt")  #usually /var/lib/apt
apt_pkg.Config.Set("Dir::State::Lists",  "./tmp_apt_varlibdpkg") #usually /var/lib/dpkg/
apt_pkg.Config.Set("Dir::State::status", "./tmp_apt_varlibdpkg/status") #If we don't set this then we will pick up packages from the local system, from the default status file.

You will need to make sure that those directories exist, along with some sub-directories.

After calling cache.update(), remember to do this, otherwise no packages will be found:

cache.open(apt.progress.OpProgress())

Getting the name of a package:

When iterating over a cache, you can do this (imagine the indenting because WordPress doesn’t want to show it to you):

for pkg in cache:
candver = cache._depcache.GetCandidateVer(pkg._pkg)
name = candver.ParentPkg.Name

Getting the full URI of a tarball:

You can get a tarball URI like so (again, imagine the indenting),

srcrec = srcrecords.Lookup(source_package_name)
if srcrec:
for (the_md5hash, the_size, the_path, the_type) in srcrecords.Files:
if(the_type == "tar"):
tarball_uri = the_path

but it only gives you the second half of the URI. To get the whole thing, if you have the latest version of python-apt from Ubuntu Edgy, do

full_uri = srcrec.Index.ArchiveURI(tarball_uri)

	

Evolution and IMAP

Evolution is great

I love Evolution. I’ve used it for years for POP3 access and it has easily handled huge amounts of email and allowed me to quickly find information. I can not use a command-line email client to read my email. I am not like that. Years ago when I first started using Linux, I had a Windows partition (actually a removable disk) just so I could boot into that to check my email with Outlook. That was tedious. Then Evolution let me use Linux all day every day. It was familiar but far more stable and faster and less confusing than Outlook.

Evolution is still great for POP3 and getting better all the time, and possibly for some other protocols that I haven’t used.

Evolution with IMAP is not so great

But about a month ago I switched Evolution from POP3 to IMAP, because I got tired of not having access to old emails when away from my main PC, for instance when I use my laptop or when I use the webmail interface. This has mostly given me what I wanted, though dreamhost’s squirrelmail times out when searching my old messages, which are archived to a separate folder after a few days for performance reasons. But it means that I don’t stumble over not having the information in an email that someone sent me yesterday, and then realise the next day that I forgot to check it while at home.

However, it’s very slow, so slow that it rarely finishes whatever it’s doing before I shut my computer down. The status bar is always showing a couple of “Working oxb1208ad8” or similar messages. I think it’s doing more things with hexidecimal names, but it only shows two at a time. Every time I click on a folder it seems to give itself lots more work, and is determined to finish whatever it started doing for the last folder, even though I’ve lost patience with that already and just hope it can forget and maybe be quicker with another folder instead.

I’m not 100% convinced that this is caused by Evolution or instead by IMAP in general, or maybe by a problem with my IMAP server, or with IMAP servers in general.

Maybe (I have no idea) IMAP is a fundamentally slow or high-latency protocol, but it seems theoretically possible for Evolution to not be slow about showing me what’s already on my local disk. If it wants to check for new stuff on the server then that should happen in the background. I’m not saying it’s easy to implement, but it should be possible.

This whole downloading local copies thing is confusing in Evolution:

  • The account setup (Edit/Preferences/Mail Accounts/Edit/Receiving Options) has an “Automatically synchronize remote mail locally” checkbox.
  • Right-clicking on a folder in the Sidebar anch choosing Properties, allows me to check “Copy folder content locally for offline operation”. Isn’t this the same thing? And is it recursive? Am I really gaining anything by tediously setting this for the 40 folders in my folder for mailing lists?
  • You can force a download of local copies by choosing File/Work Offline, but a) Didn’t I just specify in the first two ways that it should always do that, and b) I can’t check new emai while it’s doing this.

I just built Evolution 2.8 from source (I usually use 2.6 on Ubuntu Dapper) to see whether it’s faster, but I don’t notice any significant difference.

I suspect that some of the slowdown is caused by me having 40 filter rules to sort my emails into different folders for each mailing list. Unlike with POP3, it has to tell the server to move these instead of just moving them on the local disk. I guess server-based rules would be more efficient but I have no way to specify such rules with my hosted email server, and Evolution certainly doesn’t offer any way to do this. Turning off email filters or junk filtering (in the account setup) doesn’t make it noticeably faster when switching folders, but I didn’t try this for a few hours, and Evolution does seem to get slower with use.

By the way, why can’t Evolution just arrange all my mailing list emails automatically without me having to create folders and filters for each one?

It also seems slightly faster when not using encrypted IMAP (with TLS or SSL), but not fast enough, and a) I like encryption, and b) It was fast enough when using encrypted POP3.

I have tried Thunderbird. It seems slightly faster, but I haven’t tried recreating my 40 filter rules to test whether it’s faster when doing that. It does at least show a pulsing progress bar when it has to download an email, instead of showing me a blank email for a minute or so. It has the same UI problems when saying what should be downloaded locally – there’s no way to say “download everything and keep downloading everything, so you can be fast”. One great thing about IMAP is that you can easily try other email clients.

I’m not complaining about the hard-working developers. Evolution does a lot of great stuff and I’m not ready to move away from it, particularly because Evolution integrates more with my desktop than Thunderbird is ever likely to. I’m sure they’ll fix whatever they can in time – I recognise that getting an email application right is not easy. I just wanted to get my thoughts written down. I am not jumping on the bandwagon.

Update: I should give Evolution 2.8 more of a chance over the next few days. I guess it does feel a bit snappier.

Secure silent mail server

An Openismus client demanded that all email communication with them should be over TLS tunnelling, with the mail server being on-site, so my regular hosted mail accounts won’t do, and even a hosted server wouldn’t be enough. Frankly, I’d rather not have the trouble and expense of maintaining my own servers, but it’s a nice client so to please them I set up a mail server like so:

  • Buy a quiet (no fan) low-power PC. I chose an ichbinleise PC LL10. It’s not very pretty or powerful or small, but it’s enough for a mail server. Ichbinleise (MR Computertechnik), seem to use cases and parts from Silverstone.
    • The new AMD Geode-based PCs are cheaper and require much less power, but pre-built systems don’t seem to be properly on the market yet.
    • This list of mini PCs is interesting.
    • It’s not low-power and fanless, I think, but the A-Open Mini PC is pleasantly Mac Mini-like, though I suppose the new intel Mac Minis will support Linux distros some time soon.
  • Get a new DSL broadband login from Via, so I can have a static IP Address. The contrast between their web site (simple, what you need) and service (simple, quick) and that of T-Online/Deutsche-Telekom (insanely complex, obscure, beaurocratic, inflexible, expensive) is vast.
  • Get an extra domain name just for this email account. I’m not going to risk putting my regular email through this experimental system.
  • The domain registrar doesn’t let me enter IP addresses for the DNS name server, so I can’t set up my own DNS server that specifies which of the PCs on my network is the mail server, and of course I can’t create a ns.something.com without having a DNS server. This is probably normal – I wonder how this chicken and egg problem is normally solved. So, I used EasyDNS to get a DNS server, and told the domain registrar to use that name server. Then my DNS entries at EasyDNS specify my static IP address.
  • I set up port-forwarding rules on my router (a cheap wireless/DSL thing that I got from Deutsche Telekom), telling it to direct traffic on the POP and IMAP ports to the mail server PC.
  • Instead of using port forwarding, I suspect I could have a local DNS server (which would know about the IP addresses on my local subnet) in addition to the EasyDNS server (which can’t use my local IP addresses because they aren’t really assigned, being behind a NAT), but I’m not sure how that would work.
  • Now to install the mail server software:

  • I tried Ubuntu Breezy (I have one thousand CDs, so it’s easy to find one, and was too lazy to download Dapper), but installation failed half-way saying there was a problem with the CD drive.
  • Ubuntu Dapper installed perfectly.
  • Thanks to a reply to my previous blog about mail servers, I found a great page about installing an email server on Ubuntu. It explained what the point of the different parts is, and told me exactly what to do to install postfix and dovecot with support for IMAP and SMTP using TLS. There’s a scary series of commands that it tells me to type in, but if I have to do that then I’d rather not have to figure it out by myself.
  • I found it odd that the email account is defined by the linux shell account, with the same password, so I’d have to add a new shell account just to add a new email account. Those wiki documents mention something about virtual mail hosts, but they seem to be about serving mail for multiple domains. Maybe that’s also how you separate email accounts from shell accounts.

Back from Romania

So, I should finally get around to describing my trip to Romania. I returned two weeks ago, but I’ve been busy since then, and it takes ages to upload pictures on my crappy far-slower-than-advertised T-DSL line. It was an incredible three weeks, so this will be a long blog entry, even though there’s lots more that I’m not mentioning.

TimiÅŸoara / Temeschburg

We arrived first in the Banat (or the current Romanian part of it, in TimiÅŸ county), after driving though Hungary, crossing the border at Cenad, into a rich green landscape with an enormous sky and a series of people selling melons at the side of the road. In the Banat, the land is flat and the sky is both wider and higher than a regular sky.

IMG_1543 IMG_1528

Over the next few days we visited villages in the surrounding Banat, where my girlfriend’s family lived before emigrating in the late 80s, such as Bogarosch/Bulgãrus and Groß-Sankt-Peter/Sânpetru Mare. The current inhabitants welcomed us into their homes, without fear that we wanted them back. People here live from the land, growing their own food and keeping their own animals.

IMG_1568 IMG_1655

While there I started reading Richard Wagner‘s (not him) novel Habseligkeiten. He’s from Perjamosch/Periam, along the road. He covers the local history fairly well, from the point of view of the ethnic Germans, who were naively caught up in every political change that swept across them. That’s including

  • Austro-Hungarian rule before the first world war.
  • The first world war experienced at a distance, with some German men fighting far away and returning.
  • The second world war, with some German Romanians serving in the regular German army (by going to Germany), some in the SS (pretty much mandatory for Germans outside of Germany), and some in the Romanian army (which fought on the side of Germany until Romania switched sides in 1944, courageously but wisely as the Soviets started to win).
  • Transport of many Germans (mostly men, even those in the Romanian Army) to Russia in 1944 for five years of forced labour.
  • returning from Russia to Communist rule, with no private property, inefficiency, and corruption.
  • Then forced relocation of whole families to the barren Bărăgan plain in the 1950s.
  • Returning again five years later.
  • Followed by emigration to Germany during CeauÅŸescu‘s 1980s and, after the 1989 revolution, in the 1990s.

I’ve heard very similar stories from my Girlfriend’s parents and grandparents, who also mention time in Germany and in American prison camps, but the book wraps things up nicely. There’s lots of adventurous stories to be told. Like everyone, it avoids the holocaust.

TimiÅŸoara itself is part old-town and part crumbly soviet concrete. The old buildings have particular charm because they are falling apart.

IMG_1468 IMG_1426

While there, I met some of the Timisoara LUG people. Dan Damian (not pictured) had invited us, but couldn’t stay more than five minutes because he was very ill. I’m afraid I’ve already forgotten the names of everyone else, but I remember that the guy on the left was doing an anti-virus thing for Google Summer of Code, and I think he’s Török Edvin. It was interesting to speak to these guys about how they are making their way in modern Romania. I was also ill for the next four days. Maybe it was the water, or maybe Dan is to blame somehow.

IMG_1755

As I noticed later, TimiÅŸoara is entirely different to Bucharest or other parts of Romania that I saw. In Timisoara watermelons are sold from huge piles at the side of the roads in to the city and even on street corners in the city. People eat the watermelons in the street and spit out the seeds. Horse drawn carts share the roads with cars, often delivering the watermelons. Sometimes you’ll see a regular saloon car driving by, completely full up to the roof with watermelons. There are no watermelons in Bucharest as far as I can tell. It’s better where the watermelons are.

Bucharest (BucureÅŸti)

We drove from TimiÅŸoara to the capital, Bucharest, via Sibiu/Hermannstadt, on the new roads that the EU has built. The EU will be happy to know that the road is full of lorries delivering produce, but the two-lane roads are lethal.

Bucharest, is really not worth the visit except to get a more complete picture of Romania. There are a few impressive older buildings but it’s mostly large and concrete and choking in traffic. Ceacescu’s government palace is huge, but too silly to really impress. There’s a wonderful village museum, with old relocated wooden and stone buildings, but it’s nicer to see them for real outside of Bucharest.

IMGP0945 IMG_1774

Prices here are comparable to western European cities, but that doesn’t feel like value for money. Advertising has obviously taken hold, seemingly driven by a monotonous few companies selling advertising space on the sides of buildings and large video screens. Life must be strange for the people who live in Apartments whose windows are covered by 6-story-high posters. A few companies are noticeably trying to establish themselves in this new market: Orange, Coke, and Nescafe (as an upmarket brand, strangely), along with a few German food brands. There are several new shopping centers selling western products such as electronics. Credit is advertised heavily and I don’t think people are expected to buy things outright.

Hope for the future is obviously widespread, presumably due to expected EU membership, as you can tell from all the building work and the credit boom. But it seems to be only the first stage of capitalism. People are trying to make a fast buck, often at the expense of their customers instead of building long term reputations and relationships, and either competitive pressures are not being felt, or something is stopping competition from happening. The government does not seem to be fostering marketplaces or making consumers feel protected. The tourist will notice that when changing money (the exchange rates listed are generally only for amounts over 10,000 Euros) or paying extra to use his camera. The guy buying a car or washing machine on credit will notice that when he sees an extra 5% yearly interest payment for administration.

This lack of long term thought may also be responsible for how Romanians drive their cars. If not then there’s either a general failure to assess risk, or human life is considered worthless.

Oddly, government buildings are already flying the EU flag, as if Romania had already joined. That’ll look embarassing if they don’t meet the grade next year either. They probably will, but they don’t seem entirely ready for it. They also fly the NATO flag, which seems out of place on civilian institutions such as parliament or schools.

In Bucharest I met an old friend from Paderborn, Christian Schoppmeyer, who is now running the Bucharest office of Sagem Orga (formerly Orga), and some of his friends there. It’s great to see him doing well. Bucharest may be crappy, but it offers a lot more than Paderborn.

Carpathian Mountains

After visiting Sigişoara, we spent a week in the hills, in the village of Măgura, in the curve in the Carpathian mountains, near Zărneşti, near Braşov.

We stayed at Villa Hermanni run by Hermann and Katrina Kurmes, after it was recommended by Hans Gasser, a friend of a friend who had visited to do an article for the Sueddeutsche Zeitung on Romanian ecotourism. Hermann is an ethnic German from the neighbouring Vulkan village who returned to Romania after the 1989 revolution to do ecological tours of the area and its wildlife, as part of the Association for Eco Tourism in Romania.

The area, between the Bucegi and Paitra Craiului mountains, is beautiful and still inhabited by people who live from the land. There’s original forests with (unseen) brown bears and wolves still living in them.

However, poor locals are understandably selling their land for hotels and summer houses to be built, and it can’t be long before tarmac roads are built to service them, which will accelerate the destruction yet more. This isn’t allowed, because it’s officially a nature reserve, but the law is flexible in the face of corruption and beaurocracy, so it looks like this valley will be completely transformed in another couple of years. Visit now before it’s too late. We are tempted to go again in the winter for snow-shoe trecking.

A big reason for our visit was to see some of Romania’s brown bears. We only saw paw prints while walking in the hills, and in a nearby cave, but Katrina Kurmes took us to a nearby lookout hut where we could watch a bear arrive to eat food that was left for it. The bears are very shy of humans, but can lash out if surprised, and do tend to eat animals that have been left outside at night. Unlike Germany, Romanians continue to living fairly peacefully with them.

IMG_2004DSC_0148

There is an obvious problem with litter. People just drop plastic bottles and packaging on the ground, even in the most beautiful places. My brain isn’t wired like that. This seems to be part of the short-term thinking problem I mentioned above. Luckily, it’s not too late to fix this, but it needs government help to get the rubbish down to the towns.

By the way, the ÅŸ characters are like sh in English. The ă characters are a kind of “uh” sound. The final i in words is not really pronounced, unless it’s ii at the end. See Wikipedia‘s Romanian Alphabet page for more and more correct descriptions.

Automatically rotating images on Linux

Gallery doesn’t yet automatically rotate images using the EXIF orientation data that newer cameras save in the file, though it will soon. In the meantime, jhead can automatically rotate the images, like so:

jhead -autorot *.jpg

And I think that’s lossless. You can run it on already-uploaded Gallery images from the terminal.

By the way, it looks like GNOME’s Image Viewer will finally support automatic rotation soon.

Adobe Source Libraries (ASL): Adam and Eve

I finally read the introduction to Adobe’s ASL GUI system, and its Adam and Eve parts. Ralph Thomas has been using it to write his Mission Photo application, so it seems to work. It’s great to see the rationale documented publically, and I guess that Adobe will profit from the feedback. Design is so much easier when you can bounce ideas off someone. I notice that the overview document is dated December 2004, so I’m late with my feedback.

Eve seems to be a UI layout description format, much like Glade (including the use of widget identifiers that are looked up later, from code with libglade, or from Adam with ASL), but without specifying actual specific widget types, so it’s “button” instead of “GtkButton”. It’s nice that it uses generic widget names, which are later realised as actual GTK+, MacOS, or Windows UI parts. It’s annoying to me that it doesn’t use XML, which could make the structure more obvious.

The documentation makes a big deal of automatic layout, without calling it that, but that’s the only sane way that anyone does layout now anyway, so that alone is not enough to make this useful. But it’s understandable that this is an issue for them, because these guys built their code base with the awkward MacOS classic APIs, and then the crappy Win32 APIs, which demand fixed per-pixel layout and implementing your own event loop. I’ve been there. GTK+ does this for you.

Adam seems to define (declaratively) how the widgets behave. Adam files seems to be made up of sheets (maybe equivalent to a dialog, or maybe a user operation, I’m not sure), and cells (though that name is not mentioned explicitly in the syntax). Adam files specify the properties of a widget in terms of the properties of another widget, so that, for instance, a widget can be deactivated if another widget is unchecked. I guess it can do more than just simple boolean logic and mathematics, but this is where it feels like I’d need some real programming to get real things done.

But it’s a nice idea if it can work without getting in the way. We have grasped at similar things in GTK+, for instance by specifying that the keyboard mnemonic for one widget (a label) can activate another widget (an entry next to the label), plus our accessibility data that expresses higher-level relationships between the UI elements. But we haven’t generalized it. Paul Pogonyshev posted on the gtkmm list and on the libsigc++ list.recently about something like this, but I don’t think the result was useful enough, possibly due to the limitations of a statically-typed language. It would be nice to solve common problems more concisely.

Adam and eve introduce several new terms (model, command, cell, sheet, field) which leaves plenty of scope for confusion, particularly because these names seem quite arbitrary and they have several existing meanings. I think I’d use more explicit names, such as data-field, if it was my project.

As far as I understand, the system expects you to think of a UI (such as a dialog) as a way to generate a command, and ASL can represent a model of this command, though “model” is a confusing name for it. This command, produced by the view (UI) is then used to transform the document model (the data). That doesn’t sound very suitable for rich interaction, but obviously it works for Photoshop and Mission Photo. Possibly you can update your application state somehow in response to individual changes to UI widgets, to allow instant-apply.

Corrections are welcome.