glibmm 2.44.0 and gtkmm 3.16.0

I’ve just done the stable glibmm 2.44.0 and gtkmm 3.16.0 releases with the usual bunch of API additions and deprecations to keep track of the glib and gtkmm API. Thanks to Kjell Ahlstedt in particular for his many well thought-out contributions.

I’ve been maintaining gtkmm since at least some time in 2001. That’s 14 years or so. Has any GNOME maintainer maintained one module for so long?

 

 

GTK+: Aligning / Justification in text widgets

There are 3 GTK+ text widgets: GtkEntry, GtkTextView, and GtkLabel. I noticed recently that they are a little inconsistent in how they offer alignment of their text, and the API confuses the terms justification and alignment slightly. This is highly unlikely to change, because that would be disruptive to existing code, but maybe this is helpful to people trying to use the API as it is now.

I’ve put some test_gtk_text_widgets_align.c example code in github. It shows two GtkLabels (single-line and multi-line) two GtkTextViews (single line and multi-line) and a GtkEntry (single-line only). I tried this with GTK+ 3.15.9. I’ve mostly ignored deprecated API, such as GtkMisc and GtkAlignment, to simplify things.

I’ve referred to the API via the *_set_*() functions, but this API is available as properties too. There are also vertical equivalents though I only talk about horizontal alignment here.

Default

This is how it looks by default. It looks fine, though the GtkLabel, and only the GtkLabel, defaults to center alignment – not a very useful default.

Justification

Now, if you want to justify the text to the right, you might call gtk_label_set_justify() on the GtkLabels and call gtk_text_view_set_justification() on the GtkTextViews, though you’ll need to call gtk_text_view_set_wrap_mode() for that to take effect. Then you’d see something like the screenshot below.

You’d notice

  • The justification had no effect on the single-line GtkLabel (as per the gtk_label_set_justify() documentation) but it did have an effect on the single-line GtkTextView.
  • The text in the GtkLabel remains aligned in the center (the GtkLabel default) even though it’s justified within that central part.
  • We didn’t call any justification method on the GtkEntry, because it has none. However, see below about gtk_entry_set_alignment().

Justification and Alignment

To fix that center-alignment of the right-justified text in the GtkLabel, you could call gtk_widget_set_halign(). Then you’d see this:

But, strangely:

  • The center-aligned right-justified text in the multi-line GtkLabel is unchanged. It remains center-aligned – it is not flush with the right side of the widget. But see below about gtk_label_set_xalign().
  • The single-line GtkLabel now appears to be justified/aligned to the right, which is nice. But see below.
  • The GtkTextViews have changed their word wrapping for some reason. I have no idea why.
  • The GtkTextViews and GtkEntry now use only just enough space at the right for their entire widgets, not just the text that they contain. In fact, the GtkEntry is now too small to show all of its text and, strangely, that happens even if I use gtk_widget_set_hexpand() on the GtkEntry.

This shows us that gtk_widget_set_halign() affects the whole widget (as per the halign documentation), not just the text. That is more obvious if we set a background color. Then we see that that’s why the single-line GtkLabel now looks justified – it’s really just the whole widget that has been aligned to the right:

Obviously, I don’t want this effect with the GtkTextViews, and it would only be useful on a single-line GtkLabel. Luckily we don’t need to use gtk_widget_set_halign() at all to justify text:

gtk_label_set_xalign

The GTK+ developers noticed this problem with GtkWidget’s halign so in the very latest GTK+ (3.15/3.16), they added gtk_label_set_xalign(). This aligns the text in the GtkLabel, rather than the GtkLabel itself. Like so:

I just pushed an improvement to the gtk_misc_set_alignment() deprecation documentation to make that clearer.

gtk_misc_set_alignment (deprecated)

If you can’t use the last GTK+ API, you can use the deprecated gtk_misc_set_alignment() method to get the same effect as gtk_label_set_xalign(). (Ignore the GtkTextViews here – they don’t derive from GtkMisc).

screenshot_test_gtk_text_widgets_set_alignment

Entry alignment

Likewise, GtkEntry has gtk_entry_set_alignment(), which has been in GTK+ since 2.4. Like gtk_labeL_set_xalign(), it affects the text in the widget, not widget itself.

screenshot_test_gtk_text_widgets_entry_alignment

Conclusion

So, if I wanted to justify my widget’s text, without changing the size of the widget itself, I’d decide what to do like this:

  • For a GtkLabel: Use both gtk_label_set_xalign() and gtk_label_set_justify(). Use gtk_misc_set_alignment() instead of gtk_label_set_xalign() if you can’t use GTK+ 3.16.
  • For a GtkTextView: Use gtk_text_view_set_justification() with gtk_text_view_set_wrap_mode().
  • For a GtkEntry: Use gtk_entry_set_alignment().

I could be wrong, so do please tell me how.

Locking down a GNOME setting

I think the dconf documentation and GNOME’s Sysadmin lockdown documentation (There’s this GNOME dconf wiki page too) could be clearer as well as being less duplicated and scattered. I’ve started trying to improve those documents directly.

In the meantime, here are some simple instructions for locking down some settings in GNOME 3 so the user cannot change them. Thanks to Ryan for explaining this to me. I’d welcome anyone’s extra corrections.

In this example, two children, Alice and Bobby, have user accounts on the computer called “alice” and “bobby”, and I want their accounts, and only their accounts, to not allow changing the list of application launchers in gnome-shell’s Dash, and not allow changing the desktop background. (Just as an example.)

Creating a dconf profile

I create a dconf profile for children in /etc/dconf/profile/, like so:

$ sudo nano /etc/dconf/profile/child

And I put this text in that file:

user-db:user
system-db:child

That tells dconf to use the “user database” named “user” as the writable store of settings for this dconf profile. Whatever database is on this first line will be used as the writable database. “user-db” (user database) just means that dconf should look in XDG_CONFIG_HOME/dconf for this database. That typically means ~/.config/dconf/, so this is specific to the user. This is what dconf uses by default if you haven’t assigned a dconf profile or changed the default “user” profile.

This example additionally tells dconf to use our “system database” named “child” for this dconf profile. Because this is not on the first line it is not writable. Because it’s a “system database” it will be in a shared location – typically /etc/dconf/db/. So this is a good place to specify some locks to prevent changes in the “user database” by anyone with this “child” profile. We could also use this “system database” to set some default values for the “child” profile.

The dconf overview documentation explains that profile file format more fully.

Creating a dconf lock file

Then I create a lock file for that “child” system database, like so, though the actual filename doesn’t matter:

$ sudo mkdir /etc/dconf/db/child.d
$ sudo nano /etc/dconf/profile/child.d/locks/00_some_locks

I put this text in that lock file:

/org/gnome/desktop/background/picture-uri
/org/gnome/shell/favorite-apps

I then call “dconf update” to tell it about the new profile and the lock file:

$ sudo dconf update

However, when you change the lock file, you’ll often need to do this too to make “dconf update” notice (see dconf  bug #741437):

$ sudo touch /etc/dconf/profile/child.d/locks
$ sudo dconf update

Using the dconf profile

Then I assign that dconf profile to the children’s user accounts, for instance by setting the DCONF_PROFILE environment variable in Alice’s .profile file:

$ sudo mkdir /home/alice/.profile

like so:

export DCONF_PROFILE="child"

If the child is already logged in then she’ll need to log out again to make her account use the new dconf profile. Also, don’t be foolish like me and put that in the .bash_rc file instead. That way lies confusion.

Note that the Alice can change her DCONF_PROFILE by changing that .profile file, to avoid the locks in her current profile. Or she can add a .bash_profile file to override .profile, or do something else clever. Locking of dconf settings is only for convenience – it isn’t meant to provide security. So if the user knows how to change her DCONF_PROFILE, she should probably be allowed to.

It would be nice if there was some GUI for managing profiles, and the databases’ locks, and default values. That seem doable.

I also wonder if there could be any API for applications to ask for admin access before making changes. Then a parent could, for instance, add just a single application launcher to the Dash. But I guess that would need us to either remove the dconf profile from the user (which needs a logout and login) or would need us to remove the lock from the dconf profiles’ system database, which would briefly unlock the setting for all other users who have the same dconf profile.

android-galaxyzoo: Superficial porting to Android 5.0 (Material design)

Here are some notes about my experience adapting android-galaxyzoo to Material design for Android 5.0 (Lollipop) though I only used the most superficial parts of Material design.

AppCompat v21

Android 5.0 (Lollipop) has a new UI theme and some new APIs. However, for the next few years, almost everyone will use the slightly awkward AppCompat v21 compatibility API instead to achieve most of the same behavior on older devices too. Chris Banes wrote up a nice overview of AppCompat v21, some of which I mention here again for completeness.

I’m using Gradle, as should you, so I added this to the dependencies block in my app/build.gradle file. You’ll want to use the latest version.

compile "com.android.support:appcompat-v7:21.0.3"

Theme

First, I switched from the dark Holo theme to the (AppCompat) dark Material theme by changing the parent theme in my styles.xml. See the Toolbar section below about the use of the “.NoActionBar” versions of these themes.

- <style name="AppTheme" parent="android:Theme.Holo">
+ <style name="AppTheme" parent="Theme.AppCompat.NoActionBar">

If you were using the light theme, that would be:

- <style name="AppTheme" parent="android:Theme.Holo.Light">
+ <style name="AppTheme" parent="Theme.AppCompat.Light.NoActionBar">

Note that we don’t use the android: prefix with the AppCompat theme, because the theme is being bundled directly into our app via the appcompatv21 library.

I then specified  the standard colorPrimary and colorAccent colors along with some more shenanigans to get the right text and icon colors in my toolbar.

I also used the TextAppearance_AppCompat_* widget styles instead of the regular textAppearance* style attributes, because it’s recommended in the Typography section of this official “Implementing Material Design in Your Android app” blog entry. However, I didn’t notice any difference in appearance, and I wonder why we wouldn’t just get the correct styles by just using the new overall theme.

I actually created a base style and two derived styles, to support Transitions – see below.

Toolbar

The new Toolbar widget replaces the ActionBar, though the documentation doesn’t actually say that yet. Generically, they are called the “App Bar” in the Material Design document. I’m not sure that I really got any benefit from using it because my App Bar doesn’t do anything special, but I wanted to use the latest API.

To use Toolbar instead of ActionBar,  you should derive from the .NoActionBar version of the theme, such as Theme.AppCompat.NoActionBar, though I used the regular Theme.AppCompat for a long time without noticing any difference.

Then you’ll want to add a Toolbar widget to the Layout XML files for every activity. I did that by creating a toolbar.xml file

<?xml version="1.0" encoding="utf-8"?>

<android.support.v7.widget.Toolbar
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:id="@+id/toolbar"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:background="@color/color_primary">
</android.support.v7.widget.Toolbar>

and then I just including that from all the activity layouts like so:

<include layout="@layout/toolbar" />

I also specified the app:theme and app:popupTheme to get the right text and icon colors in my toolbar.

To use this toolbar as the App Bar, you need to derive from ActionBarActivity and call its setSupportActionBar() method. I did that in a utility function, like so:

Toolbar toolbar = (Toolbar) activity.findViewById(R.id.toolbar);
activity.setSupportActionBar(toolbar);

RecyclerView

Android 5.0 adds the RecyclerView widget, available for older API levels via the support.recyclerview library. For instance, I added this to the dependencies block in my app/build.gradle file. You’ll want to use the latest version.

compile 'com.android.support:recyclerview-v7:21.0.0'

RecyclerView apparently replaces ListView and GridView, though the documentation doesn’t yet actually say that, adding confusion for new developers. I replaced my use of GridView with RecyclerView to get support for simple Material Design transitions.

Unfortunately, RecyclerView has no real cursor support, so it’s hard to use it to view data from a ContentProvider. This is particularly annoying because the API of ListView and GridView previously pushed us towards moving code into a ContentProvider. My own Cursor-based RecyclerView.Adapter is terribly inefficient and totally unsuitable for a large number of items.

There’s an awful lack of practical documentation or example code for RecyclerView, even for simple tasks like responding to a click. Until there’s an easier way, you’ll need some tedious boilerplate code to derive your RecyclerView.ViewHolder from View.OnClickListener and call setOnClickListener() on one of your child views.

Transitions

I wanted the typical move-and-scale transition that we see in Material Design apps, so that clicking on one of many items makes its image expand and move into the subsequent detail activity, and shrink back again when you go back.

To achieve this, I had to specify various settings in my theme, but only Android 5.0 (Lollipop) devices support these transition items, so I created a base theme in res/styles.xml, and derived from it:

<?xml version="1.0" encoding="utf-8"?>
<resources>

    <style name="AppTheme" parent="AppTheme.Base" />

    <style name="AppTheme.Base" parent="Theme.AppCompat.NoActionBar">
        <item name="colorPrimary">@color/color_primary</item>
        ...

I then created a styles.xml file just for API Level 21, by putting it in res/v21/styles.xml, where I derived from the base theme again:

<?xml version="1.0" encoding="utf-8"?>
<resources>
    <style name="AppTheme" parent="AppTheme.Base">
        <item name="android:windowContentTransitions">true</item>
        <item name="android:windowAllowEnterTransitionOverlap">true</item>
        <item name="android:windowAllowReturnTransitionOverlap">true</item>

        <!-- specify shared element transitions -->
        <item name="android:windowSharedElementEnterTransition">
            @transition/change_image_transform</item>
        <item name="android:windowSharedElementExitTransition">
            @transition/change_image_transform</item>
    </style>
</resources>

I then defined that change_image_transform transition in my res/transition/change_image_transform.xml file, like so:

<?xml version="1.0" encoding="utf-8"?>
<transitionSet>
    <changeBounds/>
    <changeImageTransform/>
</transitionSet>

To actually use this transition on images, I needed to specify the android:transitionName on the two ImageViews in the the layout XML files for the two Activities (Fragments in my case).

Mostly this was all voodoo which I put together gradually after finding clues scattered around the internet. I haven’t found a good official example that shows this.

Unfortunately, the transition doesn’t seem to work when the user presses the Up button on the toolbar instead of using the standard Back button, even though that’s indistinguishable from Back for most users in most activities.

Metrics and Keylines

I made a fair effort to adapt my margins and padding to fit in with the Material Design Metrics and Keylines, which wasn’t too hard.

Unfortunately, the standard Android Button’s appearance is just as usless for Material design as it was for the Holo theme. It has a fake margin around its inside edges, which is part of its background graphic rather than any adjustable margin or paddng property.

So, to make the button’s sides actually flush with other widgets, and to position them properly on the layout grid, I had to specify a custom background image or color. But then I lost the nice Material Design ripple effect. I hope someone knows how to do this properly.

Activity classes without Toolbar support

There are a few helpful derived activity classes, such as AccountAuthenticatorActivity and PreferenceActivity, but these haven’t been changed to derived from ActionBarActivity, so you can’t call setSupportActionBar() on them. They can’t be changed without breaking compatibility, so you’ll have to reimplement them in your code. It’s not a lot of code but it’s an unpleasant developer experience.

 

 

android-galaxyzoo: Network I/O and UI responsiveness

While developing the Galaxy Zoo app for Android, I experienced problems with UI responsiveness while the app was downloading from, or uploading to, the remote server. I don’t mean just the usual problems caused by doing network I/O or file system access in the UI thread – mistakes that can often be caught by using Strict Mode. I mean that buttons didn’t respond to presses if the app happened to be doing I/O in an AsyncTask.doInBackground().

I solved this problem, but it was an incredibly frustrating journey. These notes might help the next person. So many Android apps are just a fancy way to interact with a website’s data, caching it locally, but the regular Android API doesn’t make that easy enough.

My app uses a ContentProvider, because the Android API seems to point us towards doing so, as I found while creating my experimental Android Glom app. The ContentProvider usually runs in the same process (and thread) as the main UI, so I don’t think this architecture was a cause of the UI responsiveness problem.

Thread+ResponseHandler versus AsyncTask

My ContentProvider needed to communicate with the remote server via REST messages, downloading images from the URLs that it then discovered and caching those images locally as well as storing data in its SQLite database. At first, I used the Thread class directly, with a UriRequestTask class derived from Runnable, that used HttpClient and accepted a (HttpClient) ResponseHandler. This was based on the example code in O’Reilly’s Programming Android (2nd Edition) book, in its “A Content Provider as a Facade for a RESTful Web Service” chapter (which you can see some of here).

Later, before I found the UI responsiveness problem, I replaced this Thread and RequestTask idea with AsyncTask because that seemed more robust and made it easier to get a result back to the main thread. I didn’t understand why the “Programming Android” book didn’t use AsyncTask in its ContentProvider. Yet later, I found that Strict Mode didn’t allow that use of AsyncTask in a SyncAdapter’s non-UI thread on some older devices (API version <15), causing an ExceptionInInitializerError with “Can’t create handler inside thread that has not called Looper.prepare()”. Maybe the authors found the same problem.

HttpURLConnection instead of HttpClient

Incidentally, I later realized that the Android developers now discourage use of HttpClient, because their version of HttpClient is buggy and they can’t update it without breaking compatibility. So I changed from HttpClient to HttpURLConnection (and here and here). Luckily, this fixed the endless redirect problem I (like others) was having with my uploads.

Again, I felt led astray by the “Programming Android” book, but books get out of date very quickly and the Android documentation apparently gives us new clues only reluctantly. Hopefully the 3rd edition will improve this.

Avoiding the user

This is when I noticed the UI responsiveness problem. I tried using Thread directly in various ways instead of AsyncTasks, fearing my own changes had caused it, but it made no difference.

I tried all kinds of tricks to avoid doing any I/O, even in the AsyncTasks, when the user was likely to be using the UI, trying to wait until it seemed that the user was idle and then doing some quick I/O before getting out of the way again. But the user is unpredictable and would still try to press a button now and then exactly when my app was doing some sneaky I/O.

After lots of imprecise testing, I was fairly sure that the problem was worse when the app happened to be doing some filesystem I/O, even just writing to the SQLite database, but it was definitely caused too by doing network I/O of even small JSON files. We should expect disk I/O to be worse than network I/O, of course.

SyncAdapter

I wanted to learn about Android’s SyncAdapter framework anyway, so I spent a week figuring that out despite its awful hand-wavy documentation. Luckily, it fixed the UI responsiveness problems. I think it helped partly because the I/O was now in a separate process, though that doesn’t feel like it should be necessary. Also, the SyncAdapter framework can decide when best to let the SyncAdapter do its work, based on the whole system. Presumably it can then prevent unrelated processes from making the UI responsiveness even worse.

Unfortunately, using SyncAdapter does mean that I lose control over when I may download items in advance. For instance, I often find that a new install of the app doesn’t fill up its initial list of ten items until five or ten minutes after the first start. In the meantime, the app requests the items directly when necessary. I call ContentProvider.requestSync(), with SYNC_EXTRAS_MANUAL, when I’m sure there’s work for the SyncAdapter to do, but it just seems to take this as a suggestion that it’s free to ignore.

Authenticator and Account

To use the SyncAdapter, I had to create an Authenticator, for which I needed an Account type, even if you are accessing your server anonymously. This account type will show up in the Settings->Accounts UI. Presumably this is why some other apps show up in the Accounts Settings, even though they don’t need any login. Maybe they are doing it just so they can use SyncAdapter.

SyncAdapter versus SharedPreferences/PreferenceFragment

So I started by storing the login details via the AccountManager instead of via SharedPreferences. I later discovered that the app’s SharedPreferences (such as how many items to download) were unavailable to the SyncAdapter’s separate process. That is documented as “currently this class does not support use across multiple processes“. So I later put other preferences in the account too. Unfortunately, the standard settings/preferences UI only uses SharedPreferences, so I have hacky code to keep the SharedPreferences and the Account in sync – see my onSharedPreferencesChange() override in my PreferenceFragment, so the account’s UserData is changed when the standard settings UI is used, via my hacky copyPrefToAccount method.

AccountManager versus main/UI thread

I also later found that you can’t use the AccountManager API from the main UI thread even though the documentation says things like “It is safe to call this method from the main thread”. The Firefox on Android developers discovered this too (1, 2, 3). So I moved that code to an AsyncTask (for example). If you don’t then you’ll get StrictMode exceptions at least on API level 15 and 16 – maybe others. StrictMode seems to have been on for Android Honeycomb, and some users seem to enable StrictMode, so you will see this as crash reports on some devices.

Volley

I also wanted to try out the Volley library. This isn’t an official part of the Android API. For instance, there’s not even an official way to add it as a dependency – I used the Volley package that mcxiaoke helpfully published to maven central. But Google use Volley in some of their own apps and they recommend Volley in the Android documentation. The video linked from there is well worth watching.

When I rewrote my HttpURL Connection-using code with Volley it seemed to make my downloads noticeably faster, probably because it avoids doing I/O in too many threads simultaneously. This was an easy win.

I couldn’t use Volley for my uploads, unfortunately, because Volley uses HttpClient on older Android versions whose HttpUrlConnection wasn’t good enough yet, causing the old endless-redirect problem again. So I use HttpURLConnection directly for that.

Picasso

I also switched to using Picasso to fill ImageViews with bitmaps from our ContentProvider or directly from the internet, instead of my own AsyncTask. This simplified the code and gave me some extra caching, particularly simplifying my ListView/RecyclerView adaptor that previously had its own custom LruCache. For instance 1, 2, 3. Image loading just generally felt much faster everywhere that I used Picasso, though I don’t know exactly why.

Just filling an ImageView without blocking the UI seems like something unexciting that the Android API should already make easy.

Galaxy Zoo for Android

I just released my Galaxy Zoo android app to the Google Play store. The Zooniverse team have adopted it as their official Galazy Zoo app for Android.

Like the Galaxy Zoo website, it asks you questions about a picture of a galaxy, with each question depending on the previous question. This “Citizen Science” helps astronomers to analyze the huge amount of images of galaxies provided, for instance, by the Hubble Space Telescope. There are several other Zooniverse projects, though most require more thought and have more complicated user interaction.

screenshot_nexus4_api21_classify_first screenshot_nexus4_api21_list

I have already released several beta versions to a handful of helpful testers, but now I’m nervous about it being publicly available. I’d rather not be bombarded with one-star reviews if there’s a bug I haven’t caught yet or if people don’t like something about the questions themselves.

There was a previous Galaxy Zoo android app that I think was in use until around 2012, but it stopped working with the latest version of the server and apparently wasn’t updated. It’s no longer available from the Google Play store. This new Galaxy Zoo android app is open source on github, so it should have more future. For instance, that would let other developers investigate problems if there are server changes, even if I don’t (though I would). Patches and pull requests are very welcome. There are even a few minor issues that I’d already like help with.

I’ve been working on this for the past few weeks, using it as a real-world exercise to learn about Android development, trying to do everything properly. I’ve enjoyed diving into something new, and it feels good to have gained the experience. However, Android development ultimately feels just as awkward as every other development environment I’ve ever worked with – no more nor less. I’ll write some follow-up blog entries about the technical parts of the adventure.

 

 

 

Android: Changing action icon colors with Android 5.0’s drawable tinting

Android 5.0 (Lollipop), which is API version 21, supports tinting of drawables. So now you don’t need to recreate your monochrome action icons (in multiple sizes) just to show them in a different color in your app bar (ActionBar or ToolBar).

Unfortunately, drawable tinting isn’t available via the AppCompat library. That makes it useless for now unless you want to target only Android 5.0 devices, but here’s how to use it anyway. One day Android 5.0 might be old enough.

This is another part of my adventure in changing the color of stuff in Android’s app bar, after already figuring out how to change the color of the app bar’s text and standard icons.

Regular Drawables

Normally you’d just have the regular multiple .png files for your action item like so:

res/drawable-hdpi/ic_action_something.png
res/drawable-mdpi/ic_action_something.png
res/drawable-xhdpi/ic_action_something.png
res/drawable-xxhdpi/ic_action_something.png
res/drawable-xxxhdpi/ic_action_something.png

You would use these in your layout XML like so:

<item
    android:id="@+id/option_menu_item_something"
    android:icon="@drawable/ic_action_something"

Or you would use them in code like this:

menutem.setIcon(android.R.drawable.ic_action_something);

Tinted Drawables

You can now add a tinted version of the existing drawable like so:

res/drawable/ic_action_something_tinted.xml

With XML contents such as this:

<?xml version="1.0" encoding="utf-8"?>
<bitmap
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:src="@drawable/ic_action_something"
    android:tint="@color/color_action_icons_tint"/>

That color_action_icons_tint could be hardcoded, but you’ll want to use the same one repeatedly via a color resource, like so:

res/values/color.xml

With XML contents like this:

<?xml version="1.0" encoding="utf-8"?>
<resources>
    <color name="color_action_icons_tint">#5a6518</color>
</resources>

You can then refer to that tinted drawable just like any other drawable,

in your layout XML like so:

<item
    android:id="@+id/option_menu_item_something"
    android:icon="@drawable/ic_action_something_tined"

or in code like this:

menutem.setIcon(android.R.drawable.ic_action_something_tinted);

Android: Changing the Toolbar’s text color and overflow icon color

Light on Dark and Dark on Light.

Android’s has normal (dark) and light themes, though it’s actually the light themes which are normally show in examples of the new Material design.

The light theme expects your App Bar1 (Toolbar or ActionBar) to have a light background color, so it gives you a dark title and dark overflow menu icon (the three vertical dots):

screenshot_toolbar_light_theme_default_cropped

The dark theme expects your App Bar to have a dark background color, so it gives you a white title and white overflow menu icon:

screenshot_toolbar_dark_theme_default_cropped

This is true of both the Holo themes and the new Material themes.

If you want to use the light theme but want your App Bar to have a dark background, or use a dark theme and want your toolbar to have a light background, things get awkward. Arguably this might be unwise design anyway, but there’s nothing in the Material design guidelines advising against it.

It’s fairly easy to change the ActionBar‘s text color, but changing the color of its overflow icon is harder. It seems normal to provide a whole new overflow icon for your app, replacing the standard one, just to get the right color.

Android’s new Toolbar, which replaces ActionBar (with some awkward code), makes it easier to change the title text color and the color of the menu overflow icon (and the Up/Back icon). So now I finally have dark text and icons on my light background in the dark theme:

screenshot_toolbar_dark_text_on_light_with_dark_theme_cropped

Toolbar theme and popupTheme

It took me ages to figure out how to do this, so hopefully the following explanation saves someone some time. I’d welcome any corrections.

My main theme derives from Theme.AppCompat (not Theme.AppCompat.Light), which is the dark Material theme for older devices, because I want most of the UI to be dark.

<style name="AppTheme" parent="AppTheme.Base" />
<!-- Base application theme.
     Defining this lets values-v21/styles.xml reuse it with changes. -->
</style>
<style name="AppTheme.Base" parent="Theme.AppCompat.NoActionBar">  
  <!-- colorPrimary is used, for instance, for the default ActionBar
       (but not Toolbar) background.
        We specify the same color for the toolbar background in 
        toolbar.xml.. -->
  <item name="colorPrimary">@color/color_primary</item>

  <!-- colorPrimaryDark is used for the status bar (with the
       battery, clock, etc). -->
  <item name="colorPrimaryDark">@color/color_primary_dark</item>

   <!-- colorAccent is used as the default value for
        colorControlActivated which is used to tint widgets. -->
  <item name="colorAccent">@color/color_accent</item>
</style>

(Update: I originally used Theme.AppCompat, manually setting windowActionBar to false, before learning of Theme.AppCompat.NoActionBar here, but I don’t notice any difference)

But that dark theme gives me light text and icons on the my light background of the App Bar:

screenshot_toolbar_standard_white_text_on_light_with_dark_theme_cropped

I want to use a light color for the toolbar background even while using the dark theme.  So I’ll need to make the text and icons on my toolbar dark instead of the default white from the light theme. Incidentally, the Material Design Color Palette page seems to agree with me, using dark title text on the Lime color I’ve chosen, but using white on almost all other colors.

So my Toolbar’s XML layout specifies a different theme (app:theme when using AppCompat, or android:theme when targetting SDK >=21), like so:

<android.support.v7.widget.Toolbar
  xmlns:android="http://schemas.android.com/apk/res/android"
  xmlns:app="http://schemas.android.com/apk/res-auto"
  android:id="@+id/toolbar"
  android:layout_width="match_parent"
  android:layout_height="wrap_content"
  android:background="@color/color_primary"
  app:theme="@style/GalaxyZooThemeToolbarDarkOverflow"
  app:popupTheme="@style/Theme.AppCompat.NoActionBar" />

That toolbar theme specifies a textColorPrimary and textColorSecondary to change the color of the title text and of the menu overflow button. You could just specify the standard Theme.AppCompat.Light.NoActionBar theme for the toolbar, to get the dark text and overflow icon,  but I wanted to derive from my own theme and make only small changes, because I have no idea what else might be affected.

<style name="GalaxyZooThemeToolbarDarkOverflow" parent="Theme.AppCompat.NoActionBar">
  <!-- android:textColorPrimary is the  color of the title text
       in the Toolbar, in the Theme.AppCompat theme:  -->
  <item name="android:textColorPrimary">@color/abc_primary_text_material_light</item>

  <!-- android:textColorPrimaryInverse is the  color of the title
       text in the Toolbar, in the Theme.AppCompat.Light theme:  -->
  <!-- <item name="android:textColorPrimaryInverse">@color/abc_primary_text_material_light</item> -->

  <!-- android:actionMenuTextColor is the color of the text of
        action (menu) items in the Toolbar, at least in the
        Theme.AppCompat theme.
        For some reason, they already get the textColorPrimary
        when running on API 21, but not on older versions of
        Android, so this is only necessary to support older
        Android versions.-->
        <item name="actionMenuTextColor">@color/abc_primary_text_material_light</item>
  <!-- android:textColorSecondary is the color of the menu
       overflow icon (three vertical dots) -->
  <item name="android:textColorSecondary">@color/abc_secondary_text_material_light</item>

  <!-- This would set the toolbar's background color,
        but setting this also changes the popup menu's background,
        even if we define popupTheme for our <Toolbar> -->
  <!-- <item name="android:background">@color/color_primary</item> -->
</style>

This gives me the dark text and icons in my App Bar while using the dark theme:

screenshot_toolbar_dark_text_on_light_with_dark_theme_cropped

That’s actually gray rather than black, and the white icons are really off-white, so you might want to use a custom color to have real black or white standard icons:

<item name="android:textColorSecondary">@color/my_black_icon_color</item>

Notice that the <Toolbar> also uses popupTheme (app:popuptheme when using AppCompat, or android:popupTheme when targetting SDK >=21). Without this, the overflow menu’s appearance is affected by the Toolbar’s style, leading to dark text on a dark background:

screenshot_toolbar_dark_text_on_light_with_dark_theme_with_menu_without_popuptheme_cropped

By specifying the Toolbar’s popupTheme, we can make its menu use our normal theme instead:

screenshot_toolbar_dark_text_on_light_with_dark_theme_with_menu_cropped

[1] “App Bar” seems to be the new official terminology. It’s used here and some other places I can’t find now.

Also see my entry about changing the colors of your own action icons with Android 5.0’s drawable tinting feature.

Looking at the Zooniverse code

Recently I’ve been looking over the Zooniverse citizen science project and its  source code on github, partly because it’s interesting as a user and partly because I thought writing an Android app for Galaxy Zoo would be a good learning exercise and something useful to open source.

So far my Android app can’t do more than show images, but I thought I’d write up some notes already. I hesitate to implement the Android App further because the classification decision tree is so tied up in the web site’s code, as I describe below.

Hopefully this won’t feel like an attack from a clueless outsider. I’m just a science enthusiast who happens to have spent years developing open source software in various communities. I’ve seen the same mistakes over and over again and I’ve seen how to make things better.

Zooniverse is organised by the Citizen Science Alliance (CSA). Incidentally, though the CSA has an organizational structure, I can’t see it’s actual legal form. Is it a foundation or a company? The zooniverse.org and citizensciencealliance.org domains are registered to Chris Lintott. Maybe it’s just a loose association of researchers with academic institutions assigning funds to work they care about, and maybe that’s normal in academia. The various Zooniverse employees actually seem to work for the member organisations such as the Adler Planetarium or the University of Oxford, though I guess some funding is for specific Zooniverse projects and some funding is for the overall Zooniverse development and hosting. That probably makes coordination difficult, like a big open source project.

Open Source

Since early 2013, the main Galaxy Zoo project has the code for it’s website on github along with the Zooniverse JavaScript library that it shares with other Zooniverse projects.

But most projects highlighted at zooniverse.org (such as Planet Four, Asteroid Zoo, Moon Zoo, or Solar StormWatch) don’t yet have their website’s code on github. This looks like a worrying trend. It doesn’t look like open sourcing has become the default as planned.

The zooniverse github repositories list is a poor overview, particularly because most of the respositories have no github description whatsover. Github should make them mandatory, even though they’d need to be updated later. Most projects don’t even have a basic description in their README.md either. Furthermore, I’d like to see a clear separation between front-ends, server-side code, and utilities (for processing data or for installing/maintaining servers.), maybe presented out on a github wiki page.

Also, they apparently have no plans to open source the server-side code (Ouroboros at api.zooniverse.org) that serves new subjects (such as galaxy images) to classify and receives classifications of these subjects. I think I’ve read that it’s a Ruby-On-Rails system. The client-side and server-side code is tightly bound, so this is a bit awkward. There is clearly room at least for some of the data structure and descriptions to be abstracted out and shared between the server, the client, and the analysis tools.

I can’t find any real documentation about the various Zooniverse code or APIs so there’s an awful chance of this blog post being the only introductory documentation that exists. I’d really welcome corrections and I’d gladly help. Neither can I find any place for public discussion of the software’s development, such as a mailing list. It’s hard for any open source project to mature without at least somewhere to discuss it.

Code

Arfon Smith at Zooniverse wrote some blog entries about the Zooniverse Domain Model, Tools and Technologies, and Server-side logic (my title).  (Arfon has since left Zooniverse to work at Github). I also found some useful  documentation at the zooniverse npm.org page. But I had to look at the code and the network traffic to get a more complete picture.

Languages, Libraries, Frameworks

The zooniverse front-end web-sites generally seem to be written in CoffeeScript (a mostly nicer language on top of JavaScript), using the Spine framework, which seems to make it easier to separate data and code into an MVC structure and to write code that deals asynchronously with the server while caching some data locally.

Some Coffeescript is written inline with the HTML, in Eco (.eco) files.

The CSS is written in the Stylus syntax, as expected by hem, which they use to bundle the code up for deployment.

I’m no JavaScript expert, but these seem like fairly wise choices.

Zooniverse web sites communicate with the Ouroboros server using RESTful GET (get subjects to classify) and POST (return a classification of a subject) HTTP requests, using JSON syntax. I think the JSON syntax is generated/parsed by the base Spine.Module.  I don’t know of any implementation-independent documentation for this web API.

The website code uses the Zooniverse library  as a helper to communicate with the server, for instance to login, to get subjects, and to submit classifications, and to support the lists of recent and favourite subjetct. The Zooniverse library is also implemented in Coffescript. Strangely, the generated JavaScript is also checked into git. The Api class seems to be most interesting..

Questions and Answers

Let’s look at the Galaxy-Zoo website though its maybe the most complicated. It allows users to classify images of galaxies. Those images may be from one of several astronomical surveys, such as Sloan or UKIDSS. Each survey has an ID and a Workflow ID listed in config.coffee (with much duplication of magic numbers). Each survey has a human-readable description and title in the list of English strings.

Each survey has a question/decision tree under app/lib, such as Galaxy-Zoo’s sloan_tree.coffee.  I wonder if this generated or duplicated from somewhere in the server software.  Why are the long question titles duplicated and used as IDs for leadsTo instead of using short codes? Is this tree validated somehow during the build?

These IDs, Workflow IDs, and decision trees are listed in the Subject class.

Question IDs

The zero-based index of the questions in the decision trees are used as IDs when submitting the classification. For instance, a submitted classification POST might contain the following parameter to show that, when classifying a Sloan image, for the “Is there any sign of a spiral arm pattern” question (sloan-3, and the 4th question asked of me) I answered “Spiral” (a-0):

classification[annotations][4][sloan-3]: "a-0"

These implicit IDs, such as sloan-3, are also used in the translations,  and throughout the code. For instance, to reuse some translation strings, to decide if there should be a talk-page link. That i18n hack in particular belongs as an association in the decision tree.

These implicit IDs are also used in the CSS (via the Stylus .styl files) to identify the relevant icons. The icons are in one workflow.png file in order to use the CSS Sprites technique for performance). The various sub-parts of that image are selected by CSS in common.styl.

This seems very fragile. It would be safer if the icon files were stored separately and then the combined file was generated, along with that .styl CSS. I guess that the icons are already stored separately somewhere, maybe as SVG. One parent file could define the decision tree and all the associated descriptions and icon files.

Ideally much of this structure would be described in configuration files separately from the code. That generalisation would allow more code reuse between Zooniverse projects and could allow reuse by other front-ends such as iPhone and Android apps. Presumably it’s this fragility that has caused Galaxy Zoo to withdraw its previous mobile apps. Even with such an improvement, you’d still need a proper release process to coordinate development of interdependent software.

Subject and Classification

Galaxy-Zoo has a Subject class, as does the Operation War Diaries project. These usually derive from the base Subject class in the zooniverse library  ,though the Snapshot Serengeti Subject class does not.

The Ouroboros server at at api.zooniverse.org provides a list of subjects for each group (a group is a survey, I think) to be classified via JSON. Here is the list of subjects for Galaxy Zoo’s Sloan survey. And here is the subjects list for Snapshot Serengeti with a simpler URI because there is only one group/survey.

The surveyId (for the group) for Galaxy Zoo is chosen randomly, though it’s currently hard-coded to always choose the Sloan survey. This JSON message contains the URLs of images for each  subject, in the list of “locations”. The Subject’s fetch() method calls the Api.get() method from the Zooniverse library and then creates Subjects for each item that the JSON message mentions.

The Subject’s constructor seems to take theJSON fragment to populate its member fields using the default Spine.Model’s AJAX functionality.

Galaxy-Zoo has a Classification class, and Snapshot Serengeti has one too. There doesn’t seem to be any common base Classification class in the zooniverse library. The Classification’s send() method calls the Classification’s toJSON() method before POSTING the message to the server via the Zooniverse library’s Api.post() method.

It’s hard to see any great commonality between the various projects.
For instance, a Galaxy Zoo classification is a series of answers to multiple-choice questions, with the questions being from a decision tree. I guess that Snapshot Serengeti’s animal classification is similar, though you can provide multiple sets of answers to the same questions about what animal it is and what it is doing, to identify multiple animals in each image. Moon Zoo and Planet Four ask you to draw specific shapes on an image and also classify each shape you’ve drawn, probably resulting in coordinates and identifications.

I wonder if the server-side code has any common model for these data structures or if the classifications just get fed into project-specific databases for project-specific data analysis later.