Tag Archives: java

BigOQuiz: C++ Standard Library Algorithms

I’ve reworked bigoquiz.com, which I created a few weeks ago. to allow multiple quizzes instead of just the default quiz about the Big-O complexity of algorithms and data structures. For some quizzes it also now offers reversed questions, asking you for the question that matches the answer.

For instance, there is now a C++ Standard Algorithms quiz. You could, for instance, check that you know the descriptions of the Sort Operations functions. Or you could check that you know the names of those Sort Operations functions based on their descriptions (or as multiple-cdon’t always hoice). Maybe this is one way to get more familiar with the many standard C++ algorithms that we often don’t realize are available. I hope this helps me as I strive to use “no raw loops“.

As before, you can track your progress in each section of the quiz, and identify problem questions that you keep getting wrong.

screenshot_bigoquiz_cpp_std_algorithms_sections

I’ve also added a Design Patterns quiz, based on the patterns mentioned in the Head-First Design Patterns book.

The user profile page also now shows overall statistics for all the quizzes that you’ve tried.

screenshot_bigoquiz_userprofile

 

Big-O Quiz: An experiment with Google AppEngine

bigoquiz.com

screenshot_bigoquiz_webOver the last few weeks I’ve been working on bigoquiz.com. I wanted to learn the Big-O algorithmic complexity of the main algorithms and data structures by heart. So, of course, I first built a system to help me do that.

bigocheatsheet.com and bigoref.com already do a good job of listing the information, but bigoquiz.com lets you check that you really know each one, and shows you the ones you keep getting wrong.

Tech I learned along the way

I find it very hard to resist a project when I see that it would let me play with multiple new things. This one was a chance to learn about:

Details

I noticed that bigoref.com had some corrections and additions compared to bigcheatsheet.com. And I made some additions of my own. I’ve listed it all here for anybody who is interested:

For instance,  bigoref.com adds Graph Search algorithms such as Dijkstra’s and Bellman-Ford (missing from bigocheatsheet.com), under “Searching“. I instead listed the graph search algorithms in bigoquiz.com under “Graph Search”  and added more graph algorithms that I’d learned about recently in my Coursera course, though I’m tempted to create a completely different graph algorithms quiz: Kruskal’s Minimum Spanning Tree, Prim’s Minimum Spanning Tree, Floyd Warshall All Pairs Shortest Path, and Johnson’s All Pairs Shortest Path.

bigoref.com also adds linear and binary search of arrays (missing from bigocheatsheet.com). For bigoquiz.com I just split Array into sorted and unsorted in “Data Structure Operations”.

I found just one error on bigoquiz.com that was corrected on bigoref.com, though there might be others:

And I noticed that bigoref.com has some additional errors so I corrected those on bigoquiz.com and  filed github issues for bigoref:

Also:

  • bigocheatsheet.com has entries for Stack, but bigoref.com does not. I kept that in bigoquiz.com.
  • bigoref.com has entries for Ternary Search Tree, apparently a kind of Trie (Prefix Tree), but bigocheatsheet.com does not. I kept this in bigoquiz.com. Curiously, bigoref.com doesn’t have entries for regular Tries (Prefix Trees), or Suffix Trees. I added them to bigoquiz.com, though I’m not completely sure about the various best/worst/average time/space complexities.
  • bigoref.com uses the term “indexing” where bigocheatsheet.com uses “access”. I’ve stuck with “access” for bigoquiz.com.
  • bigoref.com splits array into Basic Array and Dynamic Array, not allowing insertion or deletion in a Basic Array. I don’t find the distinction useful, so I kept it as just Array in bigoquiz.com.

GWTP: Model/View/Presenter for GWT with GIN/GUICE dependency injection

I’ve building a new website project with GWT, so I thought I’d take the opportunity to try GWTP, which provides an MVP framework for GWT. I’ve previously used bare GWT to do much the same thing while developing OnlineGlom, but I hoped it could be easier. Both are really for structuring client-side UI and behaviour.

Once you’ve got your GWT application built, you can almost forget how difficult it was to create the basic structure. So I’ve taken the time to describe both the old (GWT) way and the new (GWTP) way of doing things to remind myself how much better life is with GWTP. I mostly just discuss how the activities/presenters and views interact in the two systems. You would need to learn a bit more to write a full application. If you are familiar with GWT, you might skip to the GWTP section.

You’ll see that, even with GWTP, the code is far from concise, but that’s hard to avoid completely with Java, particularly when trying to split code into Presenters and Views, and splitting implementation and interface. Overall, I really wish that GWTP was just part of GWT with the old GWT Activities deprecated. Then people could focus on this one right way of doing things, and making it even better.

For GWT, I refer to my old OnlineGlom project. For GWTP, I refer to the little murrayc-gwtp-appengine-example project I created recently.

GWT: Activities/Views/Places

GWT provides the Activities and Places framework, and suggests you create Views for your UI. That page has a good overview and explanation, but I’ll go over the parts here too.

Place

GWT’s Activities and Places framework is mixed up with GWT’s history support, which lets the different states of your application have URLs that can appear in browser bookmarks and history. That’s how we have a “history token”, which describes a Place.

For instance, for each main location in your app, you would have SomethingPlace, extending GWT’s Place. Here’s an example from OnlineGlom: DetailsPlace is for a page that shows the details from a database record.

The @Prefix(details) annotation on its Tokenizer tells GWT that this place can handle URLs that begin with/ #/details, such as http:://onlineglom.something.com/#/details?table=things?primaryKey=123. The Place class takes the place’s parameters in its constructor and stores them. Those parameters usually refer to the ?something=blah parameters in the URLs.

getPlace() constructs a Place object from a token (mostly a URL), and getToken() constructs a token from a Place object. There isn’t any convenience API to avoid the tedious splitting and concatenation of strings, though we created a couple of helper methods for this in OnlineGlom.

Activity

Each place in your app will have an Activity, whose constructor will take an instance of your derived Place class. For instance, in OnlineGlom, the DetailsActivity takes a DetailsPlace. The Activity can use the Place’s getter methods to discover what the URL’s parameters have specified.

Your app will derive an ActivityMapper that maps the places to the Activities. For instance, in OnlineGlom, our DataActivityMapper‘s getActivity() method instantiates a DetailsActivity when it gets a DetailsPlace.

You’ll pass your ActivityMapper when creating an ActivityManager. For instance, in OnlineGlom, we create an ActivityManager that takes our DataActivityMapper. You’ll then call setDisplay() on your ActivityManager, passing an AcceptsOneWidget that will be used to show your activitity’s View.

This is already a lot of boilerplate code for some basic functionality.

View

Each place has a View. You’ll have one view interface and at least one implementation of that interface. For instance, OnlineGlom has its DetailsView interface and its DetailsViewImpl class that implements that interface. Your Activity can call methods on the view to make it show appropriate UI for the Activity’s state.

GWT has no base View class or interface, though we created a base View class for OnlineGlom, which has a View.Presenter interface implemented by all the Activities. Each Activity can then call setPresenter(this) on the view, giving the view a way to respond back to the Activity, such as telling the Activity that the user has clicked on something that should take the user to a new location.

Your ClientFactoryImpl, which implements your ClientFactory interface, instantiates the view implementation based on the view interface. For instance, OnlineGlom’s ClientFactoryImpl creates a DetailsViewImpl when asked for a DetailsView.

Your Activity takes your ClientFactory and uses it to create its View. For instance, in OnlineGlom’s DetailsActivity constructor, which was itself called by the DetailsActivityManager that I mentioned earlier. The actual ClientFactoryImpl is instantiated by a call to GWT.create(ClientFactory.class), using the ClientFactory to ClientFactoryImpl mapping in a .gwt.xml file. For instance, in OnlineGlom.gwt.xml:

<!-- Use ClientFactoryImpl by default -->
<replace-with class="org.glom.web.client.ClientFactoryImpl">
  <when-type-is class="org.glom.web.client.ClientFactory" />
</replace-with>

In theory, you might want different implementations of your View interface for different devices. Maybe you’d have DetailsViewMobileImpl, for instance, but that’s not a technique that I’ve personally found useful. However, allowing multiple implementations of the View also allows testing of your Activity by mocking the View,  letting you test logic without having to test UI behaviour at the same time.

Again, you can see that you have to implement lots of repetitive boilerplate code and you might wonder why you need to bother.

GWTP: Presenters/Views/Places

GWTP uses Presenters rather than Activities, but they are much the same thing: code that tells your UI View what to do without telling the UI View how to do it. But GWTP requires less repetitive code, by using dependency injection with GIN (GUICE for GWT) to tie the pieces together.

If you are returning to Java after some time away then Dependency Injection is your Tirimasu. This Google I/O 2009: Big Modular Java with GUICE talk seems to be an excellent introduction to dependency injection in general, and dependency injection with GUICE.

For instance, this lets us add a parameter to a constructor, and as long as the constructor has the @Inject annotation, and as long as we’ve told GIN (or GUICE) what implementation types to use, GIN (or GUICE) can end up calling that constructor with an appropriate instance for the parameter.

Place

WIth GWTP, you don’t need to implement any actual Place classes. Instead, you just use the @NameToken annotation on your Presenter’s proxy (declared via the @ProxyStandard annotation). For instance, in murrayc-gwtp-appengine-example’s ThingPresenter:

@ProxyStandard
@NameToken(NameTokens.THING)
interface MyProxy extends ProxyPlace {
}

That NameTokens.THING is just a string constant that I’ve put together with the others.

Presenter

Your Presenter should derive from GWTP’s Presenter, which is a generic class that takes your View interface and your Proxy interface.

For instance in murrayc-gwtp-appengine-example’s ThingPresenter:

public class ThingPresenter
  extends Presenter<ThingPresenter.MyView, ThingPresenter.MyProxy>

The presenters’s prepareFromRequest() override uses simple getParameter() calls to get the URL’s parameters, without the need for a separate Place class and manual string parsing. For instance, in ThingPresenter’s prepareFromRequest():

@Override
public void prepareFromRequest(final PlaceRequest request) {
  super.prepareFromRequest(request);
  final String thingId =
    request.getParameter(NameTokens.THING_PARAM_THING_ID, null);

View

Your View should extend GWTP’s ViewImpl and implement your Presenter’s View interface. For instance:

public class ThingView
  extends ViewImpl
  implements ThingPresenter.MyView

However, you will very often want to derive instead from GWTP’s ViewWithUiHandlers, specifying a small UiHandlers interface you’ve created, so your View can notify its presenter about user interaction. For instance, in murrayc-gwtp-appengine-example’s ThingView:

public class ThingView
  extends ViewWithUiHandlers<ThingUserEditUiHandlers>
  implements ThingPresenter.MyView {

That ThingUserEditUiHandler should also be implemented by the presenter. For instance, in ThingPresenter:

public class ThingPresenter
  extends Presenter<ThingPresenter.MyView, ThingPresenter.MyProxy>
  implements ThingUserEditUiHandlers {

And your presenters’s View interface should extend HasUiHandlers. For instance, in ThingPresenter:

interface MyView
  extends View,
  HasUiHandlers<ThingUserEditUiHandlers> {

The View can then indirectly call a method on the presenter like so:

getUiHandlers().onThingChosen("blah");

However, you must first call your view’s setUIHandler() from your presenters constructor, like so:

getView().setUiHandlers(this);

Modules

Each Presenter/View should have a Module. For instance, murrayc-gwtp-appengine-example’s ThingModule links the presenter and view together like so:

public class ThingModule extends AbstractPresenterModule {
  @Override
  protected void configure() {
    bindPresenter(ThingPresenter.class,
      ThingPresenter.MyView.class,
      ThingView.class,
      ThingPresenter.MyProxy.class);
  }
}

These sub-modules would then be combined in one module for the whole application. For instance:

public class ApplicationModule extends AbstractPresenterModule
  @Override
  protected void configure() {
    install(new ThingModule());
    install(new UserProfileModule());

This is where you can control the dependency injection. By providing a different ApplicationModule or different sub-modules, you can cause different implementations to be instantiated. For instance, to create mock Views.

Events: Communication Between Presenters

You application might have several nested presenters on a page, so one presenter might need to respond to a change to another presenter. GWTP lets us do this by defining our own GwtEvent. For instance,  murrayc-gwtp-appengine-example’s ThingUserAddedEvent:

public class ThingUserAnswerAddedEvent
extends GwtEvent<ThingUserAnswerAddedEvent.EventHandler> {

One presenter may then fire that event, by calling its fire() method, providing any parameters needed by the event.

A presenter may handle the event by implementing your event’s EventHandler interface. For instance, in murrayc-gwtp-appengine-example’s UserHistoryRecentPresenter:

public class UserHistoryRecentPresenter
  extends PresenterWidget<UserHistoryRecentPresenter.MyView>
  implements ThingUserAnswerAddedEvent.EventHandler {

And then registering the presenter as a handler for the event, with GWTP’s addRegisteredHandler(). For instance in UserHistoryRecentPresenter’s constructor:

addRegisteredHandler(ThingUserAnswerAddedEvent.TYPE, this);

You must also use GWTP’s @ProxyEvent annotation (see this too) on the handler method:

@ProxyEvent
@Override
public void onThingUserAnswerAdded(final ThingUserAnswerAddedEvent event) {

 

Simple GWTP and Objectify example with Maven

I’ve been playing with GWTP (an MVP framework for GWT) and Google’s AppEngine (via Objectify). I prefer to use the Maven build system. So I created a simple small example that does this: murrayc-gwtp-appengine-example. Improvements are very welcome. I’d like to learn how the code should be improved.

In case it’s interesting to anyone, here’s why I wanted this little example to exist:

GWT

The official GWT website’s equivalent StockWatcher example/documentation does not demonstrate best practices. It:

There is official GWT sample code that uses maven and Objectify, but it’s all mixed up with other stuff and I wanted something much simpler.

There is also an official GWT MVP example from 2010 hidden away, but it’s not in git and I didn’t even find it until I started writing this.

Objectify

Objectify’s documentation seems pretty good now that I look at it again, though I struggled at first. I think Google (and StackExchange) kept taking me to example code that used one of the various older versions of the API.

Oddly, Objectify’s documentation suggests that you use objectify-gwt with GWT. but objectify-gwt doesn’t seem to have any documentation and I didn’t seem to need it myself.

GWTP

The official GWTP website’s documentation is rather scattered, with several simple typos that make its example code snippets inconsistent with each other. The documentation seems to have been restructured from some other source. For instance, several internal links don’t take you to what is apparently intended. Some internal links take you to the page itself.  Lots of the best documentation is spread across their blog posts, but not brought together properly. They do at least show use of maven in their beginner’s GWTP example. but that example code itself is not in version control as a real project. GWTP is a nice clean API that deserves better.

Of course, it is incredibly hard to describe source code in documentation and keep that documentation and source code up to date and in sync. It needs a system. It never works if you just put the source code inline where no compiler can touch it. Although it’s nice to show source code inline (ideally taken automatically from a real source file), you also need links to the real source code so people can see the latest version.

android-galaxyzoo: Fixing typical Android bugs

My Galaxy Zoo app for android has been on the Play store for several months now and seems to be generally well liked. So far it has around 800 users, increasing slowly and linearly, probably because it isn’t linked from the galaxyzoo.org website. But even the first hundred users were enough to show me several problems that I needed to fix. I’ve written about them here, with links to the commits that fixed them in my app, because other app developers will experience them too.

These were general Android development problems, not specific to android-galaxyzoo. I guess that this is the difference between someone who understands how the Android API should be used, and someone who has actually used it with real users. It shouldn’t be quite this difficult.

Work in the main thread (Strict Mode)

Your code should not perform long-running tasks, such as network or disk IO, in the main thread, because UIs should not be unresponsive. By enabling Strict Mode in your code, you can make this cause exceptions, turning a non-responsive app into a crashing app. It would be unwise to turn this on in your app’s release version.

However, some “power” users seem to have this on via the developer options. They probably experience instability in many apps. And Android 3.0 (Honeycomb) has some Strict Mode settings on by default. So you will get crash reports via the Google Play store if you ignore Strict Mode exceptions.

As I mentioned in my previous entry, AccountManager is documented as being safe for use on the Main UI Thread, but it’s not, so I needed to use the AccountManager via AsyncTask.

If you are using a ContentProvider, you should not try to store data in the ContentProvider in the main thread. I instead did that via AsyncTask.

Likewise, avoid calling BitmapFactory.decodeStream() in the main thread, though it’s easier to just use Picasso anyway.

Automatic cache deletion

Your app probably caches some data, such as images, in its cache directory, such as the getExternalCacheDir() directory (/storage/sdcard/Android/data/com.you.yourapp/cache). However, Android will delete those files whenever it needs the space, and it won’t explicitly tell your app that it’s happened. So you need to check that your cached files really exist, one by one. You can either check the files when you first try to use them (causing UI delay) or you can check in the background, maybe trying to re-download the files.

This confused my code completely and it was hard to identify the cause of the resulting bug because the cache deletion by the system was not triggered by any particular action in my app.

By the way, although the getExternalCacheDir() documentation says that you don’t need to request permission to use that (app’s own) directory since Android 4.4 (KitKat), that’s not true – you do need to request the permission on Android 5.0 (Lollipop).

IllegalStateException: Can not perform this action after onSaveInstanceState

There are things that you cannot do in between an activity’s state being saved and that activity being resumed. It’s hard to know when that is the case but if you get it wrong then your app will crash. Here’s a full explanation, though you’ll wish you didn’t have to bother with it.

For instance, you should delay any use of AlertDialog until the parent activity’s onResumeFragments, which I did like so.

Likewise, you should not commit fragment transactions until after your activity has resumed. Again, its best to cause the commit of the fragment transaction in the parent activity’s onResumeFragments() because there’s nowhere suitable to do that in the fragment. The code could be much simpler if there was just somewhere safe in the Fragment to do this.

I believe that this problem hits every Android app developer who ever had more than a handful of users. It feels like a failure in the Android API design and the fixes feel like workarounds. If there was any clear way to structure app code to avoid this from the start then that would be mentioned early in the developer documentation.

Mutiple onLoadFinished() calls

If you use a ContentProvider (as I think you should) then you’ll use a Cursor Loader to get its data, overriding onCreateLoader() and onLoadFinished() and calling getLoaderManaged().restartLoader().

However, you will notice that your onLoadFinished() is often called more than once, sometimes with older data, confusing your app. This can be avoided by calling getLoaderManager.destroyLoader() in your onLoadFinished. For instance. I’m fairly sure this is an Android bug – at the least it is a poorly documented and unforgiving aspect of the API.

ConcurrentModificationException

I had at least one crash report with a ConcurrentModificationException, suggesting that two threads were changing the same data at the same time though I thought I had designed my code to avoid that and I thought I had made the class mostly immutable. So to avoid any chance of sharing data between threads I did some defensive copying (and here). I haven’t seen the crash since.

Toolbar’s Up button doesn’t act like the Back button

Android has a standard Back button that most users understand. It generally takes the user to the previous screen (activity), and that previous screen will look like it last looked for them.  But Android also has the concept of an Up button which few users understand.

The Up button is usually at the left of the app’s top toolbar (appbar) and it usually looks like a Back button. But it’s not a Back button. It takes the user to the parent level of the app instead of stepping back through all the previous screens that the user has traveled through since they were last at the top-level. Of course, most users don’t have any concept of the app having a hierarchy of screens in addition to a history of sibling screens. I predict that the Up button idea will be removed from Android one day.

At a second-level activity (something opened from the top-level), a user can justifiably expect the Back button and Up button to have the same effect – take me to the previous (top-level) screen/activity – even if the user knows about the Back/Up difference. However, by default the Up button will start a new parent activity instead of going back to the previous instance of that activity (as Back would). Users experience this as a loss of data – for instance this bug and this bug.

The fix for this depends on the situation:

Duplicated child fragments

My app uses child fragments (fragments in a fragment), which Android needs us to add in code rather than in the layout XML file. Strangely, I sometimes saw duplicate fragments thought it was hard to reproduce it reliably. The answer was to  always use FragmentTransaction.replace() instead of add().

 

Galaxy Zoo Android app: New Surveys

The web-based Galaxy Zoo has switched to showing subjects from two new surveys, with new sets of questions for these surveys. So I’ve updated (see in github) the Android app too and the new version is now available in the play store. These new images are less clear, and the questions are a little harder to answer. Apparently some clearer images are on the way.

Unfortunately, though the data comes from the server, all of the decision trees, icons, help text, and example images need to be in the client – The web client and the Android app at the moment. I’ve written a small text document describing how to change the surveys used by the Galaxy Zoo Android app. It should be easier, but for now documented-and-awkward is better than not-documented-and-awkward.

Looking for Work

I’ve really enjoyed the past year or so of “sabbatical”, learning new skills, doing some hobby projects, and spending more time with my kids, but it’s time to look for a proper job again. I suspect I’ll do some freelancing for a while until I find something suitable.

I’m hoping to find something still in the Linux and open source world, maybe moving into Android, and maybe with some management. I enjoyed running Openismus so I hope I can continue doing something similar: managing multiple small teams of highly skilled software developers who work with open source code and collaborative open source methods, while still doing some software development myself. I’d like to continue working with people with such high standards.

I don’t expect this to be easy because:

  • I want to stay in Munich.
  • I’d really like part time or flexible (start early, leave early) hours so I can spend some of the afternoon with my kids.
  • Running Openismus gave me project management experience, but I’m not sure it’s given me enough to call myself a manager rather than a programmer, and I don’t want to stop being a programmer.
  • GTK+ (and gtkmm) experience is apparently no longer such a popular niche. Qt still is but I’m less enthusiastic about it.
  • C++ is still broadly popular, but I’d like to stay away from MS Windows development.
  • I’ve really enjoyed doing Java Android development recently, but I hesitate to call myself an expert Java developer, compared to my C++ skills. Then again, if most Java developers are as bad as most C++ developers, then I must be better than most.
  • Although I’m learning some Algorithm analysis theory at the moment for fun, it doesn’t greatly interest me and I’ve never needed it over the last 20 years. But it seems to have become a popular interview filter.
  • I’m still not a Kernel developer and I still don’t really want to be. I like creating libraries, tools, and user experiences.

Here’s my CV if you think I can be useful.

Processing for Kids

Liam (7) has been playing a little recently with Processing, mostly drawing shapes and moving them around. The Hello Processing interactive video tutorial is an excellent introduction, for kids too. Thanks to Jon Nordby for suggesting Processing.

Liam is gradually working through the Getting Started with Processing book, typing in example code and changing it as the book suggests. Previously he has used Scratch and he’s started using the Lego Mindstorms programming environment, which is surprisingly visually complicated. But Processing is a nice introduction to real text-based programming, where you must type everything perfectly correctly or the computer will complain with incomprehensible error messages.

So far this seems to be the closest modern-day equivalent to my childhood experiences of sitting down with a Sinclair ZX81, Spectrum, or BBC Micro and trying things out from a book on BASIC. The expectations are low so you can easily feel that you’ve achieved something significant.

IMG_20141219_124554

The Processing IDE is a very simple and obvious UI and a Processing hello-world can be just one line, without any platform initialization, without specifying anything about exactly where your output should appear:

line(15, 25, 70, 90);

By default there’s just one screen that you draw on, and all functions and types appear to be in a global namespace. So you can start making things happen without the distraction of boilerplate code and without figuring out where in that mess to put your own code. You don’t need to learn about objects, inheritance, or encapsulation, though of course you should later.

Writing an iPhone or Android app might seem more interesting to modern kids, but they’d have to wade through so much kibble just to get started, while always noticing how far they are from achieving anything like the existing apps they see.

Processing is actually Java. When you write code, that code then seems to be the contents of a method in a (derived?) PApplet class, though you don’t see that other than in some compiler error messages. The functions such as size(), stroke(), point(), ellipse(), color(), strokeWeight(), etc, are actually member methods of this class. You don’t need to import any Java classes to use this API.

Java is fairly forgiving, particularly for the simple examples that people will start with. And it  offers a nice route into object orientated programming without the lower-level pain of C or C++.

Instead of just writing a bunch of code to run and then stop, you can instead define (override) setup() and draw() functions that do what you’d expect. The draw() method can make use of member variables such as mouseX and mouseY (these are not parameters of draw()). Likewise, you can implement keyPressed() and make use of keyCode. So you get a simple introduction to making a program interactive by doing event-driven programming.

Processing is not perfect, and I think you’d feel the lack of a real API and IDE as soon as your project became more serious, but it’s a great introduction to real programming.

android-galaxyzoo: Superficial porting to Android 5.0 (Material design)

Here are some notes about my experience adapting android-galaxyzoo to Material design for Android 5.0 (Lollipop) though I only used the most superficial parts of Material design.

AppCompat v21

Android 5.0 (Lollipop) has a new UI theme and some new APIs. However, for the next few years, almost everyone will use the slightly awkward AppCompat v21 compatibility API instead to achieve most of the same behavior on older devices too. Chris Banes wrote up a nice overview of AppCompat v21, some of which I mention here again for completeness.

I’m using Gradle, as should you, so I added this to the dependencies block in my app/build.gradle file. You’ll want to use the latest version.

compile "com.android.support:appcompat-v7:21.0.3"

Theme

First, I switched from the dark Holo theme to the (AppCompat) dark Material theme by changing the parent theme in my styles.xml. See the Toolbar section below about the use of the “.NoActionBar” versions of these themes.

- <style name="AppTheme" parent="android:Theme.Holo">
+ <style name="AppTheme" parent="Theme.AppCompat.NoActionBar">

If you were using the light theme, that would be:

- <style name="AppTheme" parent="android:Theme.Holo.Light">
+ <style name="AppTheme" parent="Theme.AppCompat.Light.NoActionBar">

Note that we don’t use the android: prefix with the AppCompat theme, because the theme is being bundled directly into our app via the appcompatv21 library.

I then specified  the standard colorPrimary and colorAccent colors along with some more shenanigans to get the right text and icon colors in my toolbar.

I also used the TextAppearance_AppCompat_* widget styles instead of the regular textAppearance* style attributes, because it’s recommended in the Typography section of this official “Implementing Material Design in Your Android app” blog entry. However, I didn’t notice any difference in appearance, and I wonder why we wouldn’t just get the correct styles by just using the new overall theme.

I actually created a base style and two derived styles, to support Transitions – see below.

Toolbar

The new Toolbar widget replaces the ActionBar, though the documentation doesn’t actually say that yet. Generically, they are called the “App Bar” in the Material Design document. I’m not sure that I really got any benefit from using it because my App Bar doesn’t do anything special, but I wanted to use the latest API.

To use Toolbar instead of ActionBar,  you should derive from the .NoActionBar version of the theme, such as Theme.AppCompat.NoActionBar, though I used the regular Theme.AppCompat for a long time without noticing any difference.

Then you’ll want to add a Toolbar widget to the Layout XML files for every activity. I did that by creating a toolbar.xml file

<?xml version="1.0" encoding="utf-8"?>

<android.support.v7.widget.Toolbar
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:id="@+id/toolbar"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:background="@color/color_primary">
</android.support.v7.widget.Toolbar>

and then I just including that from all the activity layouts like so:

<include layout="@layout/toolbar" />

I also specified the app:theme and app:popupTheme to get the right text and icon colors in my toolbar.

To use this toolbar as the App Bar, you need to derive from ActionBarActivity and call its setSupportActionBar() method. I did that in a utility function, like so:

Toolbar toolbar = (Toolbar) activity.findViewById(R.id.toolbar);
activity.setSupportActionBar(toolbar);

RecyclerView

Android 5.0 adds the RecyclerView widget, available for older API levels via the support.recyclerview library. For instance, I added this to the dependencies block in my app/build.gradle file. You’ll want to use the latest version.

compile 'com.android.support:recyclerview-v7:21.0.0'

RecyclerView apparently replaces ListView and GridView, though the documentation doesn’t yet actually say that, adding confusion for new developers. I replaced my use of GridView with RecyclerView to get support for simple Material Design transitions.

Unfortunately, RecyclerView has no real cursor support, so it’s hard to use it to view data from a ContentProvider. This is particularly annoying because the API of ListView and GridView previously pushed us towards moving code into a ContentProvider. My own Cursor-based RecyclerView.Adapter is terribly inefficient and totally unsuitable for a large number of items.

There’s an awful lack of practical documentation or example code for RecyclerView, even for simple tasks like responding to a click. Until there’s an easier way, you’ll need some tedious boilerplate code to derive your RecyclerView.ViewHolder from View.OnClickListener and call setOnClickListener() on one of your child views.

Transitions

I wanted the typical move-and-scale transition that we see in Material Design apps, so that clicking on one of many items makes its image expand and move into the subsequent detail activity, and shrink back again when you go back.

To achieve this, I had to specify various settings in my theme, but only Android 5.0 (Lollipop) devices support these transition items, so I created a base theme in res/styles.xml, and derived from it:

<?xml version="1.0" encoding="utf-8"?>
<resources>

    <style name="AppTheme" parent="AppTheme.Base" />

    <style name="AppTheme.Base" parent="Theme.AppCompat.NoActionBar">
        <item name="colorPrimary">@color/color_primary</item>
        ...

I then created a styles.xml file just for API Level 21, by putting it in res/v21/styles.xml, where I derived from the base theme again:

<?xml version="1.0" encoding="utf-8"?>
<resources>
    <style name="AppTheme" parent="AppTheme.Base">
        <item name="android:windowContentTransitions">true</item>
        <item name="android:windowAllowEnterTransitionOverlap">true</item>
        <item name="android:windowAllowReturnTransitionOverlap">true</item>

        <!-- specify shared element transitions -->
        <item name="android:windowSharedElementEnterTransition">
            @transition/change_image_transform</item>
        <item name="android:windowSharedElementExitTransition">
            @transition/change_image_transform</item>
    </style>
</resources>

I then defined that change_image_transform transition in my res/transition/change_image_transform.xml file, like so:

<?xml version="1.0" encoding="utf-8"?>
<transitionSet>
    <changeBounds/>
    <changeImageTransform/>
</transitionSet>

To actually use this transition on images, I needed to specify the android:transitionName on the two ImageViews in the the layout XML files for the two Activities (Fragments in my case).

Mostly this was all voodoo which I put together gradually after finding clues scattered around the internet. I haven’t found a good official example that shows this.

Unfortunately, the transition doesn’t seem to work when the user presses the Up button on the toolbar instead of using the standard Back button, even though that’s indistinguishable from Back for most users in most activities.

Metrics and Keylines

I made a fair effort to adapt my margins and padding to fit in with the Material Design Metrics and Keylines, which wasn’t too hard.

Unfortunately, the standard Android Button’s appearance is just as usless for Material design as it was for the Holo theme. It has a fake margin around its inside edges, which is part of its background graphic rather than any adjustable margin or paddng property.

So, to make the button’s sides actually flush with other widgets, and to position them properly on the layout grid, I had to specify a custom background image or color. But then I lost the nice Material Design ripple effect. I hope someone knows how to do this properly.

Activity classes without Toolbar support

There are a few helpful derived activity classes, such as AccountAuthenticatorActivity and PreferenceActivity, but these haven’t been changed to derived from ActionBarActivity, so you can’t call setSupportActionBar() on them. They can’t be changed without breaking compatibility, so you’ll have to reimplement them in your code. It’s not a lot of code but it’s an unpleasant developer experience.

 

 

android-galaxyzoo: Network I/O and UI responsiveness

While developing the Galaxy Zoo app for Android, I experienced problems with UI responsiveness while the app was downloading from, or uploading to, the remote server. I don’t mean just the usual problems caused by doing network I/O or file system access in the UI thread – mistakes that can often be caught by using Strict Mode. I mean that buttons didn’t respond to presses if the app happened to be doing I/O in an AsyncTask.doInBackground().

I solved this problem, but it was an incredibly frustrating journey. These notes might help the next person. So many Android apps are just a fancy way to interact with a website’s data, caching it locally, but the regular Android API doesn’t make that easy enough.

My app uses a ContentProvider, because the Android API seems to point us towards doing so, as I found while creating my experimental Android Glom app. The ContentProvider usually runs in the same process (and thread) as the main UI, so I don’t think this architecture was a cause of the UI responsiveness problem.

Thread+ResponseHandler versus AsyncTask

My ContentProvider needed to communicate with the remote server via REST messages, downloading images from the URLs that it then discovered and caching those images locally as well as storing data in its SQLite database. At first, I used the Thread class directly, with a UriRequestTask class derived from Runnable, that used HttpClient and accepted a (HttpClient) ResponseHandler. This was based on the example code in O’Reilly’s Programming Android (2nd Edition) book, in its “A Content Provider as a Facade for a RESTful Web Service” chapter (which you can see some of here).

Later, before I found the UI responsiveness problem, I replaced this Thread and RequestTask idea with AsyncTask because that seemed more robust and made it easier to get a result back to the main thread. I didn’t understand why the “Programming Android” book didn’t use AsyncTask in its ContentProvider. Yet later, I found that Strict Mode didn’t allow that use of AsyncTask in a SyncAdapter’s non-UI thread on some older devices (API version <15), causing an ExceptionInInitializerError with “Can’t create handler inside thread that has not called Looper.prepare()”. Maybe the authors found the same problem.

HttpURLConnection instead of HttpClient

Incidentally, I later realized that the Android developers now discourage use of HttpClient, because their version of HttpClient is buggy and they can’t update it without breaking compatibility. So I changed from HttpClient to HttpURLConnection (and here and here). Luckily, this fixed the endless redirect problem I (like others) was having with my uploads.

Again, I felt led astray by the “Programming Android” book, but books get out of date very quickly and the Android documentation apparently gives us new clues only reluctantly. Hopefully the 3rd edition will improve this.

Avoiding the user

This is when I noticed the UI responsiveness problem. I tried using Thread directly in various ways instead of AsyncTasks, fearing my own changes had caused it, but it made no difference.

I tried all kinds of tricks to avoid doing any I/O, even in the AsyncTasks, when the user was likely to be using the UI, trying to wait until it seemed that the user was idle and then doing some quick I/O before getting out of the way again. But the user is unpredictable and would still try to press a button now and then exactly when my app was doing some sneaky I/O.

After lots of imprecise testing, I was fairly sure that the problem was worse when the app happened to be doing some filesystem I/O, even just writing to the SQLite database, but it was definitely caused too by doing network I/O of even small JSON files. We should expect disk I/O to be worse than network I/O, of course.

SyncAdapter

I wanted to learn about Android’s SyncAdapter framework anyway, so I spent a week figuring that out despite its awful hand-wavy documentation. Luckily, it fixed the UI responsiveness problems. I think it helped partly because the I/O was now in a separate process, though that doesn’t feel like it should be necessary. Also, the SyncAdapter framework can decide when best to let the SyncAdapter do its work, based on the whole system. Presumably it can then prevent unrelated processes from making the UI responsiveness even worse.

Unfortunately, using SyncAdapter does mean that I lose control over when I may download items in advance. For instance, I often find that a new install of the app doesn’t fill up its initial list of ten items until five or ten minutes after the first start. In the meantime, the app requests the items directly when necessary. I call ContentProvider.requestSync(), with SYNC_EXTRAS_MANUAL, when I’m sure there’s work for the SyncAdapter to do, but it just seems to take this as a suggestion that it’s free to ignore.

Authenticator and Account

To use the SyncAdapter, I had to create an Authenticator, for which I needed an Account type, even if you are accessing your server anonymously. This account type will show up in the Settings->Accounts UI. Presumably this is why some other apps show up in the Accounts Settings, even though they don’t need any login. Maybe they are doing it just so they can use SyncAdapter.

SyncAdapter versus SharedPreferences/PreferenceFragment

So I started by storing the login details via the AccountManager instead of via SharedPreferences. I later discovered that the app’s SharedPreferences (such as how many items to download) were unavailable to the SyncAdapter’s separate process. That is documented as “currently this class does not support use across multiple processes“. So I later put other preferences in the account too. Unfortunately, the standard settings/preferences UI only uses SharedPreferences, so I have hacky code to keep the SharedPreferences and the Account in sync – see my onSharedPreferencesChange() override in my PreferenceFragment, so the account’s UserData is changed when the standard settings UI is used, via my hacky copyPrefToAccount method.

AccountManager versus main/UI thread

I also later found that you can’t use the AccountManager API from the main UI thread even though the documentation says things like “It is safe to call this method from the main thread”. The Firefox on Android developers discovered this too (1, 2, 3). So I moved that code to an AsyncTask (for example). If you don’t then you’ll get StrictMode exceptions at least on API level 15 and 16 – maybe others. StrictMode seems to have been on for Android Honeycomb, and some users seem to enable StrictMode, so you will see this as crash reports on some devices.

Volley

I also wanted to try out the Volley library. This isn’t an official part of the Android API. For instance, there’s not even an official way to add it as a dependency – I used the Volley package that mcxiaoke helpfully published to maven central. But Google use Volley in some of their own apps and they recommend Volley in the Android documentation. The video linked from there is well worth watching.

When I rewrote my HttpURL Connection-using code with Volley it seemed to make my downloads noticeably faster, probably because it avoids doing I/O in too many threads simultaneously. This was an easy win.

I couldn’t use Volley for my uploads, unfortunately, because Volley uses HttpClient on older Android versions whose HttpUrlConnection wasn’t good enough yet, causing the old endless-redirect problem again. So I use HttpURLConnection directly for that.

Picasso

I also switched to using Picasso to fill ImageViews with bitmaps from our ContentProvider or directly from the internet, instead of my own AsyncTask. This simplified the code and gave me some extra caching, particularly simplifying my ListView/RecyclerView adaptor that previously had its own custom LruCache. For instance 1, 2, 3. Image loading just generally felt much faster everywhere that I used Picasso, though I don’t know exactly why.

Just filling an ImageView without blocking the UI seems like something unexciting that the Android API should already make easy.