Building a solid JSON REST client in Android using Volley, Gson and Otto

Questions like "How do you make an HTTP request on Android" or "How do you do async/background JSON requests on Android", are question that often arise on Stack Overflow and blogged about elsewhere. Often the answers contain somewhat low-level code who leave a lot undecided for the developer. Firing an AsyncTask for the request to allow the processing to happen off the UI thread is good, but what if the UI element who fired the request is destryed when the response arrives? And what about parsing the resulting JSON? What about caching and throttling? These are questions that will be addressed in this post. A complete working project will provided at the end.

A word on Volley

Volley is a library by Google introduced at Google IO 2013 which set out to make REST client implementation simpler and faster. Unfortunately Google hasn't seemed to embrace their own project to the max, as it still is undocumented and no references to it seems to be found on developer.android.com. That doesn't stop it from being a powerful and effective library - enough blog posts exist about how to get started using it. This post goes a step further by introducing Gson for the simplest Json parsing. I also focus on 100% guarantee of any response being processed, which can be a challenge when dealing with Android activity lifecycles.

While most people would start out using AsyncTask for the request processing, Volley uses a thread pool of 4 threads by default. This has several advantages.

1. You don't need to spin up a thread each time a new request is sent. It's already there, waiting to serve you. Improves performance.
   
2. A limit of 4 ensures you don't congest the Android operating system while still allowing you for a fair amount of parallellism
   

If you do need more than 4 threads, you may download the Volley source code and tweak it yourself. If you need a lot more than 4 threads, you probably have a use case more suited for libraries like Ion. You can still apply most of the techniques in this blog post to that framework though.

In addition to this, Volley provides optional caching of GET-requests, primarily intended for image caching. You might want to look into that depending on your needs.

Gson instead of JsonObject

On top of Volley I'll be using Gson and an extension to the Request class allowing for seamless Json-response-to-Pojo-mapping. If you don't know what Gson is, it's a framework by Google allowing for automatic parsing and mapping of Json strings to plain java objects. Example:

Json:

{
        "age" : 14,
        "name": "ralph"
    }
    

Java class:

public class Person {
        public int age;
        public String name;
    }
    

This example uses fields, but you can use Java-bean properties if you prefer that (I don't).

The code to parse the Json to the object in this case would be a one-liner:

String jsonString = "{\"age\" : 14,\"name\": \"ralph\"}";
    Person person = new Gson().fromJson(jsonString, Person.class); // assuming jsonString contains the json above

Combining the power of Gson with Volley, we can make a GsonRequest-class which allows us to get a callback with the strongly typed object directly - never dealing with the intermediate parsing:

public class GsonRequest<T> extends Request<T> {
    private final Gson gson = new Gson();
    private final Class<T> clazz;
    private final Map<String, String> headers;
    private final Listener<T> listener;
    
    public GsonRequest(String url, Class<T> classType, Map<String, String> headers,
                       Listener<T> listener, ErrorListener errorListener) {
        super(Method.GET, url, errorListener);
        this.clazz = classType;
        this.headers = headers;
        this.listener = listener;
    }
    
    public GsonRequest(String url, Class<T> classType,
                       Listener<T> listener, ErrorListener errorListener) {
        super(Method.GET, url, errorListener);
        headers = null;
        this.clazz = classType;
        this.listener = listener;
    }

    @Override
    public Map<String, String> getHeaders() throws AuthFailureError {
        return headers != null ? headers : super.getHeaders();
    }

    @Override
    protected void deliverResponse(T response) {
        listener.onResponse(response);
    }

    @Override
    protected Response<T> parseNetworkResponse(NetworkResponse response) {
        try {
            String json = new String(response.data, HttpHeaderParser.parseCharset(response.headers));
            T parsedObject = gson.fromJson(json, clazz);
            return Response.success(parsedObject, HttpHeaderParser.parseCacheHeaders(response)); 
        } catch (UnsupportedEncodingException e) {
            return Response.error(new ParseError(e));
        } catch (JsonSyntaxException e) {
            return Response.error(new ParseError(e));
        }
    }
}

Otto and the issue with device rotation and destroyed activities

I'm going to outline the following scenario of a poker game app for Android where a callback approach is uses for requests:

1. User opens the activity where he'll be presented with the cards dealt to him
   
2. The Activity fires the request to fetch the cards
   
3. The user realizes this view is best presented in landscape mode and rotates the device
   
4. Android kills the Activity which "owns" the request and creates a new one
   
5. The request completes and calls back to the now dead activity.
   
6. The dead activity is either smart enough to realize it's dead, so it discards the request, or it tries to process it and the application crashes when tryin to updated a dead UI.
   

This may sound as a far-fetched scenario or a rare edge case, but it is infact a real-life scenario I regularly encountered in production for a game I made.

The solution is based on introducing an Event Bus, also known as the Pub/Sub-Pattern. I've grown fond of the Otto library from Square Inc, which does a fine job at that. Instead of having callbacks to the Activity, we rely on a dispatcher to publish the response on the event bus once any response arrives. Whether or not it is the original or a new instance of the Activity then listening, is irrelevant.

Illustration of timeline when the user rotates the device before the response arrives

Now a second challenge arise because there is a small gap between the time where the old activity is killed and the new is created. Context switches might occur here, responses may arrive and be lost. To amend for this I have in addition made a class called OttoGsonResponseBuffer, which will receive and buffer any requsts arriving in this gap. This way we ensure that no response is ever lost when the user rotates the device right after a request is sent.

The response buffer handles responses arriving in the time gap where no activity lives

The way this functions is that Otto supports producing of events through the @Produce attribute, which will cause the response data to be resubmitted to the event bus once a new listener for that type of event subscribes.

A word of warning here: Be careful when you use this buffer with regards to memory use. There is no mechanism of timeout or automatic flushing of the buffer as it is implemented now. Do not activate the buffer when user presses the back button (or flush it after he does), only when rotating the device.

Source code

A complete working example and a proof of concept can be found at my OttoVolleyDoneRight github repo. It may also be used as a library by following the simple steps describe at the bottom of the README there.

When launching the example you are presented with a simple button labeled "HTTP Request", which will send off a request to http://httpbin.org/delay/1. This service always delays one second before responding, which allows you to test rotating the device while waiting. You can up the number 1 to whatever you like if 1 second is too short. Notice how the request is processed in the new activity instance after the device is done rotating and the response arrives.

Going further

The example provided here is a good start, but there are some properties you should be aware of which might not scale well or should be done different in your application, depending on your needs:

1. There is just one pair of HTTP resonse message-bus class defined: VolleyRequestSuccess/Failed. This means that you must check the type of response in the onHttpResponseReceived-methods of your listener classes. Due to type erasure in Java, you can only have one listener method in each class as it is now, and it will receive any kind of response.
   
2. The OttoGsonResponseBuffer doesn't autoflush. Potential memory leak. Be careful with where you place the
   ResponseBuffer.startSaving()-method call.
   

In any case I believe this pattern provides a solid foundation for a JSON-based HTTP REST client for Android, and you'll have an application with fewer potential bugs than if you follow most other guides I've read so far online. Feel free to argue against me or come with suggestions for improvements in the comment below.