Massimo Carli speech slides at Turin DrodCon

1. Mail Online Android App
Massimo Carli
Head of Mobile

2. Who is Massimo Carli
• Born in Rovigo (Italy)
• Graduated at Padua
University
• Co-founder of Mokabyte
• http://www.mokabyte.it
• Teacher for Sun/Oracle
• Author of the first Italian book
about Android
• Head of Mobile for Mail
Online
• Twitter: massimocarli

3. What is Mail Online
• UK-born MailOnline (www.dailymail.co.uk) is the
world’s largest English-language newspaper
website
– over 59.7 million unique monthly visitors globally
• The third largest newspaper website in America
– over 21 million unique monthly U.S. visitors
(comScore, November 2013).
• Offices in London, NewYork, Los Angeles and
Sidney
• 700 articles and 8000 photos every day

4. Mail Online Android App
• One of Google’s “Best Apps of
2013” with over 1.9 million
downloads
• Over 15 channels of must-read
articles and pictures
• Over 600+ stories every day
• Offline mode
• Content related to location
• Comments
• Different display options

5. Architecture Goals
• Universal Application
– Both smartphone and tablet supported
– Easy evolution on TV
• Leverage on existing Android Components
– Avoid reinvent the wheel
– Performance improvements
• Create a common library
– Optimize tools for commons stuff
– Opportunity to create new things

6. Design Principles
• Avoid classic Antipattern
– Spaghetti code
– Golden Hammer
– Reinvent the wheel
• Design for change
• Open Close Principle
– Opened for the extension
– Closed for the changes
• Test first

7. Key Components
• Data Synchronization
– Article data should be persisted to permit offline
reading
– Download should NOT reduce application
responsiveness
• Image download and optimization
– Prevent classic Android OutOfMemory Error
• Content personalization
– See WHAT the user wants to see and WHEN

8. High Level Architecture
User Interface
Text Custom
Components
getData()
AsyncImageView
getMetadata()
getImage()
MOL ContentProvider
Image ContentProvider
SQLite
FS

9. Article Data Synchronization
SyncAdapter
DataManager
REST Method Library
Profile Settings
Profile Manager
Profile
Data
Internet
MOL
Content

10. Rest Method Library
• REST Services requests
– GET, PUT, POST and DELETE supported
• Delegate result management to a Deserializer<?>
• Http Error code management
• Use of HttpClient and UrlConnection based on the
Api Level
• Traffic measures
• Executed in the caller Thread
– @See Command Framework later

11. An Example: Get Request
RestCommand postCommand =
RestCommandBuilder.get(url)
.addParam(”param1”,”value 1”)
.addParam(”param2”,”value 2”)
.addHeader(”header1”,”value 1”)
.addHeader(”header2”,”value 2”)
.build();
RestCommandResult<LoginData> result =
RestExecutor.get().execute(ctx, command,
loginDataDeserializer);
LoginData loginData = result.getResult();

12. An Example: Post Request
RestCommand postCommand =
RestCommandBuilder.post(url)
.withStringDocument(inputJson)
.asJson()
.build();
RestCommandResult<LoginData> result =
RestExecutor.get().execute(ctx, command,
loginDataDeserializer);
LoginData loginData = result.getResult();

13. Deserializer<?>
// Abstraction of the object who read bytes from an
// InputStream and create, if possible, an instance of
// type E
// It’s “Android Context” dependent
public interface Deserialiser<E> {
E realise(InputStream inputStream, Context context)
throws IOException;
}

14. Deserializer for String
public final class StringHttpDeserialiser
implements HttpDeserialiser<String> {
--@Override
public String realise(InputStream inputStream, Context context)
throws IOException {
String encodedString = null;
if (TextUtils.isEmpty(mEncoding)) {
encodedString = IOUtils.toString(inputStream);
} else {
encodedString = IOUtils.toString(inputStream, mEncoding);
}
IOUtils.closeQuietly(inputStream);
return encodedString;
}
}

15. Deserializer for Bitmap
// Deserializer implementation that reads Bitmap data from the
// input stream
// Warning: Possible OutOfMemoryError!!!
public final class BitmapDeserialiser implements
HttpDeserialiser<Bitmap> {
--@Override
public Bitmap realise(InputStream inputStream, Context context)
throws IOException {
Bitmap image = BitmapFactory.decodeStream(inputStream);
return image;
}
}

16. Save Bitmap on File System
// Data are read from the inputstream and written directly to
// the File System. No extra memory used for Bitmap allocation
public final class FileDeserializer implements Deserializer<Void> {
@Override
public Void realise(InputStream inputStream, Context context) throws IOException {
// Prepare streams
try {
tmp = new FileOutputStream(tempDestinationFile);
flushedInputStream = new BufferedInputStream(inputStream);
fos = new BufferedOutputStream(tmp);
IOUtils.copy(flushedInputStream, fos);
} finally {
// Close everything
}
return null;
}
}

17. Read data as JSon Object
// Using StringDeserializer we create and return a JSONObject
public class JsonDeserializer implements Deserializer<JSONObject> {
@Override
public JSONObject realise(InputStream inputStream, Context context) throws IOException
{
final String jsonString = StringDeserializer.getDefault().realise(inputStream, context);
try {
final JSONObject jsonObject = new JSONObject(jsonString);
return jsonObject;
} catch (JSONException e) {
e.printStackTrace();
}
return null;
}
}

18. RestMethod GET example
// We create a simple GET request for a given URL an we use
// the RestExecutor to execute it. We get a RestCommandResult
// object with the data or the error informations
RestCommand getCommand = RestCommandBuilder.get(url).build();
RestCommandResult<String> result =
RestExecutor.get().execute(getContext(), getCommand,
int statusCode = result.getStatusCode();
if (HttpStatus.SC_OK == statusCode) {
String strResult = result.getResult();
} else {
Log.e(TAG_LOG, ”Error :” + result.getStatusMessage());
}

19. RestMethod POST example
// We send data with a POST and parse the result into a specific
// object of type LoginData.LoginResponseDeserializer which is
// a JSON deserializer specialisation
final LoginData.LoginResponseDeserializer deserializer =
new LoginData.LoginResponseDeserializer();
final RestCommand command =
RestCommandBuilder.post(loginUrlStr)
.addParam("email", username)
.addParam("password”,password)
.addParam("deviceId", deviceId)
.build();
final RestCommandResult<LoginData> result =
RestExecutor.get().execute(ctx, command, deserializer);

20. Bitmap Fetching with RestMethod
// We send data with a POST and parse the result into a specific
// object of type LoginData.LoginResponseDeserializer which is
// a JSON deserializer specialisation
BitmapDeserializer bitmapDeserializer =
BitmapDeserializer.get();
RestCommand command = RestCommandBuilder.get(url).build();
final RestCommandResult<Bitmap> result =
RestExecutor.get().execute(ctx, command, bitmapDeserializer );
// If ok
Bitmap bitmap = result.getResult();

21. The Decorator Design Pattern
http://en.wikipedia.org/wiki/Decorator_pattern

22. Measure Traffic
public class TrafficCounterDecorator<T> implements Deserializer<T> {
private Deserializer<? extends T> mDecoratee;
private long mDataCount;
public TrafficCounterDecorator(final Deserializer<? extends T> decoratee) {
this.mDecoratee = decoratee;
mDataCount = 0L;
}
public final long getDataCount() {
return mDataCount;
}
public final void reset() {
mDataCount = 0;
}
public final Deserializer<? extends T> getDecoratee() {
return mDecoratee;
}
}

23. Measure Traffic cont.
public T realise(final InputStream inputStream, final Context context) throws IOException {
return mDecoratee.realise(new InputStream() {
@Override
public int read() throws IOException {
mDataCount++;
return inputStream.read();
}
@Override
public int read(final byte[] buffer) throws IOException {
final int dataRead = super.read(buffer);
mDataCount += dataRead;
return dataRead;
}
@Override
public int read(final byte[] buffer, final int offset, final int length)
throws IOException {
final int dataRead = super.read(buffer, offset, length);
mDataCount += dataRead;
return dataRead;
}
}, context);
}

24. Android Command Library
• Framework we use to execute tasks in background
• It’s based on Android Service Component
– It stops when there’s anything to do
• May use Queue and priority Queue
• Callback and result Management
public interface Command {
void execute(Context ctx, Intent intent);
}

25. Android Command Library
http://en.wikipedia.org/wiki/Command_pattern

26. Android Command Library
// In a resource of type XML
<actions>
<action name="testCommand"
class="uk.co.mailonline…command.SimpleCommand"/>
</actions>
// In the AndroidManifest.xml
<service android:name="uk.co…service.CommandExecutorService”
android:exported="false">
<meta-data
android:name="uk.co…command.COMMAND_CONFIG"
android:resource="@xml/commands" />
</service>

27. A very simple Command
// A simple Command
public class SimpleCommand implements Command {
public static final String TAG_LOG =
SimpleCommand.class.getName();
@Override
public void execute(Context ctx, Intent intent) {
Log.i(TAG_LOG, "SimpleCommand executed!");
}
}

28. Executor
// We use the normal Executor to execute the testCommand
// without any parameters
public static class Executor {
public static void execute(Context ctx, String name,
Bundle inputData){}
public static void executeIfConnected(Context ctx, String name,
Bundle inputData){}
public static void executeAtLeastAfter(Context ctx,
CommandBag commandBag,
String name, Bundle inputData){}
}

29. Executor
// We can execute a simple Command given its name and
// input Bundle (null in this case)
Command.Executor.execute(this, "testCommand", Bundle.EMPTY);

30. Audit Executor
// We can execute a Command and get information on the
// result. This is a simple Command that takes a while and
// then completes
public void execute(Context ctx, Intent intent) {
int times = intent.getExtras().getInt(InputBuilder.TIME_EXTRA, 1);
String message = intent.getExtras()
.getString(InputBuilder.MESSAGE_EXTRA);
if (TextUtils.isEmpty(message)) {
message = DEFAULT_MESSAGE;
}
for (int i = 0; i< times ; i++) {
Log.i(TAG_LOG, message + "#" + i);
try{
Thread.sleep(500L);
}catch(InterruptedException e){}
}
}

31. Audit Executor
// We can execute a Command and get a feedback.
Bundle inputParams = InputParamsCommand.InputBuilder.create("Hello World!")
.withNumberOfTimes(10).build();
Command.AuditExecutor.execute(this, "simpleAuditCommand", inputParams, new
CommandListener() {
@Override
public void commandStarted(long commandId) {
// Do something into the UI thread when the command starts
mProgressBar.setVisibility(View.VISIBLE);
}
@Override
public void commandCompleted(long commandId, Bundle result) {
// Do something into the UI thread when the command ends
mProgressBar.setVisibility(View.GONE);
}
});

32. Audit Executor with Handler
// If we need it we can receive the callback into a different
// thread associated with a given Handler
public static class AuditExecutor {
public static long execute(final Context ctx, String name,
Bundle inputData, final CommandListener commandListener,
final boolean tracked){...}
public static long execute(final Context ctx, String name,
Bundle inputData, final Handler handler,
final boolean tracked){...}
}

33. Audit Executor with Handler
// If we need it we can receive the callback into a different
// thread associated with a given Handler
// Different WHAT are used
final android.os.Handler handler = new android.os.Handler() {
public void handleMessage(Message msg) {
switch (msg.what) {
case COMMAND_STARTED_WHAT: // Command starts
break;
case COMMAND_PROGRESS_WHAT: // Command progress
break;
case COMMAND_COMPLETED_WHAT: // Command completes
// Invoked when Command completes
}
}
};

34. Audit with BroadcastReceiver
• The same events are notifier using a
LocalBroadcastManager
– Useful to manage configuration changes during
Command execution
– Sometime better to use Fragment with
setRetainState(true)
• Implemented for completeness

35. Audit Executor with Result
// With an AuditCommand it’s possible to get the result of
// the Command
/**
* All the AuditCommand should be able to return it's result
* at the end of their execution
* @return The Bundle with the result of the command. Can be null.
*/
public Bundle getResult();

36. Audit Executor with Progress
// The AuditCommand interface extends the Command interface
// adding functionality for callback using a
// LocalBroadcastManager injected by the AuditExecutor
public interface AuditCommand extends Command {
public void setLocalBroadcastManager(LocalBroadcastManager
localBroadcastManager);
public void setRequestId(long requestId);
public Bundle getResult();
}

37. Audit Executor with Result
// Into the commandCompleted() callback method we can have the
// result of the Command. We can use the getResultValue()
// utility method to get that information
@Override
public void commandCompleted(long requestId, Bundle result) {
long value = ResultAuditCommand.getResultValue(result);
mProgressView.setText("Result: " + value);
mProgressBar.setVisibility(View.GONE);
}

38. Audit Executor with Progress
// To manage progressState we implemented the
// AbstractAuditCommand abstract class with the
// setProgressingState() utility method
public class ProgressAuditCommand extends AbstractAuditCommand {
--public void execute(Context ctx, Intent intent) {
// Read input paameters...
for (int i = 0; i< times ; i++) {
Log.i(TAG_LOG, message + "#" + i);
sendProgressingState(getRequestId(), new Bundle());
try{
Thread.sleep(800L);
}catch(InterruptedException e){}
}
}
}

39. Queue Executor
// Commands can be executed into a specific Queue defined into
// a XML resource file
// Different types of Queues for different types of data
<?xml version="1.0" encoding="UTF-8"?>
<queues>
<queue name="commandQueue"
class=“…queue.SynchronizedCommandQueue" />
<queue name="articleCommandQueue"
class=”…queue.SynchronizedCommandQueue" />
<queue name="imageCommandQueue"
class=”…queue.ImageCommandQueue" />
</queues>

40. Different Queues
// Commands can be executed into a specific Queue defined into
// a XML resource file
// Different types of Queues for different types of data
<?xml version="1.0" encoding="UTF-8"?>
<queues>
<queue name="commandQueue"
class=“…queue.SynchronizedCommandQueue" />
<queue name="articleCommandQueue"
class=”…queue.SynchronizedCommandQueue" />
<queue name="imageCommandQueue"
class=”…queue.ImageCommandQueue" />
</queues>

41. Queue Executor
// Commands can be executed into a specific Queue defined into
// a XML resource file
public static class QueueExecutor {
public static void execute(Context ctx, String queueName,
String commandName, Bundle inputData) {…}
}

42. ORM Android Library
• We created a library to manage ORM into Android
– An abstraction of ContentProvider
– A way to map Entities into DB Table or
ContentProvider
– Convention over configuration when possible
• Prebuilt query into XML documents
• Smart way to execute query

43. ORM Android Library
• We didn’t use it!!!
• All the entities were in memory
– Android Cursor implementation is a better solution
– Reinvented the Wheel
• Other ORM tools have the same problems
– We used Android specific APIs
• Is it useful abstract information into model
entities?
– Probably yes but only for better code style and for
OCP (Open Close Principle)

44. Images and AsyncImageView
setImageUrl()
Image Not Cached
Image Cached
DnlImageCommand
showImage()

45. Images and Memory
•
•
•
•
We don’t create Bitmap instance during download
Images are Cached only if needed
We resize images based on screen size
Use of inPurgeable flag for devices < Api Level 11
• If Bitmap memory needs to be reclaimed it is. If the
Bitmap needs to be reused it is re-encoded
• Is not ignored for Api Level >= 11
• Less memory -> more CPU -> less responsiveness

46. Two approaches for Article Details
• Unique Layout file with all the different sections
• Easier to implement
• Bad usage of memory. The user not always scroll to the bottom of the
news.
• Bad responsiveness when user swipes the ViewPager
• A ListView with different types of cells
• More verbose implementation
• Better usage of memory
• Better responsiveness when user swipes the ViewPager

47. Battery Usage
• Mail OnLine Android App needs to download a lot of
data
• 14 Channels with 50MB of data each during
synchronization
• Each article can have between 3 and 100 images
• Our readers want to read articles in offline mode
• We need to optimize the quantity of data based on
• User preferences
• Connection Type (more on WiFi, less on 3G)
• Our Command Executor needs to run only when
necessary
• When the queue is not empty

48. Tracking and Ads
• Our application is not free and needs Ads
• Tracking is important for
• User traffic statistics
• Ads targeting
• We also use a tool for Crash reporting
• Our crash rate is less that 0,03%
• All these third parties tools and SDKs make adds
things our app has to manage
• It’s very important to optimize everything

49. The ‘Probe’ Project
• To optimize the app we need to have information
about the devices states
• System information (memory, free space, etc.)
• Find solution for specific problems
• Send notification for specific article based on user
preferences
• We use Profile informations
• Send sounds, images, actions, etc
• Security is important

50. The ‘Probe’ Project
Internet
Node.js
GCM
Mongo
DB

51. Our Projects on GitHub
• Rest Method Library
• https://github.com/MailOnline/android_rest_framework
• Command Library
• https://github.com/MailOnline/android_command_framework
• Stay tuned!!

52. Question?