Présentation donnée à la Droidcon Paris 2015
Constructeur, opérateur, grosse entreprise avec une flotte d'objets connectés ou tout simplement curieux, Pierre-Olivier soulève le capot et vous explique comment une OTA fonctionne de A à Z :
- le recovery
- le paquet de mise à jour
- le service de mise à jour
- les chinois
- etc.
8. Générer un paquet de mise à jour
% . build/envsetup.sh && lunch tardis-eng
% mkdir dist_output
% make dist DIST_DIR=dist_output
% ./build/tools/releasetools/ota_from_target_files
dist_output/tardis-target_files.zip ota_update.zip
9. Ce que ça contient
META-INF
CERT.RSA
CERT.SF
com
android
metadata
otacert
google
android
update‐binary
updater‐script
MANIFEST.MF
10. mount("ext4", "EMMC", "/dev/block/platform/dw_mmc.0/by-‐name/system", “/system");
file_getprop("/system/build.prop", "ro.build.fingerprint") ==
"google/mantaray/manta:4.4.3/KTU84L/1148727:user/release-‐keys" ||
file_getprop("/system/build.prop", "ro.build.fingerprint") ==
"google/mantaray/manta:4.4.4/KTU84P/1227136:user/release-‐keys" ||
abort("Package expects build fingerprint of google/mantaray/manta:4.4.3/KTU84L/
1148727:user/release-‐keys or google/mantaray/manta:4.4.4/KTU84P/1227136:user/release-‐keys; this
device has " + getprop("ro.build.fingerprint") + ".");
getprop("ro.product.device") == "manta" || abort("This package is for "manta" devices; this
is a "" + getprop("ro.product.device") + "".");
ui_print("Verifying current system...");
11. Full vs. incremental
% ./build/tools/releasetools/ota_from_target_files
-i PREVIOUS-tardis-target_files.zip
dist_output/tardis-target_files.zip incremental_ota_update.zip
Hey hi everyone and thanks for coming to our talk.
Today we are going to talk about OTA.
But first, let me introduce ourselves.
Je ne sais pas quand votre téléphone Samsung sera mis à jour vers Android Guimauve et je m’en fous.
Je ne sais pas non plus comment on le root.
It is not an app update (though it can update apps), it is a system update
Means Over-the-air
It’s an update that you receive by the Internet
It doesn’t - and shouldn't - require any technical manipulations by an end-user
Means you don’t have to plug your Android
Means it costs less in logistics
We are not advertising any of theses companies, but this is an example of a few ones who are using or could benefit from a well designed OTA update system
You are a mobile carrier : you may be distributing a specific version of an android phone
You are a cable operator : your Set Top Box is running Android
You are a device maker
Phones & tablets
Other Android devices (like the Neptune watch)
(and basically any connected device)
You are a large corporation and you have custom devices for your employees (HydroQuébec, UPS, Amazon, …)
You are a an independant ROM maker
So if you recognize your logo (or one of your competitors), you can come and meet us at the end of this presentation ;)
Mettre à jour un terminal, c’est cool, mais c’est pas juste proposer une version plus récente de l’OS
C’est combler les failles de sécurité
C’est déployer de nouvelles fonctionnalités à vos utilisateurs
C’est aussi vous permettre de commercialiser votre terminal plus tôt : la durée de mise en marchée (time to market) est donc réduite
Vous libérez votre services de support et donc vous faites des économies
Dépendre d’une installation manuelle n’est pas une bonne solution :
La plupart des utilisateurs sont cons. Ou bourrés. Ou vieux. Ou sont des chats.
La plupart des utilisateurs manquent de confiance en eux et ne savent pas trop ce qu’ils font. Ils ont peur d’installer des virus, de briquer leur téléphone, etc.
Il est nécessaire de développer un outil de mise à jour (et donc pour plusieurs OS) et de le maintenir.
Qui se souvient de Samsung Kies ? Qui a déjà réussi à s’en servir ?!?
Procéder à une OTA plutôt qu’une MàJ manuelle, c’est donc économiser sur le support
Les utilisateurs ne voient pas la MàJ comme risquée et n’ont donc pas d’apréhension à l’appliquer.
Tout ça, c’est pareil avec les objets connectés et autres. Lorsqu’on gère une flotte de terminaux, on ne veut pas faire une mise à jour manuelle sur genre plus de 100k terminaux.
Quand une voiture a un problème, on fait un rappel de produit. Ça coûte des milliards de dollarsé
Qui veut faire ça avec son bidule-connecté quel qu’il soit ?
«challenges»
Les défis dans ce genre de projets ne sont pas de gros défis. Le truc, c’est plutôt qu’on n’a pas le droit à l’erreur parce que si on a merdé en “prod”, c’est terminé.
Dans les trucs que l’on doit donc gérer :
Corruption de fichier (hash, …)
Securité au travers du réseau
La sécurité sur le terminal:
battery check
network preferences (you don’t want to kill your users data plan)
Dealing with multiple device models
Dealing with multiple versions compatiblity
There are always users that are one or more versions late
Plongeons un peu dans les aspects techniques d’une OTA
Si vous n’avez jamais builder une image d’Android, ne vous inquiétez pas, ce n’est pas si complexe et c’est toujours cool à apprendre.
Fabriquer un paquet de mise à jour, ça se fait avec les outils qui sont inclus dans AOSP : il y a juste un script à rouler et il fait tout le travail
If you try to look inside an update package, you will always find the following tree.
The updater-script is highlited, because it is the script which will perform the update.
It is interpreted by the updater-binary, which holds the basic features needed during the update process
mount, unmount and format partitions
access properties
change files by applying patches
install applications
extract archives content
change rights, create, update, delete, link ... every operation you can perform on files
Aren't you happy not to have to build it by hand ;) ?
Est-on obliger de générer une mise à jour complète ?
Are we supposed to always provide a complete version of Android as an update ?
The answer is no :
En tant qu’éditeur de ROM, vous ne voulez pas forcément déployer de gros paquets, surtout lorsqu’il s’agit de corriger de petits bugs.
The tools provided in AOSP allow you to create smaller versions of the update package called incremental updates.
They are lighter in weight and will only make specific changes to the system.
Building these is pretty easy when you have the previous update package available.
Bon, on a vu que générer un paquet de mise à jour était finalement assez simple.
Maintenant on passe à l’installation.
Comment faire ?
Ça passe par la création d’une application “SystemUpdate”
C’est elle qui va s’occuper de contacter le serveur, vérifier la disponibilité d’une mise à jour, la télécharger et demander au système d’installer la mise à jour.
Dans notre cas, on utilise AlarmManager pour se réveiller à chaque semaine et contacter le serveur.
Here is how you can tie your OTA app to the settings menu.
This code shows how the activity is linked to the settings menu.
It is an extract from Android's source code.
But how is the activity selected by the Settings application ?
The Settings app requests the PackageManager for a specific activity.
This activity can be started with the intent filter you can see on this screen.
You need to include it in your Manifest to make sure the Settings app will find your activity.
Make also sure the priority is high, or the Google Play Services will be used instead of your application.
The last important step is to put your application in the /system/priv-app so it will gain the necessary permissions.
(parenthèse chinois)
This is a pretty usual process to download the update, though you have to ensure a few things :
use the appropriate connectivity, as mobile users will not like to see their data plan being killed
download in background,
make sure you can restart the download if the connection is lost
Here is an exemple using the download manager natively available in the OS, which supports most of the features needed :
background
wifi only
not disturbing the user
The only drawback is that you will be forced to download the update on a publicly accessible data partition.
(you can create you own version of the DL manager in order to avoid that)
(Parenthèse chinois)
Do not forget a few things such as permissions and a broadcast receiver which will be notified when the download finishes.
Make sure you perform some kind of integrity check after the download is complete :
double checking doesn’t cost much and can prevent your device from becoming a nice brick
storage may be defective and your file corrupted
Again, make sure this is well tested and nothing wrong may happen here !
Once the update was downloaded, you have to perform a few operations :
check the validity of the update package
reboot the system in order to install the update package => this is why your application needs to be privileged (ie. installed in /system/priv-app)
Easy to say … and easy to do !
There is no hidden API, though it is not the most documented one…
(parenthèse chinois)
Do you know that Android is not the only OS on your device ?
The other one is called the recovery.
In order to load each of the OS, the bootloader is used (see it as the BIOS of your device)
The bootloader is always loaded on boot, it is responsible for loading the OS which will be used on the device.
In our case, it will either load the recovery or Android.
There is no action required here, but it is still interesting to know what's going on when your device reboots :
the bootloader starts as usual
it checks the reason for which the device is restarted
as we need the device to install the update, recovery is the reason
recovery is started !
Once recovery is loaded, it will read commands from a set of files in the cache partition, and will start the update process :
Check (again !) the file signature
Decompress the update
Run the updater script, which will perform the specifics of the update
This screen is not the one the user will see during the update, although, it reflects the available functions in the Recovery.
The updater script will perform various operations, such as
mounting file system - even formatting them
installing new apks or binaries
setting new system properties
updating rights
…
Depending on the content of the update, the duration may change.
Make sure your updater is notified when the device boots up, it will help you perform a few basic tasks :
Delete update from storage, as you do not want the user to see its storage filled with update packaged
Call the back-end which will keep track of a successful update !
… and you are done with the update !
How can we do this ? It is pretty straightforward : we use a simple broadcast receiver to start our service on boot.
Now that we have seen how the update process works on the device; we will explain how we worked on the back-end.
You may want to host a simple back-end with a description file and the updates.
It may be useful if you are an independent developer, and provide your ROM to a few users.
Pretty straightforward, but it will not fit the needs of a larger distribution
If you are distributing ROMs to a various range of devices, you may have to go for a solution which will help you doing so.
Here are a few things we took in consideration while designing ours :
Make sure you can distribute large files to a large number of devices
Create a simple yet efficient API which will provide the devices with the correct updates
Allow the possibility of a multitude of devices
Handle multiple distribution branches
Give feedback (statistics) about installation and requests
Soooooo let’s talk about the challenges we are facing.
This is our device, it has its own characteristics.
Hourray, I sold lots of it !
And I start to sell new models, and different kinds of devices : phones, tablets, smartwatches, thermostats, smoke detectors, ...
you may want to grab some of dev, like the one you have for dev purpose, and update them over the air without affecting the rest of the users
Another use case: sometimes you don’t have the choice, your terminal are sent to your support team. Those guys also have the right to test your corrective maintenance easily.
Specific information makes it possible to create groups which will be provided with specific versions of the ROM while using the same devices :
developers, who'd like to have the latest version of an OS in order to test their applications,
beta users, who will provide feedback to your company,
enterprise users, who may have specific use cases with dedicated ROMs,
support centers, who will need to be trained before the official release,
… basically any kind of group you could think of !
In order to make sure we can identify a device specifically, we send as many information as possible to our back-end : model, IMEI, MAC address, serial number, ROM version, etc.
All this data allows us to determine if the device has to be updated, and which version is to be used.
=> Passe la main à Simon
Time goes by, users will update - or not - their devices, which will lead to fragmentation.
Compatibility awareness is a pain in the ass:
A version you distribute can be compatible with just one other version or with several version. You don’t really want to force a non updated tablet through all incremental versions you manage until the very last one. Maybe just serving the last full compatible version does a better job.
Here is a screen capture of the main dashboard of our solution running, which reflects everything we said before.
Because of the fine granularity we use, we can quickly and easily picture the state of a fleet of devices.
Our main sources of information are the following :
Android's source code : when you are working on a ROM, just keep reading AOSP's code and you'll find what you want !
StackOverflow and Google : sadly / luckily, we were not the first people to look into this, so you may find a nice amount of information about OTA using the usual tools
Reverse Engineering the Play Services : have a look at what Google does !
As we now have a nice way to update Android-based devices, we work on updating other devices : wearables, camera, beacons, home automation, ...
[Questions]
Thank you for your attention, if you want us to help you providing updates to your devices, feel free to meet us at our booth after this talk.
Enjoy the rest of the Droidcon, and we look forward to drink a beer with you after the event !