Session ID: SFO17-506 Session Name: AOSP Devboard update and recent/future pain points - SFO17-506 Speaker: Track: LMG ★ Session Summary ★ Cover an outline of the AOSP Devboard effort with HiKey and now HiKey960, work on upstreaming, generic HAL implementations for Linux common hardware interfaces, graphics efforts, along with issues coming from the transition to Treble, and from upstream kernel work colliding with legacy Android kernel interfaces. This presentation will be similar to one given earlier at Linux Plumbers but will go a bit deeper on issues and will provide some context to the discussion had at Linux Plumbers. --------------------------------------------------- ★ Resources ★ Event Page: http://connect.linaro.org/resource/sfo17/sfo17-506/ Presentation: Video: --------------------------------------------------- ★ Event Details ★ Linaro Connect San Francisco 2017 (SFO17) 25-29 September 2017 Hyatt Regency San Francisco Airport --------------------------------------------------- Keyword: http://www.linaro.org http://connect.linaro.org --------------------------------------------------- Follow us on Social Media https://www.facebook.com/LinaroOrg https://twitter.com/linaroorg https://www.youtube.com/user/linaroorg?sub_confirmation=1 https://www.linkedin.com/company/1026961

1. John Stultz <john.stultz@linaro.org>
LMG Kernel Team
AOSP Devboard Update &
Recent/Future Pain Points

2. ENGINEERS AND DEVICES
WORKING TOGETHER
Now there are two: https://source.android.com/source/devices
HiKey HiKey960

3. ENGINEERS AND DEVICES
WORKING TOGETHER
Hardware overview
HiKey
● HiSilicon Kirin 620 SoC
● 64bit ARM A53
● 8 cores
● 1 or 2 GB RAM
● 4 or 8 GB eMMC
● Mali 450 (Utgard) Graphics
● USB2 Type A & USB-OTG
● $100-120
HiKey960
● HiSilicon Kirin 960 SoC
● 64bit ARM 4xA53 + 4xA73
● 8 core big.LITTLE design
● 3 GB RAM
● 32 GB UFS Storage
● Mali G71 (Bifrost) Graphics
● PCIe
● USB3 Type A & USB-C (USB2)
● 5GHz WiFi & BT 4.1
● $240

4. ENGINEERS AND DEVICES
WORKING TOGETHER
Remaining out-of-tree patches (4.14-rc)
HiKey
● Minor USB improvements
● ION ABI revert HACK
● Mali gpu
HiKey960
● regulators
● ufs
● mailbox
● cpufreq/thermal
● usb
● iommu
● drm driver
● dts
● Mali gpu
● ...

5. ENGINEERS AND DEVICES
WORKING TOGETHER
96Boards.org
Vendor neutral hardware standard for developer boards
Allows ecosystem of peripherals that can be shared

6. ENGINEERS AND DEVICES
WORKING TOGETHER
96Boards.org
Vendor neutral hardware standard for developer boards
Allows ecosystem of peripherals that can be shared

7. ENGINEERS AND DEVICES
WORKING TOGETHER
Neonkey
Neonkey is certified 96Boards mezzanine: https://source.android.com/source/devices#neonkey
● CPU: STM32F11CE
● NOR Flash: 512 KB
● SRAM: 128 KB
● Pressure sensor: BMP280
● ARM Hall sensor: MRMS501A
● Temp/Humidity sensor: SI7034-A10
● ALS/Proximity sensor: RPR-0521RS
● LED driver with 15LEDs: LP3943
● Accel/Gyro+Geomagnetic sensors:
○ BMI160+BMM150
● 4 GPIO-driver LEDs
● I2C expansion
● GPIO (2 lines) expansion
● JTAG connector
● 96Boards LS Expansion connector

8. Why this matters

9. ENGINEERS AND DEVICES
WORKING TOGETHER
v3.10 Released
June 2013
2015 Nexus devices
(Oct 2015)
v3.18 Released
Dec 2014
v4.4 Released
Jan 2016
New Phones, Old Kernels
2 years, 4 months
v4.9 Released
Dec 2016

10. ENGINEERS AND DEVICES
WORKING TOGETHER
v3.10 Released
June 2013
2015 Nexus devices
(Oct 2015)
v3.18 Released
Dec 2014
v4.4 Released
Jan 2016
New Phones, Old Kernels
2 years, 4 months
v4.9 Released
Dec 2016
2016 Pixel Phone
(Oct 2016)
1
year, 10
m
onths

11. ENGINEERS AND DEVICES
WORKING TOGETHER
v3.10 Released
June 2013
2015 Nexus devices
(Oct 2015)
v3.18 Released
Dec 2014
v4.4 Released
Jan 2016
New Phones, Old Kernels
2 years, 4 months
1 year, 3 months
v4.9 Released
Dec 2016
2016 Pixel Phone
(Oct 2016)
1
year, 10
m
onths
Early 2016 Flagship
(Mar 2016)
1 year, 4 months
Early 2017 Flagship
(April 2017)

12. ENGINEERS AND DEVICES
WORKING TOGETHER
v3.10 Released
June 2013
2015 Nexus devices
(Oct 2015)
v3.18 Released
Dec 2014
v4.4 Released
Jan 2016
A Year Ahead
2 years, 4 months
1 year, 3 months
v4.9 Released
Dec 2016
2016 Pixel Phone
(Oct 2016)
1
year, 10
m
onths
Early 2016 Flagship
(Mar 2016)
1 year, 4 months
Early 2017 Flagship
(April 2017)
HiKey on 4.4
(Apr 2016)

13. ENGINEERS AND DEVICES
WORKING TOGETHER
v3.10 Released
June 2013
2015 Nexus devices
(Oct 2015)
v3.18 Released
Dec 2014
v4.4 Released
Jan 2016
A Year Ahead
2 years, 4 months
1 year, 3 months
v4.9 Released
Dec 2016
2016 Pixel Phone
(Oct 2016)
1
year, 10
m
onths
Early 2016 Flagship
(Mar 2016)
1 year, 4 months
Early 2017 Flagship
(April 2017)
HiKey on 4.4
(Apr 2016)
HiKey on 4.9
(Feb 2017)

14. ENGINEERS AND DEVICES
WORKING TOGETHER
2015 Nexus devices
(Oct 2015)
Kernel/Device Release Latencies
2 years, 4 months
1 year, 10 months
2016 Pixel Phone
(Oct 2016)
Early 2016 Flagship
(Mar 2016)
1 year, 3 months
Early 2017 Flagship
(April 2017)
1 year, 4 months
2014 2015 2016 2017
HiKey on 4.4
(April 2016)
3m
2m
v3.10
(Jun 2013)
v3.18
(Dec 2014)
v4.4
(Jan 2016)
v4.9
(Dec 2016)
HiKey on 4.9
(Feb 2017)

15. ENGINEERS AND DEVICES
WORKING TOGETHER
SoC Android Trees (circa ~2015)

16. HiKey 4.9 Kernel Tree (~July)

17. HiKey 4.9 Kernel Tree (~July)

18. ENGINEERS AND DEVICES
WORKING TOGETHER
2015 Nexus devices
(Oct 2015)
Kernel/Device Release Latencies
2 years, 4 months
1 year, 10 months
2016 Pixel Phone
(Oct 2016)
Early 2016 Flagship
(Mar 2016)
1 year, 3 months
Early 2017 Flagship
(April 2017)
1 year, 4 months
2014 2015 2016 2017
HiKey on 4.4
(April 2016)
3m
2m
v3.10
(Jun 2013)
v3.18
(Dec 2014)
v4.4
(Jan 2016)
v4.9
(Dec 2016)
HiKey on 4.9
(Feb 2017)

19. ENGINEERS AND DEVICES
WORKING TOGETHER
2015 Nexus devices
(Oct 2015)
Kernel Release Latencies
2 years, 4 months
1 year, 10 months
2016 Pixel Phone
(Oct 2016)
Early 2016 Flagship
(Mar 2016)
1 year, 3 months
Early 2017 Flagship
(April 2017)
1 year, 4 months
2014 2015 2016 2017
HiKey on 4.4
(April 2016)
HiKey960 on 4.4
(April 2017)
3m
2m
v3.10
(Jun 2013)
v3.18
(Dec 2014)
v4.4
(Jan 2016)
v4.9
(Dec 2016)
HiKey on 4.9
(Feb 2017)
HiKey960 on 4.9
(Aug 2017)
1 year, 4 months
8 months

20. HiKey 4.9 Kernel Tree (after HiKey960)

21. Benefits

22. ENGINEERS
AND DEVICES
WORKING
TOGETHER
Benefits to Android Devs
● Inexpensive Android development board
● Amount of board-specific patches is small
● Helps development of new Kernel features
○ Android Verity
○ Hibernation
○ Sdcardfs
○ SELinux
● Validation of Kernel patches / -stable updates
○ Wide range of supported kernels
○ Example: 3.18, (4.1), 4.4, 4.9
● Development of Android runtime libraries
○ Wide range of supported kernels

23. ENGINEERS
AND DEVICES
WORKING
TOGETHER
Benefits to Android Devs (cont)
● Development of new Bootloader features
○ UEFI bootloader is provided in source
○ Boot to RAM
● Development and Support of new Hardware modules
○ Display boards
○ Camera modules
○ Sensor modules ( ex: Neonkey sensorhub)
○ Connectivity modules
● Testing
○ Manual
○ Automation

24. ENGINEERS AND DEVICES
WORKING TOGETHER
Upstream Testing
Have a AOSP target that can run mainline kernels.
● LKFT efforts
● Weekly manual testing every mainline -rc release w/ HiKey
● Manually testing mainline ~daily during merge window
○ Small enough patchset that rebase-bisection isn’t bad
Found and reported numerous regressions against upstream
(SELinux, drm, binder, ipv6 networking, etc)
Also reported a number of issues that AOSP needs to address to work with
upstream kernels (New SELinux policies, dma_buf fences, ION destaging ABI
break)

25. Recent & Future Efforts

26. ENGINEERS AND DEVICES
WORKING TOGETHER
Recent efforts
Bring-up and validation of android-4.9 kernel tree
HiKey960 migration to v4.9
Migration to Android O
Transition to Generic Linux Bluetooth HAL (hikey & hikey960)
Parameterized EAS PowerHAL to make it shared/reusable
Gralloc binderization & migration to opensource gralloc on hikey960
Migration to new mali driver, which works w/ dma-buf fence SYNC_FILE interface
In Progress:
libION/Gralloc support for old and new (4.12+) ION ABIs
Hibernation proof-of-concept support on HiKey
Cadence HIFI3 DSP enablement

27. ENGINEERS AND DEVICES
WORKING TOGETHER
Cadence HiFi DSP SDK for Android
• A development and demonstration platform for Audio and Speech Applications
• Target Applications
– Audio and speech codecs
– Voice pre-processing such as acoustic echo cancellation, noise reduction
– Multi-microphone processing
– Audio post processing including object audio processing and rendering
– Immersive audio, 3D audio, AR/VR
– Speech recognition
– Neural network based processing

28. ENGINEERS AND DEVICES
WORKING TOGETHER
List of Demonstrations
• App processor to HiFi 3 DSP IPC (Inter processor communication)
• PCM Playback to USB Speaker via HiFi 3 DSP
• PCM Record from USB Microphone via HiFi 3 DSP
• PCM Playback to HDMI Speaker via HiFi 3 DSP
• MP3 Decoder on HiFI 3 DSP
09-29-2017
AOSP
App processor Code Release
AOSP
DSP Code Release
11-10-2017
Cadence
HiFi DSP SDK Launch
TBD
Multi-Microphone support
TBD
Linaro Connect Demo
10-13-2017
Roadmap

29. ENGINEERS AND DEVICES
WORKING TOGETHER
Direction forward
Utilize HiKey as arm64 target for lots of upstream and LTS testing
HiKey960 upstreaming
android-4.14 bringup and validation
UEFI transition(hikey960), updates & improvements (A/B, ?)
drm_hwcomposer
Proving ground for upstream EAS
Improved treble implementation
HiKey960 accelerated video encoding/decoding integration
Upstream DT fragments/overlay solution

30. ENGINEERS AND DEVICES
WORKING TOGETHER
Generic Build
● Kconfig based
● Supports multiple devices/architectures from single build
system
● db410c, db820c, nexus7, qemu-x86_64/aarch64, raspi3...
● Need more community involvement
Work on open-graphics stack
(mesa3d, gbm_gralloc, drm_hwcomposer, libdrm)
Other Work

31. Pain Points

32. ENGINEERS AND DEVICES
WORKING TOGETHER
AOSP pain points
Devboards aren’t really mobile devices (no panel, touch, baseband, etc)
Devboards aren’t “product” level targets internally, so AOSP changes break them
fairly often.
Support Effort/Load (bugs/feature requests):
● HiKey: ~240 over 2 years
● HiKey960: ~ 60 over 0.5 years
● ~1 issue every 3 days
Mali blobs make us dependent on ARM for continued support of hardware
No real space for developing shared/common HALs (cross-project dependencies
avoided)
Very difficult process to add/update project trees

33. ENGINEERS AND DEVICES
WORKING TOGETHER
Community pain points
Support multiple older kernels (v3.18, v4.4, v4.9, v4.14)
Difficult to develop clean solutions that cross all those points.
Example: drm_hwcomposer HWC2 implementation needs changes in v4.10+
Still need something for v3.18, v4.4 kernels
Android Sync/dma-buf fence transition, ION ABI changes, etc have been difficult
Would be nice if upstream efforts provided better migration paths

34. ENGINEERS AND DEVICES
WORKING TOGETHER
Vendor pain points
Treble trouble
● Wifi / Bluetooth / gralloc all needed work
● Bluetooth was broken for 3 weeks during the transition
○ In part motivated the generic linux bluetooth HAL work
● Gralloc changes required changes into libGLES_mali.so
○ Required ARM to respin
We want to provide a good reference implementation, so feedback on that would
be good.

35. ENGINEERS AND DEVICES
WORKING TOGETHER
Things to watch out for...
ION ABI (4.12+) changes will affect lots of vendors with custom implementations
● We really need to solve this right, but so far low involvement from vendors in
community discussions!
● GET INVOLVED!
Start aligning with open graphics stack!
● Work with community to resolve issues that keep you on custom
implementations

36. ENGINEERS AND DEVICES
WORKING TOGETHER
Questions/Discussion
Price vs Power preference?
Upstreaming board support takes a frustratingly long time!
Multiple boards (and supporting multiple kernels) is hard to scale!
Still not getting to the point where we’re really pushing some of the advanced
Android functionality upstream (offloaded sensor collection, wifi/bt scanning, etc).
Standard Kernel interfaces for Android are becoming a smaller issue than
standard bootloader interfaces for Android (Reboot reason, dtb handling,
fastboot flashing, secure boot, A/B updates, etc). Some work going on in UEFI,
but with limited resources.

37. Thank You
#SFO17
Questions: John Stultz <john.stultz@linaro.org>
SFO17 keynotes and videos on: connect.linaro.org
For further information: www.linaro.org

38. Backup Slides

39. ENGINEERS AND DEVICES
WORKING TOGETHER
Hardware Architecture of Cadence HiFi 3 DSP in Hikey960
App Processor
Audio DSP (HiFi 3)
Hi3660 SoC
LPDDR4
Shared Memory
HiFi 3 DSP Core
(500 MHz)
DRAM
IRAM
D-Cache I-Cache
SoC
I/O

40. ENGINEERS AND DEVICES
WORKING TOGETHER
DSP Software Architecture of the Hikey960
Kernel
App Processor
HiFi 3 DSP
XAF
Xtensa Audio
framework
Audio HALUSB HAL
Host Side XAF
Audio Signal Processing component
DSP Side XAF
Your App SW
IPC Inter-processor
connect
(Shared Memory)
DSP Developers API
(enables direct control of
components)
HiFi
Driver
App Developers API
(abstracts DSP with few APIs)