This briefly covers the goals, challenges, progress, and future work remaining for the buildout of a custom Firefox OS/B2G (Boot2Gecko) test-automation lab, running our Flame reference devices.

1. B2G Phone-Automation
Lab
Goals, Challenges, Accomplishments
July 31, 2014
Stephen Donner
sdonner@mozilla.com

2. Goals
● greater capacity:
○ keep up with b2g-inbound builds for UI + perf (esp.
the latter), as much as possible
○ multi-node mapping/support
● high reliability
● easier device
troubleshooting/administration

3. From this
...wait for it!

4. 1st Buildout

5. To this
...getting there; closer there

6. 2nd Buildout

7. And here we are
Bam!

9. And still going...

10. Challenges (1)
● balancing buildout with current-rig’s maintenance
● near-constant task (re)prioritization
● real carriers/networks (AT&T/T-Mobile), real problems:
○ spam calls/voicemail
○ Amber alerts
○ low signal/data-connection resets
■ slow throughput
■ DNS-lookup failures

11. Challenges (2)
● multi-node ADB mapping
○ solution had to work with both UI and perf tests
○ also required:
■ Jenkins-config changes (job/system)
■ physical reconfiguration
● active SIMs vs. inactive SIMs
○ reduce interference/cost/maintenance cost
● SecOps/NetOps requirements
○ new office, new problems (new policies)
■ port-by-port, host-by-host firewall pass-through
○ VPN/Jenkins (CI) access granted on a user-by-user basis

12. Challenges (3)
● RF (Radio Frequency)
○ Wi-Fi
■ ateam vs. Mozilla Mobile
● capacity + speed + reach
○ FM tuner interference
■ requires cabling magic
● ...and perhaps a low-power, local FM transmitter?
○ more devices, more interference?
● physical space/configuration
○ shelving space
○ power-strip capacity

13. Challenges (4)
● remote teams requested live Air Mozilla
streams of the phones
○ prototype works, but doesn’t immediately scale
bandwidth (# streams, codecs/encoders)
■ informs # of encoder boxes/# of channels
○ live (remote) demo
● “keeping the light on”
○ camera test failures due to low light
■ ...but intermittent

14. Currently*, we have:
● ~23 active Flame nodes (phones)
○ 12 dedicated to UI / 11 perf
○ 26 potentially
● attached to 13 active Mac Minis
* number of active nodes fluctuates due to
investigational/maintenance work, etc.

18. What’s Left?
● remainder of phones (20) to come online in
the next few weeks
● MozPool for managing devices, including:
○ taking devices offline
○ power measurement through ammeter
○ remote-reboot capability through power harness
● Puppetization for managing node
configurations

19. Comments? Questions?
● Mailing list: mozwebqa@mozilla.org
● Team page: https://quality.mozilla.
org/teams/web-qa/
● IRC: #mozwebqa on irc.mozilla.org