Codemash 2012 presentation

1. CI in the Mobile World
Godfrey Nolan
RIIS LLC
godfrey@riis.com

2. Agenda
• What is CI
• Benefits of CI in Mobile World
• CI Configuration
• Android
• iPhone
• Still a work in progress
• Perfecto Mobile
• Test Flight App
• Executable Requirements

3. Spam #1
• RIIS LLC
• Based in Southfield, MI
• Mobile Development
• Fandango
• DTE Outage Maps
• BroadSoft Front Office Assistant
• Other Clients
• BondDesk
• Comerica
• BCBSM

4. Spam #2

5. What is CI?
• Automate everything
• Soup to nuts
• Build, test and deploy
• Build as often as possible
• Build early, build often
• Typically on every checkin
• Alternatively nightly builds
• Practice, practice, practice

6. What is CI?
• Automate ….
• Builds
• Unit testing
• Functional testing.
• Mobile Device Testing
• Code Coverage
• Deployment

7. Benefits of CI
• Fewer Errors
• iPhone vs. Android story
• Faster Feedback
• Repeatable and Reliable
• No more manual testing
• Cheaper
• Built in regression testing

8. Benefits of CI in a Mobile World
• Less complex
• Mobile in its infancy
• Typically smaller projects and teams
• Lots of CI mobile plugins available
• Easy deployment options
• Testflight App (iPhone)
• Email apk (Android)
• Too many Android devices to manually test
• Automated Device testing is the holy grail

9. Mobile Device Testing

10. Goal

11. Goal

12. It’s all about your perspective

13. CI Configuration
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

14. CI Server
• Jenkins is de-facto industry standard
• Currently the best choice
• Number of plugins
• Ease of use
• Good for both Android and iPhone
apps.

15. CI Server
• Download from http://jenkins-ci.org/.
• "java -jar jenkins.war" from cmd line
• load Jenkins in browser at
http://localhost:8080
• RIP Hudson

16. CI Server

17. CI Server
• Under Manage Jenkins, Configure System.
• Add ant, JDK, and android SDK paths
• Under Manage Jenkins, Plugin Manager.
• Install XCode, GitHub and TestFlight

20. Requirements for CI
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

21. Source Code Repository
• Provides
• Safety
• Version Control etc.
• Subversion (SVN) or GitHub
• SVN for the Android project
• GitHub for iPhone project

22. Android and iPhone projects

23. SVN
• Create a new project on your SVN host

24. SVN
• Using tortoise svn simply right click and say import.
• Note that using the standard trunk tags and
branches of svn will force you to specify your
build.xml location in Jenkins because it will not be
in the project root.

25. GitHub
• Download Git from github.com
• Create a new ssh key by entering the
following command
ssh-keygen –t rsa –C “your-email”
• Enter a passphrase when prompted.
• Click on GitHub.com account settings, SSH
public keys, and click add a SSH Key.
• Copy the contents of the id_rsa.pub and
hit add key.

27. GitHub Cont.
• To create a repository
• Click create a repository on github.com
• Fill in the requested fields.
• git init in source code dir
• git add (file(s) to be added)
• git commit –m “message”
• git push to push to GitHub

28. Requirements for CI
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

29. Creating a Android Test Project
• Create a new project AndroidCIDemo and select
free style project.

30. Configuring a Android Test
Project
• Enter SVN details
• Jenkins will alert you if it cannot access the URL
• Be sure to enter credentials correctly

31. Configuring a Android Test Project
Cont.
• Under the Build tab
• Add commands to build/run the project.
• Click on Add build step and select Execute shell.
• Run android update project must be run in order to create the
local.properties file.
[path_to_android_sdk]/tools/android update project –p
./[path_to_build.xml]
• If android tools is already on your path and the build.xml file is in
root than the command would be:
Android update project –p ./
• Add another build step Invoke Ant and add the target debug.
• If build.xml file not in root hit advanced and specify its location in
the build file field.

33. Build
Now build the project and hopefully it runs with no errors.

34. Requirements for CI
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

35. Unit Tests
• For unit tests in Android use built in JUnit framework.

36. Functional Tests
• For Android Functional tests use Robotium
• To implement Robotium all you have to do is add it to
your test projects build path.
• GUI based, i.e. brittle

37. Modifications to the Android Project
• Both Unit and Functional tests are run in the same way.
• To run android tests run an emulator during the build.
• Under build environment choose Run an Android emulator during build.
If you have already created an emulator using the android avd than you
can choose to run that here, otherwise choose run emulator with
properties.
• Specify the OS version, Screen
density and Screen resolution
• Uncheck the Show emulator
window box under Common
emulator options to run headless.

38. Modifications Cont.
• Add another Execute shell command:
[path_to_android_sdk]/tools/android update test-project
–p ./[path_to_test_project] –m ./[path_to_main_project].
• And change the ant target to debug install test (don’t
forget to change the build file if its not in root).

39. Requirements for CI
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

40. Multi Configuration Job
• For multiple device testing create matrix jobs
• Specify variables to run multi test combinations
• Run multiple emulators to make sure the app
behaves same across different devices

41. Configuration Matrixes
• Create a new project in Jenkins and choose
matrix configuration job.
• Same setup except we have to add one or more
configuration matrixes.

42. Configuration Matrixes Cont.
• Under the Configuration Matrix tab choose add
axis.
• Jenkins will than ask for a name and values. For
example you can choose to create an emulator
for different OS versions and screen density's.
• This matrix runs job on four different emulators.

43. Configure the Emulator
• Now use variables instead of hard coded values.

44. Requirements for CI
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

45. Android Deployment
• Manage Jenkins->manage plugins.
• Download email-extension plugin

46. Email Configuration
• Go to email notification tab and check the box.
• Choose a recipient, subject, and message. Under
attachments give it the argument **/*.apk
• Add trigger on success to choose who it’s sent to.

47. Requirements for CI
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

48. Creating a iPhone Test Project
• Create a new job IPhoneCIDemo with the option build free
style

49. Configuring the IPhone Test
Project
• Go to the source control tab and select GitHub.
• Copy your GitHub into Repository URL field
• Specify a branch if not in the default branch.

50. Configuring the IPhone Test Project
Cont.
• Go to the build tab and add a new build step
XCode.
• Change the XCode target to the target you want
to build, and supply XCode project directory and
file if not in root.
• Fill in the XCode SDK field and change the
Configuration field to default or debug.

52. Requirements for CI
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

53. iPhone Unit Tests
• Use standard OCunit testing framework.
• Jenkins uses Christian Hedlin’s Ocunit2junit ruby
script
• Converts the Xcode OCunit tests to JUnit tests.
• Cannot run any tests that touch the UI.

54. Modifications to the IPhone Project
• Remove test host values under the build settings
• Add a new build step:
xcodebuild -target yourtarget -configuration Debug -sdk
iphonesdk | [path_to_ruby_script]/ocunit2junit.rb
• Choose publish junit test results and give it the value test-
reports/*.xml

55. Requirements for CI
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

56. iOS Functional Tests
• Use FoneMonkey to run functional tests from CI.
• Create automated tests by recording
scripts and adding verification steps
• Brittle GUI testing again 

57. FoneMonkey
• To implement FoneMonkey first create a new target by
duplicating the primary target.
• Download the FoneMonkey Zip File and add it to the new
target
• Under the build phases tab of the FoneMonkey target go to
the link with binary with libraries tab.
• Click on the + button and add the CoreGraphics.framework
and QuartzCore.framework.
• You will also need to add the libxml2.dylib and
SenTestingKit.framework
• Lastly On the Build Settings tab, scroll down to the Linking
section and set Other Linker Flags to -all_load

58. FoneMonkey Cont.
• To be able to run FoneMonkey from the command line without having
to start Xcode we need to create a TestRunner
• Create a new target off of your FoneMonkey target
• Add a run script to the TestRunner target under build phases
export FONEMONKEY_HOME=~/[Location_of_Fone_Monkey]
export FAMILY=[iphone|ipad]
export LOG_FILE="$PROJECT_DIR/$TARGETNAME.log“
export FONEMONKEY_ENV=$FONEMONKEY_HOME/FoneMonkeyRunner.plist rm -f "$LOG_FILE"
$FONEMONKEY_HOME/bin/iphonesim launch "$CODESIGNING_FOLDER_PATH" -verbose -sdk
$IPHONEOS_DEPLOYMENT_TARGET -family $FAMILY -stderr “ $LOG_FILE" -env
"$FONEMONKEY_ENV“
• You can now run this target from the command line and it will run
your FoneMonkey tests for you.

59. FoneMonkey
• The command to run the FoneMonkey TestRunner is
xcodebuild –target TestRunnerTargetName –sdk
iPhonesimulatorsdk –configuration Debug

60. Requirements for CI
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

61. iPhone Deployment
• Use Test Flight App
• Over the air beta testing
• Test Flight app Jenkins plugin doesn’t work
• But…all is not lost
• Add a new build step and choose Xcode
• Fill in the normal fields and choose to build IPA
and unlock keychain as well.
• Add keychain password.

62. iPhone Deployment
• Add a build step execute shell and add the following
commands:
cd $WORKSPACE/build/Release-iphoneos/ curl
http://testflightapp.com/api/builds.json-F file=@[IPA_FILE_NAME] -F
api_token='[API_TOKEN]' -F team_token='[TEAM_TOKEN]' -F
notes='This is an auto deploy build from Jenkins!' -F notify=True -F
distribution_lists='[DISTRIBUTION_LIST]
• Team token and api token can be gotten from your
testflight account

63. Requirements for CI
Android IPhone
Requirement
CI Server
Source Code Repository
Project Build
Unit Testing
Functional Testing
Mobile Device Testing
Deployment

64. Conclusion
• Work in Progress
• Executable Requirements
• Perfecto Mobile
• Test Flight App
• Demo
• Followup
• github.com/RIIS/RIIS-codemash.git
• godfrey@riis.com
• Android Application Testing Guide

65. Thanks
• RIIS
• Derek Verlee
• David Armstrong
• Brad Dunlap
• Nayan Hajratwala