1.
Automating the Gaps of
Unit Testing Mobile Apps
Effectively Utilizing Xcode’s Instruments

2.
@ggeoffreggeoffre.com
20+ years of experience in the development, marketing, sales,
and leadership of computer software development
A C T S

3.
Preparing your QA team for mobile testing
http://www.slideshare.net/ggeoffre

4.
Where did the term "bug"
come from?
66 years ago on
September 9th
1947, operators of
the Mark II Aiken
Relay Computer
being tested at
Harvard University,
found something
curious trapped
between points at
Relay #70, Panel
F.

5.
Bugs not Defects!

6.
Bugs not Defects!

7.
Bugs not Defects!

8.
Bugs not Defects!

9.
Bugs not Defects!
Outside the scope of most acceptance or unit testing
Tend to design around them not to them
Defensive programming techniques employed
More likely hardware based than software based
Environment plays a major role discovering them
Magnified greatly with mobile based development

10.
Controlled Environments

11.
Today’s Topics
Reference Counting
Battery Usage
Network Monitoring
Communicating Build and Version Numbers
Xcode Server Initial Setup and Bot Creation
Test Deployment Options to Devices
Crash and User Analytics Reporting
DevOps
UITesting
Instruments
Automation Instrument
Calabash Appium
UIAutomation XCTest

12.
Xcode ▶︎Open Developer Tool ▶︎Instruments

13.
Reference Counting

14.
[Objective-C]
alloc/init
retain
copy
release
autorelease

15.
Properties and @IBOutlets
atomic
nonatomic
strong
weak
retain
assign
unsafe_unretained
copy
readonly
readwrite

16.
Automatic Reference Counting
Edit => Convert => To Objective-C ARC
Project => Build Settings

17.
Swift
When to use ? and when to use !
When to use “if let”?
When to use as? and when to use as

18.
WWDC 2015
Optimizing Swift Performance
https://developer.apple.com/videos/play/wwdc2015-409/

19.
WWDC 2016
Understanding Swift Performance
https://developer.apple.com/videos/play/wwdc2016/416/

20.
Xcode 8 InstrumentsI

21.
Leaks
The Leaks profiling template uses the Allocations
and Leaks instruments to measure general
memory usage in your app and check for leaks—
memory that has been allocated to objects that
are no longer referenced and reachable.
https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/FindingLeakedMemory.html
iOS Developer Library - Instruments User Guide

22.
Leaks
https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/FindingLeakedMemory.html
iOS Developer Library - Instruments User Guide

23.
Allocations
The Allocations profiling template uses the
Allocations and VM Tracker instruments to
measure general and virtual memory usage in
your app. However, to track down abandoned
memory that’s been allocated but isn’t needed
again, you’ll focus strictly on the Allocations
instrument. This instrument measures heap
memory usage and tracks allocations, including
specific object allocations by class.
https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/FindingAbandonedMemory.html
iOS Developer Library - Instruments User Guide

24.
Allocations
https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/FindingAbandonedMemory.html
iOS Developer Library - Instruments User Guide

25.
Zombies
The Zombies profiling template uses the
Allocations instrument to measure general
memory usage in your app, with a focus on the
detection of overreleased “zombie” objects—that
is, objects that are called after they’ve been
released and no longer exist.
https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/EradicatingZombies.html
iOS Developer Library - Instruments User Guide

26.
Zombies
https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/EradicatingZombies.html
iOS Developer Library - Instruments User Guide

27.
Energy Savvy Users
Battery Life is very important to
users
iOS Now Displays Apps in
order of Battery Usage
Be sure to test in Low Power
Mode

28.
Energy Gauge

29.
Energy Diagnostics
Settings > Developer >
Instruments
Enable Energy Recording
Start Recording
Xcode > Instruments
New Energy Log
File > Import Legged Data
from Device

30.
Energy Diagnostics

31.
WWDC 2015
Debugging Energy Issues
https://developer.apple.com/videos/play/wwdc2015-708/

32.
Data Savvy Users
Data caps are real
iOS displays cellular data
usage and allows users to turn
on/off apps
Be sure to test with Network
Conditioner

33.
Network Conditioner
Settings > Developer >
Network Link Conditioner
Enable
Choose a Profile

34.
Network Instrument

35.
Network Monitoring

36.
Network Monitoring
Use iTunes to find your UDID
UDID

37.
Network Monitoring
Use Xcode Devices Window to find your UDID
UDID

38.
Network Monitoring
rvictl -s <UDID>
ifconfig rvi0
tcpdump -s 0 -A 'tcp dst port 80 and
(tcp[((tcp[12:1] & 0xf0) >> 2):4] =
0x504f5354)'
...
rvictl -x <UDID>

39.
HTTPS Server Trust Evaluation
Technical Note TN2232
App Transport Security Exceptions
<key>NSAppTransportSecurity</key>
<dict>
<key>NSExceptionDomains</key>
<dict>
<key>yourdomain.com</key>
<dict>
<!--Include to allow subdomains-->
<key>NSIncludesSubdomains</key>
<true/>
<!--Include to allow HTTP requests-->
<key>NSTemporaryExceptionAllowsInsecureHTTPLoads</key>
<true/>
<!--Include to specify minimum TLS version-->
<key>NSTemporaryExceptionMinimumTLSVersion</key>
<string>TLSv1.1</string>
</dict>
</dict>
</dict>

40.
macOS Sierra Apps
WireShark
tastycocoabytes.comwireshark.org
Cocoa Packet Analyzer

41.
WWDC 2015
Your App and Next Generation Networks
https://developer.apple.com/videos/play/wwdc2015-719/

42.
Asynchronous Programming
Concurrent code execution on multicore
hardware
Grand Centeral Dispatch
Blocks
Closures
UIView Animations
NSNotifications
Local and Remote Push Notifications

43.
Blocks [Objective-C]
dispatch_async(/* a queue /*, ^{
/* your code here */
});
dispatch_get_main_queue()
dispatch_get_global_queue(
DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)

44.
dispatch_async(/* a queue /*, {
/* your code here */
})
Closures Swift
dispatch_get_main_queue()
dispatch_get_global_queue(
DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)

45.
func perform_on_main(block: dispatch_block_t?)
{
dispatch_async(
dispatch_get_main_queue(), block)
}
func perform_in_background(block: dispatch_block_t?)
{
dispatch_async(
dispatch_get_global_queue(
DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), block)
}
Closures Swift

46.
perform_on_main {
/* your code here */
}
perform_in_background {
/* your code here */
}
Closures Swift

47.
[UIView animateWithDuration:1.0
delay: 1.0
options: UIViewAnimationOptionCurveEaseIn
animations:^{
/* your code here */
} completion:^(BOOL finished){
/* your code here */
}];
[Objective-C]UIView Animation

48.
UIView Animation
UIView.animateWithDuration(0.7,
delay: 1.0,
options: .CurveEaseIn,
animations: {
/* your code here */
}, completion: { finished in
/* your code here */
})
Swift

49.
NSNotifications
[[NSNotificationCenter defaultCenter]
addObserver:self
selector:@selector(someMethod:)
name:@"SOME_NOTIFICATION"
object:nil
];
[[NSNotificationCenter defaultCenter]
postNotificationName:@"SOME_NOTIFICATION"
object:self
];
[[NSNotificationCenter defaultCenter]
removeObserver:self
];
[Objective-C]

50.
NSNotifications
NSNotificationCenter.defaultCenter()
.addObserver(self,
selector:"someMethod",
name:"SOME_NOTIFICATION",
object:nil)
NSNotificationCenter.defaultCenter()
.postNotificationName("SOME_NOTIFICATION",
object: self)
NSNotificationCenter.defaultCenter()
.removeObserver(self)
Swift

51.
WWDC 2015
Building Responsive and Efficient Apps with GCD
https://developer.apple.com/videos/play/wwdc2015-718/

52.
WWDC 2015
Designing with Animation
https://developer.apple.com/videos/play/wwdc2015-803/

53.
WWDC 2011
Using Local And Push Notifications
https://developer.apple.com/videos/play/wwdc2011-517/

54.
WWDC 2015
What's New in Notifications
https://developer.apple.com/videos/play/wwdc2015-720/

55.
WWDC 2016
Advances in UIKit Animations and Transitions
https://developer.apple.com/videos/play/wwdc2016/216/

56.
WWDC 2016
Thread Sanitizer and Static Analysis
https://developer.apple.com/videos/play/wwdc2016/412/

57.
Testing the User Interface

58.
iOS Human Interface Guidelines
https://developer.apple.com/ios/human-interface-
guidelines/overview/design-principles/

59.
UIKit Catalog (iOS)
https://developer.apple.com/library/content/samplecode/UICatalo
g

60.
Accessibility for Developers
https://developer.apple.com/accessibility/

61.
Accessibility Audit
Xcode => Open Developer Tool => Accessibility Inspector

62.
Accessibility Attributes
name Derived from the
accessibility label.
value The current value of the
control, for example, the text in a
text field.
elements Any child elements
contained within the current
element, for example, the cells in
a table view.
parent The element that contains
the current element.
The name property is one of four properties of these
elements that can be very useful in your test scripts.
Select and Set

63.
Accessibility Inspector
Xcode => Open Developer Tool => Accessibility Inspector
Accessibility Label

64.
WWDC 2015
iOS Accessibility
https://developer.apple.com/videos/play/wwdc2015-201/

65.
Auditing Your Apps for Accessibility
https://developer.apple.com/videos/play/wwdc2016/407/
WWDC 2016

66.
AUTOMATION
Behavior Driven Development
Acceptance Tests
JavaScript
Tool Based (Command Line Execution)
Instruments

67.
AUTOMATION
Install Xcode from the App Store
https://itunes.apple.com/us/app/xcode/id497799835?mt=12
Xcode => Open Developer Tool => Instruments
Choose “Automation”
Select the target app to test with
Record/Run

68.
AUTOMATION
Accessibility Label

69.
TuneupJS.org
Tune-up is a collection of JavaScript utilities that
builds upon and improves the UIAutomation library
provided by Apple for testing iOS applications via
Instruments (get it? "tune-up"? Instruments? get
it?).
http://www.tuneupjs.org

70.
T XCUIApplication()
Behavior Driven Development
Acceptance Tests
Swift or Objective-C (XCTest)
Tool Based, Project Test Target, (Command Line
Execution)
Xcode, Xcode Server Bot

71.
T XCUIApplication()
Install Xcode from the App Store
https://itunes.apple.com/us/app/xcode/id497799835?mt=12
Create a new Project
Product => Test
Record

72.
T XCUIApplication()
let app = XCUIApplication()
let someTextField = app.textFields["MyCustomLabel"]
Find Things
someTextField.tap()
someTextField.typeText("qwerty")
Do Some Stuff
let expectaion = self.expectationWithDescription("Something")
expectaion.fulfill()
self.waitForExpectationsWithTimeout(5.0, handler: nil)
Set Expectations

73.
T XCUIApplication()
Xcode

74.
T XCUIApplication()
xcodebuild test
-project SampleProject.xcodeproj
-scheme SampleProject
-destination
'platform=iOS Simulator,
name=iPhone 6s'
Command Line

75.
Behavior Driven Development
Acceptance Tests
Ruby/Cucumber/Bash
Command Line Scripting
RubyMine (optional)

76.
Clone the iso-smoke-test-app repo…
https://github.com/calabash/ios-smoke-test-app
Install the calabash sandbox… (setup Ruby)
https://github.com/calabash/install
Build a .app for the Simulator…
Run your first cucumber test:

77.
Scenario: Touch OK button on alert
When I touch the show alert button
Then I see an alert
Then I see the "Smoke Test!" alert
And I wait for all animations to stop
And I can dismiss the alert with the OK button
When(/^I touch the show alert button$/) do
wait_for_element_exists("view marked:'show alert'")
touch("view marked:'show alert'")
end
Features
Steps
Ruby
Gherkin

78.
Behavior Driven Development
Acceptance Tests
Earl Grey

79.
WWDC 2015
UI Testing in Xcode
https://developer.apple.com/videos/play/wwdc2015-406/

80.
Managing Simulators
Visible and Meaningful Version
Numbers
Xcode Server Bots for CI
iTunes Connect Crash Analytics
Flurry User Analytics and Reports
Fabric Crash Analytics Support
HockeyApp QA and Beta Testing
Some DevOps Tips

81.
Manage Simulators
Window => Devices

82.
Add a new Run
Script Phase
Reset All Simulators
osascript -e 'tell application "iOS Simulator" to quit'
osascript -e 'tell application "Simulator" to quit'
xcrun simctl erase all
Project => Target => Build Phases

83.
Run Script Git Version Info
Build Version Number
buildVersion=$(/usr/libexec/PlistBuddy -c "Print CFBundleShortVersionString”
“${PROJECT_DIR}/${INFOPLIST_FILE}")
buildNumber=$(git rev-parse --verify HEAD | cut -c1-5)
buildString=$buildVersion' ('$buildNumber')'
/usr/libexec/PlistBuddy -c "Set PreferenceSpecifiers:1:DefaultValue $buildString”
"${SRCROOT}/Settings.bundle/Root.plist"
/usr/libexec/PlistBuddy -c "Set :CFBundleVersion $buildNumber"
"${PROJECT_DIR}/${INFOPLIST_FILE}"

84.
Build Version Number

85.
File => New => Settings Bundle
Build Version Number

86.
Settings Bundle Root.strings
Build Version Number
/* A single strings file, whose title is specified in your preferences schema. The strings
files provide the localized content to display to the user for each of your preferences. */
"Group" = "SampleProject";
"Name" = "Version";

87.
Settings Bundle Root.plist
Build Version Number
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"><dict>
<key>PreferenceSpecifiers</key>
<array>
<dict>
<key>Title</key>
<string>About</string>
<key>Type</key>
<string>PSGroupSpecifier</string>
</dict>
<dict>
<key>DefaultValue</key>
<string>1.0</string>
<key>Key</key>
<string>version_preference</string>
<key>Title</key>
<string>Version</string>
<key>Type</key>
<string>PSTitleValueSpecifier</string>
</dict>
</array>
<key>StringsTable</key>
<string>Root</string>
</dict></plist>

88.
Install OS X Server and Xcode on the target Mac
Enable the Xcode service (administrator account)
Add the remote repository for the project
Create a “BOT” to build and test the project
View the results in the browser
Setup Xcode Server

89.
Enable Xcode Server

90.
Create Bot for Project

91.
WebView Screen Saver
GitHub / liquidx / webviewscreensaver

92.
WWDC 2016
Advanced Testing and Continuous Integration
https://developer.apple.com/videos/play/wwdc2016/409/

93.
Crash Analytics
Technical Note TN2151
Understanding and Analyzing Application Crash Reports
https://developer.apple.com/library/content/technotes/tn2151/_index.html
App Distribution Guide - Analyzing Crash Reports
https://developer.apple.com/library/content/documentation/IDEs/Conceptual/
AppDistributionGuide/AnalyzingCrashReports/AnalyzingCrashReports.html

94.
WWDC 2015
Getting the Most out of App Analytics
https://developer.apple.com/videos/play/wwdc2015/303/

95.
Flurry
Measure, track and analyze app performance, user
acquisition and activity with Flurry Analytics. Get a deep
understanding of app performance metrics and everything
your users are doing.
Create a Yahoo account and get your API Keys
Add the Flurry framework to your project
Integrate Flurry Analytics into your App
https://developer.yahoo.com/analytics/

96.
Fabric hooks deeply into your existing build process and
automatically upload your dSYMs so you don’t have to. Now
you can sleep well, knowing you’ll get the crash reports you
need to keep your app stable and your users happy.
Create and confirming your Fabric account
Select the project you want to integrate Fabric into
Add the Fabric framework to your project
Fabric
https://developer.apple.com/testflight/

97.
Distribute beta versions, collect live crash reports, get
feedback from real users and analyze test coverage. Bring
DevOps to your apps by integrating with your existing build
and work item management.
Create an Account to Manage Teams and Apps
Uploading a Build for an App
Set Up and Invite Testers
HockeyApp
http://hockeyapp.net/features/

98.
TestFlight Beta Testing makes it easy to invite users to test
your iOS, watchOS, and tvOS apps before you release them
on the App Store. You can invite up to 2,000 testers using
just their email address.
Create an App Store iTunes Connect Record
Uploading a Build for an App
Set Up and Invite Testers
TestFlight
https://developer.apple.com/testflight/

99.
ggeoffre

100.
ggeoffre
slideshare.net
http://www.slideshare.net/ggeoffre