The first ever QA Summit for Digital Healthcare Marketing took place in October 2013 and more than 20 agencies and organizations attended. This guide includes presentations from all the speakers at the event. Speakers: Jim Spillson, Joe Shields, Ben Currie, Elizabeth Estes, Michael Morowitz, Shachar Schiff

1. Presentation Guide
Thursday, October 24, 2013

2. Agenda
8:30 AM Breakfast - Pearson Room
9:00 AM Welcome & Introduction
Jim Spillson (AbelsonTaylor)
9:10 AM Setting Context for Quality in Healthcare Marketing
Joe Shields (AstraZeneca)
9:40 AM Shift Left
Jim Spillson (AbelsonTaylor)
10:00 AM Mind the Gap
Ben Currie and Elizabeth Estes (GA Communication Group)
10:30 AM Break
10:40 AM “Roundtable” Discussion
12:00 PM Lunch - Buckingham Room
1:00 PM Keynote Presentation
Testing is not Assurance: QA in Digital Marketing
Michael Morowitz (R/GA)
1:30 PM Client Panel Discussion
2:40 PM Shift to Mobile
Shachar Schiff (Bad Testing)
3:00 PM Break / Afternoon Snack
3:20 PM Best Practices Document Working Session
4:50 PM Closing - Jim Spillson

3. Welcome

4. Introductions
Jim Spillson – Host
Manager of Quality Assurance, AbelsonTaylor
Dale Taylor – Co-Host
President, CEO, AbelsonTaylor
Mark Goble – Co-Host
President, COO, GA Communication Group
Ben Currie – Co-Host
Group Director Digital Solutions, GA Communication Group
Joe Shields – Moderator
Digital Marketing Leader, AstraZeneca
2

5. QA: We Hit a Nerve
3

6. We All Face the Same Questions
iOS management How much testing is enough?
4
QA team responsibility
Jailbreaking (iOS)
Simulators
Automation
3rd party vendors
PC operating systems
MDM deployments
Are some bugs not worth fixing?
What about mobile testing?
Bug tracking systems
Test devices
What about Android?
Xcode
Crowd source testing
Constant release of browsers
client ITBrowsers too old?

7. Response has been Overwhelming
5

8. Agenda
Morning Session
9:00 – Welcome & introductions
[Jim Spillson]
9:10 – Setting Context for Quality
in Healthcare Marketing
[Joe Shields]
9:40 – Shift Left [Jim Spillson]
10:00 – Mind the Gap [Ben Currie &
Elizabeth Estes]
10:30 – Break
10:40 – “Roundtable” Discussion
6
Afternoon Session
12:00 – Lunch
1:00 – Testing is not Assurance:
QA in Digital Marketing
[Michael Morowitz]
1:30 – Client Panel Discussion
2:40 – Shift in Mobile
[Shachar Schiff]
3:00 – Break
3:20 – Best Practices Working
Session
4:50 – Closing

9. Our Goal
Best Practices Document
• Gather all information today
• Review, refine, participant approval/
endorsement
• Publish and distribute
7

10. Next Stop
8

11. Discussion Format and Miscellaneous
• Participation
– Ask questions: working session with real outcomes
– Q&A for panel discussion, keynote and presentations will be at the end
– Best Practices template in front of you
• Considerations
– Place phones on vibrate
– Respectful of others
– Microphones for everyone
• Miscellaneous
– Breaks and lunch in Buckingham Room
– Charging station
– Special requests (hosts/hotel)
9

12. Future Events
10
Information
• Publish Best
Practices
• YouTube channel
• White paper
• Publications
• LinkedIn group
• StateofQA.com
2014 QA SummitSXSW 2014
• QA Best Practices
in Today’s Digital
Agencies

13. Setting Context for Quality in
Healthcare Marketing
Joe Shields
Digital Marketing Leader

14. Disclaimers
• The views expressed are my own & not necessarily those of my
employer
• Please note that all images in this presentation are borrowed from the
Internet, and cited where possible
• There may be a Google Glass reference today. Sorry
2

15. Everything I know about quality is from film shoots
3
ALMOST

16. “We’ll fix it in post” never really happened
4

17. Managing creativity is hard
“When the job is to conjure the next
brilliant idea out of thin air, against
deadline, via a combination of
inspiration, hard work, experience,
intuition, and confidence, getting the
best work out of creative people on a
consistent and efficient basis can be
tricky business.”
5
Source: Rae Ann Fera, “10 Tips for Managing Creative People,” Fast Company, September 17, 2013.

18. We struggle with being subjective vs. objective
6

19. We struggle with being subjective vs. objective
7

20. The world has changed….

21. Customers live in a multiscreen world
• Screens are more than just objects we look at to
consume content. We can also:
–Use voice+gestures to instruct them
–Shoot pictures+video
–Play+write music
–Get+give directions
–Pay for lattes+shoes
–Message friends+foes
–Control apps+devices
9

22. Connectivity is now a basic human need
10
SELF-
ACTUALIZATION
SELF-ESTEEM
BELONGING-LOVE
SAFETY
PHYSIOLOGICAL
WIFI

23. Technology is fundamentally
changing human behavior
• “One in six Americans with recent
dating experience have broken
up with someone—or had
someone break up with them—by
text message, email, or sending a
message online”
• “People high in FOMO … feel
less competent, less autonomous
& less connected with others than
people who don't worry about
being left out”
11
Sources: Pew Internet & American Life Project: Online Dating & Relationships, October 21, 2013, and
“No More FOMO: Fear of Missing Out Linked to Dissatisfaction,” Stephanie Pappas,
LiveScience.com, May 14, 2013.
Image credit: Hubspot.

24. These human behaviors are also
healthcare behaviors
•Consume
•Create
•Connect
•Collaborate
•Contextualize
•Care
12
Image credit: Boyle-Associates.com.

25. HCPs are close to the digital tipping point
• In 2014, the majority of EU HCPs will be digitally native
13
Source: Manhattan Research, TTP, ePharma HCP 2012.
Image credit: LBi Digitas.

26. Clinicians are now ‘digital omnivores’
• Digital omnivores routinely use tablet,
smartphone & laptop/desktop for
professional purposes
• Omnivores are becoming the dominant
type
– 2012: 28% of clinicians
– 2014: 82% of clinicians
• Mobile devices extend the digital day
– Computer use drops at 5 PM, but mobile use
for professional reasons remains high until 9
PM
14
Source: “Mobile Trends Report,” ePocrates, 2013.

27. Changing healthcare context has made
‘Healthcare Anywhere’ a reality for many
Ubiquitous
Connectivity
HEALTHCARE
ANYWHERE
Empowered
Patients
Technology-
enabled
Outcomes -
driven
Cost
Containment






28. Investments in healthcare must
yield better quality outcomes
• “Quality measures are tools that help us measure or quantify
healthcare processes, outcomes, patient perceptions, and
organizational structure and/or systems that are associated with the
ability to provide high-quality health care and/or that relate to one or
more quality goals for health care
• These goals include: effective, safe, efficient, patient-centered,
equitable & timely care”
16
Source: Centers for Medicare & Medicaid Services, cms.gov.

29. We have been slow to respond to
these dramatic changes….

30. We are no longer in the advertising business
18
It was fun while it lasted.

31. We need more content for more devices, and faster
19
RESPONSIVE ADAPTIVE
C D E F
A B
C
D E F
A B A
B
C
D
C D E F
A B
C D
A G
E
G
H
B C
< optimal
viewing
experience
optimal user
experience >

32. So we must methodically plan content
20
Operating System (OS)
Screen Size
Platform or Channel

33. Although the possibilities seem endless
21
Operating System (OS)
Screen Size
Platform or Channel
+ MILLIONS OF
CUSTOMER JOURNEYS
+ SEVERAL BRANDS
+ MANY FILE FORMATS
+ PUSH VS. PULL
+BUSINESSRULES

34. We must be better at technology
22
visualization
connectivity
memory
processing
battery life
integration
Image credit: Samsung.
Fitness or
medical
device

35. We can’t do it ourselves
23
“The Marketing department
in my company spends
more on IT services than the
IT department.”
-- Overhead at a recent
pharma industry conference

36. Yet we’re confused about the current & future role of
medical communications agencies
24
Source: “Omnipotent, or omnishambles?” The Economist, August 3, 2013.

37. And stuff like this isn’t helping
"Go back 70 years, go ahead 50 years,
a human is a human. There are
primal things that will always drive us:
Will this product be better, will it help
me succeed, will it make me more
attractive? So the technology and the
way we to speak to people will
change, but those fundamentals will
never change.“
David Lubars, North American
chairman & chief creative officer,
BBDO
25
Source: “The Future of Advertising: Three Agencies' Visions,” Fast Company, November 19, 2010.

38. Or this
26
• Email on Friday from one of my agency friends, who is a VP >

39. Quality is more important than ever….

40. We are all in the patient care &
healthy behaviors business
28
Image credit: Regrounding.Wordpress.com.

41. Together we build complex,
technology-driven systems
29
Image credit: MasterNewMedia.org.

42. Quality matters in everything we do,
not just in software
• Except when it doesn’t
30
Source: Twitter post from Forrester Ideas conference, October 3, 2013.

43. The lifecycle management of these products &
systems happens over years, not months
31
Image credits: PSDgraphics.com, Apple Corporation.
forever.

44. Finally, we must still manage the
tension among quality, speed & cost
• “The ‘cost of quality’ isn't the price of creating a quality product or
service. It's the cost of NOT creating a quality product or service. Every
time work is redone, the cost of quality increases.”
32
Source: American Society for Quality, ASQ.org. Image credit: Jokeroo.com.

45. So let’s talk.

46. Shift Left

47. Why Shift QA to the Left?
2
QA starts too late in the process – like at the end.
Build things that aren’t broken in the first place.
The longer a bug hangs around, the more it costs to fix.

48. Typical Agency Process
3
Job start Requirements Documentation
Concept/
design
Routing
Client review Revisions Development Routing Revisions
Testing
Programming
revisions
Testing
Regulatory
review
Repeat (2x)
Release

49. Where Defects Come From
56%
27%
10%
7%
Where Defects Originate
Requirements
Code
Other
Design
4
The Journal of Defense Software Engineering

50. Shift Left
5
Requirements ReleaseTestDevelopmentDesign
Phase View of
Quality
Project Life Cycle
NumberofDefectsJobOpens
‘Shift Left’
Early Detection of Defects
Project Life Cycle
Shift Left
Life Cycle

51. Financial Impact of Shifting Left
6
Cost to fix a defect
Time detected
Requirements Doc/Creative Development Testing Post-release
Time
introduced
Requirements 1x 3x 5-10x 10x 10-100x
Doc/Creative – 1x 10x 15x 25-100x
Development – – 1x 10x 10-25x

52. Quality Assurance Really Isn’t A Luxury
7
“It can be tempting to see quality assurance as an
overhead. When schedules start to slip then it can be
tempting to cut down on quality assurance, but this is a
false economy. A project that is beset by quality
assurance difficulties is a grueling experience. It’s also an
expensive one as you end up pouring resources into fixing
bugs that could and should have been caught earlier in
the development process. It blows a hole in profitability,
damages the reputation of your business, undermines user
confidence and demoralizes development teams.
Quality assurance really isn’t a luxury.”
10 Myths About Quality Assurance in Software Development – Ben Morris

53. It’s an Ongoing Process
8
Shift in
mindset
• Support from
the top
• Client
involvement
• Collaboration at
all levels
Enable teams
• Training
• Proper
Integration
• Empowerment
Continuous
improvement
• Keeping up with
tech
• Tracking defect
origination
Process
Changes
• Bug tracking
throughout
• Automation tools
• Best practices
document

54. Shift Left – What it Means to Your Business
9
Improved
overall quality
Reduction in overall
Software Development LifeCycle (SDLC) cost
Transparency and
awareness of quality in SDLC
Increased
client satisfaction
Improved adherence
to timing and scopes of work

56. Who Are We?
2
Name: Ben Currie
Title: Director of Marketing
Technology for GA
Communication Group
Fav QA Quote: “A foolish consistency is the
hobgoblin of little minds.” -
Emerson
Name: Elizabeth Estes
Title: EVP/Chief Strategy Officer
for GA Communication
Group
Fav QA Quote: “If you don’t have time to do
it right, you must have time
to do it over.” - Unknown

57. Yin/Yang
3

58. Goal: Get the Conversation Going
4
• Let’s Look At It From Inside Out
• Part Presentation/Part Discussion
• Debate vs. Answers
• Catalyst for Best Practices Doc

59. Our General Thinking
5
• Quality Requires More Than a
Technical Discipline at the End
of a Project
• Determining “Critical to Quality”
Factors at the Beginning Is Vital
• There Is a Language Barrier
Between the Disparate Groups

60. Why Is It Hard?
6
“I need an automated
email that tells people
what the weather will be
on the day of their
check up”
“I need their email
address, location and
date of their
appointment”

61. Let’s Understand the Players
7
http://blog.mindjet.com/wp-content/uploads/2012/05/Between-Minds-An-Ongoing-Taxonomy-of-Team-Dynamics1.jpg

62. We Are Different Now
8
http://www.mdgadvertising.com/blog/the-modern-marketer-infographic/

63. Team Dynamics Are Complex(er?)
9
http://esdandassociates.com/2012/06/the-dynamics-of-marketing-success/

64. So Are the Options and Opportunities
10
http://www.mergesocialmedia.com/

65. So Are Expectations
11
http://www.strangeloopnetworks.com/resources/infographics/mobile-infographics/poster-mobile-infographics/
• Consumer Tolerance Limits
• On-Demand World
• Personalization Expected
• User Experience Vital

66. Under Pressure (Da-Da-Da-DaDa-Da-Da)
12
• Expected to Know More
and Do More in Less Time
• While Coordinating More
Specialists
• Without Sacrificing Quality

67. Kinda Like...
13

68. So What Are The Components of Shift Left?
14

69. How Do We…?
15
• Address the “Human Side” Systematically
• Start at the Top
• Involve Every Layer
• Identify “Critical to Quality” Moments
• Make the Formal Case
• Create Ownership
• Communicate the Message
• Assess the Cultural Landscape
• Address Culture Explicitly
• Prepare for the Unexpected
• Speak to the Individual

70. “Reach Right” to Help the Shift
16

71. Identifying Issues
17
Communication Process Strategy Time & Budget Functionality Maintenance

72. Outside of Functionality
18
Communication Process Strategy Time & Budget Functionality Maintenance

73. And Close the Gap
19

74. Communication Considerations
20
• Internal
– Project Kickoffs
– Ongoing Team
Reviews
– Post Project
• External
– Client Involvement
– Client Communication
• Lack of Context
– Language Barriers
– Empathy Chips/Perspective

75. Process – Issues to Consider
21
• PPDS
• Identifying “Critical to Quality” Moments
• Documentation
• How Locked Down Should It Be?
• Acknowledged Shortcut Areas?
• Managing Change

76. Strategic Considerations
22
• Feasibility?
• Clear Direction?
• Correct Channel?
• Correct Technology?
• Pulled Through To End?

77. Time & Budget Considerations
23
• Enough Time?
• Enough Money?
• Is the Project Fully
Understood To Determine
Time & Budget?

78. Functionality Considerations
24
• Critical to Business Requirements?
• Critical to User?
• Critical to Success?
• How to Handle Distractions That Do Not
Meet Critical Goals But Jeopardize
Success?

79. Maintenance Considerations
25
• Opposite of Software Updates
• Where Are the Checks and
Balances?
• Who Is Responsible?
• How Is Value Communicated
and Understood?

80. What Is YOUR Single Biggest Issue?
26
It’s Your Turn Soon!

81. 27
And Finally….

112. The Shift to Mobile

113. The Shift to Mobile.
• Shachar Schiff
• Work out of 1871, the technology center at Chicago's Merchandise
Mart, designed to help launch and grow early-stage tech start-ups.
• BAD TESTING® offers an advanced and versatile skill set in software
quality assurance, testing, and configuration to ensure effective and on
time product releases.

115. Mobile | Overtakes Desktop

116. Mobile | QA
• Mobile-device testing involves a set of activities from monitoring and
trouble shooting mobile application, content and services on real
handsets.
• Testing includes verification and validation of hardware devices and
software applications.

117. Mobile | Designs
• You need to know about responsive, adaptive, mobile, and native
app design.
• Raises the bottom line on site's final cost.
• Review analytics data.
• Know what device your audience is on.
• Important to look at site traffic and target audience before determining.

118. Mobile | Estimation
• Project estimations for responsive/adaptive websites need to be treated
as separate projects for each Break-point.
• Including development and QA.

119. Mobile | Responsive & Adaptive
Responsive
• "Fluid" layouts shrink and grow based on the browser size.
– Content is sized using percentages, adjusts as the browser size changes.
Adaptive
• Content reconfigured at predetermined "breakpoints".
• Easier to control how content will flow
• 1,2,3 site redesign increases costs, time.

120. Mobile | Responsive

121. Mobile | Adaptive

122. Mobile | Specific & Native App
Mobile-specific
• Mobile-specific site separates your site's desktop experience from the
mobile experience.
• Desktop URL on mobile redirects to separate URL (m.yourdomain.com)
Native App
• Device-specific features (GPS, camera, accelerometer, NFC)
• Difficult topic due to development costs
• iOS is different than Android:
• Alienates 50% of potential users
• 2x costs to develop (iOS, Android)

123. Mobile | Desktop Ownership Decreasing

124. Mobile | Pew Internet Project Highlights
As of May 2013:
• 91% of American adults have a cell phone
• 56% of American adults have a smartphone
• 28% of cell owners own an Android; 25% own an iPhone; 4% own a
Blackberry
As of September 2013:
• 24% of Americans ages 16 and older own an e-reader
• 35% of Americans ages 16 and older own a tablet computer

125. Mobile | Device Shipments

126. Mobile | Device Shipment Highlights
• Today feature phones (non smartphones) outsell smartphones; and
PCs and portable PCs (notebooks/laptops) both outsell tablets.
• In the future that is expected to change.
• Both analysts predict that marginally more smartphones will be sold in
2013 than feature phones.

127. Mobile | Desktop Limited to Working Hours

128. Mobile | Mobile Web Statistics
• 61% of people have a better opinion of brands when they offer a good
mobile experience.
• 25.85% of all emails are opened on mobile phones, and 10.16% are
opened on tablets.
• Almost half a billion tablets will ship in 2013 and 2014 alone.
• Mobile-based searches make up one quarter of all searches.

129. Mobile | Summary
• The reason shift left is so important to mobile is because customers and
employees are making the mobile mind shift.
• They expect that any information or service they desire be available to
them on any device, in context, at their moment of need.