1. Using Market & Customer Insight to Make Better Decisions Jodie Slaughter McKinley Advisors (presenting) American Ceramics Society Jackie Oppenheim & Russ Raman American Society of Mechanical Engineers Marc Beebe, CAE IEEE Corporate Strategy and Communications Peter Turner (moderator) MCI Group Remote Meeting Technology provided by

2. What does it take to grow a T&D business?

4. Research as a Catalyst for Sound Association Decision Making

6. Research you can conduct yourself with no/little budget

7. When to call in a professional

9. Allows you to hear from new audiences

10. Validates or contradicts whims and curtails lurching

11. Surfaces new product ideas/unmet needs

12. Provides insights on competitors

13. Tests ideas and assumptions

14. Determines product/service pricing, positioning and distribution

17. Research 101—Definitions Primary research: Collecting data that doesn’t already exist Secondary research: Collecting data that already exists Qualitative: Deeply descriptive and anecdotal information Quantitative: Findings that can be expressed numerically and applied to confidence intervals (statistical reliability)

18. Research 101 Qualitative Research +Quantitative Research =Sound Research

19. Research You Can Do Yourself No Money…No Problem

21. Electronic survey instruments*

23. When do you need a professional?

24. CESSE Member Mini-Case Study Budget: ~ $6,000,000 Members: 5,862 – 6,120 (20% growth in corporate) Why: New exec loves data, needed myth-busting and voices from broader audiences, never did it before What: 69 interviews, two focus groups, electronic survey (19.5% response rate), enviro scan When: Late 2007 What’s next: Repeating in 2011

26. Very small (older) segment revered sacred cows

27. Time-consuming and potentially troublesome elections were not necessary

28. Academics had been over-represented in association decisions

30. Effective when contemplating key decisions

31. Major change

32. Strategic planning

33. Program development

34. Must be informed by qualitative phase

35. Seeks a measure of “practical significance”

39. CESSE Member Mini-Case Study

40. CESSE Member Mini-Case Study In their own words: “As a result of the information we gained from the survey, we were able to better identify our current audience as well as identify our potential audience and in one short year we were able to increase our total membership by 3%. “ Megan Bricker Director, Marketing & Membership Services

41. Thank youJodie Slaughter, FASAEjslaughter@mckinley-advisors.com

42. “If there is one phrase that sets remarkable associations apart from their counterparts, it’s “Data, data, data.. [In remarkable associations, ] whether qualitative or quantitative, research is always put to use, not put on the shelf.” -7 Measures of Success – What Remarkable Associations Do that Others Don’t

43. Do you want to be remarkable?

44. ASME Training & Development 26 Using Market and Custom Insight to Make Better Decisions2 Primary Research Case StudiesE&T3Wednesday July 201.40pm – 3.00pm

45. 27 Jackie Oppenheim Russ RamanDirector, Training & Development Marketing ManagerASME ASMETel: +1 212 591 7526 Tel: +1 212 591 7745Email: oppenheimjs@asme.org Email: ramanr@asme.org ASME Training & Development

47. Offers professional development and continuing education courses

48. Internationally recognized industrial and manufacturing standards enhance public welfare and safety

49. Educational and technical society of mechanical engineers

50. Not-for-profit organization founded in 1880

51. 120,000 members worldwide, including 24,000 student members

52. Staff of over 350: headquarters in New York City with offices in Washington, Houston, Atlanta, Brussels, India and BeijingASME Training & Development

53. 29 What are the two case studies? Nuclear Training Opportunities Study - higher cost, more detailed, longer time frame Workshop Participant Engagement Study - lower cost, less detailed, quicker ASME Training & Development

54. For each Case Study we’ll address: Background and Purpose Approach and Methodology Summary of Findings Conclusions and Business Implications What would / should we do differently now? 30 ASME Training & Development

55. Background and Purpose Focus on energy; specifically nuclear Understand the current and projected training and skills assessment needs of the nuclear power industry Would help guide us in what additional training courses, Certificate and Certification programs to pursue Nuclear Training Opportunities Study ASME Training & Development 31

56. Nuclear Training Opportunities Study 2. Approach and Methodology Conducted in-depth telephone interviews Targeted different industry segments to get balanced viewpoint Average length of interview 40–45 minutes. Interviewers had discussion guide outline Had specific questions to answer (see next slide) Also had ability to listen to unsolicited information All respondents were pre-recruited for the interview Not initially identified as an ASME study ASME Training & Development 32

57. ASME Training & Development 33 DISCUSSION GUIDE OUTLINE A. Opening & Comfort Building Purpose of interview Confidentiality – Private study for trade association Audio-taping B. Company Background Products produced (Suppliers only) Operating units and new plants (Utilities) Number of employees Percent of business in nuclear industry (Suppliers only) C. Respondent Background Respondent’s position/responsibilities Length of career in nuclear industry Supervision of other employees Involvement with operating plants and/or new construction Positions held throughout career Training received throughout career How maintain/refresh skills D. Current Training Requirements Involvement with training employees receive in nuclear industry Main categories of workers requiring specific training Specific job training E. View of Nuclear Industry View of nuclear industry over next 3 years View of nuclear industry over next 10 years Areas of growth/contraction Future needs for training F. Skill Level and Certifications Industry-wide certifications Peer review systems Certificates/Certifications that are important How training should be provided G. Training Providers Major training providers in nuclear industry Types of training they provide Strengths/Weaknesses How training is offered/assessed Satisfaction H. ASME Training Awareness of ASME Would you look to ASME for training? Role of ASME for training Specific type of training ASME should provide

58. Nuclear Training Opportunities Study 3. Summary of Findings Growing need for training & education in the nuclear power industry over next 5-10 years. ASME is perceived as a well-respected, well-known association with national and international awareness ASME should continue dialog with consumers of training as they build their portfolio Primary employees that need to be trained Partnering opportunities Increase in the need for international training within this industry ASME Training & Development 34

59. Nuclear Training Opportunities Study 4. Conclusions and Business Implications Continued discussions with industry players to carry on development of a Certificate Program for Nuclear Codes and Standards Additional course development within the nuclear industry in live courses, workshops and eLearning Building a network of global partners to deliver ASME courses around the world ASME Training & Development 35

60. Nuclear Training Opportunities Study 5. What would / should we do differently now? Define expectations of results Avoid designing the questionnaire with too many stakeholders Follow up survey would have been good to supplement some of the areas – more statistical focus If we knew then what we know now, would we still carry out the study? ASME Training & Development 36

61. ASME Training & Development 37

62. Workshop Participant Engagement Study Background and Purpose In early 2010 ASME launched a series of educational workshops held in conjunction with quarterly meetings of code committee volunteers Goals: Increase revenues from training and exhibit sponsorships Explore and test new topics Identify and evaluate prospective new instructors Incite interest of attendees to volunteer for participation in future code committee activities Use market research to validate achievement of Goal #iv ASME Training & Development 38

63. Workshop Participant Engagement Study Approach and Methodology Online survey software (email) to query 150 past educational workshop attendees Similar to Zoomerang or Survey Monkey software Pros: easy, quick and affordable Cons: “self-selection bias,” 10-15% response rate Questionnaire to assess participants’ reaction to, interest inand intended involvement withvoluntary code committee activities Requires skills to draft questions / analyze results Good to have trained survey research professional handle Field survey electronically, tab results and analyze ASME Training & Development 39

64. Workshop Participant Engagement Study Summary of Findings Total respondents: 38 (25.3%) Workshops: Very high satisfaction level; would recommend to friends or colleagues Reason for attending: “educational / earn CEU’s”; only 12.5% cited “to learn more about ASME Code Week Committee Activities” Result: Two-thirds of respondents attended one or more Code Committee meetings –– almost half were first time attendees ASME Training & Development 40

65. Workshop Participant Engagement Study Summary of findings (continued) Code Committee Meeting attendance: found “highly useful;” 80% said future attendance was “highly likely” Reasons for non-attendance: “scheduling conflicts” or “unaware that Code Committee meetings were open to the public” g) Strong interest in new “Introduction to ASME Code Committees” workshop ASME Training & Development 41

66. Workshop Participant Engagement Study Conclusions and Business Implications Overall satisfaction levels with Code Week Workshops – high Objective of attracting new participants to Code Committee activities – achieved Should ASME continue holding educational workshops co-located with Code Committee meetings? – yes ASME Training & Development 42

67. Workshop Participant Engagement Study What would / should we do differently now? Raise awareness among workshop registrants that Committee Meetings are “open to the public” Develop and launch new “Introduction to ASME Code Committees” workshop offering Aggressively seek and schedule “hot topic” subject matter to draw more workshop participants Encourage participants to recommend attendance to peers and colleagues ASME Training & Development 43

68. Conclusions Use research to understand your customers’ wants, needs and preferences – adjust products and services accordingly Clearly identify the objectives and purposes Research is scalable – depending upon importance, budget and time And perhaps most important… Only use research if you are prepared to act upon results – don’t waste time and resources ASME Training & Development - January 2011 44

69. Cheers! ASME Training & Development 45 Thank you! We welcome any questions or comments….. Goodbye! Russ Jackie

70. IEEE Education Research Marc Beebe Sr. Manager, Strategic Research

71. Goal for Today Case study on IEEE Education Research and how you can apply some of our lessons to your own activities 7/15/2011 47

72. About IEEE World's leading professional association for the advancement of technology 38 technical societies, 325 geographic Sections 148 transactions, journals & magazines 1,300 sponsored/co-sponsored conferences each year 1,200 active standards Approximately $390 million annual budget 400,000 members in more than 160 countries 52% in US, 48% in all other countries 25% students (102,000) 51% belong to a technical society Research Area 3 Full-time research staff Budget to hire external vendors

73. IEEE is….

74. BUT…. 7/15/2011 50

75. We still need diverse strategies for doing research More projects than internal staff can handle Lack of funding Ability to prioritize research projects 7/15/2011 51

76. Case Study: Continuing Education in Singapore

77. Business questions we wanted to answer Is there a market for Continuing Education in Singapore? What form/topic areas of continuing education would be most useful Would our members consider IEEE as a provider? What is the competitive landscape like in that country? If needed, who could we partner with? 7/15/2011 53 Member Survey Environmental Scan Interviews

78. Member Survey All non-student, non-retired member in Singapore Conducted in-house, using existing resources 364 respondents, margin of error of ±4.5%, response rate of 23.1% 7/15/2011 54

79. In-person and customized most of interest 7/15/2011 55

80. Practical Technical Topics of most interest 7/15/2011 56

81. Singapore members more interested in continuing education from IEEE 7/15/2011 57

82. Environmental Scan General background about Singapore Size of the Relevant Industry Market Continuing Education Industry structure and general trends Current state of the market Specific opportunities 7/15/2011 58

83. Overview The Singapore CET industry is a complex market with many players Private education was an unregulated industry until late 2009, with new legislation gazetted and the set up of Council for Private Education (CPE) Singapore There are the usual types of ‘bigger players’ Universities Vocational institutions (polytechnics, ITEs) Industry Associations/ Unions with a training arm Private for profit providers (e.g. Kaplan) There are many niche small providers with narrow fields of offerings E.g. 60 smaller Accredited Training Organizations (ATOs) in soft skills segment E.g. 90 smaller ATOs in professional skills segment 7/15/2011 59 Sources: NTUC Learning Hub, Company websites

84. Key Characteristics of some players in the market – tertiary institutes 7/15/2011 60 Sources: Company websites and brochures

85. Interviews Individual Interviews/Discussions with government officials and industry leaders Served two purposes: Gather information Make connections Used semi-structured interview format 7/15/2011 61

86. Gremlins to avoid Not doing research, when it’s necessary Doing research that’s not necessary Using non-representative samples Volunteers Panels that aren’t like your members Doing bad research 7/15/2011 62

87. “Bad” Research 7/15/2011 63 Via Brian F. Singh: http://yfrog.com/kkffwyeyj

88. Thank you! Marc Beebe Sr. Manager, Strategic Research IEEE marc.beebe@ieee.org +1 732.465.5891 7/15/2011 64

89. Thank you