Best Practices for Collecting Career Outcomes Data using The Outcomes Survey

1. Staying Ahead of the
Curve
Best Practices for
Collecting Career Outcomes Data Using

2. Presenters
Matt Berndt
Vice President,
Career Services & Research
CSO Research, Inc.
Max Wartel
Head of Research & Analysis
CSO Research, Inc.
25+ combined years collecting and analyzing career outcomes data
for colleges and universities

3. 3

5. Staying Ahead of the Curve
AGENDA
 The College Scorecard
 The Outcomes Survey®
 Six Best Practices for
Outcomes Data
Collection
 Q&A
 Next Steps

6.  How many
graduates get jobs
 What kinds of jobs
they get
 How much they earn

7. At One Year At Five Years
How many graduates get jobs?
What kinds of jobs do they get?
How much do they earn?

9. R&D: 2012-2013
O Monitored state and federal initiatives and NACE efforts
around first destination career outcomes data collection
O Reviewed/studied reporting standards and best practices
O Gathered input/feedback from career services professionals
nationwide
O Developed, tested and revised beta version of The Outcomes
Survey
O 16 colleges and universities joined on as Partners in
Innovation prior to launch

10. 128+ Schools in 38 States

11. National Outcomes Report
O Data at 6-months and at One Year
O Blends anonymized TOS client response
data and Chegg response data
O Students at TOS client schools do not
receive Chegg survey
The only national career outcomes report based on a nationally
standardized data collection and reporting methodology.

12. National Data

13. CAREER OUTCOMES QUESTIONS*
O Employment/Self-Employment/Entrepreneur
O Graduate School/Additional Education
O Military Service, Service Program
O Unemployment (seeking & non-seeking)
O When did you begin your search?
O Where did you look for an occupation?
ENGAGEMENT QUESTIONS (optional)
O Job search resources used . . .
O Use of career services . . .
O Helpfulness of services . . . .
EXPERIENTIAL EDUCATION (optional)
O Internships, Co-ops, Assistantships
O Field Experiences, Practicums
O Clinical Experiences, Research
O Service Engagement, Volunteering
O And more
SATISFACTION QUESTIONS (optional)
O Relation of your occupation to your major
O Importance of correlation to major
O Relation of your occupation to your career goals
O Initial satisfaction
MOTIVATION QUESTIONS (optional)
O Income
O Employment opportunity
O Prestige
O Personal enrichment
O Job satisfaction
O Ability to impact the world
CUSTOM QUESTIONS (optional & local)
O Unique and editable
O School-specific
O Limited in scope
*Exceed College Scorecard data. Address anticipated federal reporting needs.
Meet MBA CSEA & NACE reporting standards.

14. Features
O Takes from 3-7 minutes to complete
O Outcomes module only: avg. 8 questions
O All modules: avg. 20-30 questions
O Skip logic and conditional questions optimize the survey
experience
O Responses updateable throughout the entire survey period
O Graduates are prompted four times to respond/update
their responses
O Web-based and mobile-friendly
O Secure, SSL-based system for students and
administrators
O Informed Consent Statement based on IRB best
practice examples
O Real-time data collection and reporting

15. How it works
~2 Months
Prior to
Graduation
University
Provides
Student
Data
1 Month
Prior to
Graduation
Surveying
begins
At
Graduation
1st set of
responses
3 Months
After
Graduation
2nd set of
responses
6 Months
After
Graduation
3rd set of
responses
One Year
After
Graduation
Final set of
responses
Conduct The Outcomes Survey®
Update Student Data National Report National ReportImport Data
24/7 Admin Dashboard provides:
 Real-time access to alumni database, data and reporting tools
 Mass email tools, response tracking tools
 Multiple administrators/permission levels (fall 2015)
 Administrator data entry tool for 2nd and 3rd party sourced data
 Downloadable reports and data files

16. O Response Data: 30-100% expected
Key Variables: Quality of email addresses provided and historical
alumni engagement/affinity; Client engagement in data collection
process and use of TOS tools and resources
O TOS Tools and Resources
O Request my Link Tool
O Mass and Direct Email Tools & Analytics
O Best Practice Marketing Resources
O Admin-Entry and Data Upload Tools
O Optional Call Center Support (at additional expense from CSO
partner Campbell-Rinker)
O Knowledge Rate: >65% is achievable
Primary Variable: Client engagement in 2nd and 3rd party data
collection
Expectations

17. Six Best Practices
for staying ahead of the curve
1) Apply sound research methodology to ensure valid
and reliable data collection
2) Integrate data collection into graduation activities
3) Involve key influencers in awareness-building and
marketing
4) Offer incentives while student are still on campus
5) Conduct targeted follow-up with non-responders
6) Integrate into overall communication and marketing
activities and platforms

18. Best Practice #1
Apply sound research
methodology to ensure valid
and reliable data collection

19. Gold Standard Methodology

20. Our Methodology Works!
O “Gold Badge” institutions obtained response rates
as high as 100% (average response rate: 52%)
O 89% of respondents who start TOS complete TOS
O 75% of respondents provide salary/compensation
information
O 23% of respondents provide updates after
graduation
O Institutions that survey in real time obtain better
results

21. Collect multiple data types
O Response Data
Collected in a standardized manner directly from respondents
O Valid, reliable and consistent
O Defensible by Institutional Research
O Secondary Source Data
Collected indirectly and in a non-standardized manner from
faculty, family, social media, and other 2nd and 3rd parties.
O General Information
O Anecdotal and typically incomplete

22. FEMALE MALE
GENDER
NOT
REPORTED
HISPANIC/
LATINO
NATIVE
AMERICAN
ASIAN
BLACK/
AFRICAN
AMERICAN
NATIVE
HAWAIIAN
/ PACIFIC
ISLANDER
WHITE/
CAUCASIA
N
INTERNATI
ONAL
MULTIETH
NIC/
OTHER
NOT
LISTED
ETNICITY
NOT
REPORTED
POPULATION (N= 62,102) 47.0% 30.1% 22.9% 7.7% 0.7% 5.1% 5.6% 0.5% 48.4% 4.3% 6.4% 21.3%
SAMPLE (n = 12,687) 55.9% 27.2% 16.8% 8.5% 0.6% 3.6% 5.8% 0.5% 51.8% 3.3% 5.6% 20.4%
0%
10%
20%
30%
40%
50%
60%
CLASS OF 2014-2015 DEMOGRAPHICS OF RESPONDENTS AS OF GRADUATION
Analysis shows respondent demographics are representative of the student population.
By gender, ꭓ2
(df = 2) = 3.59, p > .10 By ethnicity, ꭓ2
(df = 8) = 1.99, p > .975
Collect representative data

23. Best Practice #2
Integrate data collection into
graduation activities

24. Flagler College

25. Roger Williams University

26. Best Practice #3
Involve key influencers in
awareness-building and
marketing

27. Northeastern State University

28. Strategies
O Engaging a C-level champion (President,
Provost, Vice President, Dean, etc.)
O Involving key departments in survey design
and custom question development
O Sending letters/emails from department
heads and favorite faculty encouraging their
students to respond
O Generating department/major-specific
reports to demonstrate value

29. Best Practice #4
Offer incentives while
students are still on campus

30. O Tabling in advance of
graduation
O Coffee mug &
goodies giveaway
O Use of “Request My
Link” tool

31. Prize Drawings

32. Best Practice #5
Conduct targeted follow up
with non-responders

33. Mass Email Tool

34. O SJFC reported a 5-10% boost in response
rates for each interim follow up
O Mass mailing tools allow rapid follow up
with non-responders between scheduled
emails
O Follow up can be customized to for
different majors, degrees, departments,
and more
Targeted Follow up

35. Best Practice #6
Integrate into overall
communication and
marketing activities and
platforms

36. Message Integration

37. Email signatures
Including the “Request my Link” URL in email
signature blocks

38. Six Best Practices
1) Apply sound research methodology to ensure
valid and reliable data collection
2) Integrate data collection into graduation
activities
3) Involve key influencers in awareness-building
and marketing
4) Offer incentives while student are still on
campus
5) Conduct targeted follow-up with non-
responders
6) Integrate into overall communication and
marketing activities and platforms

39. Usage/Purpose
 Accreditation
 Admissions recruiting
 Advising: Academic and career
 Alumni relations
 Curriculum development
 Media inquiries
 Reporting: Federal, State,
Association, Other
 Strategic Planning
Reporting Intervals
 Before Graduation
 At Graduation
 At 3 Months
 At 6 Months
 At One Year
 3-, 5-, 10-Years+
Define Your School’s Needs & Priorities | Tell Your Graduates’ Stories
Collection Standards
 Response Data
 Secondary Source Data
 IRB Approval/Sign-off

40. Questions

41. The National Standard
for Career Outcomes Data
Collection
outcomes@csoresearch.com
1-866-705-4201, ext. 3