Outline of evaluation approach focus on networking

1. ULearning Skillnet
Overview of Evaluation Approach
Rob Cosgrove & Kieran Swail
Monday 8th September 2008
Shannon Business Park

2. Role & Approach
• Role = monitor, evaluate and report
• Approach = Listen, observe, engage,
advise, recommend and review

3. Scope of evaluation
• 2 reports Interim early 2009, Final early
2010
• Key data & indicators set by Skillnet
– Mandatory, Strongly recommended, Optional
– Kirkpatrick scale of evaluating training Level
1-3 & Phillips Level 5 see Guidelines handout.
– Innovation & Networking = Opportunity to
stand out

4. Phillips ROI Model
The Phillips ROI methodology
is a 10-step process preferred
by Skillnets
We need to select suitable
course/modules
Consider Impact Measurement
Skillnet Training for this
initiative and next round.

6. Proposed ROI Method
• Identify module(s) to target for measurement
• Criteria
– Multiple students doing same module in same
company failing that – similar companies closely
spaced
– preference for larger companies
– preference for modules with concrete skill
deliverables
– Interview module lecturer for modules chosen for
ROI measurement.
– In depth interviews with reps from each provider
– In depth interviews/ liaison with trainers in
modules selected for ROI assessment

7. Progress to date
• Attended Launch in Shannon
• Met with Project Co-ordinator and
members of Steering Group in Dublin
• Met with Skillnets Liaison Officer in
Dublin
• Evaluators met and agreed joint
approach

8. Methodology / Outputs to Interim Report
• Attend Steering Group Meetings Sep, Oct – Dec 08
• f2f Consultations with select no of network member
companies Oct – Dec 08
• Review Lead Generation Process & General Marketing
Activity
• Identify course/module (s) for ROI assessment
• Students/Trainees
– Pre Module Start
• Subset 10-15 minute telephone interviews
• eSurvey or integrate with AUA pre- commencement data
capture
– Interview Student Cohort Nov 08
– Review End Module feedback sheets
• Draft Report Dec 08 & Interim Report Feb 09

9. Added Value & Support
• Wiki
• Spot Evaluation of an event - pre & post
survey – process and content review Nov/Dec
08?
• Ideas
– Calendar of Events Network Activities &
promotional activities
– Online Forum?
– Analysis of sales funnel – lead generation to sales
conversion

10. Risk Area Severity Probability Mitigation
ROI may not be easily Evaluation Moderate High Select modules for
measurable via Phillips. ROI assessment
Degree level ROI less with care.
tangible
ROI slower to accrue
Softer skills harder to
measure credible ROI.
Training days targets may Marketing High High
not be hit.
Network may seem Administrative ?? Moderate Ensure network
supply led rather than members are the
demand led – a sales key driver of
channel for universities. course content –
not providers.
Regional Focus on the Marketing High Moderate Retarget marketing
west misses key markets activity to Greater
– Dublin - is less effective Dublin Area
Marketing focus Marketing High Moderate Monitor & Evaluate
unstructured – not leading marketing/sales
to closed sales. process

11. Q How do you attract new Apply W5
network members and trainees? Who?, What?, When?,
Where?, Why?
Trainees
• Telephone – cold v warm / Network Members
• Text Mkting -Marketeers call the •Network - Regular themed events
3rd screen •Branding & Merchandising
• Direct Mail – postcard mkting •Trade shows, Exhibitions
• Competitions
•Discussion Forums HR/CPD
• Demos - “wow” factor
• PR / Word of Mouth
www.linkedin.com
• Testimonials / Referrals •Conferences
• Web Marketing – Web 2.0 •Market Research
– email, blog, podcasts, social •Joint Venture e.g. IRCSET - the Irish
networking, Flickr YouTube, Research Council for Science,
Facebook, Bebo, MySpace RSS
feeds.. Engineering and Technology
• Advertising •Journalists Visits – education, science
– Radio, newspapers,
supplements, journals. Local & technology
business circulars, Magazines, •Sponsorship
Billboards, Google Adwords,

12. ULearning Skillnet
Thankyou
Rob Cosgrove & Kieran Swail
Monday 8th September 2008