5. Fantasy Football Today explodes*
• Live, streaming fantasy football show Week 1 Week 2 W/W Growth
featuring expert advice & analysis Streams 2x,xxx Xxx,xxx 2527%
• Auto-played Minutes 2xx,xxx 1,xxx,xxx 362%
• Streams from 11:00am – 12:30pm, EST each Minutes/Stream XX.X X.X -82%
Sunday on FFT Microsite & Commissioner
league home pages
FFT Microsite Commissioner League Home
• Few streams (new show, low traffic • Not available in Week 1
page, developing audience) • Lots of streams (high traffic page)
• High engagement. Users come to watch • Low engagement (users come to set
video; they watch longer lineup, not watch video)
• Possible bad user experience (forced to turn
off video if not interested)
Source: Akamai
6. FFT: the numbers
Content Starts 9/7/2008 9/14/2008 W/W Growth The addition of the Fantasy
FFT Main Page Xx,xxx Xx,xxx 32% Football Today video on
FFT League Home 0 Xxx,xxx -
FFT Total Xx,xxx Xxx,xxx 2527%
league home pages increased
streams to Xxx,xxx in week 2
Minutes 9/7/2008 9/14/2008 W/W Growth (+2527%)
FFT Main Page Xxx,xxx Xxx,xxx -17%
FFT League Home 0 Xxx,xxx -
FFT Total Xxx,xxx X,xxx,xxx 362%
Streaming minutes also
Estimated Viewers 9/7/2008 9/14/2008 W/W Growth increased to over 1MM in
FFT Main Page Xx,xxx Xx,xxx 24% week 2 (+362%)
FFT League Home 0 Xxx,xxx -
FFT UV Total * Xx,xxx Xxx,xxx 2172%
Minutes per
Mins/Content Start 9/7/2008 9/14/2008 W/W Growth
FFT Main Page XX.X X.X -37% stream, however, decreased -
FFT League Home - X.X - 82% W/W. This is likely the
FFT Total XX.X X.X -82% result of users coming to
their leagues to set their
*UV total aggregates viewers from the FFT main page & the League Home
Page (i.e. assumes no duplication) lineups, not watch video
Source: Akamai
8. Slow start gives way to strong week 2
Mobile Week 1 Y/Y Growth Week 2 Y/Y Growth
Avg. Daily Unique Visitors Xx,xxx -15% Xx,xxx 22%
Visits Xxx,xxx 5% Xxx,xxx 13%
Page Views X,xxx,xxx 0% X,xxx,xxx 35%
Total Mins X,xxx,xxx N/A X,xxx,xxx N/A
Minutes/Visit XX.X N/A XX.X N/A
Page Views/Visit XX.X -5% XX.X 19%
Minutes per visit
Avg. daily unique visitors Though flat week 1, increased from XX.X in
decreased 15% Y/Y in page views during week week 1 to XX.X in week
week 1, but rebounded two reached X.X MM 2. Minutes per visit on
+22% Y/Y in week 2 (+35% Y/Y) Sundays were nearly 2X
that of other days
Source: Omniture
9. 08/31/08
09/01/08
09/02/08
TV
09/03/08
09/04/08
Thursday
09/05/08
Season Opener
09/06/08
09/07/08
Mobile
Week 1
09/08/08
Sunday, Monday
09/09/08
09/10/08
Football forges cross platform synergy
09/11/08
09/12/08
Internet
09/13/08
09/14/08
Week 2
09/15/08
Sunday, Monday
11. Outline of Contents
Purpose of the study
Participants
Methodology
Observations
Usability survey results
Next steps
11
12. Purpose of the study…
• To establish a usability and user experience baseline of
CBSSports.com
• To establish a foundation for a method of continued
observation and evaluation
• To discover strengths and weaknesses of the current website
• To provide recommendations for design alternatives
12
13. Outline of Contents
Purpose of the study
Participants
Methodology
Observations
Usability survey results
Next steps
13
14. Participants
• 7 participants (all male) were recruited by CBS Television City, NV based on
their interest in Sports and Fantasy products and membership.
• 4 Fantasy users and 3 non-Fantasy users were recruited.
• Fantasy memberships on sports sites included:
Yahoo (3 participants)
CBS (2 participants)
ESPN (1 participant)
Fox (1 participant)
• All participants were comfortable and competent using the Internet.
• Average age: 31 (min=21, max=43)
• Education: Post Graduate (2); College (2); Some College (1), High School (1)
14
15. Outline of Contents
Purpose of the study
Participants
Methodology
Observations
Usability survey results
Next steps
15
16. Methodology
• Participants were greeted by a moderator and given a verbal overview of
the study.
• To begin, each participant was given an opportunity to freely explore and
interact with the site – while doing so they were asked to talk aloud about
their thoughts and reactions.
• Next, participants were directed to complete a set of tasks, for example:
Choose and read a story
Choose and play a video
Interact with the video player controls
Do a Search for a topic of interest
Get Sports information (team, schedule, scores, player stats)
Complete registration
Engage Community features
• At the end of the session, a usability survey was given to each participant
asking them to rate certain design statements about CBSSports.com.
16
17. Outline of Contents
Purpose of the study
Participants
Methodology
Observations
Usability survey results
Next steps
17
18. Observations – Key Takeaways
• Participants who are Fantasy oriented (be it on CBS or somewhere else)
rated the site much higher than non-Fantasy oriented participants.
• Non-Fantasy participants tended to be negatively impacted by the amount
of Fantasy information on the site. These participants were simply
interested in getting general information about their team or sport –
especially true on the Video page.
• Video content (main thumbnail and page links) were not readily apparent
to all participants.
The Presence of a consistent
pre-roll ad clearly a buzz kill but
accepted as standard practice.
• Player controls responded as expected and were generally very easy to use.
18
19. Observations – Key Takeaways, cont…
• Participants liked the prominent stack of headlines on Home page.
• Main story thumbnail navigation confused some participants.
• Most participant assumed that clicking on the big picture would open the full story – not
so. Others thought the story would open simply on one click.
• Some confusion occurred when a participant clicked on the small thumbnail and the big
picture did not match. “Did I click on the right picture?”
• The link for the full story is hidden in the text description – hard to find/see.
19
20. Observations – Key Takeaways, cont…
• Basic registration was considered straightforward and typical.
• But, after registering; participants often stumbled through the process for:
– Setting up and using My Quicklinks
– Customizing the site to highlight Your teams and players
• The presence of Community was readily apparent to participants.
• Orange comment callouts were understood but participants thought they
should be clickable.
• Navigating to Community aspects was relatively easy.
20
21. Observations – Key Takeaways, cont…
• Search may be an area for improvement as you will see by the
usability survey ratings that follow.
• Depending on the query entered, Search returned credible
results.
• A participant’s initial keyword search often under-delivered;
however, subsequent searches were much better - trial and
error learning.
• Non-Fantasy participants were somewhat perplexed
(sometimes put off) by the amount of Fantasy results returned
by a query.
21
22. Outline of Contents
Purpose of the study
Participants
Methodology
Observations
Usability survey results
Next steps
22
23. Usability Survey Results
Survey Ratings: Fantasy vs. Non-Fantasy Participants
Completely 7 Fantasy and non-Fantasy participants
Agree
rated CBSSports.com very differently!
6
On average, non-Fantasy ratings
across all survey metrics are 1.5 points
5 lower compared to Fantasy ratings.
Average Rating
Good News: Visitors that are Fantasy
4 oriented are served fairly well.
3 Bad News: Visitors to the site looking
for general information (not Fantasy)
are not being served nearly as well.
2
Completely Non-Fantasy participants
Disagree 1 rated the apparentness of
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14
video, navigation, and the
Fantasy 5.5 6.8 5.5 6.3 4.5 6.0 6.5 6.8 6.5 6.0 5.5 5.3 5.0 6.3 organization of the site the
Overall 4.9 5.9 4.3 5.7 4.1 5.1 5.3 6.0 5.9 5.6 5.1 4.7 4.6 5.7 lowest.
Non-Fantasy 4.0 4.7 2.7 5.0 3.7 4.0 3.7 5.0 5.0 5.0 4.7 4.0 4.0 5.0
Fantasy participants also
Survey Questions: 1. The amount of information on the Home page is just right. rated navigation the lowest
2. The design of Home page enables me to easily get the Sports information I want. but not as severely (maybe
3. Videos on the Home page were readily apparent to me. due to site familiarity).
4. The Video player controls responded exactly as I expected.
5. I had no difficulty navigating the web site.
6. The website is interesting and engaging to me. Overall participants rated the
7. The organization of information on the Home page is just right.
design of the Home
8. It was easy for me to get information on my favorite Sport.
9. It was easy for me to get information on my favorite Team. page, video player
10. Getting the score for my favorite Team was easy. controls, and ease of getting
11. Getting stats for my favorite Player was easy. favorite sports team
12. The Search feature enabled me to get the information I wanted. information the highest.
13. The Search results returned (based on my keyword/phrase) matched my expectations.
14. The amount of advertising on the Home page is just right: 23
24. Usability Survey Results, cont…
Survey Question: Thinking about your interests and needs in Sports Survey Question: Based on the experience you had on the web site you
entertainment, how does this web site fit you? just used, would you recommend it to a friend?
Completely 7 Would definitely 7
Fits me 6.5 recommend
6 5.8 5.6
6 5.6 5.3
Average Rating
Average Rating
5 5
4.3
4 4
3 3
2 2
Completely Would definitely
Does not fit me 1 NOT recommend 1
Fantasy Non-Fantasy Overall Fantasy Non-Fantasy Overall
• Again, the difference between Fantasy • Participants in general might or would
and non-Fantasy participants probably recommend CBSSports.com.
demonstrates that CBSSports.com is a
better fit for Fantasy users. • The difference between Fantasy and
non-Fantasy participants is not as
strong regarding the propensity to
recommend CBSSports.com.
24
25. Outline of Contents
Purpose of the study
Participants
Methodology
Observations
Usability survey results
Next steps
25
26. Next Steps
• Review findings with the Sports team.
• Prepare a more detailed analysis and presentation
• Discuss strategy of website – how do we improve user
experience?
• Discuss design alternatives.
• Set a schedule for iterative Usability tests.
26
28. 0 to 30 days
• Interview key sports personnel Kint, Herde, Ruhl, Swanson, etc. to understand
current reporting, must have data and future needs.
• Reach out to Sales (Calacci & Fixer) to determine immediate needs. Research team
participating in weekly Sales calls. Move towards proactive deliverables (industry
one sheets, website & product strengths/weaknesses comparisons, etc.) Leverage
Magid industry study findings to close deals (Magid presenting next week)
• Quick, meaningful win. Commissioner league conversion project
31-90 days
• Revamp current report suite(s) to better-align w/ BU reporting needs.
• Primary research around community
• Primary research around usability (is the redesign working as intended. Test new
concepts)
91 days +
• User segmentation project. Understand how users migrate from segment to
segment in the value chain
