Many of today’s websites have recognised the importance of mobile friendly pages to keep users engaged and to provide a satisfying user experience. However, next to the experience provided by the sites themselves, advertisements, when clicked, present users with landing pages that are not necessarily mobile friendly. We explore what type of features are able to characterise the mobile friendliness of sponsored search ad landing pages. To have a complete understanding of the mobile ad experience in terms of layout and visual appearance, we also explore the notion of the ad page aesthetic appeal. We design and collect annotations for both dimensions on a large set of ads, and find that mobile friendliness and aesthetics represent different notions. We perform a comprehensive study of the effectiveness of over 120 features on the tasks of friendliness and aesthetics prediction. We find that next to general page size, HTML, and resource usage based features, several features based on the visual composition of landing pages are important to determine mobile friendliness and aesthetics. We demonstrate the additional benefit of these various types of features by comparing against the mobile friendliness guidelines provided by W3C. Finally, we use our models to determine the state of landing page mobile friendliness and aesthetics on a large sample of advertisements of a major internet company. These are the slides of work presented at WWW 2017 in Perth: M. Bron, M. Redi, F. Silvestri, H. Evans, M. Chute and M. Lalmas. Friendly, Appealing or Both? Characterising User Experience in Sponsored Search Landing Pages, 26th International World Wide Web Conference (WWW 2017), Industrial Track, Perth, Australia, 3-7 April, 2017.

1. Friendly, Appealing or Both? Characterising User
Experience in Sponsored Search Landing Pages
Marc Bron, Miriam Redi, Fabrizio Silvestri, Hue Evans, Mahlon Chute and Mounia Lalmas

2. Motivation and goals
• Bad post-click experience with ads results in loss of daily active users
and as a consequence revenue
• Two dimensions: mobile friendliness and aesthetic appeal
• Help advertisers improving quality of their ads
● create ground truth
● develop features and learn classifliers
● understand the post-click experience
Search ads

3. What is “mobile friendliness”?
A web page is mobile friendly if it
has a good user experience on a
mobile device.
Good experience is a combination
of great performance and mobile
specific experience.

4. What is “ad mobile friendliness”?
A web page that is mobile friendly as previously defined
and
it makes it easy for the user to understand what the ad is
about (e.g. the product advertised) and allows the user to
convert (e.g. purchase the product advertised).

5. What we did
Features
Learning a
classifier
editorial
guidelines
Ground truth
Aesthetics
appeal
HTLM
Visual
Random forest
700,000 ad search landing pages:
Bad: 6% Fair: 50% Good: 35% Excellent: 9%
What does this
tell us?

6. Ground truth

7. Mobile Friendliness – 4 levels
EXCELLENT GOOD FAIR BAD
Mobile-optimized
Conversion easiness
Good product/
business experience
Mobile optimized
Simplified navigation
Good readability
High usability
Missing good mobile friendliness criteria
Fair: missing ONE criteria
Bad: missing 2 OR MORE criteria (e.g. a desktop page!)

8. Aesthetically pleasing checkbox
● Good color and shape harmony
● Photographically beautiful images (if any)
● Colors and fonts tend to harmonize with the theme of the site, without being centers of attention
● The layout of each page tends to direct eye movement to the key part of the page

9. Annotation results
4,025 landing pages
63% agreements on mobile friendliness
Of the 37% disagreements:
3% by one rating level
5% by two or more rating levels
2% Not Judged rating (page not longer present)
5% aesthetically pleasing
Spearman's rho = 0.2 between mobile friendliness
and aesthetically pleasing (p < 0.001)

10. Features

11. Family of HTML features (63)
• Mobile optimized: whether a page is specifically designed for mobile or
whether it is a desktop page.
• Window size: aspects of the size of the rendered html image and used
to detect whether the size of a page is suitable for mobile devices.
• Readability: identify the formality of the language used in the landing
page text. Intuition is that dense and formal texts may be less pleasant to
read on a mobile device.
• Input: number and type of input elements in a page. Intuition is pages
that require users to provide information through many forms may be
considered less mobile friendly.
• Navigation: proportion of internal links, external links, and text contained
in a page. E.g., mobile friendly pages may provide access to different
sections of a page through internal links.

12. Family of visual features (66)
Color Distribution
Features:
Hue, Saturation, Brightness
Rule of Thirds: Image
Composition and
Layout
Emotional Response
Features:
Pleasure, Arousal, Dominance
Depth of Field: Sharpness
contrast between
foreground and background
Objective Quality
Features:
Sharpness, Noise, JPEG
quality, Contrast Balance,
Exposure Balance

13. Learning classifiers

14. Classification tasks
ad landing pages
1,2,3,4
0/1
Random forest
split the data
based on
advertisers
10-fold cross
90% and 10%

15. Results

16. Feature family and level of mobile friendliness
Feature family AUCw AUCbad AUCfair AUCgood AUCexcellent
readability
navigation
input
window size
mobile optimised
layout
texture & contrast
color
image quality
.589
.642
.674
.731
.752
.707
.706
.727
.727
.518
.695
.682
.850
.797
.825
.741
.795
.855
.662
.677
.697
.741
.769
.722
.734
.766
.719
.553
.566
.618
.707
.739
.668
.662
.676
.696
.786
.626
.661
.629
.687
.620
.632
.623
.627
all .788 .890 .800 .762 .693
Easier to predict bad then fair, less easy to predict excellent.

17. Aesthetics
HTML feature
category
AUCw Visual feature
category
AUCw
input
navigation
mobile optimised
readability
.541
.551
.566
.570
layout
quality
texture
color
.633
.680
.688
.695
Using all features: AUCw = 0.661, which is compared to state-of-the-art
on image beauty classification in computational aesthetics research

18. Feature analysis

19. Mobile friendliness
• Most discriminative features
related to window size +
mobile optimized
• Cleaner and less pixelated
pages are more mobile
friendly
• Low quality pages more
often have very bright
colour combinations
• Presence of a small number of
bright colors with an otherwise
moderate use of brightness
associate well with mobile
friendliness

20. Aesthetics
• Presence of objects in
certain focus areas
better for aesthetic
appeal
• Aesthetic appeal
mostly explained with
visual features, in
particular image
brightness
• Too bright pages are
not good, while pages
with mild brightness
more aesthetically
pleasing

21. Accounting for mobile friendliness &
aesthetic can provide easier
navigation & inspiring and
convincing ad experience
Mobile friendliness vs. aesthetic appeal
HTML features
• mobile optimized and structure important
for mobile friendliness
• width image negatively correlated to mobile
friendliness, but positively correlated to
aesthetics
Visual features
• smooth and rougher pages are
aesthetically pleasing
• symmetry very important for mobile
friendliness
• importance of brightness for aesthetics less
prominent compared to mobile friendliness
• uniqueness more important for mobile
friendliness than aesthetics

22. Study dimensions of post-click experience for search ads, mobile
friendliness and aesthetic appeal à to “consume” the ad
Good and excellent ad landing pages positively associated with long clicks
but not necessarily with conversions.
Few visually attractive landing pages but these were positively associated
with conversions.
A/B testing with bad landing pages filtered out
decrease of -6.04% in short clicks
increase of +1.15% in ad click-through rate
Final results
700,000 ad search landing pages:
Bad: 6% Fair: 50% Good: 35% Excellent: 9%

23. Examples of excellent mobile friendly and
aesthetically appealing landing pages
keep it simple, with one nice background, and a simple call to action