SEO by Hypothesis

SEO BY HYPOTHESIS
Recommend → Hypothesise & Test
@TomAnthonySEO

Cooking nuts in mud
Replicated it, but devil in details
Google black box - can’t always replicate
#dmsso@TomAnthonySEOwww.madeinindia.tv

recommendation!
WE GET IDEAS FROM ALL OVER…
#dmsso@TomAnthonySEO
blog
post
worked
before
colleague
google
guidelines
new
idea
tried for
another client

FROM THERE, WE MAKE RECOMMENDATIONS
“Add ‘Free Shipping’ to title tags.”

DO WE UNDERSTAND
THE IMPACT
OF THESE CHANGES
?

WE SURVEYED 2000+ SEOS…
future of tv advertising
Which URL ranks higher for the search:

MORE EXPERIENCED SEOS DO BETTER…
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO

… BUT WE DON’T DO GREAT
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip

WE CANNOT PREDICT
WHAT RANKS
WITH ANY ACCURACY
!

WE’VE BEEN A/B TESTING…
NBED - we have started testing

RECOMMENDATION → TEST
“Add ‘Free Shipping’ to title tags.”
“Let’s test ‘Free Shipping’ in title tags.”

NOTHING HAPPENED

Before
After
query search volume
which tv should i buy 250
query search volume
which tv to buy 1,300

THAT MADE THINGS WORSE

NADA… NOTHING… NO IMPACT…

“Stand back guys,
I got this.”

OH GOD…

COULD BE OUR
RESULTS
AREN’T ACCURATE
?

NOPE, THE PLATFORM IS SOLID. WE SUCK.

HARDER THAN ANTICIPATED TO PICK WINNERS

CAN WE MAKE MORE
EFFECTIVE
RECOMMENDATIONS
?

LAST YEAR: RECOMMENDED TESTING

FOLLOW THE CRO EXPERTS - dis.tl/cro-hypothesis

MEASURE
THE RESULT
GATHER
SOME DATA
BUILD A
HYPOTHESIS

IT REQUIRES BEING SPECIFIC
Changing <abc> will have an impact on <xyz>
& thus improve organic search performance.
Based on <evidence>, we hypothesise…

FOR EXAMPLE..
Changing <title formats, by…> will improve <ctr>
& thus improve organic search performance.
Based on <survey data>, we hypothesise…

HYPOTHESIS BENEFITS
Made to be more consistent.
It ensures there is clarity.
To support it we need data.

Control
Influence
We have the same
data as Google.
Google has data
that we don’t.

Control
Influence
Keyword
targeting
Internal
links
We have the same
data as Google.
Google has data
that we don’t.

Control
Influence
Website
‘quality’
We have the same
data as Google.
Google has data
that we don’t.

Control
Influence
Usage
Data
We have the same
data as Google.
Google has data
that we don’t.

Control
Influence
External
Links
We have the same
data as Google.
Google has data
that we don’t.

Control
Influence
External
Links
Usage
Data
Keyword
targeting
Internal
links
Website
‘quality’
We have the same
data as Google.
Google has data
that we don’t.

Control
Influence
External
Links
Usage
Data
Keyword
targeting
Internal
links
Website
‘quality’
Test &
Model
We have the same
data as Google.
Google has data
that we don’t.

Control
Influence
External
Links
Usage
Data
Keyword
targeting
Internal
links
Website
‘quality’
Survey &
Study
We have the same
data as Google.
Google has data
that we don’t.

Control
Influence
External
Links
Usage
Data
Keyword
targeting
Internal
links
Website
‘quality’
Analyse
Better
We have the same
data as Google.
Google has data
that we don’t.

INFORMATION ARCHITECTURE

INFORMATION ARCHITECTURE
• Few sites are this simple.
• Can’t measure impact of
changes.

CURRENT BEST APPROACH
Calculate internal PageRank
Use interactive visualisations

VISUALISATIONS - USEFUL, BUT RARELY
Credit: Paul Shapiro

INTERNAL PAGERANK - POWERFUL, BUT QUIRKY
Credit: Paul Shapiro

PEOPLE SEE REAL RESULTS FROM BIG CHANGES TO IA
See Alex’s fascinating Mozcon talk [PDF]

BUT REAL-WORLD
CHANGES ARE:
HARD TO MAKE &
HARD TO UNDO
!

OUR INTUITION
IS REALLY BAD.
!

WE ASKED PEOPLE, GIVEN A SITE…

ADD A NEW PAGE - LINKED FROM HOMEPAGE

IT LINKS TO SAME PAGES AS HOMEPAGE

WHAT HAPPENS TO PAGERANK?
PageRank
Change?

WILL ASKED ON TWITTER…

…3/5 PEOPLE GOT IT WRONG
NOPE

GIVEN THAT,
HOW CAN WE
HYPOTHESISE
THE IMPACT?
?

FETCH & NORMALISE MOZRANK
Raw MozRank Normalised MozRank
3.67E-13 1.0%
3.35E-11 94.2%
1.71E-13 0.5%
1.64E-13 0.5%
1.59E-13 0.4%
3.28E-13 0.9%
6.88E-14 0.2%
2.45E-13 0.7%
7.12E-14 0.2%
3.12E-13 0.9%
1.67E-13 0.5%

OVERLAY MOZRANK ON TO INTERNAL PR ALGO
8.2
5.3 4.3 4.8

MODEL EXACTLY WHAT WOULD HAPPEN
8.05
5.3 4.3 4.8
4.05

FIND THE WHOLE PROCESS HERE
http://dis.tl/ia-pagerank

Website
‘quality’
Survey
& Study
Usage
Data

USAGE DATA: WE CAN’T GUESS
It is impossible to guess what
people prefer.

SERP TURKEY

• How do you communicate it to clients / bosses?
• How do you benchmark it against competitors?
• How do you figure out if proposed changes improve it?
QUALITY?

Gather human rater
information
Google employs
thousands of human
quality raters to answer
questionnaires about
many kinds of website

Train ML models
Google uses the human
questionnaires as training
data for ML models of
“quality”
Gather human rater
information
Google employs
thousands of human

2011
Panda released
The Panda quality
algorithm starts being
used as a batch process
modifying the regular core
algorithm
Train ML models
“quality”
Gather human rater
information
Google employs
thousands of human

2011
Panda released
The Panda quality
algorithm starts being
used as a batch process
modifying the regular core
algorithm
Train ML models
“quality”
Gather human rater
information
Google employs
thousands of human
2016
Panda goes real-time
“Quality” becomes a first-
class ranking factor in the
core algorithm

QUALITY - RANKING FACTOR IS A PROXY
Google use Machine Learning to
predict what we can just ask
people.
Actually
measure: 
ML PROXY
FOR QUALITY
Want to
measure:
QUALITY

2011: WILL SUGGESTS RUNNING PANDA SURVEYS

PANDA SURVEY
Client site 1 Client site 2
Would you trust information from this website? 72% 64%

PANDA SURVEY
Client site 1 Client site 2 Key competitor
Would you trust information from this website? 72% 64% 81%

PANDA SURVEY
Is this website written by experts? 50% 46% 65%

PANDA SURVEY
Is this website written by experts? 50% 46% 65%
Would you give this site your credit card details? 29% 21% 43%
Are there any noticeable errors on this page? 6% 4% 1%
Does this page provide original content or info? 76% 72% 85%
Would you recognize this site as an authority? 44% 33% 58%
Does this website contain insightful analysis? 72% 62% 81%
Would you consider bookmarking pages on this site? 44% 38% 56%
Are there excessive adverts on this website? 2% 2% 8%
Could pages from this site appear in print? 54% 54% 59%

DECK - HOW YOU CAN DO SPLIT-TESTING
A
GUIDE TO
SPLIT-TESTING
THE NEXT
ERA
OF SEO
dis.tl/diy-ab-test

HYPOTHESIS
CATEGORY PAGE HTML IS VERBOSE,
AND CONTAINS A LOT OF CRUFT.
ADDING A LIST OF PRODUCTS AS
STRUCTURED MARKUP WILL MAKE
THE INTENT CLEARER.

ADD STRUCTURED MARKUP

AN 11% UPLIFT

REPEATED IN ANOTHER REGION - 11% AGAIN

THE HYPOTHESIS DID NOT
FIT THESE OTHER SITES.
!

FINAL EXAMPLE - FRESHNESS

COMPETITORS - MORE FREQUENT DATE ANNOTATIONS

GOOGLE IDENTIFIES FRESHNESS AS
IMPORTANT IN THESE RESULTS
REGULARLY MAKING SMALL UPDATES
FOR THIS CLIENT’S CONTENT WILL HELP
GOOGLE RECOGNISE FRESHNESS.
HYPOTHESIS

ADDED & UPDATED DATE MODIFIED
<time itemprop="dateModified"
datetime="{{date}}">
Updated on: {{date}}
</time>

AN 8% UPLIFT

THE MAJORITY OF SEO
RECOMMENDATIONS
HAVE LITTLE IMPACT
OBSERVATION

NULL/NEGATIVE CHANGES SLOW YOU DOWN

EVEN EXPERTS STRUGGLE
TO PREDICT THE IMPACT
OF CHANGES
OBSERVATION
ME INCLUDED!

PAGERANK IS ITERATIVE & NOT INTUITIVE
NOPE

IN ISOLATION RANKINGS ARE HARD TO PREDICT
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip

MORE IMPACT & QUICKER RESULTS
RUN TESTSBUILD A HYPOTHESISGATHER DATA

BEGIN TESTING, START HERE…
distilled.net/diy-splittester

BEGIN HYPOTHESISING, START HERE…
http://dis.tl/pandaQ
http://dis.tl/serpturkey
http://dis.tl/ia-pagerank

THANK YOU!
@TomAnthonySEO @WillCritchlow @Dom_Woodman

SEO by Hypothesis

Recommandé

Recommandé

Contenu connexe

Similaire à SEO by Hypothesis

Similaire à SEO by Hypothesis (20)

Plus de Tom Anthony

Plus de Tom Anthony (18)

Dernier

Dernier (20)

SEO by Hypothesis