Zipf's Law is prevalent throughout many forms of data and that includes the internet at large and within sectors of the internet, websites and web pages plus linguistics. How does this impact SEO if at all?

1. Does Zipfian Distribution
Explain SEO?
Presented by: Dawn Anderson
#Pubcon@dawnieando

2. Does%Zipfian%
Distribution%
Explain%SEO?
@dawnieando

3. We#Know#
That#SEO#Is#
All#About
Words
Links
Crawling
Indexing
Ranking
Importance
Popularity
Probability;determination
Disambiguation

4. There%is%a%
phenomenon%
‘hiding%in%plain%
sight’%(blending%in)%
which%impacts%all%of%
these%elements

5. Zipf’s'Law'+ The'Principle'of'
Least'Effort'is'Everywhere

6. We#know#words#&#linguistics#have#
unusual#patterns#e.g.
“You#shall#know#a#word#by#the#
company#it#keeps”#(Firth,#1957)#(coE
occurrence)

7. Zipf’s Law*is*another*
one*of*those*strange*
linguistic*phenomena…*
but*MUCH*more*than*
that

8. Information*Retrieval*Lecture*2 University*
of*Freiburg*– Lecture*1*– Zipf’s Law

9. Vsauce Video*– The*Zipf Mystery*5 You*should*watch*it

10. Mind%blown

11. So…$Just$What$is$Zipf’s Law?

12. Zipf’s Law*is*about*
frequency*distribution,*
rank*law,*&*is*often*
associated*with linguistics

13. Zipf’s Law*is*Prevalent*
When*It*Comes*to*
‘Popularity’*(Rank)*
Distributions

14. Zipf’s Law*is*
Empirical*(Widely*
accepted).*Not*
just*a*theory

15. Popularised by.George.
Kingsley.Zipf
• George&Kingsley&Zipf (1902&5 1950)
• Linguist&&&philologist
• Popularised ‘Zipf’s Law’&but&did&not&claim&to&have&discovered&it
• Observed&patterns&in&word&frequency&distribution
• Extended&Zipf’s Law&to&data&types&other&than&words
• Developed&‘Principle&of&Least&Effort’&(Path&of&least&resistance)
• Image&attribution:&Freeport&High&School,&Freeport,&Illinois.,&Public&
domain,&via&Wikimedia&Commons

16. ‘Normal’)is)
definitely)
something)
which)Zipf’s
Law)is)NOT

17. A"‘Standard"Deviation"Curve’"(Normal"Distribution)
This%file%is%licensed%under%the Creative%Commons Attribution%2.5%Generic license.

18. In#a#‘Normal’#Distribution
Very%few%observations%with%very%low%values
Quite%a%Few%observations%with%%low%values
Many%observations%with%average%values
Quite%a%Few%observations%with%high%values
Very%few%observations%with%very%high%values

19. AKA#‘A#Bell#
Curve’

20. A"Zipfian"
Distribution"is"VERY"
Different

21. A"Zipfian"Distribution"is"Very"
Skewed"&"Heavily"Biased

22. ‘Zipfian’)Distribution
Very%few%observations%with%very%high%values
The%most%frequent%item%(e.g.%word)%is%twice%as%
frequent%as%the%second%most%frequent%item
The%third%most%frequent%item%is%1/3%the%value%of%the%
first%most%frequent%item…%and%so%on
Very,%very%many%observations%with%low%values

23. A"Long"Tail"
Distribution

24. Of#Course.#As#SEOs#We#Are#Used#To#Seeing#‘Long#Tail#
Distributions’.#In#Search#Behaviour

25. But$It$Goes$Much$
Further$Than$Search$
Demand$Curves

27. In#Linguistics#+ By#Zipfs Law#– The#
Frequency#of#Any#Word#Is#Inversely#
Proportional#To#Its#Rank#In#The#
Frequency#Table

28. Zipfian'
Distribution
Layman’s(terms(– In(any(count(of(
the(frequency(of(words(in(a(large(
body(of(content,(the(probability(of(
occurrence(of(words(or(other(items(
starts(high(and(tapers(off.(
A"few"occur"very"often"
while"many"others"occur"
rarely.

29. ’The’%is%the%most%used%word%in%the%
English%language%(7%%frequency)
‘The’%is%used%twice%as%often%as%the%next%
most%used%word%in%English%(3.5%)
And%three%times%as%much%as%the%next%
most%used%word
Etcetera,%etcetera,%etcetera%infinitium

30. The$Most$Used$
Words$In$The$
English$Language$
Follows$a$Zipfian$
Distribution

31. Brown&Corpus&– 1&Million&Word&Text

32. It#starts#off#
looking
something
like this
0
10000
20000
30000
40000
50000
60000
70000
80000
The Of And To A In That Is Was He For It With As His On
Occurrence
Occurrence

33. Till$it’s$this$)
Brown$Corpus$
Words$Ranked$
by$Frequency$
Distribution
'The’

34. And$it$doesn’t$just$apply$to$
English$either

35. It#applies#to#ALL#
languages

36. Even%languages%
untranslated%as%yet

37. Zipf’s Law*&*
30*Languages*
of*Wikipedia*
(Image*credit*Wikiwand)

38. By Zipfs Law
A"few"words"
will"be"used"
very"frequently
But"account"for"
a"large"part"of"
the"vocabulary
Many"words"
will"be"used"
infrequently
But"account"for"
a"small"part"of"
the"vocabulary

39. In#most#
languages#the#
most#popular#
100#words#make#
up around 50%#
of#all text

40. “In$the$Brown$Corpus,$consisting$of$over$one$
million$words,$half$of$the$word$volume$
consists$of$repeated$uses$of$only$135$words.”$
(Fagan$&$Gencay,$2011)

41. Historically,,of,course,,
many,of,these,words,
were,considered,to,
add,no,value,(stop,
words)

42. ASIDE&' Any&value
from most of&these&
words&is&from&
‘contextual&glue’&in&
natural&language&
‘post&BERT&and&
friends’

43. Zipf’s Law*is*Aligned*with*Pareto’s*80:20*
Rule*Law*(The*Law*of*The*Vital*Few)

44. By#Pareto#* 20%#of#the#words#
account#for#80%#of#the#vocabulary

45. Pareto'is'also'EVERYWHERE'too'– Examples
20%$of$drivers$cause$80%$of$all$traffic$accidents.
20%$of$a$company’s$products$represent$80%$of$the$sales.
20%$of$employees$are$responsible$for$80%$of$the$results.

46. 20%$of$pea$pods$produce$
80%$of$the$peas

47. Even…20%)of)a)
carpet)receives)
80%)of)the)wear

48. So#Why#Does#This#
‘Zipfyness’#Exist?

51. Zipf’s'Law'
Appears'to'
Apply'to...Lots'
and'lots'of'
types'of'data

52. Zipf’s Law*&*The*Population*of*Cities*In*Countries
Appears'to'always'hold'
true'only'when'the'cities'
are'economically'related'
(e.g.'in'the'same'country'
as'each'other)

53. • Zipf%rank%plot%for%276%
metropolitan%areas%in%the%United%
States,%after%results%of%the%census%in%
2000.%Source:%factfinder.census.gov.%
The%straight%line%has%slope%1.11.

54. This%biased%spread%holds%true%
for%income%distribution%!

56. On#the#Internet

57. So…$There$are these$same$
predictable$patterns$across$many$
aspects$of$the$web,$within$ccTLDs$
&$even$website$&$webpage$
features$themselves

58. Pretty&much&every&aspect&follows&a&
Power&Law

59. And$Zipfs Law$is$a$Power$Law
“A power law*is*a*functional relationship between*two*quantities,*where*a*relative*change*in*one*
quantity*results*in*a*proportional*relative*change*in*the*other*quantity,*independent*of*the*initial*
size*of*those*quantities:*one*quantity*varies*as*a power of*another.”*(Source:*Wikipedia)

60. These%Laws%Don’t%Change%In%‘Scale%Free’%Networks%Such%
as%The%Web%Grows%or%Ages%Either
This%file%is%licensed%under%the Creative%Commons Attribution6Share%Alike%3.0%Unported license.

61. A"Small"Example"in"All"The"Words"of"a"Blog"Site

62. Etymology(Nerd’s(
Study
• 490$page$blog
• Top$40$most$frequently$occurring$words$ordered$by$rank
• Followed$a$Zipfian$distribution

63. But$It$Goes$Well$Beyond$Just$
The$‘Words’$on$The$Internet

64. Zipfian'Distribution'Examples'On'The'Internet
Number'of'
inlinks'to'
pages
Number'of'
outlinks from''
pages
Size'of'
websites
No.'of'pages'
in'websites
Number'of'
words'on'a'
page
Number'of'
videos,'images'
&'audio'files

65. In#word#
distribution#on#
a#ccTLD
In#internal#link#
distribution#on#
within#a#ccTLD
In#external#link#
distribution#
within#a#ccTLD
In#the#
popularity#of#
sites
In#the#
popularity#of#
queries
In#the#
popularity#of#
web#pages

66. Zipfs Law)and)The)
Internet
Whilst)the)data)is)older)
Zipfs Law)still)prevails
Adamic,)L.A.)and)
Huberman,)B.A.,)2002.)
Zipf's law)and)the)
Internet. Glottometrics,
3(1),)pp.143G150.

67. In…$EVERYTHING

68. I"asked"an"eminent"search"engineer"to"be"sure"
this"was"correct."He"replied"yes

69. Some%Baeza)Yates%Papers%on Web%Characteristics
Characterization,
of,National,Web,
Domains,(2007)
Link,Analysis,in,
National,Web,
Domains,(2005),
Characteristics,
of,the,Web,of,
Spain,(2005)
Bias,on,The,
Web,(2018)

70. Some%Caveats

71. It#is#true.#All#aspects#of#the#
web#show#this#power#law#
but#it#is#less#precise#on#
webpages#at#the#very#far#
ends#of#the#distribution#
(Baeza<Yates,#2005,2007)

72. Shame&Law&in&web&
pages&(Minimal&
Shame)&Baeza5Yates,&
2007
Where%authors%or%organisations do%some%
minimal%work%because%they%don’t%want%to%be%
ashamed%of%their%work%(minimal%effort).%Makes%
Zipfs Law%less%precise%on%the%small%pages%or%
websites.

73. What%About%At%The%
Long,%Long,%Long%Tail?
Ricardo(explained(it’s(because(the(are(too(many(
unique(value(for(it(to(be(exact

74. ccTLD%Power%
Laws%Versus%a%
‘Whole%
Internet’%
Power%Law
• Each%ccTLD%will%have%a%different%power%law%
(different%exponent)%
• Although%in%general%the%sum%of%ccTLD%power%
laws%is%not%a%power%law,%in%many%cases%it%is%
because%you%have%a%dominant%one.%
• For%example%for%the%whole%Web,%the%USA%will%
dominate.%Experimental%data%shows%this.
Many%thanks%to%Ricardo%BaezaJYates%for%providing%
this%response when questioned

75. Given&a&known&Power&Law&
Distribution&in&EVERYTHING&on&the&
web&how&might&this&be&reflected&in&
search&engine&systems?

76. Given&That&By&Zipf’s &&Pareto’s&Law
A"few"sites"will"be"
sites"will"be"visited""
very"frequently
But"account"for"a"
large"part"of"
search"traffic
Many"sites"will"be"
visited"
infrequently
And"account"for"a"
small"part"of"
overall"search"
traffic

77. They%
Know%ALL%
About%
Zipfs%Law
“No$paper$on$statistics$of$web$pages$
is$complete$without$a$graph$
showing$a$power6law$distribution.”$
(Fetterly,$2005)

78. Zipf’s'Law'Helps'
with'Search'
Systems'Built'for'
Scale

79. Since&EVERYTHING&Is&
‘Naturally’&Much&More&
Predictable

80. Priorities(are(VERY(
obvious(via(Zipf’s
Law

81. Given&That&By&Zipf’s &&Pareto’s&Law
A"few"sites"will"be"
sites"will"be"visited""
very"frequently
But"account"for"a"
large"part"of"
search"traffic
Many"sites"will"be"
visited"
infrequently
But"account"for"a"
small"part"of"
search"traffic

82. Some%‘things’%are%simply%NOT%as%important%as%
others Cities,'towns'&'
villages'
(Population)
Many'Low'quality'
content'URLs
Sites:'Think'CNN'
vs'Mom'&'Pop'
Blog
Few'High'vs'Many,'
Many'Low'or'No'
PageRank'URLs
Head'terms'versus'
long'tail'search'
terms

83. Much%
Literature%
Confirms%
Zipf’s Law%is%
Considered%
in:
Crawling
Indexing
Caching
Ranking
Quality5Determination

84. Zipfs&Law&&&Crawl&
Frequency&
Scheduling&– AKA&
‘Crawl&Budget’?
Crawl&
budget
John&
Mueller
Gary&
Illyes
Link&
Juice
Meme&
with&
cats
Duplicate&
content

85. Crawl&Budget&is&
absolutely&tied&to&
Zipfian&Skewed&
‘Demand’

86. ‘Most’'Sites'Will'NOT'Have'High'Crawl'
Budget…'EVER
https://support.google.com/webmasters/answer/9689511?hl=en

87. The$Rate$of$Change$on$
Sites$Will$Likely$Follow$a$
Power$Law$Too$
(Predictable)

88. As#Will#The#Rate#of#
Content#Creation#on#
Sites#(Predictable)

89. ‘Quality’*Issues*Can*Arise*as*a*
Result*of*Too*Many*
Automatically*Generated*
Pages*Defying*Zipf’s*Law*(The*
rate*of*creation*is*obviously*
not*by*human*hand)

90. Zipfian'Quality'Thresholds'
Meeting'will'‘probably’'shift'
for'the'site.'Quality;
impacted;Crawling'will'
‘probably’'be'Impacted

91. You$Threw$a$‘Long$Tail’$
of$Low,$Low$Importance$
URLs$Into$The$Mix

92. You$Just$
Bought$
Some$Places$
At$The$Back$
of$The$‘Crawl$
Schedule’$
(Or$Didn’t$
Make$The$
Grade$At$All)

93. 'Crawl'Budget’'Issues'
are'Often'OVERALL'‘Site'
Quality’'Issues

94. Since&You’re&
Being&Judged&
on&The&‘Whole’&
Cake

95. Aside&from&’Demand’&&&
Overall&Quality

96. Crawl&‘budget’&
is mostly
about
‘Importance’(
which&follows&
Zipf’s Law)&
(aside&from&
host&load&
capabilities)
Demand
PageRank+(or+
equivalent)
Inclusion+in+
sitemap
Internal+links
External+links Canonical+tag
Redirections
Quality+
parents

97. All#of#These#
Things#Likely#
Are#Impacted#
by#Zipf’s Law

98. The Zipfian Distribution of Many Things

99. What%About%Zipfyness%in%
Indexing%Systems?

100. One$Index.$Multiple$
Tiers$Within$It$(Probably)
• “Each&tier&is&the&next&level&of&document&popularity&
and&usually&popularity&is&similar&to&a&Zipf&distribution&in&
its&central&part”&
• (This&is&Ricardo’s&answer&to&me&when&I&asked&about&
this&last&week)

101. What%About%Web%
Caching?

102. Given&That&By&Zipf’s &&Pareto’s&Law
A"few"sites"will"be"
sites"will"be"visited""
very"frequently
But"account"for"a"
large"part"of"
search"traffic
Many"sites"will"be"
visited"
infrequently
But"account"for"a"
small"part"of"
search"traffic

103. Search'
Engine'Web'
Caching'&'
Zipfian'
Distribution
Web$caching$policies$make$use$of$
Zipfs Law$(many$papers$confirm)
A$small$amount$of$pages$will$be$
very$popular$(stored$in$near$
computer$memory)
A$lot$of$pages$will$be$rarely$called$
(stored$on$disk)$and$probably$in$a$
low$level$tier

104. Web$Caching$Uses$Zipf’s$Law$Too
“It$is$important$to$note$the$
effectiveness$of$caching$
relies$heavily$on$the$
existence$of$Zipf’s$law”$
(Adamic$&$Huberman,$2002)$$
Adamic,$L.A.$and$Huberman,$
B.A.,$2002.$Zipf's$law$and$the$
Internet. Glottometrics, 3(1),$
pp.143L150.

105. What%About%Zipfian%
Distribution%in%Ranking?

106. Yep.%Web%
Ranking%is%
Zipfy%Too

107. By#Zipf’s#Law#in#
Ranking#Position#2#is#
‘probably’#judged#
‘half’#as#relevant#as#
Position#1

108. Despite'Some'Debate'PageRank'is'Largely'
Thought'To'Follow'a'Zipfian'Type'Power'Law
“We$suggest$that$power/law$distributions$of$PageRank$in$Web$graphs$have$
been$observed$because$the$typical$damping$factor$used$in$practice$is$
between$0.85$and$0.90.”$(Becchetti &$Castillo,$2006)

109. On#’Long#Tail’#
SERPs#that#half#is#
‘probably’#very#
small#though

110. ‘Could'be’'Why'
‘Same1Site'
Same1Intent’'
URLs'Appear'
For'The'Same'
Query
THEORY'
ALERT

111. Regardless)of)Theory)
Search)Engines)Absolutely)
Use)Prioritised ‘Thinning)
Out’)Systems)For)Rankings

112. Top$K&Ranking&Systems&
(2&(or&multi)&Stage&
Shortlisting)

113. Stage&1&(Full&Ranking)
Relevance(Inclusion((Is(the(page(relevant(at(all?)
Costs(less
Using(‘scale(systems’
Gather(a(Top>K(bunch(of(URLs(for(re>ranking
Stage&2&(Re2Ranking)
Precision(refinement(amongst(Top>K((top(x(number(of(
results(from(stage(1
Utilising more(computationally(expensive(machine(
learning(resources
Likely(judging(Top>K(on(further(features((added(value)

114. First&Stage&Ranking&Gets&Rid&of&
MOST&of&the&Candidates&For&The&
Top&10&/&20

115. So why&does&all this even&
matter?

116. And what(can(we(do(
about(it?

117. Zipfs Law)Is)Aligned)
with)An)
Exceptionally)Strong)
bias)of)‘Importance)
Identification’)
Beyond)The)‘Norm’

118. You$Are$Being$Judged$On$
The$‘Whole’$Cake

119. 'Overall’)
Matters

120. The$‘Whole$Pie’$
‘Probably’$
Includes$EVERY$
URL$You$Ever$
Created

121. Whilst'‘Your'
Index'
Inventory’'
‘Importance’'
Is'Paramount

122. Google&Might&Think&You’re&a&Time4Waster&Aside&From&
What&Is&Indexed

123. Particularly*
Problematic*
on*‘Long*Tail’*
Content

124. Much%of%That%‘Discovered%Not%Indexed’%Content%Is%
Often%In%The%‘Unimportant’%End%of%The%Zipfian%Curve
A"few"URLs"which"can"satisfy"many"users
URLs that can satisfy few users"needs"but"
there’s"a"lot"of"them"(long,"long"tail)
The"majority"of"URLs"–
Medium"importance

125. As#A#Site#Grows#You#
Can#End#Up#With#
Messy#‘Entrails’

126. If#You#Keep#Throwing#These#Unimportant#
URLs#into#The#Mix#You#Will#NEVER#Get#
Enough#Googlebot#Crawl#to#Reconsolidate

127. As#sites#grow#the#challenge#is#
around#healthy#‘index5
worthy’#‘inventory’#(of#
varying#content#types)#
management

128. Keeping'A'Strong'
‘Importance’'Heart'
Within'A'Site

129. At#this#stage#SEO#is#as#much#about#saying#
what#is#NOT#important#as#it#is#about#saying#
what#IS#important

130. You$Need$To$Think$
Carefully$About$How$You$
Will$Manage$Content$as$It$
Grows,$Ages$&$Expires

131. Simply'Deleting'Content'
is'Rarely'The'Answer

132. It#Can#Be#A#
Disaster

133. If#You#Change#Anything in#Your#
Site#You#Change#The#Zipfian#
Distribution#– EVERYTHING
Matters

134. For$every$action$
there$is$an$equal$and$
opposite$reaction$is$
TRUE$– Newton’s$
Third$Law

135. If#You#Prune#
Without#Thought#
You#Change#the#
Zipfian#Distribution#
in#Your#Site
In#the#words
In#the#internal#links
In#the#out#links#from#pages
In#the#‘topical#themes’#prevalent#in#
your#site
In#the#‘needs#met’#prevalence

136. Instead(of(deletion(try(‘importance’
suppression

138. You$Archive

139. Archiving)Can)Retain)Categorical)Value)Whilst)
Dampening)‘Importance’
• https://www.searchenginejo
urnal.com/google5john5
mueller5rank5important5
pages/345192/

140. What%About%
Redirection?

141. Strong'‘Entity’'
Redirection'is'Powerful

142. But$Inconsistent$
Redirections$
Cause$Confusion

143. Be#Careful#You#Do#Not#
Keep#Redirecting#To#
Products#Which#Also#
Go#Too

144. Consider)Redirection)To)
‘Entity1Aligned’)Specific)
Subcategories)Which)
WILL)NOT)Go)Away

145. Build&‘Legacy&
Redirection&Review&&&
Consolidation’&Into&
Your&‘Business&As&
Usual’&Activities

146. Often&You&Will&Find&
‘Forked&&&Split’&
Inconsistent&Legacy&
Redirection&Patterns

147. So#Continually#Gather#Everything#Back#Together

148. Point&Everything&
In&The&Right&
Direction

149. Be#Aware#That#Often#On#The#Long#
Tail#Google#Will#Let#You#Handle#The#
Redirection#On#Your#Side#Without#
Updating#The#SERPs

150. And$Will$Use$
Page$Titles$&$
Snippets$
From$
‘Redirected:
From’$URLs$
with$
‘Redirected:
To’$URLs

151. When%SHOULD%You%Delete%
Content?

152. When%You%Stop%
Providing%The%
Service%or%Product%
Category%Altogether

153. You$Mostly$Can’t$Rank$
For$Things$Which$Have$
No$Presence$In$Your$Site

155. OR…$Improve$the$
template$with$
minimum$boilerplate$
&$maximum$added$
value$overall

156. A"GREAT"dynamic"
template"built"for"
scale"can"be"super"
powerful"on"many"
levels."Particularly"
the"long"tail

157. BUT…%It%MUST%be%done%well%
(not%spam)%&%designed%to%meet%
ALL%the%important%specific%
intent%needs

158. An#excellent#
product#or#
subcategory#
template#is#a#
super#‘long6
tail#net’

159. BEWARE&OR&
DELIGHT?
It’s&easy&to&
add&(or&lose)&
at&scale&
‘value?add’&
on&templates

160. A"small"change"
dynamically"to"many"
pages"can"kill"or"cure

161. Badly&Designed&Templated&
Dynamic&Menus&&&Filters&
Can&Be&Disastrous

162. Plan%Programmatic%
Approaches%With%
Care.%Lots%of%Testing%
&%Tentative%Scaling

163. If#Things#Go#Wrong#
They#Will#Likely#Go#VERY#
Wrong#7 Yikes

164. Employing*Zipfs*Law*Thinking*in*Quality*
Thresholds
• Skewed&proportions??&Reset&
the&quality&proportions
• If&you&have&less&control&over&
one&section&improve&another
• ‘Overall’&matters
• Consider&‘everything&that&
has&gone&before’&too&(legacy&
cruft&can&be&devastating)&

165. Pro$Tip:)Don’t)Just)Wait)
For)Googlebot)in)Log)
File)Analysis)When)
Trying)to)Find)&)Fix)
Dynamic)Cruft

166. Use$The$Crawling$
Patterns$of$Other$Bots$
to$Find$Crufty9URLs$
From$the$Past.$
Redirect$Them$Before$
Googlebot$Arrives$Next$
Time$Round

167. You$Will$Rarely$Find$Long$Legacy$Cruft$From$A$Crawl$of$
A$Current$Site$or$in$GSC.$They$are$NOT$individually$
important$enough$to$make$the$1k$sample

168. Microsoft)Index)Explorer)is)a)Gem)For)Legacy)Fixing)in)
a)Well)Organised Map)of)The)Site)Past)&)Present

169. Become&a&
Cruftbuster

170. Employing*Zipfs*Law*Thinking*in*Quality*
Thresholds
• Skewed&
proportions??&
Improve the&quality&
proportions
• If&you&have&less&
control&over&one&
section&improve&
another

171. And$Don’t$JUST$
Have$Dynamic$
Content

172. If#All#You#Can#Offer#is#
Thin#Product#Pages#
How#Are#You#Any#
More#Important#Than#
Others?

173. Quality(Supplements(– Redeem(your(site(
with(‘high(quality’(sections(to(supplement(
less;creative(transactional(pages

174. You$Are$Building$More$Than$An$
Ecommerce Site
You$are$building$a$valuable$resource$on$a$domain$
of$knowledge.$The$by9product$is$you$monetise
with$ecommerce

175. You$need$to$
meet$as$many$
niche0relevant$
informational$
needs$as$possible

176. Since&You&Are&Being&Judged&On&
The&‘Whole’&‘Needs9Meeting’&
Cake

177. But$Be$Careful$You$Do$Not$‘Inadvertently’$Morph$Into$a$
Different$Type$of$Site
https://www.sistrix.com/blog/disciplined5how5dailymail5co5uk5got5placed5
amongst5peers/

178. Whole&Pie&Proportions&Matter&Too

179. Build&Strong&Theme&Clusters&&&
'Little&Bow&Ties’&(Strongly&
Connected&Components)&(Hubs)

180. Whole&Pie&Quality&is&
Paramount

181. Keep$Google$(&$Users)$Away$From$Poor$
Quality$&$Too$Much$‘Low$Importance’

182. Since&Some&‘things’&are&simply&NOT&as&
important&as&others
Cities,'towns'&'
villages'
(Population)
Many'Low'quality'
content'URLs
Sites:'Think'CNN'
vs'Mom'&'Pop'
Blog
Few'High'vs'Many,'
Many'Low'or'No'
PageRank'URLs
Head'terms'versus'
long'tail'search'
terms

183. Indicate)‘IMPORTANCE’)&)Quality
• Avoid&linking&internally&
to&low&importance&
pages
• Avoid&linking&to&
products&which&are&out&
of&date&or&out&of&stock
• Avoid linking to ‘far&too&
long&a&tail’&too&much

184. Internally)Linking)to)
Zero)Inventory)is)a)Bad)
Sign)– But)
Programmatic)
Solutions)Help)With)
Resource)Limitations
If#stock#level#<#0#don’t#link#in#
relatedness

185. “Stay&out&of&the&black&and&into&the&
red,&Nothing&in&this&game&for&two&
in&a&bed.”&(Jim&Bowen,&Bullseye)

186. Zipf’s'Law'Means'Even'We'
Need'Scaleable'Solutions

187. Employ'Programmatic'Internal'
Linking'in'Approaches'On'
Ecommerce'Templates'For'
Scale

188. But$NOT$Spam$
– Plan$Well$–
Map$to$
Demand

189. Your%Important%Pages%Should%Be%Like%‘Brazil%Nuts’

190. But$Flat$
Architectures$Send$a$
Signal$That$
Everything$is$of$Equal$
Importance

191. That’s'Simply'
NOT True

192. Undertake)Recency,)
Frequency)&)
Monetary)Value)
Analysis)– The)Results)
with)be)Zipfy

193. Some%‘things’%are%simply%MUCH%MORE%important%than%
others
In#‘local#search’#
human#population
Seasonal#product#
importance#
(Halloween#does#not#
matter#in#March)
Product#importance#
to#the#business#(Some#
offerings#make#a#loss)
Audience#importance#
to#the#business#(Some#
CLTVs#are#lossD
making)
Product#importance#
to#top#audience
A#select#'few'#top#
tasks#of#that#audience#
(top#pains#&#gains)

194. It’s%Important%
to%VERY%MUCH%
Emphasise%
Importance

195. Employing*‘Zipfs*Law*
Thinking’*in*Internal*
Linking*in*Ecommerce
Important)categories)absolutely)must)
enjoy)emphasized)internal)linking

196. Wrong&Page&Ranking&
– Probably&’Skewed’&
Importance&Signals

197. Or…$Over'
optimization$(Google$
finds$another$page$
to$rank)

198. If#You#Insist#on#Flat#
Architectures#&#Few#
Categories#– Then#You#Must#
Use#Linking#&#Pagination#Very#
Well

199. Employing*Zipfs*Law*in*Pagination
If#you#must#use#pagination#utilize#programmatic#approaches#to#keep#the#
most#important#products#high#in#the#paginated#series:
• E.g:
• By#(genuine)#availability
• By (genuine)#popularity
• By seasonality

200. Automated)Categorical)
&)Subcategorical)
Clustering)From)
Product)Pages)is)Both)
Powerful)&)Scaleable

201. Quality(Sub+
categorization(Is(Gold(
and(Can(Help(with(
Avoiding(Many(Issues

202. Too#Many#Products#in#Too#Few#Categories
Could#well be#one#of#the#reasons#why#
deep#paginated#results#appear#in#SERPs#–
Google’s#attempts#to#surface#inventory#in#
search

203. Such%as:%
Competing%
with%
Yourself%in%
eCommerce
Products)equally)important
Competing)with)each)other
Competing)with)their)category
Old)product)floating)around)in)SERPs)– Entity)
redirect)to)highly)relevant)subcategory

204. Quality(Sub+
categorization(Is(Specific(
Enough(for(Quality(
‘Entity(Redirection’

205. And$Specific$
Enough$To$Rank$
Against$Product$
Pages

206. Nudge&Forward&a&Clear&Winner&– The&Subcategory

207. But$Utilise$
‘Overflow$
SEO’
Don’t&create&subcategories&
just&because&you&can
Nobody&wants&a&
subcategory&or&category&
with&zero&products
Grow&and&contract&into&
subcategorization&as&your&
inventory&dictates

208. Don’t&Try&To&Wear&
Shoes&Bigger&Than&
Your&True&‘Value&Add’&
Zipfian&Footprint

209. And$Finally…

210. Zipf argued+Zipf’s Law+
could+be+in+part+to+do+
with+‘laziness’+(Zipf’s
Law+AKA ‘The+Principle+
of+Least+Effort’)

211. Zipf’s
Principle-of-
Least-Effort-
alludes-to:
Make%it%easy%to%find%what%I%
need
Make%sure%there%is%
consistency%so%I%can%get%
quicker%at%finding%what%I%need%
easily%through%repetition

212. Consider)‘The)Law)of)Least)Effort’)in)
Copy

213. Zipfian'
Approaches'in'
Primary'
Navigations
‘Top%Task’%
Identification
Limited%‘High%
Importance’%
categories
Links%to%‘View%
all%x’%in%menu%
structures

214. Use$popular$
words$
people$use.$
Not$your$
company$
jargon

215. So#Does#Zipfian#
Distribution#
Explain#SEO?

216. Let’s&Look&At&The&Evidence…&Zipfs Law&applies&
to: Word%frequency%distribution
The%internet%at%large
Link%graphs%(internal%&%external)
Crawl%frequency%(probably)
Web%caching%policies%and%likely%index%inclusion
Probably%first%stage%ranking%at%least
'Importance’%(and%unimportance)%identification

219. Thank&You

220. References

221. • Adamic,(L.A.(and(Huberman,(B.A.,(2002.(Zipf's law(and(the(Internet. Glottometrics, 3(1),(
pp.143C150.
• BaezaCYates,(R.,(Castillo(Ocaranza,(C.(and(López(Martínez,(V.,(2005.(Characteristics(of(the(
Web(of(Spain.
• BaezaCYates,(R.,(Gionis,(A.,(Junqueira,(F.,(Murdock,(V.,(Plachouras,(V.(and(Silvestri,(F.,(
2007,(July.(The(impact(of(caching(on(search(engines.(In Proceedings0of0the030th0annual0
international0ACM0SIGIR0conference0on0Research0and0development0in0information0
retrieval (pp.(183C190).
• BaezaCYates,(R.,(Castillo,(C.(and(Efthimiadis,(E.N.,(2007.(Characterization(of(national(web(
domains. ACM0Transactions0on0Internet0Technology0(TOIT), 7(2),(pp.9Ces.
• BaezaCYates,(R.,(Boldi,(P.(and(Chierichetti,(F.,(2015,(May.(Essential(web(pages(are(easy(to(
find.(In Proceedings0of0the024th0International0Conference0on0World0Wide0Web (pp.(97C
107).
• BaezaCYates,(R.,(2018.(Bias(on(the(web. Communications0of0the0ACM, 61(6),(pp.54C61.
• Becchetti,(L.(and(Castillo,(C.,(2006,(May.(The(distribution(of(PageRank(follows(a(powerC
law(only(for(particular(values(of(the(damping(factor.(In Proceedings0of0the015th0
international0conference0on0World0Wide0Web (pp.(941C942).
• Zipf,(G.K.,(2016. Human0behavior and0the0principle0of0least0effort:0An0introduction0to0
human0ecology.(Ravenio Books.

222. Appendix

223. Zipfian'
Distribution'
&'Efficiency'
Scaling'in'
SEO
There%is%no%shame%in%using%well%thought%
through%and%intelligent%programmatic%
approaches%for%scale%in%SEO
On%long%tail%content%(low%search%volume%
but%many%combined%searches)%it%is%a%
‘realistic’%approach
Build%‘high%quality’%scalable%templates

224. Subcategorisation.
&.Zipfs.Law
• Target'‘torso'terms’'(e.g.'black'dresses,'
diamante'shoes,'lacy'dresses)
• Internally'linking'subcategory'to'subcategory'
can'be'super'relevant
• Subcategories'provide'an'excellent'way'to'
‘dampen’'importance'and'raise'importance'to'
‘fewer’'categories'(moving'‘things’'down'to'
subcategory'level'in'the'architecture)
• Often'a'more'natural'match'to'queries
• Provides'a'more'natural'featureErich'‘ontology’
• Builds'out'the'semantic'richness'of'a'site'in'
internal'anchors

225. Maximal
Shame*Law*
&*Zipfian*
Distribution
(With&regards&to&Zipfian&distribution&
in&web&pages):&“To&be&precise&it&is&
not&true&at&the&beginning&(e.g.,&small&
pages)&because&what&I&call&the&
shame&law&(you&do&some&minimal&
work)&and&it&is&not&true&at&the&end&of&
the&long&tail&because&it&is&very&long&
(e.g.,&many&unique&values).”&(BaezaF
Yates,&2020)

226. Maximal'Shame'Power'Law'(Baeza3Yates
“One%phenomenon%that%has%appeared%before%in%our%own%studies%and%now%is%
completely%clear%is%the%smaller%power%law%exponent%at%the%beginning%of%several%of%
the%measures%presented.%In%fact,%this%happens%for%file%sizes%up%to%25Kb,%pages%perA
site%up%to%(15–30),%pages%perAdomain%up%to%10%(except%South%Korea),%number%of%
outlinks in%a%page%up%to%10%to%40,%and%average%number%of%internal%links%persite up%to%
15%to%30,%where%a%range%is%given%to%show%the%variability%for%different%countries.%We%
argue%that%this%is%due%to%another%empirical%power%law%that%we%call%maximal%shame%
which%forces%people%to%work%a%bit%more%than%the%minimum%until%they%feel%good%
about%their%work.%Notice%that%this%maximal%shame%can%be%for%an%individual%or%for%a%
group%(e.g.,%in%the%case%of%a%Web%site).”
BaezaAYates,%R.,%Castillo,%C.%and%Efthimiadis,%E.N.,%2007.%Characterization%of%national%
web%domains. ACM$Transactions$on$Internet$Technology$(TOIT), 7(2),%pp.9Aes.

227. Automated)Internal)
Linking)&)Strong)
Templates
There%is%no%shame%in%using%well%thought%
through%and%intelligent%programmatic%
approaches%for%scale
Avoid%linking%internally%to%products%
which%have%a%short%shelf%life%unless%you%
have%programmatically%set%up%the%
system%to%change%based%on%availability
Internally%linking%to%unavailable%
products%is%probably%not%a%good%quality%
signal

228. Be#Prepared#
To#Give#Some#
Things#Up
Some%‘things’%WILL%matter%Zipfianly%
less%than%others