52. How rendering works with Google
Bartosz: So, you look at the
difference between the initial
HTML, and, then, if after
rendering you see extra content?
53. How rendering works with Google
Bartosz: So, you look at the
difference between the initial
HTML, and, then, if after
rendering you see extra content?
Martin Splitt: Yeah.
54. The Heuristics
There are certain heuristics, that,
if we see after a while, like, oh, this
page, actually, the renderer does not
diff as much or doesn’t diff, it looks
the way before
55. There are certain heuristics, that,
if we see after a while, like, oh, this
page, actually, the renderer does not
diff as much or doesn’t diff, it looks
the way before
The Heuristics
I still haven’t fully, like, grasped
what exactly triggers the
heuristics
67. Pretty much every website, when
we see them for the first time, goes
to rendering. So there’s no indexing
before it hasn’t been rendered.”
ALL NEW SITES
GET RENDERED
68. But what is
a new website exactly?
What is a new website?
69. But what is
a new website exactly?
What is a new website?
70. What if
a new website doesn’t have
any comments yet?
79. Benchmarking Google's heuristics:
Our own JS Budget Experiment (2019)
3 domains with <a href> links in the initial HTML response
3 other domains with <a href> links generated with external JavaScript files
80. Benchmarking Google's heuristics:
Our own JS Budget Experiment (2019)
3 domains with <a href> links in the initial HTML response
Each domain runs on a different IP (UK)
3 other domains with <a href> links generated with external JavaScript files
81. Benchmarking Google's heuristics:
Our own JS Budget Experiment (2019)
3 domains with <a href> links in the initial HTML response
Each domain runs on a different IP (UK)
Content generated using Articoolo
3 other domains with <a href> links generated with external JavaScript files
84. List of experiment websites
Time needed to reach the 7th level of nested content
(server logs)
htmlcrawling.party
first 6 levels: 6 minutes. We had to wait as much as 6
days for the 7th level.
jscrawling.party 17 minutes
htmlcrawling.wine 4 minutes
jscrawling.wine
We had to wait 1 day for Google to reach the 2nd level,
but then it went through the remaining levels within
minutes.
jscrawling.pizza 8 minutes
htmlcrawling.pizza 7 minutes
Results – 2019
85. I was right
Results – 2019
List of experiment websites
Time needed to reach the 7th level of nested content
(server logs)
htmlcrawling.party
first 6 levels: 6 minutes. We had to wait as much as
6 days for the 7th level.
jscrawling.party 17 minutes
htmlcrawling.wine 4 minutes
jscrawling.wine
We had to wait 1 day for Google to reach the 2nd level,
but then it went through the remaining levels within
minutes.
jscrawling.pizza 8 minutes
htmlcrawling.pizza 7 minutes
96. Raw data: Most of the JavaScript content indexed after
a half-hour (!)
HTML to JS ratio experiment- results
97. Raw data: Most of the JavaScript content indexed after
a half-hour (!)
With the exception of 5 URLs.
HTML to JS ratio experiment- results
98. Raw data: Most of the JavaScript content indexed after
a half-hour (!)
With the exception of 5 URLs.
HTML to JS ratio experiment- results
99. Raw data: Most of the JavaScript content indexed after
a half-hour (!)
With the exception of 5 URLs.
But here, Google didn’t even crawl it,
so it’s not related to rendering at all
HTML to JS ratio experiment- results
100. Raw data: Most of the JavaScript content indexed after
a half-hour (!)
With the exception of 5 URLs.
But here, Google didn’t even crawl it,
so it’s not related to rendering at all
After 4 hours: 29/30
JavaScript content indexed.
HTML to JS ratio experiment- results
101. Raw data: Most of the JavaScript content indexed after
a half-hour (!)
With the exception of 5 URLs.
But here, Google didn’t even crawl it,
so it’s not related to rendering at all
After 4 hours: 29/30
JavaScript content indexed. After 8 hours: 30/30
JavaScript content indexed.
HTML to JS ratio experiment- results
110. Google in 2017 vs Google in 2019.
Good job,
Google!
0:3
111. But what about popular websites having some
content generated by JS?
Martin Splitt was right
about the new websites.
112. Martin Splitt was right
about the new websites.
Let's find out!Can Google deal with them?
But what about popular websites having some
content generated by JS?
113. The State of JavaScript Indexing
Experiment 2019
114. The State of JavaScript Indexing
Experiment 2019
115. The State of JavaScript Indexing
Experiment 2019
116. The State of JavaScript Indexing
Experiment 2019
117. The State of JavaScript Indexing
Experiment 2019
141. 2 WAVES
O F I N D E X I N G - T I M E L I N E
142. 2 WAVES
O F I N D E X I N G - T I M E L I N E
143. I expect, eventually rendering,
crawling and indexing will come
closer together
We are going like: ”Oh, all right, we
are gonna skip rendering.” It is not
as frequently happening anymore.
(...) Many websites even if they do
not run JavaScript, they might still
go through the render phase,
because it doesn’t make a difference
as much
144. Crawling
and indexing
to come
together
I expect, eventually rendering,
crawling and indexing will come
closer together
We are going like: ”Oh, all right, we
are gonna skip rendering.” It is not
as frequently happening anymore.
(...) Many websites even if they do
not run JavaScript, they might still
go through the render phase,
because it doesn’t make a difference
as much
190. JavaScript SEO is important. But other
parts of technical SEO are important too.
191. JavaScript SEO is important. But other
parts of technical SEO are important too.
I mentioned The Guardian has some issues with JavaScript SEO.
192. JavaScript SEO is important. But other
parts of technical SEO are important too.
I mentioned The Guardian has some issues with JavaScript SEO.
But Google indexes their HTML content really fast.
193. JavaScript SEO is important. But other
parts of technical SEO are important too.
I mentioned The Guardian has some issues with JavaScript SEO.
Sample of 1300 URLs.
Number of pages indexed after 1 day: 1280 = 98.5%
But Google indexes their HTML content really fast.
194. JavaScript SEO is important. But other
parts of technical SEO are important too.
I mentioned The Guardian has some issues with JavaScript SEO.
What about the rest?
Sample of 1300 URLs.
Number of pages indexed after 1 day: 1280 = 98.5%
But Google indexes their HTML content really fast.
198. A quick check of 100 URLs of Medium...
Only 70% of them are indexed in Google.
*Tomek’s joke :D
Medium is...medium*
199. A quick check of 100 URLs of Medium...
Only 70% of them are indexed in Google.
*Tomek’s joke :D
Medium is...medium*
200. Spoiler alert: 50% of the indexed URLs has JS content indexed.
That's the...medium value**.
A quick check of 100 URLs of Medium...
Only 70% of them are indexed in Google.
*Tomek’s joke :D**
Medium is...medium*
201. Spoiler alert: 50% of the indexed URLs has JS content indexed.
That's the...medium value**.
A quick check of 100 URLs of Medium...
Only 70% of them are indexed in Google.
*Tomek’s joke :D**
Medium is...medium*