Log File Analysis is rarely touched on by the average SEO... but you're missing out on data that is pure gold. Imagine not "guesstimating" how Googlebot and other crawlers behave on your website, but actually knowing how bots behave? Imagine not having to play by the 1000 row limit in Search Console (without delving into the coding or the API) and having unlimited data? Imagine being able to find out what those crawl anomalies actually are? Log File Analysis can reveal all this and more, and this talk will show you how…. Where other talks on Log File Analysis cover theory, this talk is highly actionable with quick, 15 minute techniques and free downloadable copy/paste scripts to identify issues and quick wins. Faisal Anderson - Technical SEO Consultant, LiveArea

1. Imagine if you could Spy on Google...

2. Understand exactly
how your site is Crawled

3. Quickly identify Crawling
and Indexation Issues

4. Reveal MASSIVE wins

5. What really is Log File Analysis?

7. Log File

8. HUGE Potential.

9. What about
Search Console?

10. 1)
Data Accuracy is
NOT Guaranteed

11. 2)
Only 1000 Rows
Available in the GUI

12. 3) Google Owns That Data

14. What about
Crawling Tools?

17. Log File Analysis

18. Six Step Process.
1 2 3
A B C
Data Preparation.
Data Analysis.

19. Six Step Process.
1 2 3
A B C
Data Preparation.
Data Analysis.

20. Data Preparation.
1
A B C
Retrieve The Log Files

21. 1
A B C
Hardest Part.

22. 1
A B C

23. 1
A B

24. 1
1 Determine The Hosting Setup

25. 11
Common Scenarios

26. 11

27. 11

28. 11

29. 11

30. 1
1
2
Determine The Hosting Setup
Ask Whether The Hostname is In The Data

31. 12
Livearea.com

32. 12
https://

33. 12

34. 12
/faisal/loves/log-files

35. 1
1
2
Determine The Hosting Setup
Ask Whether The Hostname is In The Data
3 Get 6 Months of Data

36. 13

37. 1
1
2
Determine The Hosting Setup
Ask Whether The Hostname is In The Data
3 Get 6 Months of Data

38. 1
A B

39. 1
A B

40. 1
A B

41. 1
A B
copy .*log mylogfile.csv
Bash$ cat file*.txt > mergedfile.csv

42. Data Preparation.
1
2
A B C
Retrieve The Log Files
Choose the Best Tool

43. A B
2

44. A B
2
Excel

45. A B
2
Easy To Use

46. A B
2

47. A B
2

48. A B
2
Screaming Frog
Log File Analyser

49. A B
2

50. A B
2

51. A B
2
Fairly Easy To Use

52. A B
2
Can't Create Custom Reports

53. A B
2

54. A B
2
£0.00

55. A B
2
?!

56. 2

57. 2

58. 2

59. A B
2

60. Data Preparation.
1
2
3
A B C
Retrieve The Log Files
Choose the Best Tool
Choose the Best Method of Analysis

61. Six Step Process.
1 2 3
A B C
Data Preparation.
Data Analysis.

62. Crawl Behavior.
A
B

63. Crawl Budget
Waste.
B
B

64. Site Health.
B
C

65. B

66. A
B

67. A
B
12

68. A
B

69. A
B

70. Crawl Frequency by
Depth and Internal
Links
A
B

71. A
B

72. A
B

73. A
B

74. A
B

75. A
B

76. A
B

77. A
B

78. A
B

79. A

80. Crawl Budget
Waste.
B
B

81. B
B

82. B
B

83. B
B

84. B
B

85. B
I > Love > Breadcrumbs
B

86. B
I > Love > Breadcrumbs
B

87. B
Breadcrumbs
Breadcrumbs
Breadcrumbs
B

88. B
B

89. B
B

90. B
Breadcrumbs
Breadcrumbs
Ain't Got No
Time For
Loops!
B

91. B
B

92. B
Crawl Frequency
by Subdirectory
B

93. B
B

94. B
B
?search=quality+content

95. B
B

96. B
B

97. Site Health.
B
C

98. B
C

99. C
SMARTPHONE
DESKTOP

100. C

101. C

102. C

103. C
Log File Analysis

104. C
Monday Friday
200 500

105. C

106. C

107. C

108. C

109. C

110. C

111. C

112. C

113. C

114. B
C

115. C

116. B

117. B
Bit.ly/log-file-python

118. B
https://www.screamingfrog.co.uk/22-ways-to-analyse-log-files/

119. B
https://builtvisible.com/log-file-analysis/