In his session, Andor will talk about what is important to know when talking about crawling as well as showing real log file analysis data. This way he wants to provide ideas how to improve in that area.

You are going to learn:

• How robots.txt can become an information security nightmare
• Little-known robots.txt pitfalls that even experienced SEOs often overlook, potentially costing rankings and traffic
• Ideas on how to use robots.txt in the age of genAI
• How Google Search Console can mislead you, and what to do about it - by applying little known facts for your monitoring.