Course content
During the workshop we’ll take a deep dive into using the SEO Spider, including looking at:
- How to set up the SEO Spider using the configuration options to make your crawl as efficient or comprehensive as you need, including when to use the list mode
- How to approach crawling very large sites (from over 100,000 pages to millions of URLs)
- Identifying and crawling websites with JavaScript a framework including rendering the page and finding missing resources
- Using the SEO Spider to analyse a variety of technical SEO problems, including hreflang implementation, AMP and structured data validation, thin and near-duplicate content issues, XML sitemaps, canonical tag use, redirect chains and more
- Integrating external data from Google Analytics, Search Console, Ahrefs, Majestic and Moz and when this data can be invaluable
- How to crawl password protected and development sites
- Common crawling issues and ways you can self-diagnose and debug problems with your crawl
- Custom extraction - using the SEO Spider as an incredibly powerful scraper
- Generating crawl visualisations to help spot structural and technical issues
- Using the Spider to conduct page speed analysis
We’ll also get practical and take a look at some sites with the SEO Spider to find issues or accomplish a specific task.
At the end of the course, you’ll be able to confidently crawl complex websites, get the data you need and have a host of new ideas for your next SEO audit.