What to expect...
This detailed, fast-paced but easy-to-follow training course dives deep into the fundamentals, from basic tactics to advanced strategies. Perfect for building a strong technical SEO foundation, you’ll learn to effectively manage how search engines crawl and index your site, and all in plain language anyone can understand.
Gain practical know-how and confidence in leveraging various crawling & indexing tools, tactics and insights. Plus, explore advanced tactics such as crawling Javascript sites, using log files and finding and fixing content quality issues. All in a friendly environment where you can ask any questions you can think of.
By the end of the course, you’ll know how to use crawling, indexing and log file data to optimise your SEO roadmap and create a higher quality, leaner and more efficient site… And walk away with the urge to immediately run back to the office, dive into the relevant data, and put everything into action.
Oh, and you just might score some Screaming Frog swag too…
Bringing a laptop to the session is recommended, but not required.
Who should attend this course?
If you’ve ever wondered how to get your pages crawled & indexed or diagnose content quality issues, this course is for you.
This one-day course shares everything you need to know about how search engines visit, interpret and index your site. Ideal for SEO teams managing large or complex websites, as well as agency-side junior and intermediate digital marketers looking to improve their technical SEO skills.
Here's what one of our past training course attendees had to say:
“Essential for all SEOs - There is no point in creating great content if Google can't index it! This course ensures that you get your content 'in the game' and is tailored to suit your needs.”
Why should you attend this course?
Join this Managing Crawling and Indexing training course if you want to…
- Control how search engines interact with your site
- Create efficient, easily crawlable sites
- Be able to find, diagnose and fix content quality issues
- Confidently identify indexed pages, crawling frequency & site issues
- Improve (and control) content discoverability
- And get answers to all your crawling and indexing questions!
Course Content
This jam-packed training course will be split into four key sessions plus breaks for coffee, lunch and post-training drinks.
Here’s a look at what you can expect to learn in this course:
1: Introductions & what crawl budget means
- Getting to know you – your skills, challenges & goals
- An introduction to crawling & indexing
- How Google crawls, assesses & indexes your pages
- What happens when indexing goes wrong & how you can take control
- Crawling vs indexing problems
- Crawl budget – what is it, do I need to worry about it & how can you improve it?
2: Crawl analysis & management
- Different crawler types – the advantages of each & how to choose one
- Your Screaming Frog SEO Spider questions answered
- How to analyse your crawl
- How to find crawl errors & what to do about them
- How to audit your site architecture & internal link structure to optimise crawling
- The power of log file analysis – how to get the data & what you can discover
- Exploring Search Console’s Crawl Stats report
- Spotting crawl traps
- Using the robots.txt file to control which URLs can be crawled
- Using XML sitemaps to help content discovery & encourage crawling
- Using nofollow to stop links from being followed
- Faceted navigation – the risks, best practices & what to do
- Crawling Javascript-based websites (and why it’s not as scary as you might think)
- How to use page rendering to mimic the way Google ‘sees’ & understands your content
3: Controlling indexing
- Mobile-first indexing
- The options we have at our disposal
- Using Robots tags to control crawling & indexing
- Canonical tags – how they work & when to use them
- Pagination controls – to do or not to do? (and how to do it)
- The different ways you can “ask” Google to index your pages
- When & how to deindex, retire, or redirect pages
- Tools you can use for indexing analysis
4: Indexing issues, monitoring & Q&A
- Identifying & monitoring indexing
- Using the URL Inspection tool in Search Console
- How to spot crawling and indexing patterns for your site
- What the crawling and indexing of your site can tell us about potential content quality issues
- Search Console’s Page Indexing reports – everything you need to know about Error, Valid with Warning, Not Indexed and Excluded pages & what to do
- Why doesn’t Google want to index my page?
- How to find rubbish content
- Managing duplication
- Putting together an indexing analysis plan
- Q&A
What's included in your ticket?
Book onto this training course for April 2025 and you will get:
- A full day's training on your chosen topic on Wednesday 9 April
- Lunch included on your training day
- Full access to brightonSEO and Hero Conf conferences and all the talks on Thursday 10 & Friday 11 April
- Bag of merch
- Video bundle to stream all the conference talks post event
- Access to conference networking events and activities
--
We may cancel courses which don't have enough enrolments, and if so we’ll let you know and give you the opportunity to swap to an alternative course or have a refund.
