Table of Contents
What is a Site Migration?
Depending on who you ask, a site migration can mean a number of different things. Generally, it describes when a website undergoes substantial changes that can impact search engine visibility. These types of changes can include:
- Domain name changes
- Moving to a new CMS or platform
- Site consolidations
- Site hierarchy and information architecture updates
- Redesigns or rethemes
For our purposes as SEOs, we will be focusing on migrations that are more than just cosmetic and have changes to URLs, pages or structure.
Pre-Launch Checklist
The below steps involve checking both the current website and staging website prior to the new site being pushed live.
1. Crawl the Site + Sitemap
Before any site migration occurs, gather a list of all URLs that exist on the site. This can be done using a tool like Screaming Frog. Once your crawl is complete, separate this list into two categories: indexable and non-indexable URLs.
2. Analyze Indexable URLs
Once this list of all indexable URLs that return a 200 response code is compiled, pair it with Google Analytics or Search Console data to view metrics like organic traffic, clicks, impressions, average rank, conversions and conversion rate. This will help identify which pages are important and absolutely need to be carried over during the migration. This will also help with the eventual creation of the new sitemap.
* Don’t forget to check the Google Search Console coverage report and make note of any indexed, not in sitemap URLs. These URLs also need to be handled correctly during the migration process.
3. Check Non-Indexable URLs
Non-indexable URLs include:
- Canonicalized pages
- No-indexed pages
- Parameter, internal search, or filter URLs
- Pages blocked by robots.txt
The goal here is to determine the purpose that these URLs serve, whether they need to be brought over, and what these types of pages look like on the new site. Then, ensure that these pages are actually not indexed. Google can ignores canonicals and robots.txt, so if these pages have made their way into the index, then they may need to be redirected or handled in some way.
* If the pages are truly not indexed, then pageviews in Google Analytics can be a useful metric to determine if these pages are helpful for users.
4. Measure Baseline Metrics
Since the overall goal here is to NOT see a drop in performance, collect a list of baseline metrics such as keyword rankings, traffic, site speed or indexed pages that can be compared pre vs. post migration. This will let us know what impacts changes have had and where areas of improvement may lie post migration.
5. Determine the New Site’s Information Architecture
When referring to information architecture, look at folder structure, permalinks, main navigation and URL structure. These will be the nuts and bolts that comprise the new site hierarchy.
Sometimes changes will be minor like the consolidation of a few pages or the editing of blog permalinks. Other times, the changes will be major, such as a domain change or complete folder reorganization. Determine the scale of the changes being made and map them out appropriately.
* Take extra care if pages are being consolidated on a website. This tends to be one of the number one issues that leads to traffic loss and deceased rankings!
6. Set Up a Redirect Map
Any indexable URL that is changing or being removed will need a 301 redirect set up. This is also where checking those non-indexable URLs is important. If these pages have never been indexed, theoretically we won’t need to set up redirects from them. However, redirects for non-indexable pages may be needed if:
- They are internally linked throughout the site.
- The page has external links pointing to it.
- It is a page that users viewed frequently.
- Google has ignored the no-indexing directive (ie canonicalization or robots.txt) and the pages made their way into the index.
* Pro Tip: Be cognizant of server strength and site performance. Some websites will experience issues when you are talking about 10,000+ redirects.
Can I Let Some URLs 404?
The short answer is, yes but only in certain situations.
Post migration, inevitably there will be some URLs that 404. More often than not, you will want to set up redirects for these. However, if these pages were low importance, do not have links pointing towards them, or never should have been indexed in the first place (think filter pages or certain canonicalized URLs), it can be OK to let these pages 404 and drop out of the index naturally over time rather than setting up mass redirects and potentially harming server performance.
7. Check the Robots.txt File
After a redirect map has been set up, check the robots.txt file. If the site structure is changing, do the directives in the current file still need to be there? Do new directives need to be added for the new site? Google’s Search Console’s robotx.txt tester is a great tool for testing different directives.
8. Create a New .xml Sitemap
Most CMS’s are going to generate a sitemap automatically, but it’s still a good idea to have an idea of the URLs that should be showing up in the sitemap. Also check if the .xml sitemap can be customized for things like crawl frequency and priority.
9. Check Written Content
Although bring the content over will be a task for the development team, ensure that content for blog posts, product descriptions, and category level pages are being brought over to the new site. The goal here is to be aware of any changes to page content as well as URLs and site structure.
10. Check Canonicals & Metadata
In addition to page content, be sure to confirm that elements like title tags, meta descriptions, schema, robots and open graph are being brought over to the new site as well. Check these elements across different page types, such as the homepage, blog posts, category level pages, and product detail pages.
Post-Launch Checklist
The below steps involve checking the new website after it has been pushed live.
1. Check Analytics Tags
Once the site has launched, the first thing to do is to check that analytics is set up properly and correct data is being captured. More often than not, this means simply copying and pasting the same GTM container in the header from the old website. Some common questions to ask yourself include:
- Are we receiving hits on the same property/view as before?
- Have duplicate codes been implemented?
- Is bounce rate dropping towards zero?
- Are we seeing inflated sessions?
Monitor this data over the next few days to make sure there are no analytics issues.
2. Recrawl the Site & Sitemap
Just as the site was crawled pre-launch, crawl the site again after launch and check for any glaring issues. Pages should return a 200 response code with correct canonicals. Check for any broken internal links and make a note to clean these up. If everything looks good, submit the new sitemap in places like Google Search Console and Bing Webmaster Tools.
3. QA Redirect Map
If URLs changed over the course of the migration, run the redirect map through a tool like Screaming Frog and ask yourself:
- Do redirects end up at their correct new destination?
- Do the redirects return a 301 response code?
- Are there any redirect chains?
4. QA Non-Indexable Pages
Once the sitemap and redirects have been checked, turn your attention to pages that are not meant to be in the index. Are pages that are supposed to be canonicalized, no-indexed or blocked by robots.txt still that way? Pay special attention to things like filter pages, URLs with parameters, or URLs generated by internal site search.
5. QA Metadata
This can be done using the same crawl of the site and sitemap. Check that title tags, meta descriptions and headings were carried over properly. Is there any duplicate metadata or are pages missing metadata altogether?
6. QA Written Content
In addition to metadata, ensure that blog content, product descriptions, and category level content was carried over properly. Check on formatting, spacing and the overall status of content by cherry picking a handful of pages and reviewing them manually.
7. Clean Up Broken Internal Links
If URLs changed during the migration, chances are there will still be internal links to old URLs that either 301 or 404. Using a previous crawl, identify and clean up any broken internal links by either removing them or replacing them with URLs that return a proper 200 response code.
8. Check Google Search Console
Google Search Console will take a some time to populate any errors after a migration, but come back to check on things like:
- The Coverage report for any Errors or Warnings
- The Page Experience report for any issues with Core Web Vitals and Mobile Usability
- Schema errors
- The URL parameters tool for the addition of any parameters that have been generated by the new site
* Don’t forget to submit for validation after any fixes have been made!
9. Review Baseline Metrics
In the weeks following the migration, check back on the baseline metrics that were noted pre migration. Is the site seeing any drops in traffic, rankings, or site speed? Is the index expanding or decreasing rapidly? If so, investigate possible causes and provide recommendations.
* Keep in mind that if the domain changed or numerous redirects were set up, there will be a temporary dip in performance as search engines crawl through these redirects and index the new pages.
10. Continue to Monitor Performance
It may take weeks or months for search engines to crawl and recognize all of the changes that have occurred on the website. Continue to check baseline metrics, Search Console and analytics over the next few months to monitor any performance issues.
Migration Gap Analysis
The above process only works if planning is able to be done prior to the site migration. However, many times a site migration will happen and only after the site is experiencing decreased performance will help be enlisted to figure out what went wrong. Situations like these call for a migration gap analysis.
1. Compile a List of the Old Site’s URLs
In order to see what happened with the migration, attempt to recreate the old site from a URL perspective. The best way to do this is to:
- Go into Google Analytics and look at the organic traffic report.
- Set the date range to the months (or year) before the migration and get a list of all the top performing organic landing pages.
- Export this list of URLs and run them through a crawler like Screaming Frog. The goal here is to see what happened to these old URLs.
- Were important pages not brought over?
- Were important pages not redirected to their new destination?
- Were pages consolidated?
One you know what happened to the old URLs, recommendations will become more clear.
2. Click/Query Gap Analysis
Another useful exercise after a migration gone wrong is a click gap analysis. This is done using Google Search Console and looking at the decrease in clicks and impressions for queries that the site ranks (or used to rank) for.
Similar to what was done in Google Analytics, set the date ranges for a few months before the migration and compare it to a few months after the migration. This should clearly reveal losses for specific keywords. Reasons that losses occur include:
- A page was not brought over or redirected properly.
- Metadata, targeting, content or some other on-page feature changed.
- A page is no longer internally linked well or all of the internal links to that page are broken.
- Search engines are still working through redirects from the old site to new and rankings have not returned to normal levels.
This process can provide valuable information when determining next steps and fixes.
3. Wayback Machine
Aside from just looking at URLs and queries, it can be useful to see what the previous website physically looked like by using the Wayback Machine. This can also help identify any changes or issues as they relate to:
- Navigation menus
- Internal linking
- Anchor text
- Images
Combined with Google Analytics and Search Console data, this should be everything needed to gain a clear understanding of the old website.
4. Recommend Fixes
Now that the traffic gap analysis, tracing of redirects, click/query gap analysis, and use of the Wayback Machine is complete, it is time to recommend fixes. Some common fixes after a migration gap analysis can include:
- Setting up redirects from old pages that result in 404s for the sake of traffic, lost link equity, rankings, or indexation cleanup.
- Recreating old pages that used to be successful but were not brought over.
- Breaking apart pages that were incorrectly consolidated.
- Addressing internal linking and anchor text issues.
Have an Upcoming Site Migration?
Site migrations are the number one cause of organic traffic loss and if done improperly, impacts can be felt for months or even years. Don’t let this happen to your business. Contact NextLeft and ask about how our website migration services can help you avoid any issues today!
