We love to help folks make awesome websites. For a while now, we've been answering questions from developers, site-owners, webmasters, and of course SEOs in our office hours hangouts, in the help forums, and at events. Recently, we've (re-)started answering your questions in a video series called #AskGoogleWebmasters on our YouTube channel.
To ask a question, just use the hashtag #AskGoogleWebmasters on Twitter. While we can't get to all submissions, we regularly pick up the questions there to use in future episodes. We pick questions primarily about websites & websearch, which are relevant to many sites. Want to stay in the loop? Make sure to subscribe to our channel. If you'd like to discuss the questions or other important webmaster topics, feel free to drop by our webmaster help forums and chat with the awesome experts there.
At its core the update is a switch from Chrome 41 as the rendering engine to the latest stable Chromium. Googlebot is now using the latest stable Chromium to run JavaScript and render pages. We will continue to update Googlebot along with the stable Chromium, hence we call it "evergreen".
A JavaScript-powered demo website staying blank in the old Googlebot but working fine in the new Googlebot.
What this means for your websites
We are very happy to bring the latest features of the web platform not only to Googlebot but to the tools that let you see what Googlebot sees as well. This means websites using ES6+, Web Components and 1000+ new web platform features are now rendered with the latest stable Chromium, both in Googlebot and our testing tools.
While the previous version of the mobile-friendly test doesn't show the page content, the new version does.
What the update changes in our testing tools
Our testing tools reflect how Googlebot processes your pages as closely as possible. With the update to the new Googlebot, we had to update them to use the same renderer as Googlebot.
The change will affect the rendering within the following tools:
We tested these updates and based on the feedback we have switched the tools listed previously to the new evergreen Googlebot. A lot of the feedback came from Googlers and the community. Product Experts and Google Developer Experts helped us make sure the update works well.
Note: The new Googlebot still uses the same user agent as before the update. There will be more information about an update to the user agent in the near future. For now, Googlebot's user agent and the user agent used in the testing tools does not change.
Use server-side or hybrid rendering so users receive the content in the initial payload of their web request.
Always ensure your URLs are independently accessible:
https://www.example.com/product/25/
The above should deep link to that particular resource.
If you can’t support server-side or hybrid rendering for your Progressive Web App and you decide to use client-side rendering, we recommend using the Google Search Console “Fetch as Google tool” to verify your content successfully renders for our search crawler.
Don’t:
Don't redirect users accessing deep links back to your web app's homepage.
Additionally, serving an error page to users instead of deep linking should also be avoided.
Provide Clean URLs
Why? Fragment identifiers (#user/24601/ or #!user/24601/) were an effective workaround for browsers to AJAX new content from a server without reloading the page. This design is known as client-side rendering.
However, the fragment identifier syntax isn’t compatible with some web tools, frameworks and protocols such as Facebook’s Open Graph protocol.
The History API enables us to update the URL without fragment identifiers while still fetching resources asynchronously and therefore avoiding page reloads -- it’s the best of both worlds. The AJAX crawling scheme (with its #! / escaped-fragment URLs) made sense at its time, but is now no longer recommended.
Provide clean URLs without fragment identifiers (# or #!) such as:
https://www.example.com/product/25/
If using client-side or hybrid rendering be sure to support browser navigation with the History API.
Avoid:
Using the #! URL structure to drive unique URLs is discouraged:
https://www.example.com/#!product/25/
It was introduced as a workaround before the advent of the History API. It is considered a separate pattern to the purely # URL structure.
Don’t:
Using the # URL structure without the accompanying ! symbol is unsupported:
https://www.example.com/#product/25/
This URL structure is already a concept in the web and relates to deep linking into content on a particular page.
Specify Canonical URLs
Why? The best way to eliminate confusion for indexing when the same content is available under multiple URLs (be it the same or different domains) is to mark one page as the canonical, and all other pages that duplicate that content to refer to it.
Best Practice:
Include the following tag across all pages mirroring a particular piece of content:
If you are supporting Accelerated Mobile Pages be sure to correctly use its counterpart rel=”amphtml” instruction as well.
Avoid:
Avoid purposely duplicating content across multiple URLs and not using the rel="canonical" link element.
For example, the rel="canonical" link element can reduce ambiguity for URLs with tracking parameters.
Don’t:
Avoid creating conflicting canonical references between your pages.
Design for Multiple Devices
Why? It’s important that all your users get the best experience possible when viewing your website, regardless of their device.
Make your site responsive in its design -- fonts, margins, paddings, buttons and general design of your site should scale dynamically based on screen resolutions and device viewports.
Small images scaled up for desktop or tablet devices give a poor experience. Conversely, super high resolution images take a long time to download on mobile phones and may impact mobile scroll performance.
Use “srcset” attribute to fetch different resolution images for different density screens to avoid downloading images larger than the device’s screen is capable of displaying.
Scale your font size and line height to ensure your text is legible no matter the size of the device. Similarly ensure the padding and margins of elements also scale sensibly.
Don't show different content to users than you show to Google. If you use redirects or user agent detection (a.k.a. browser sniffing or dynamic serving) to alter the design of your site for different devices it’s important that the content itself remains the same.
Use the Search Console “Fetch as Google” tool to verify the content fetched by Google matches the content a user sees.
For usability reasons, avoid using fixed-size fonts.
Develop Iteratively
Why? One of the safest paths to take when adding features to a web application is to make changes iteratively. If you add features one at a time you can observe the impact of each individual change.
Alternatively many developers prefer to view their progressive web application as an opportunity to overhaul their mobile site in one fell swoop -- developing the new web app in an isolated environment and swapping it with their existing mobile site once ready.
When developing features iteratively try to break the changes into separate pieces. For example, if you intend to move from server-side rendering to hybrid rendering then tackle that as a single iteration -- rather than in combination with other features.
Both approaches have their own pros and cons. Iterating reduces the complexity of dealing with search indexability as the transition is continuous. However, iterating might result in a slower development process and potentially a less innovative overhaul if development is not starting from scratch.
In either case, the most sensitive areas to keep an eye on are your canonical URLs and your site’s robots.txt configuration.
Best Practice:
Iterate on your website incrementally by adding new features piece by piece.
For example, if don’t support HTTPS yet then start by migrating to a secure site.
Avoid:
If you’ve developed your progressive web app in an isolated environment, then avoid launching it without checking the rel-canonical links and robots.txt are setup appropriately.
Ensure your rel-canonical links point to the real site and that your robots.txt configuration allows crawlers to crawl your new site.
Don’t:
It’s logical to prevent crawlers from indexing your in-development site before launch but don’t forget to unblock crawlers from accessing your new site when you launch.
Use Progressive Enhancement
Why? Wherever possible it’s important to detect browser features before using them. Feature detection is also better than testing for browsers that you believe support a given feature.
A common bad practice in the past was to enable or disable features by testing which browser the user had. However, as browsers are constantly evolving with features this technique is strongly discouraged.
Service Worker is a relatively new technology and it’s important to not break compatibility in the pursuit of progress -- it's a perfect example of when to use progressive enhancement.
Best Practice:
Before registering a Service Worker check for the availability of its API:
if ('serviceWorker' in navigator) {
...
Use per API detection method for all your website’s features.
Don’t:
Never use the browser’s user agent to enable or disable features in your web app. Always check whether the feature’s API is available and gracefully degrade if unavailable.
Avoid updating or launching your site without testing across multiple browsers! Check your site analytics to learn which browsers are most popular among your user base.
Test with Search Console
Why? It’s important to understand how Google Search views your site’s content. You can use Search Console to fetch individual URLs from your site and see how Google Search views them using the “Crawl > Fetch as Google“ feature. Search Console will process your JavaScript and render the page when that option is selected; otherwise only the raw HTML response is shown
Google Search Console also analyses the content on your page in a variety of ways including detecting the presence of Structured Data, Rich Cards, Sitelinks & Accelerated Mobile Pages.
Best Practice:
Monitor your site using Search Console and explore its features including “Fetch as Google”.
Provide a Sitemap via Search Console “Crawl > Sitemaps” It can be an effective way to ensure Google Search is aware of all your site’s pages.
Annotate with Schema.org structured data
Why?Schema.org structured data is a flexible vocabulary for summarizing the most important parts of your page as machine-processable data. This can be as general as simply saying that a page is a NewsArticle, or as specific as detailing the location, band name, venue and ticket vendor for a touring band, or summarizing the ingredients and steps for a recipe.
The use of this metadata may not make sense for every page on your web application but it’s recommended where it’s sensible. Google extracts it after the page is rendered.
There are a variety of data types including “NewsArticle”, “Recipe” & “Product” to name a few. Explore all the supported data types here.
Check that the data you provided is appearing and there are no errors present.
Don’t:
Avoid using a data type that doesn’t match your page’s actual content. For example don’t use “Recipe” for a T-Shirt you’re selling -- use “Product” instead.
Annotate with Open Graph & Twitter Cards
Why? In addition to the Schema.org metadata it can be helpful to add support for Facebook’s Open Graph protocol and Twitter rich cards as well.
These metadata formats improve the user experience when your content is shared on their corresponding social networks.
If your existing site or web application utilises these formats it’s important to ensure they are included in your progressive web application as well for optimal virality.
Don’t forget to include these formats if your existing site supports them.
Test with Multiple Browsers
Why? Clearly from a user perspective it’s important that a website behaviors the same across all browsers. While the experience might adapt for different screen sizes we all expect a mobile site to work the same on similarly sized devices whether it’s an iPhone or an Android mobile phone.
While the web can be perceived as fragmented due to number of browsers in use around the world, this variety and competition is part of what makes the web such an innovative platform. Thankfully, web standards have never been more mature than they are now and modern tools enable developers to build rich, cross browser compatible websites with confidence.
Why? The faster a website loads for a user the better their user experience will be. Optimizing for page speed is already a well known focus in web development but sometimes when developing a new version of a site the necessary optimizations are not considered a high priority.
When developing a progressive web application we recommend measuring the performance of your page load speed and optimizing before launching the site for the best results.
Best Practice:
Use tools such as Page Speed Insights and Web Page Test to measure the page load performance of your site. While Googlebot has a bit more patience in rendering, research has shown that 40% of consumers will leave a page that takes longer than three seconds to load..
Avoid leaving optimization as a post-launch step. If your website’s content loads quickly before migrating to a new progressive web application then it’s important to not regress in your optimizations.
We hope that the above checklist is useful and provides the right guidance to help you develop your Progressive Web Applications with indexability in mind.
App install banners are less intrusive and preferred
App install banners are supported by Safari (as Smart Banners) and Chrome (as Native App Install Banners). Banners provide a consistent user interface for promoting an app and provide the user with the ability to control their browsing experience. Webmasters can also use their own implementations of app install banners as long as they don’t block searchers from viewing the page’s content.
If you have any questions, we’re always happy to chat in the Webmaster Central Forum.
Posted by Daniel Bathgate, Software Engineer, Google Search.
April 21st’s mobile-friendly update boosts mobile search rankings for pages that are legible and usable on mobile devices.
This update:
Affects only search rankings on mobile devices
Affects search results in all languages globally
Applies to individual pages, not entire websites
While the mobile-friendly change is important, we still use a variety of signals to rank search results. The intent of the search query is still a very strong signal -- so even if a page with high quality content is not mobile-friendly, it could still rank high if it has great content for the query.
To check if your site is mobile-friendly, you can examine individual pages with the Mobile-Friendly Test or check the status of your entire site through the Mobile Usability report in Webmaster Tools. If your site’s pages aren’t mobile-friendly, there may be a significant decrease in mobile traffic from Google Search. But have no fear, once your site becomes mobile-friendly, we will automatically re-process (i.e., crawl and index) your pages. You can also expedite the process by using Fetch as Google with Submit to Index, and then your pages can be treated as mobile-friendly in ranking.
1. Will desktop and/or tablet ranking also be affected by this change?
No, this update has no effect on searches from tablets or desktops. It affects searches from mobile devices across all languages and locations.
2. Is it a page-level or site-level mobile ranking boost?
It’s a page-level change. For instance, if ten of your site’s pages are mobile-friendly, but the rest of your pages aren’t, only the ten mobile-friendly pages can be positively impacted.
3. How do I know if Google thinks a page on my site is mobile-friendly?
Individual pages can be tested for “mobile-friendliness” using the Mobile-Friendly Test.
Test individual URLs in real-time with the Mobile-Friendly Test.
To review site-level information on mobile-friendliness, check out the Mobile Usability report in Webmaster Tools. This feature’s data is based on the last time we crawled and indexed your site’s pages.
Mobile Usability in Webmaster Tools provides a snapshot of your entire site’s mobile-friendliness.
4. Unfortunately, my mobile-friendly pages won’t be ready until after April 21st. How long before they can be considered mobile-friendly in ranking?
We determine whether a page is mobile-friendly every time it’s crawled and indexed -- you don’t have to wait for another update. Once a page is mobile-friendly, you can wait for Googlebot for smartphones to naturally (re-)crawl and index the page or you can expedite processing by using Fetch as Google with Submit to Index in Webmaster Tools. For a large volume of URLs, consider submitting a sitemap. In the sitemap, if your mobile content uses pre-existing URLs (such as with Responsive Web Design or dynamic serving), also include the lastmod tag.
5. Since the mobile ranking change rolls out on April 21st, if I see no drop in traffic on April 22nd, does that mean that my site’s rankings aren't impacted?
You won't be able to definitively determine whether your site’s rankings are impacted by the mobile-friendly update by April 22nd. While we begin rolling out the mobile-friendly update on April 21st, it’ll be a week or so before it makes its way to all pages in the index.
6. I have a great mobile site, but the Mobile-Friendly Test tells me that my pages aren't mobile-friendly. Why?
If a page is designed to work well on mobile devices, but it’s not passing the Mobile-Friendly Test, the most common reason is that Googlebot for smartphones is blocked from crawling resources, like CSS and JavaScript, that are critical for determining whether the page is legible and usable on a mobile device (i.e., whether it’s mobile-friendly). To remedy:
Check if the Mobile-Friendly Test shows blocked resources (often accompanied with a partially rendered image).
The most common reason why a mobile page fails the Mobile-Friendly Test is that Googlebot for smartphones is blocked from crawling resources, like CSS and JavaScript, that are crucial for understanding the page’s mobile-friendliness.
To reiterate, we recommend that site owners allow Googlebot to crawl all resources for a page (including CSS, JavaScript, and images), so that we can properly render, index, and in this case, assess whether the page is mobile-friendly.
7. What if I link to a site that’s not mobile-friendly?
Your page can still be “mobile-friendly” even if it links to a page that’s not mobile-friendly, such as a page designed for larger screens, like desktops. It’s not the best experience for mobile visitors to go from a mobile-friendly page to a desktop-only page, but hopefully as more sites become mobile-friendly, this will become less of a problem.
8. Does Google give a stronger mobile-friendly ranking to pages using Responsive Web Design (which uses the same URL and the same HTML for the desktop and mobile versions) vs. hosting a separate mobile site (like www for desktop and m.example.com for mobile)?
No, mobile-friendliness is assessed the same, whether you use responsive web design (RWD), separate mobile URLs, or dynamic serving for your configuration. If your site uses separate mobile URLs or dynamic serving, we recommend reviewing the Mobile SEO guide to make sure Google is properly crawling and indexing your mobile pages.
9. Will my site / page disappear on mobile search results if it's not mobile-friendly?
While the mobile-friendly change is important, we still use a variety of signals to rank search results. The intent of the search query is still a very strong signal -- so even if a page with high quality content is not mobile-friendly, it could still rank high if it has great content for the query.
Specialized FAQs
10. What if my audience is desktop only? Then there’s no reason to have a mobile site, right?
Not exactly. Statistics show that more people are going “mobile only” -- either because they never had a desktop or because they won’t replace their existing desktop. Additionally, a non-mobile-friendly site may not see many mobile visitors precisely for that reason.
The mobile-friendly update will apply to mobile searches conducted across all sites, regardless of the site’s target audiences’ language, region, or proportion of mobile to desktop traffic.
11. I have pages showing mobile usability errors because they embed a YouTube video. What can I do?
We suggest paying close attention to how the YouTube video is embedded. If you are using the “old-style” <object> embeds in the mobile page, convert to <iframe> embeds for broader compatibility. YouTube now uses the HTML5 player on the web by default, so it’s mobile-friendly to embed videos using the <iframe> tags from the “share” feature on the watch page or from the YouTube iFrame API. If you have a more complex integration, that should also be mobile-friendly, since it’ll instruct the device to use the device’s native support.
For Flash content from sites other than YouTube, check if there is an equivalent HTML5 embed tag or code snippet to avoid using proprietary plugins.
Yes, we suggest a minimum of 7mm width/height for primary tap targets and a minimum margin of 5mm between secondary tap targets. The average width of an adult's finger pad is 10mm, and these dimensions can provide a usable interface while making good use of screen real estate.
13. To become mobile-friendly quickly, we’re thinking of creating a very stripped down version of our site (separate mobile pages) until our new responsive site is complete. Do you foresee any problems with this?
First, keep in mind that we support three mobile configurations and that your website doesn't have to be responsive to be mobile-friendly. In response to your question, please be cautious about creating a “stripped down” version of your site. While the page may be formatted for mobile, if it doesn’t allow your visitors to easily complete their common tasks or have an overall smooth workflow, it may become frustrating to your visitors and perhaps not worth the effort. Should a temporary mobile site be created, once the RWD is live, be sure to move the site properly. For example, update all links so they no longer reference the separate mobile URLs and 301 redirect mobile URLs to their corresponding RWD version.
