BusinessWebsite Success

JavaScript and SEO – Best Practice Guide

Want create site? Find Free WordPress Themes and plugins.

JavaScript and SEO weren’t compatible for a long time, because search engines couldn’t crawl the Javascript, so content in JavaScript was useless for the Googlebot. Now, Google can crawl JS. In this article, we show you how to implement JS to ensure that your SEO isn’t affected.

Can Search Engines Crawl JavaScript?

This question isn’t very simple, as there is always a risk involved when using JavaScript for design and functionality, because the the crawler might incorrectly interpret the content, and generate results you did not intend.

One search engine that can deal with JavaScript is Google. In the United States, Google has a market share of over 60%, so there are many more bots that want to crawl your website, from Bing to Facebook to certain SEO tools. Many bots cannot deal with JavaScript, which might mean that backlink tools links on your website won’t be recognized, or that the crawler might not be able to create an XML sitemap. Google is technologically far ahead of other suppliers in crawling JS.

Even though there are a series of tests that successfully prove that, for example, Google can read JavaScript very well, this only says a little bit about the daily practice of millions of websites that use JS. This shouldn’t scare you away from JavaScript, but from am SEO perspective, it should (still) be used with reservations.

Will JavaScript increase a website’s loading time?

JavaScript websites must first be rendered by a client or a bot so that they can be displayed. This rendering takes time. Therefore, JS sites have a higher loading time than pure HTML sites, but with certain tools (see next section), loading times can be optimized even with JavaScript.

If JavaScript is used in the form of a tracking code or another code from third parties, it is recommended to allow the code to load at the end and asynchronously, so that the page speed is not slowed down.

How does Google see JavaScript?

Google announced in the webmaster central blog in October 2015 that the search engine is able to render and understand websites and modern browsers. Among these are JavaScript and CSS. A requirement for this is that no JavaScript or CSS is excluded from the crawling. In the updated webmaster guidelines, it is recommended that no important site components are excluded from crawling via the robots.txt. In the same blog post, Google nevertheless indicates that it prefers the principle of Progressive Enhancement, an approach involving the successive improvement of HTML websites, but JavaScript is not excluded from being crawled.

To summarize: Google is the best search engine when it comes to dealing with JavaScript, crawling, and rendering files. However, you should not develop your websites exclusively based on JavaScript, but use JS elements.

Google itself has promoted projects such as Google AMP and Progressive Web Apps (PWA), which focus less on JavaScript, and much more on HTML and HTML5.

Why is it difficult for search engines to crawl JS?

It’s difficult for search engines to crawl JS due to computing capacity. HTML, which functions with PHP, CSS, and other technologies, can be directly read by a crawler – in this case, the source code is rendered immediately when the URL is called up.

With JavaScript, this direct access is not possible. First, the crawler must analyze the DOM (Document Object Model). The DOM’s code is loaded and listed, and only when this is done can the website be rendered. Every browser does this work automatically when surfing the web. If you want to test JavaScript, you can use so-called “headless browsers.” This is a browser without a graphic user interface that is operated by a command line.

5 Rules for JavaScript and SEO

  • So that JavaScript content makes sense for search engines, the content must be indexable within the load event.
  • Content dependent on user events is not indexable.
  • JS sites require an indexable URL with server-side support.
  • The same SEO best practices used for HTML sites apply to rendering JS sites.
  • There should be no contradiction between HTML and JavaScript versions of the website.

Basics for the development of JS websites.

JavaScript is often more complex than HTML. In order to know what is important for SEO, you should understand the behavior of JavaScript:

1. Initial Request: The browser and the search engine bot start a GET request for the HTML code of the website and its affiliated assets.

2. DOM rendering: The JS script site delivers the DOM (Document Object Model) to the browser or the bot. The document shows how the content will be formed on the website and what the connections are between the individual elements on the site. The browser renders this information and makes it visible and usable for the user.

3. IDOM load: While the target site is being processed, the browser triggers events that belong to the DOMContentLoaded. The initial HTML document is then loaded and stored. The browser or the bot is now ready to implement JavaScript elements.

4. JavaScript implementation: JavaScript elements can now change the contents or functions of the website without the HTML source code having to be changed, removed, or expanded.

5. Load Event: As soon as the resources and the JS resources dependent on these are loaded, the browser starts the load, and the site is finished.

6. Post Load Events: After the JS site has been loaded, further content or functional elements can be changed or adapted by the user.

Search engines such as Google use so-called headless browsers in order to simulate access to a traditional browser. In contrast to the “normal” browser, the headless browser calls up the code via the DOM, in order to render a website from it. In this way, the Googlebot can, for example, check which elements JavaScript inserts in order to modify the HTML site. After the rendering, the Googlebot can analyze and index the rendered elements like an HTML source.

With JavaScript, there are two versions for crawlers, the pre-DOM HTML code and the rendered post-DOM HTML code.

Important events from JavaScript that have an influence on SEO

Load events and user events can clearly influence your SEO. Here you will learn why that is.

  • Load Event: The load event is “fired” by the browser when a site has been completely loaded. Search engine robots imitate common browsers in rendering JavaScript. The load event is therefore important for them in order to take a snapshot of the rendered content. It is assumed that all elements that are loaded after the load event of the browser are not necessarily considered in JavaScript crawling and they are therefore no longer indexed. This assumption is correct because JavaScript can quickly change site content. This is especially true for news sites or social networks feeds, such as Twitter and Facebook.

The time frame of the DOMContentLoaded can be measured with the Google development tool:

  • User Events: After the load event, further events can be triggered via JavaScript. Among these, for example, are the so-called “onClick-Events.” These are user-triggered events, such as the restriction of site content or interactive navigation. However, this content, which is created or changed by user events, is usually not indexed by search engines, because it takes place after the event load.

Errors that you should avoid with JavaScript

If you use JavaScript on your website, Google can now render the elements after the load event quite well and can finally read and index the snapshot like a traditional HTML site.

Actually, most problems with JavaScript and SEO result from improper implementation. Many common SEO best practices can therefore also be used for JavaScript SEO. In a Google+ post, John Müller likewise gave an indication of what webmasters should be aware of in the use of JS and AJAX.

1. Indexable URLs: Every website requires unique and distinctive URLs so that the sites can be indexed at all. A pushState, as is created with JavaScript, however, does not generate a URL. Therefore, your JavaScript site also requires its own web document that can give a status code 200 OK as a server answer to a client or bot inquiry. Every product presented with JS (or each category of your website realized with JS) must therefore be equipped with a server-side URL so that your site can be indexed.

2. pushState errors: With the pushState method, JavaScript URLs can be changed. Therefore, you must absolutely ensure that the original URL is relayed with server-side support. Otherwise, you risk duplicate content.

3. Missing Metadata: With the use of JavaScript, many webmasters or SEO forget the basics and do not transfer meta data to the bot. However, the same SEO standards hold for JavaScript content as for HTML sites. So, think about title and meta description of ALT tags for images.

4. a href and img src: The Googlebot requires links that it can follow so that it can find further sites. Therefore, you should also provide links with href- or src-attributes in your JS documents.

5. Create unified versions: Through the rendering of JavaScript, preDOM and postDOM versions arise. Ensure that, if possible, no contradictions slip in and, for example, that canonical tags or paginations can be correctly interpreted. In this way, you will avoid cloaking.

6. Create access for all bots: Not all bots can deal with JavaScript like the Googlebot. It is therefore recommended to place title, meta information, and social tags in the HTML code.

7. Don’t disable JS over robots.txt: Ensure that your JavaScript can also be crawled by the Googlebot. For this, the directories should not be excluded in the robots.txt.

8. Use a current sitemap: In order to show Google any possible changes in the JS contents, you should always keep the attribute “lastmod” current in your XML sitemap.

Check JS websites: What To Do in an Audit

A JS website audit is primarily a manual inspection of individual elements. Current SEO tools have (still) not yet mastered the analysis of JavaScript.

In order to check JavaScript, you can use the Google Developer Tools from Google Chrome as well as the Web Developer Expansion for Chrome.

1. Visual inspection

In order to gain a feel for how a visitor will see the website, you should divide the content on the website into:

  • Visible contents on the site
  • Content that requires an interaction
  • Hidden content
  • Content that come from third parties, such as ratings
  • Content that contain product recommendations

In this way, you can narrow down the selection to elements that can be realized with JavaScript. You should be checking the JavaScript elements with the goal of making these elements likewise crawl-able.

2. Check HTML code

With web developer tools you can turn off CSS, JavaScript, and cookies. You can see what is missing on the site in the remaining code. These elements are controlled by JavaScript.

Thereafter, you can control meta elements such as the title and website description. So that bots can index these elements, they must be accessible via the load event. In general, however, only Google can currently read these elements. It is therefore recommended to write title and meta tags in the HTMl code even with JS sites.

3. Check rendered HTML

Load the site with deactivated JavaScript and CSS anew. Finally, right-click on the site and choose “Examine element” in the Chrome menu. On the right side, another window will appear. Click on the HTML tag. With a right click, an options menu will appear. Choose “Copy External HTML” here. Finally, insert the code into an editor. This code can then be indexed by search engines such as Google.

The Google Search Console offers an additional opportunity to test your JS sites. A prerequisite for the inspection of your website is that your domain be stored and confirmed in the Search Console. In the Search Console, click first on “Crawling” and then on “Fetch as Google.” Finally, insert the directory that has to be edited and click on “Fetch and render.”

Google then shows you possible problems in the rendering of your site and gives notices of missing JavaScript elements or elements blocked by the crawling.

Test Javascript sites in the Google Search Console

Figure 1: Test JS sites in the Google Search Console.

Further things to consider

  • Indexable URLs: In order for a website to be indexed and ranked at all, it requires an indexable URL. If your website uses versions of JavaScript that cannot be called up via their own URL, it is not possible for search engines to index this site. If a URL exists but cannot be linked in JS code, the site similarly cannot rank.
  • pushState: If an improper URL is produced by a pushState, duplicate content can arise. Therefore, be especially sure to check internal links that are created with JavaScript.
  • Data attributes: On JS sites, further resources such as images can be stored in data attributes. However, in general, these resources cannot be loaded, rendered, and indexed by bots. It is therefore recommended that you incorporate resources, such as images, traditionally via HTML.

Practical tools that help your JS site be crawled

There are various tools that you can use in editing or creating JavaScript content.

1. prerender.io

prerender.io is an open-source software that optimizes the rendering of a JS site. With this, the site is cached after rendering and can be pulled up more quickly when accessed by a bot.

2. Brombone

This program downloads your website and renders it in a web browser. In this way, you can check whether AJAX retrievals and JavaScript are functioning correctly. DOM changes can be tested in the same way. If the rendering is working, these sites are stored as HTML. If a crawler is accessing your site, you can allow the rendered JS sites to be issued by a proxy from Brombone. In order to correctly implement the tool, you also require an XML sitemap.

3. ANGULAR JS

With Angular JS, HTML snapshots can be prerendered so that the Googlebot can more quickly grasp and index JS sites.

4. SEO.JS

With this program, JS code is likewise rendered as HTML and made crawl-able by Google. The program code is hereby transferred to your server. Having your own dashboard will help you to manage those of your JS elements and sites that have to be rendered. Moreover, the tool creates an XML sitemap with your JS sites.

5. Google Search Console

With its Search Console, Google helps you check JS elements by rendering individual sites. Finally, the tool shows you possible crawling problems.

6. Isomorphes JavaScript.

With isomorphic JavaScript, applications can be carried out on the part of the server or the client. Back-end and front-end thus share the same code. By implementing the missing rendering, JavaScript is less error-prone with regards to SEO.

Conclusion

JavaScript can enormously expand the functionality of your website. So that the Googlebot and other search engines can actually render JavaScript, several settings that you have found in this article must be taken into account. Against the background of AMP or progressive web apps, you should always ask yourself whether the use of progressive enhancement is more sensible than the development of complete JavaScript sites.

Practice makes perfect!
Analyze your JavaScript files with Ryte for FREE

Start now!

Did you find apk for android? You can find new Free Android Games and apps.
Advertisements
Show More

Related Articles

Close
Close