What Are Crawl Errors And How To Remove Them Using GWT?
October 5th, 2014 | by Ravi Chahar |
While handling your website you may get some errors. You can check about crawl errors by visiting Google Webmaster Tools (GWT). You all know that Google bots crawl your website and if they face any problem in crawling then GWT is the one which will let you know about those crawl errors.
Many bloggers ask for proper guidance regarding errors but do they all know what are crawl errors?
Let me tell you. When Google tries to crawl your website and doesn’t able to crawl any particular page or your website then these errors are termed as crawl errors. As its name is signalling us about its meaning related to crawling.
How to fix crawl errors in WordPress?
After getting any error many bloggers get afraid, but there is nothing to panic about. Though there are mainly two types of crawl errors categorized as:-
1). Site Errors
2). URL errors
Site Errors :- Let me tell you about both types of errors. When Google bots crawl your whole website and find an error related to the server, domain name or robots.txt file then these type of errors are considered as site errors. Google Webmaster Tools shows you the data of crawling of last 90 days and you can check all the errors there.
-> DNS error :- As you, all know that DNS stands for Domain Name System. This error is shown by GWT when Google is not able to connect your DNS server which is the company from which you have bought the domain name of your website. In most of the cases, your DNS and web hosting is same because most of the bloggers get one free domain name with their web hosting.
You may get this error after moving your website from one web host to another one. Though Google stats that main cause of getting DNS error is the lost connection from DNS server. If you want to fix this error then you must contact to your DNS provider to check the connectivity for your domain name.
If your website is getting this error then you must try to fetch the URL of your website using “fetch as Google”. Though this error is not a problem for your website because it doesn’t stop Google to crawl your website but it may create a problem for your readers while visiting your website.
-> Server connectivity errors :- These errors may be shown to you because Google is facing problem in connecting the web host of your website. The server may be down or may be too slow for which Google drop the request and shows server connectivity errors. There are many messages you may see during this period.
When you try to open your website then you may get connection timeout, connection failed, truncated response and many error messages like these. These errors are shown just because of the web hosting service of your website. You may notice that when any web hosting company faces downtime then at that time you can’t access the website which is placed on their server.
You must know how to fix crawl errors in WordPress? And server connectivity errors are major to panic about because while getting these errors no one can visit your website because of server failure.
You can fix these errors by contacting your web hosting provider to provide you quality service. You must take care that home page size of your website shouldn’t be high because it can cause a problem while crawling. Lower the page size, great will be the speed of your website.
Many people stop Google to crawl their website. I am not talking about all the Google bots, there are few to be ignored. Choose your web hosting provider wisely.
-> Robots.txt fetch errors :- You all are well aware of the robots.txt file which tells Google bots what to crawl and what not to crawl in your website? When Google bots start crawling any website, the first thing to see the robots.txt file which guides then about crawling of the website. When Google bots don’t get this file then they return back and wait for some time according to their crawling schedule and try to crawl your website again.
When they again don’t get the robots.txt file to tell about crawling for your website then Google Webmaster Tools shows you this error. You all must know that robots.txt file is the only reason for getting this error. You all must check this file in regular interval of time.
If you don’t want any particular folder to be crawled by Google bots then add it in your robots.txt file. If you want to crawl everything in your website then you don’t need to have this file. Though Google bots adds 404 error but still they will continue to crawl your website.
If you are using any firewall to control crawling for your website then you must keep in mind that it shouldn’t be adjusted to Google blocking.
URL Errors :- These errors are related to particular pages of Desktop, Smartphone, Featured phone and Android phone. Google uses different mechanism while crawling your website for desktop, smartphones and for many other devices. You may have read about choosing a good theme for your website which is mobile friendly. Its the main reason because Google bots crawl your website for mobile phones too.
If they get any kind of difficulty while crawling pages of your website in reference to all the devices then GWT shows URL errors. There may be any certain page which is not good for any particular device to show then GWT shows an error for that page.
If you want to fix URL errors then you must always redirect with 301 redirects. Maintain your sitemap. If you get any URL error then after going through it, you should mark them as fixed in Google Webmaster Tools.
Keep in your mind that deep links for any device should be maintained properly so that Google bots can crawl them easily.
Now it’s your turn to fix Google errors :-
Do you know what are crawl errors? How to fix crawl errors in WordPress? How to fix Google crawl errors using GWT?
There are many beginners who used to ask these questions because they need to know about Google crawling and all the errors. After going through this post you may get an idea about all the errors show by Google Webmaster Tools.