Robots.txt Usage Distribution in Chile

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Google AdsBot Disallow	24,679	22.24
Googlebot Disallow	20,744	18.69
Google AdsBot Mobile Disallow	14,154	12.75
Ahrefs Bot Disallow	10,703	9.64
Robots Disallow	5,647	5.09
Anthropic Claude Bot Disallow	3,784	3.41
Baidu Bot Disallow	3,659	3.3
Microsoft BingBot Disallow	2,538	2.29
Some Robots Disallow	2,177	1.96
Yandex Bot Disallow	1,976	1.78
Yahoo Slurp Disallow	1,901	1.71
Sogou Bot Disallow	1,865	1.68
Alexa Bot Disallow	1,802	1.62
Soso Bot Disallow	1,504	1.36
GPTBot Disallow	1,222	1.1
Common Crawl Bot Disallow	1,215	1.09
AppleBot Disallow	1,140	1.03
Google Bard Disallow	1,108	1
Facebook Bot Disallow	1,082	0.97
Apple AI Bot Disallow	1,059	0.95
Perplexity	918	0.83
Google AdsBot Mobile Apps Disallow	898	0.81
Majestic Bot Disallow	894	0.81
Cohere AI Disallow	890	0.8
Semrush Bot Disallow	807	0.73
Google Media Partners Disallow	744 ^*	0.67
Microsoft MSNBot Disallow	410	0.37
Exalead Bot Disallow	390	0.35
Webz Omgili Disallow	245	0.22
Google Image Bot Disallow	227	0.2
ByteDance Bytespider Disallow	115	0.1
360 Bot Disallow	64	0.06
Youdao Bot Disallow	56	0.05
Google News Bot Disallow	50	0.05
DuckDuckGo Bot Disallow	46	0.04
Google Video Bot Disallow	43	0.04
Google Favicon Disallow	43	0.04
BuiltWithBot Disallow	41	0.04
Google API Bot Disallow	37	0.03
Google DuplexWeb Disallow	36	0.03
Diffbot Disallow	27	0.02
YOU Disallow	26	0.02
Pipl Disallow	10	0.01