Robots.txt Usage Distribution in Argentina

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	17,833	14.93
Google AdsBot Disallow	10,985	9.19
Google AdsBot Mobile Disallow	9,859	8.25
Baidu Bot Disallow	8,352	6.99
Robots Disallow	8,063	6.75
Anthropic Claude Bot Disallow	6,740	5.64
Sogou Bot Disallow	6,621	5.54
Yandex Bot Disallow	5,587	4.68
Microsoft BingBot Disallow	5,154	4.31
Yahoo Slurp Disallow	5,081	4.25
Alexa Bot Disallow	4,960	4.15
Soso Bot Disallow	4,687	3.92
Some Robots Disallow	3,929	3.29
Ahrefs Bot Disallow	2,336	1.96
GPTBot Disallow	1,582	1.32
Majestic Bot Disallow	1,570	1.31
Common Crawl Bot Disallow	1,526	1.28
Semrush Bot Disallow	1,523	1.27
Google Bard Disallow	1,343	1.12
AppleBot Disallow	1,337	1.12
Google Media Partners Disallow	1,332 ^*	1.11
Facebook Bot Disallow	1,318	1.1
Cohere AI Disallow	1,216	1.02
Apple AI Bot Disallow	1,198	1
Perplexity	1,132	0.95
Google AdsBot Mobile Apps Disallow	1,053	0.88
Microsoft MSNBot Disallow	574	0.48
Google Image Bot Disallow	412	0.34
Exalead Bot Disallow	358	0.3
Pipl Disallow	342	0.29
Webz Omgili Disallow	326	0.27
ByteDance Bytespider Disallow	306	0.26
360 Bot Disallow	161	0.13
BuiltWithBot Disallow	127	0.11
Diffbot Disallow	117	0.1
YOU Disallow	100	0.08
DuckDuckGo Bot Disallow	63	0.05
Google Video Bot Disallow	60	0.05
Google News Bot Disallow	52	0.04
Google Favicon Disallow	42	0.04
Google API Bot Disallow	41	0.03
Google DuplexWeb Disallow	41	0.03
Youdao Bot Disallow	31	0.03
Google FeedFetcher Disallow	2	0