Robots.txt Usage Distribution in Jamaica

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Google AdsBot Disallow	266	11.98
Googlebot Disallow	178	8.01
Google AdsBot Mobile Disallow	147	6.62
Ahrefs Bot Disallow	128	5.76
Robots Disallow	101	4.55
Common Crawl Bot Disallow	93	4.19
GPTBot Disallow	92	4.14
AppleBot Disallow	92	4.14
Anthropic Claude Bot Disallow	92	4.14
Google Bard Disallow	92	4.14
Facebook Bot Disallow	91	4.1
Apple AI Bot Disallow	90	4.05
Cohere AI Disallow	90	4.05
Perplexity	89	4.01
Google AdsBot Mobile Apps Disallow	89	4.01
Baidu Bot Disallow	73	3.29
Yandex Bot Disallow	56	2.52
Sogou Bot Disallow	52	2.34
Yahoo Slurp Disallow	52	2.34
Microsoft BingBot Disallow	49	2.21
Alexa Bot Disallow	46	2.07
Some Robots Disallow	43	1.94
Soso Bot Disallow	42	1.89
Majestic Bot Disallow	20	0.9
Google Media Partners Disallow	17 ^*	0.77
Microsoft MSNBot Disallow	11	0.5
Google Image Bot Disallow	11	0.5
Semrush Bot Disallow	9	0.41
Exalead Bot Disallow	4	0.18
ByteDance Bytespider Disallow	2	0.09
Webz Omgili Disallow	1	0.05
360 Bot Disallow	1	0.05
BuiltWithBot Disallow	1	0.05
Diffbot Disallow	1	0.05