Robots.txt Usage Distribution in Saint Vincent and the Grenadines

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	1,720	15.21
Baidu Bot Disallow	1,486	13.14
Sogou Bot Disallow	1,452	12.84
Some Robots Disallow	1,171	10.36
Yandex Bot Disallow	430	3.8
Microsoft BingBot Disallow	428	3.78
Alexa Bot Disallow	426	21.07
Yahoo Slurp Disallow	421	3.72
Soso Bot Disallow	420	3.71
Google AdsBot Disallow	415	3.67
Google AdsBot Mobile Disallow	410	3.63
Robots Disallow	301	2.66
GPTBot Disallow	207	10.24
Google Bard Disallow	198	9.79
Common Crawl Bot Disallow	193	9.55
AppleBot Disallow	193	1.71
Anthropic Claude Bot Disallow	193	9.55
Facebook Bot Disallow	193	9.55
Apple AI Bot Disallow	191	9.45
Cohere AI Disallow	184	9.1
Perplexity	183	9.05
Google AdsBot Mobile Apps Disallow	177	1.57
Google Media Partners Disallow	159 ^*	1.41
Semrush Bot Disallow	37	0.33
Majestic Bot Disallow	36	0.32
Ahrefs Bot Disallow	32	0.28
Webz Omgili Disallow	16	0.79
ByteDance Bytespider Disallow	11	0.54
Diffbot Disallow	6	0.3
YOU Disallow	6	0.3
Exalead Bot Disallow	3	0.03
360 Bot Disallow	3	0.03
Microsoft MSNBot Disallow	1	0.01
Google Image Bot Disallow	1	0.01
BuiltWithBot Disallow	1	0.01
Google News Bot Disallow	1	0.01
DuckDuckGo Bot Disallow	1	0.01
Pipl Disallow	1	0.01
LLMS Text	1	0.01