Robots.txt Usage Distribution in Martinique

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	681	51.28
Ahrefs Bot Disallow	100	7.53
Robots Disallow	75	5.65
GPTBot Disallow	31	2.33
Yahoo Slurp Disallow	30	2.26
Anthropic Claude Bot Disallow	28	2.11
Majestic Bot Disallow	27	2.03
Common Crawl Bot Disallow	26	1.96
Some Robots Disallow	24	1.81
Yandex Bot Disallow	24	1.81
Google Bard Disallow	24	1.81
AppleBot Disallow	23	1.73
Facebook Bot Disallow	23	1.73
Cohere AI Disallow	23	1.73
Google AdsBot Mobile Apps Disallow	23	1.73
Apple AI Bot Disallow	22	1.66
Perplexity	22	1.66
Semrush Bot Disallow	21	1.58
Microsoft BingBot Disallow	19	1.43
Sogou Bot Disallow	16	1.2
Alexa Bot Disallow	15	1.13
Soso Bot Disallow	15	1.13
Google Image Bot Disallow	15	1.13
DuckDuckGo Bot Disallow	5	0.38
Microsoft MSNBot Disallow	3	0.23
Webz Omgili Disallow	3	0.23
ByteDance Bytespider Disallow	2	0.15
Exalead Bot Disallow	2	0.15
Google News Bot Disallow	1	0.08
Google Video Bot Disallow	1	0.08
Google Favicon Disallow	1	0.08
Youdao Bot Disallow	1	0.08
Google API Bot Disallow	1	0.08
Google DuplexWeb Disallow	1	0.08