Robots.txt Usage Distribution in French Polynesia

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Robots Disallow	65	23.38
Yandex Bot Disallow	18	6.47
Ahrefs Bot Disallow	14	5.04
Semrush Bot Disallow	14	5.04
GPTBot Disallow	13	4.68
Common Crawl Bot Disallow	12	4.32
Anthropic Claude Bot Disallow	12	4.32
Google Bard Disallow	12	4.32
Google AdsBot Mobile Apps Disallow	12	4.32
AppleBot Disallow	11	3.96
Facebook Bot Disallow	11	3.96
Apple AI Bot Disallow	11	3.96
Cohere AI Disallow	11	3.96
Perplexity	11	3.96
Google Image Bot Disallow	8	2.88
Some Robots Disallow	6	2.16
Majestic Bot Disallow	6	2.16
Microsoft BingBot Disallow	4	1.44
Yahoo Slurp Disallow	4	1.44
Alexa Bot Disallow	3	1.08
Soso Bot Disallow	3	1.08
ByteDance Bytespider Disallow	2	0.72
Exalead Bot Disallow	2	0.72
Microsoft MSNBot Disallow	2	0.72
Google News Bot Disallow	2	0.72
Google Video Bot Disallow	2	0.72
Google Favicon Disallow	2	0.72
360 Bot Disallow	1	0.36
BuiltWithBot Disallow	1	0.36
DuckDuckGo Bot Disallow	1	0.36
Google API Bot Disallow	1	0.36
Google DuplexWeb Disallow	1	0.36