Robots.txt Répartition des utilisations en Polynésie française

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Robots Disallow	62	28,18
Yandex Bot Disallow	10	4,55
Ahrefs Bot Disallow	10	4,55
GPTBot Disallow	10	12,5
Common Crawl Bot Disallow	10	12,5
Semrush Bot Disallow	10	4,55
AppleBot Disallow	9	4,09
Anthropic Claude Bot Disallow	9	11,25
Google Bard Disallow	9	11,25
Facebook Bot Disallow	9	11,25
Apple AI Bot Disallow	9	11,25
Perplexity	9	11,25
Cohere AI Disallow	8	10
Google AdsBot Mobile Apps Disallow	8	3,64
Some Robots Disallow	6	2,73
Majestic Bot Disallow	5	2,27
Google Image Bot Disallow	4	1,82
Microsoft BingBot Disallow	3	1,36
Alexa Bot Disallow	3	3,75
Yahoo Slurp Disallow	3	1,36
Soso Bot Disallow	3	1,36
ByteDance Bytespider Disallow	2	2,5
Exalead Bot Disallow	2	0,91
Microsoft MSNBot Disallow	1	0,45
Webz Omgili Disallow	1	1,25
360 Bot Disallow	1	0,45
BuiltWithBot Disallow	1	0,45
Google News Bot Disallow	1	0,45
Google Video Bot Disallow	1	0,45
Google Favicon Disallow	1	0,45