Robots.txt Répartition de l'utilisation dans Sainte-Hélène

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Googlebot Disallow	1 642	16,35
Baidu Bot Disallow	1 534	15,27
Some Robots Disallow	1 262	12,57
Robots Disallow	911	9,07
Sogou Bot Disallow	532	5,3
Yandex Bot Disallow	434	4,32
Microsoft BingBot Disallow	414	4,12
Yahoo Slurp Disallow	404	4,02
Alexa Bot Disallow	403	4,01
Soso Bot Disallow	382	3,8
Google Media Partners Disallow	313 ^*	3,12
Google AdsBot Disallow	187	1,86
Ahrefs Bot Disallow	175	1,74
GPTBot Disallow	164	1,63
Majestic Bot Disallow	111	1,11
Google Bard Disallow	103	1,03
Common Crawl Bot Disallow	102	1,02
Semrush Bot Disallow	100	1
Anthropic Claude Bot Disallow	87	0,87
Google AdsBot Mobile Disallow	86	0,86
Facebook Bot Disallow	83	0,83
AppleBot Disallow	81	0,81
Apple AI Bot Disallow	69	0,69
Webz Omgili Disallow	63	0,63
Perplexity	58	0,58
Cohere AI Disallow	55	0,55
ByteDance Bytespider Disallow	44	0,44
Google AdsBot Mobile Apps Disallow	32	0,32
Diffbot Disallow	31	0,31
Microsoft MSNBot Disallow	27	0,27
YOU Disallow	25	0,25
Google Image Bot Disallow	23	0,23
Exalead Bot Disallow	15	0,15
Google News Bot Disallow	14	0,14
DuckDuckGo Bot Disallow	14	0,14
Google Video Bot Disallow	12	0,12
Pipl Disallow	12	0,12
Google Favicon Disallow	11	0,11
Google API Bot Disallow	9	0,09
Google DuplexWeb Disallow	9	0,09
360 Bot Disallow	7	0,07
BuiltWithBot Disallow	2	0,02
Youdao Bot Disallow	1	0,01