Robots.txt Répartition de l'utilisation dans Îles Vierges britanniques

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Yandex Bot Disallow	8 897	12,49
Microsoft BingBot Disallow	8 891	12,48
Yahoo Slurp Disallow	8 891	12,48
Soso Bot Disallow	8 891	12,48
Googlebot Disallow	8 788	12,33
Baidu Bot Disallow	8 780	12,32
Sogou Bot Disallow	8 768	12,31
Some Robots Disallow	8 705	12,22
Alexa Bot Disallow	239	0,34
Robots Disallow	79	0,11
GPTBot Disallow	29	0,04
Google Bard Disallow	27	0,04
Common Crawl Bot Disallow	26	0,04
AppleBot Disallow	26	0,04
Anthropic Claude Bot Disallow	26	0,04
Facebook Bot Disallow	26	0,04
Apple AI Bot Disallow	26	0,04
Cohere AI Disallow	23	0,03
Google AdsBot Mobile Apps Disallow	23	0,03
Perplexity	22	0,03
Google Media Partners Disallow	17 ^*	0,02
Majestic Bot Disallow	12	0,02
Semrush Bot Disallow	10	0,01
Ahrefs Bot Disallow	9	0,01
Webz Omgili Disallow	4	0,01
Google Image Bot Disallow	3	0
ByteDance Bytespider Disallow	2	0
Exalead Bot Disallow	2	0
Microsoft MSNBot Disallow	2	0
360 Bot Disallow	2	0
Diffbot Disallow	2	0
Google News Bot Disallow	1	0
Google Video Bot Disallow	1	0
DuckDuckGo Bot Disallow	1	0
Google Favicon Disallow	1	0
Google API Bot Disallow	1	0
Google DuplexWeb Disallow	1	0
Pipl Disallow	1	0