Robots.txt Nutzungsverteilung In Britische Jungferninseln

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Yandex Bot Disallow	8.897	12,42
Microsoft BingBot Disallow	8.891	12,42
Yahoo Slurp Disallow	8.891	12,42
Soso Bot Disallow	8.891	12,42
Googlebot Disallow	8.888	12,41
Baidu Bot Disallow	8.880	12,4
Sogou Bot Disallow	8.870	12,39
Some Robots Disallow	8.757	12,23
Alexa Bot Disallow	239	0,33
Robots Disallow	79	0,11
GPTBot Disallow	29	0,04
Google Bard Disallow	27	0,04
Common Crawl Bot Disallow	26	0,04
AppleBot Disallow	26	0,04
Anthropic Claude Bot Disallow	26	0,04
Facebook Bot Disallow	26	0,04
Apple AI Bot Disallow	26	0,04
Cohere AI Disallow	23	0,03
Google AdsBot Mobile Apps Disallow	23	0,03
Perplexity	22	0,03
Google Media Partners Disallow	17 ^*	0,02
Majestic Bot Disallow	12	0,02
Semrush Bot Disallow	10	0,01
Ahrefs Bot Disallow	9	0,01
Webz Omgili Disallow	4	0,01
Google Image Bot Disallow	3	0
ByteDance Bytespider Disallow	2	0
Exalead Bot Disallow	2	0
Microsoft MSNBot Disallow	2	0
360 Bot Disallow	2	0
Diffbot Disallow	2	0
Google News Bot Disallow	1	0
Google Video Bot Disallow	1	0
DuckDuckGo Bot Disallow	1	0
Google Favicon Disallow	1	0
Google API Bot Disallow	1	0
Google DuplexWeb Disallow	1	0
Pipl Disallow	1	0