Robots.txt Nutzungsverteilung In Turks- und Caicosinseln

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	104	8,04
Google AdsBot Disallow	89	6,88
Robots Disallow	76	5,88
Google Media Partners Disallow	68 ^*	5,26
GPTBot Disallow	65	5,03
Common Crawl Bot Disallow	62	4,8
Some Robots Disallow	61	4,72
Baidu Bot Disallow	61	4,72
Google Bard Disallow	61	4,72
Anthropic Claude Bot Disallow	60	4,64
Facebook Bot Disallow	59	4,56
AppleBot Disallow	58	4,49
Apple AI Bot Disallow	58	4,49
Google AdsBot Mobile Apps Disallow	58	4,49
Perplexity	57	4,41
Cohere AI Disallow	56	4,33
Sogou Bot Disallow	47	3,63
Yandex Bot Disallow	33	2,55
Alexa Bot Disallow	27	2,09
Yahoo Slurp Disallow	27	2,09
Microsoft BingBot Disallow	25	1,93
Soso Bot Disallow	23	1,78
Ahrefs Bot Disallow	13	1,01
Microsoft MSNBot Disallow	7	0,54
Majestic Bot Disallow	6	0,46
Semrush Bot Disallow	4	0,31
Google Image Bot Disallow	4	0,31
ByteDance Bytespider Disallow	3	0,23
Webz Omgili Disallow	3	0,23
Exalead Bot Disallow	2	0,15
360 Bot Disallow	2	0,15
Google News Bot Disallow	2	0,15
Google Video Bot Disallow	2	0,15
DuckDuckGo Bot Disallow	2	0,15
Google Favicon Disallow	2	0,15
Google API Bot Disallow	2	0,15
Google DuplexWeb Disallow	2	0,15
BuiltWithBot Disallow	1	0,08
Google FeedFetcher Disallow	1	0,08