Robots.txt Nutzungsverteilung in Ålandinseln

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	96	27,35
AppleBot Disallow	20	5,7
Anthropic Claude Bot Disallow	20	5,7
GPTBot Disallow	19	5,41
Common Crawl Bot Disallow	18	5,13
Google Bard Disallow	18	5,13
Facebook Bot Disallow	18	5,13
Apple AI Bot Disallow	18	5,13
Perplexity	18	5,13
Cohere AI Disallow	17	4,84
Robots Disallow	15	4,27
Google AdsBot Mobile Apps Disallow	14	3,99
Ahrefs Bot Disallow	7	1,99
Yandex Bot Disallow	5	1,42
Majestic Bot Disallow	5	1,42
ByteDance Bytespider Disallow	5	1,42
Webz Omgili Disallow	5	1,42
Semrush Bot Disallow	4	1,14
Diffbot Disallow	4	1,14
YOU Disallow	4	1,14
Microsoft BingBot Disallow	3	0,85
Alexa Bot Disallow	3	0,85
Yahoo Slurp Disallow	3	0,85
Soso Bot Disallow	2	0,57
Microsoft MSNBot Disallow	2	0,57
DuckDuckGo Bot Disallow	2	0,57
Google Image Bot Disallow	1	0,28
Google News Bot Disallow	1	0,28
Google Video Bot Disallow	1	0,28
Google Favicon Disallow	1	0,28
Google API Bot Disallow	1	0,28
Google DuplexWeb Disallow	1	0,28