Robots.txt Nutzungsverteilung In Grönland

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Robots Disallow	110	9,58
Googlebot Disallow	108	9,41
Google AdsBot Disallow	106	9,23
Google AdsBot Mobile Disallow	83	7,23
Baidu Bot Disallow	54	4,7
Ahrefs Bot Disallow	46	4,01
GPTBot Disallow	44	3,83
Common Crawl Bot Disallow	41	3,57
Anthropic Claude Bot Disallow	39	3,4
Google Bard Disallow	38	3,31
Some Robots Disallow	37	3,22
Facebook Bot Disallow	37	3,22
AppleBot Disallow	34	2,96
Apple AI Bot Disallow	34	2,96
Cohere AI Disallow	32	2,79
Perplexity	32	2,79
Google AdsBot Mobile Apps Disallow	32	2,79
Yandex Bot Disallow	28	2,44
Microsoft BingBot Disallow	27	2,35
Sogou Bot Disallow	26	2,26
Alexa Bot Disallow	24	2,09
Yahoo Slurp Disallow	24	2,09
Majestic Bot Disallow	24	2,09
Soso Bot Disallow	22	1,92
Google Media Partners Disallow	15 ^*	1,31
Semrush Bot Disallow	8	0,7
Webz Omgili Disallow	8	0,7
ByteDance Bytespider Disallow	7	0,61
Microsoft MSNBot Disallow	5	0,44
Google Image Bot Disallow	4	0,35
Exalead Bot Disallow	3	0,26
Diffbot Disallow	2	0,17
Google News Bot Disallow	2	0,17
Google Video Bot Disallow	2	0,17
DuckDuckGo Bot Disallow	2	0,17
Google Favicon Disallow	2	0,17
Google API Bot Disallow	2	0,17
Google DuplexWeb Disallow	2	0,17
360 Bot Disallow	1	0,09
BuiltWithBot Disallow	1	0,09