Robots.txt Nutzungsverteilung In Liechtenstein

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	1.848	11,81
Baidu Bot Disallow	1.423	9,09
Robots Disallow	1.322	8,45
Sogou Bot Disallow	1.008	6,44
Microsoft BingBot Disallow	832	5,32
Yandex Bot Disallow	809	5,17
Some Robots Disallow	772	4,93
Google AdsBot Disallow	701	4,48
Yahoo Slurp Disallow	694	4,44
Alexa Bot Disallow	655	4,19
Soso Bot Disallow	635	4,06
Google AdsBot Mobile Disallow	613	3,92
Anthropic Claude Bot Disallow	330	2,11
Google Media Partners Disallow	304 ^*	1,94
GPTBot Disallow	287	1,83
Google Bard Disallow	251	1,6
Common Crawl Bot Disallow	247	1,58
Ahrefs Bot Disallow	234	1,5
AppleBot Disallow	231	1,48
Facebook Bot Disallow	231	1,48
Apple AI Bot Disallow	215	1,37
Cohere AI Disallow	202	1,29
Perplexity	195	1,25
Majestic Bot Disallow	193	1,23
Semrush Bot Disallow	191	1,22
Google AdsBot Mobile Apps Disallow	171	1,09
Webz Omgili Disallow	128	0,82
ByteDance Bytespider Disallow	110	0,7
Diffbot Disallow	89	0,57
Microsoft MSNBot Disallow	87	0,56
YOU Disallow	85	0,54
Google Image Bot Disallow	81	0,52
DuckDuckGo Bot Disallow	75	0,48
Google News Bot Disallow	67	0,43
Google Video Bot Disallow	67	0,43
Google API Bot Disallow	64	0,41
Google Favicon Disallow	63	0,4
Google DuplexWeb Disallow	63	0,4
Exalead Bot Disallow	32	0,2
360 Bot Disallow	22	0,14
BuiltWithBot Disallow	13	0,08
Pipl Disallow	6	0,04
Google BIG Bench Disallow	1	0,01