Robots.txt Nutzungsverteilung In Österreich

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	51.904	13,22
Google AdsBot Disallow	42.870	10,92
Google AdsBot Mobile Disallow	36.604	9,32
Baidu Bot Disallow	22.180	5,65
Robots Disallow	18.171	4,63
Sogou Bot Disallow	14.460	3,68
Ahrefs Bot Disallow	14.311	3,64
Yandex Bot Disallow	12.658	3,22
Some Robots Disallow	12.203	3,11
Microsoft BingBot Disallow	11.611	2,96
Yahoo Slurp Disallow	11.044	2,81
Alexa Bot Disallow	10.989	2,8
Soso Bot Disallow	9.112	2,32
GPTBot Disallow	8.973	2,28
Common Crawl Bot Disallow	8.811	2,24
Anthropic Claude Bot Disallow	8.590	2,19
Google Bard Disallow	8.068	2,05
AppleBot Disallow	7.950	2,02
Facebook Bot Disallow	7.900	2,01
Google AdsBot Mobile Apps Disallow	7.899	2,01
Apple AI Bot Disallow	7.639	1,94
Cohere AI Disallow	7.562	1,93
Perplexity	7.530	1,92
Majestic Bot Disallow	6.258	1,59
Semrush Bot Disallow	5.726	1,46
Google Media Partners Disallow	5.005 ^*	1,27
ByteDance Bytespider Disallow	3.963	1,01
Google Image Bot Disallow	2.703	0,69
Microsoft MSNBot Disallow	2.427	0,62
Google News Bot Disallow	1.919	0,49
DuckDuckGo Bot Disallow	1.887	0,48
Google Video Bot Disallow	1.859	0,47
Google Favicon Disallow	1.852	0,47
Webz Omgili Disallow	1.834	0,47
Google API Bot Disallow	1.823	0,46
Google DuplexWeb Disallow	1.821	0,46
Diffbot Disallow	1.401	0,36
YOU Disallow	1.334	0,34
Exalead Bot Disallow	1.032	0,26
360 Bot Disallow	537	0,14
BuiltWithBot Disallow	272	0,07
Pipl Disallow	50	0,01
Youdao Bot Disallow	16	0
Google FeedFetcher Disallow	2	0
Google Read Aloud Disallow	1	0
Google BIG Bench Disallow	1	0