Robots.txt Distribución de uso en Austria

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Googlebot Disallow	52.212	14,14
Google AdsBot Disallow	43.233	11,71
Google AdsBot Mobile Disallow	36.908	10
Baidu Bot Disallow	22.221	6,02
Robots Disallow	15.514	4,2
Ahrefs Bot Disallow	13.507	3,66
Some Robots Disallow	12.304	3,33
Yandex Bot Disallow	11.433	3,1
Microsoft BingBot Disallow	10.436	2,83
Sogou Bot Disallow	10.413	2,82
Yahoo Slurp Disallow	9.932	2,69
Alexa Bot Disallow	9.922	2,69
GPTBot Disallow	8.646	2,34
Soso Bot Disallow	8.123	2,2
Common Crawl Bot Disallow	7.969	2,16
Google AdsBot Mobile Apps Disallow	7.737	2,1
Anthropic Claude Bot Disallow	7.709	2,09
AppleBot Disallow	7.379	2
Google Bard Disallow	7.238	1,96
Facebook Bot Disallow	7.087	1,92
Apple AI Bot Disallow	6.833	1,85
Cohere AI Disallow	6.748	1,83
Perplexity	6.729	1,82
Majestic Bot Disallow	5.904	1,6
Semrush Bot Disallow	5.352	1,45
Google Media Partners Disallow	4.138 ^*	1,12
ByteDance Bytespider Disallow	2.995	0,81
Google Image Bot Disallow	2.666	0,72
Microsoft MSNBot Disallow	2.389	0,65
Google News Bot Disallow	1.888	0,51
DuckDuckGo Bot Disallow	1.858	0,5
Google Video Bot Disallow	1.828	0,5
Google Favicon Disallow	1.821	0,49
Google API Bot Disallow	1.792	0,49
Google DuplexWeb Disallow	1.790	0,48
Webz Omgili Disallow	1.178	0,32
Exalead Bot Disallow	1.037	0,28
Diffbot Disallow	741	0,2
YOU Disallow	702	0,19
360 Bot Disallow	542	0,15
BuiltWithBot Disallow	278	0,08
Pipl Disallow	44	0,01
Youdao Bot Disallow	16	0
Google FeedFetcher Disallow	2	0
Google Read Aloud Disallow	1	0
Google BIG Bench Disallow	1	0