Robots.txt Distribución de uso en Eslovenia

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Googlebot Disallow	3.238	10,5
Google AdsBot Disallow	3.167	10,27
Google AdsBot Mobile Disallow	2.215	7,18
Robots Disallow	1.794	5,82
Ahrefs Bot Disallow	1.631	5,29
Baidu Bot Disallow	1.586	5,14
Yandex Bot Disallow	1.143	3,71
Yahoo Slurp Disallow	956	3,1
Sogou Bot Disallow	942	3,05
Majestic Bot Disallow	935	3,03
Microsoft BingBot Disallow	899	2,92
Alexa Bot Disallow	857	2,78
GPTBot Disallow	850	2,76
Common Crawl Bot Disallow	819	2,66
Soso Bot Disallow	710	2,3
Some Robots Disallow	700	2,27
Anthropic Claude Bot Disallow	690	2,24
Google Bard Disallow	660	2,14
AppleBot Disallow	657	2,13
Semrush Bot Disallow	657	2,13
Facebook Bot Disallow	649	2,1
Apple AI Bot Disallow	623	2,02
Google AdsBot Mobile Apps Disallow	604	1,96
Cohere AI Disallow	596	1,93
Perplexity	580	1,88
Google Media Partners Disallow	521 ^*	1,69
Microsoft MSNBot Disallow	356	1,15
Webz Omgili Disallow	251	0,81
YOU Disallow	183	0,59
Google Image Bot Disallow	175	0,57
360 Bot Disallow	169	0,55
Exalead Bot Disallow	152	0,49
ByteDance Bytespider Disallow	149	0,48
Google News Bot Disallow	106	0,34
Google Video Bot Disallow	106	0,34
DuckDuckGo Bot Disallow	101	0,33
Google API Bot Disallow	96	0,31
Google Favicon Disallow	87	0,28
Google DuplexWeb Disallow	80	0,26
Diffbot Disallow	74	0,24
BuiltWithBot Disallow	41	0,13
Google FeedFetcher Disallow	16	0,05
Youdao Bot Disallow	7	0,02
Pipl Disallow	6	0,02
Google Read Aloud Disallow	4	0,01