Robots.txt Distribución de uso en Mongolia

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Googlebot Disallow	459	14,54
Google AdsBot Disallow	302	9,57
Google AdsBot Mobile Disallow	253	8,01
Robots Disallow	244	7,73
Baidu Bot Disallow	196	6,21
Some Robots Disallow	140	4,43
Sogou Bot Disallow	139	4,4
Yandex Bot Disallow	134	4,24
Microsoft BingBot Disallow	132	4,18
Alexa Bot Disallow	125	3,96
Yahoo Slurp Disallow	125	3,96
Soso Bot Disallow	123	3,9
GPTBot Disallow	119	3,77
Ahrefs Bot Disallow	60	1,9
Common Crawl Bot Disallow	50	1,58
AppleBot Disallow	49	1,55
Anthropic Claude Bot Disallow	48	1,52
Facebook Bot Disallow	46	1,46
Apple AI Bot Disallow	46	1,46
Google Bard Disallow	44	1,39
Perplexity	43	1,36
Cohere AI Disallow	42	1,33
Google AdsBot Mobile Apps Disallow	42	1,33
Google Media Partners Disallow	30 ^*	0,95
Microsoft MSNBot Disallow	27	0,86
Semrush Bot Disallow	24	0,76
Majestic Bot Disallow	20	0,63
ByteDance Bytespider Disallow	12	0,38
Google Image Bot Disallow	12	0,38
Webz Omgili Disallow	10	0,32
360 Bot Disallow	8	0,25
DuckDuckGo Bot Disallow	8	0,25
Google News Bot Disallow	7	0,22
Google Video Bot Disallow	7	0,22
Google Favicon Disallow	7	0,22
Google API Bot Disallow	6	0,19
Google DuplexWeb Disallow	6	0,19
Exalead Bot Disallow	5	0,16
Diffbot Disallow	4	0,13
BuiltWithBot Disallow	2	0,06
Youdao Bot Disallow	1	0,03