Robots.txt Distribución de uso en Montserrat

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Googlebot Disallow	259	14,2
Baidu Bot Disallow	237	12,99
Some Robots Disallow	151	8,28
Robots Disallow	151	8,28
Sogou Bot Disallow	122	6,69
Yandex Bot Disallow	103	5,65
Microsoft BingBot Disallow	103	5,65
Yahoo Slurp Disallow	102	5,59
Alexa Bot Disallow	100	5,48
Soso Bot Disallow	96	5,26
Google Media Partners Disallow	74 ^*	4,06
Google AdsBot Disallow	41	2,25
GPTBot Disallow	31	1,7
Common Crawl Bot Disallow	22	1,21
Google Bard Disallow	22	1,21
AppleBot Disallow	20	1,1
Anthropic Claude Bot Disallow	20	1,1
Facebook Bot Disallow	20	1,1
Ahrefs Bot Disallow	17	0,93
Apple AI Bot Disallow	17	0,93
Perplexity	16	0,88
Google AdsBot Mobile Apps Disallow	16	0,88
Cohere AI Disallow	13	0,71
Majestic Bot Disallow	9	0,49
Semrush Bot Disallow	8	0,44
Microsoft MSNBot Disallow	8	0,44
Google Image Bot Disallow	6	0,33
Webz Omgili Disallow	5	0,27
ByteDance Bytespider Disallow	4	0,22
Google News Bot Disallow	4	0,22
Google Video Bot Disallow	4	0,22
DuckDuckGo Bot Disallow	4	0,22
Google Favicon Disallow	4	0,22
Youdao Bot Disallow	4	0,22
Exalead Bot Disallow	3	0,16
Google API Bot Disallow	3	0,16
Google DuplexWeb Disallow	3	0,16
Diffbot Disallow	1	0,05
YOU Disallow	1	0,05