Robots.txt Distribución de uso en Costa de Marfil

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Robots Disallow	262	16,93
Googlebot Disallow	204	13,18
Baidu Bot Disallow	97	6,27
Ahrefs Bot Disallow	80	5,17
Sogou Bot Disallow	76	4,91
Yandex Bot Disallow	76	4,91
Soso Bot Disallow	68	4,39
Some Robots Disallow	61	3,94
Microsoft BingBot Disallow	60	3,88
Yahoo Slurp Disallow	60	3,88
Alexa Bot Disallow	59	3,81
Common Crawl Bot Disallow	49	3,17
GPTBot Disallow	36	2,33
Google AdsBot Mobile Apps Disallow	31	2
Majestic Bot Disallow	29	1,87
Google Bard Disallow	28	1,81
Semrush Bot Disallow	28	1,81
AppleBot Disallow	27	1,74
Anthropic Claude Bot Disallow	27	1,74
Facebook Bot Disallow	27	1,74
Apple AI Bot Disallow	27	1,74
Cohere AI Disallow	26	1,68
Perplexity	26	1,68
Exalead Bot Disallow	15	0,97
360 Bot Disallow	13	0,84
BuiltWithBot Disallow	13	0,84
Google Image Bot Disallow	7	0,45
Microsoft MSNBot Disallow	5	0,32
Google News Bot Disallow	5	0,32
Google Video Bot Disallow	5	0,32
DuckDuckGo Bot Disallow	5	0,32
Google Favicon Disallow	5	0,32
Google API Bot Disallow	5	0,32
Google DuplexWeb Disallow	5	0,32
Webz Omgili Disallow	1	0,06