Robots.txt Distribución de uso en India

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Googlebot Disallow	122.498	15,78
Google AdsBot Disallow	112.924	14,54
Baidu Bot Disallow	61.387	7,91
Google AdsBot Mobile Disallow	61.142	7,87
Ahrefs Bot Disallow	55.147	7,1
Robots Disallow	52.448	6,75
Sogou Bot Disallow	36.301	4,67
Yandex Bot Disallow	34.561	4,45
Microsoft BingBot Disallow	33.983	4,38
Alexa Bot Disallow	33.772	4,35
Some Robots Disallow	33.166	4,27
Yahoo Slurp Disallow	32.881	4,23
Soso Bot Disallow	31.403	4,04
GPTBot Disallow	6.393	0,82
Google Media Partners Disallow	6.222 ^*	0,8
Majestic Bot Disallow	5.506	0,71
Common Crawl Bot Disallow	5.490	0,71
Google Bard Disallow	4.826	0,62
Anthropic Claude Bot Disallow	4.724	0,61
AppleBot Disallow	4.631	0,6
Facebook Bot Disallow	4.584	0,59
Apple AI Bot Disallow	4.222	0,54
Semrush Bot Disallow	4.149	0,53
Perplexity	3.938	0,51
Google AdsBot Mobile Apps Disallow	3.545	0,46
Cohere AI Disallow	3.526	0,45
BuiltWithBot Disallow	3.084	0,4
Google Image Bot Disallow	2.024	0,26
Microsoft MSNBot Disallow	1.795	0,23
Webz Omgili Disallow	1.228	0,16
ByteDance Bytespider Disallow	1.148	0,15
Exalead Bot Disallow	1.043	0,13
360 Bot Disallow	490	0,06
DuckDuckGo Bot Disallow	414	0,05
Google News Bot Disallow	365	0,05
Google Video Bot Disallow	297	0,04
YOU Disallow	254	0,03
Google Favicon Disallow	248	0,03
Google API Bot Disallow	176	0,02
Youdao Bot Disallow	174	0,02
Google DuplexWeb Disallow	169	0,02
Diffbot Disallow	156	0,02
Pipl Disallow	81	0,01
Google FeedFetcher Disallow	13	0
Google Read Aloud Disallow	3	0
Google Web Light Disallow	1	0