Robots.txt Distribución de uso en Bielorrusia

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Yandex Bot Disallow	6.323	34,16
Googlebot Disallow	2.459	13,28
Robots Disallow	2.415	13,05
Semrush Bot Disallow	706	3,81
Microsoft BingBot Disallow	582	3,14
Ahrefs Bot Disallow	528	2,85
Majestic Bot Disallow	497	2,68
Alexa Bot Disallow	494	2,67
Some Robots Disallow	470	2,54
GPTBot Disallow	378	2,04
Common Crawl Bot Disallow	377	2,04
Anthropic Claude Bot Disallow	322	1,74
Google Bard Disallow	312	1,69
Google Media Partners Disallow	298 ^*	1,61
Baidu Bot Disallow	293	1,58
Yahoo Slurp Disallow	292	1,58
Microsoft MSNBot Disallow	291	1,57
Webz Omgili Disallow	284	1,53
Sogou Bot Disallow	216	1,17
Soso Bot Disallow	207	1,12
Google AdsBot Disallow	135	0,73
Google AdsBot Mobile Disallow	107	0,58
Exalead Bot Disallow	100	0,54
Google Image Bot Disallow	84	0,45
AppleBot Disallow	46	0,25
Facebook Bot Disallow	38	0,21
Apple AI Bot Disallow	32	0,17
Google AdsBot Mobile Apps Disallow	31	0,17
360 Bot Disallow	30	0,16
Cohere AI Disallow	25	0,14
Perplexity	23	0,12
DuckDuckGo Bot Disallow	17	0,09
Google News Bot Disallow	16	0,09
ByteDance Bytespider Disallow	14	0,08
Google Video Bot Disallow	14	0,08
Google API Bot Disallow	12	0,06
BuiltWithBot Disallow	11	0,06
Google Favicon Disallow	11	0,06
Google DuplexWeb Disallow	11	0,06
Diffbot Disallow	4	0,02
YOU Disallow	2	0,01
Pipl Disallow	2	0,01
Google FeedFetcher Disallow	2	0,01
Youdao Bot Disallow	1	0,01