Robots.txt Répartition des utilisations en Biélorussie

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Yandex Bot Disallow	4 662	31,47
Robots Disallow	2 580	17,41
Googlebot Disallow	1 699	11,47
Semrush Bot Disallow	552	3,73
Microsoft BingBot Disallow	458	3,09
Ahrefs Bot Disallow	448	3,02
Majestic Bot Disallow	429	2,9
Alexa Bot Disallow	399	20,67
Some Robots Disallow	361	2,44
Google Media Partners Disallow	332 ^*	2,24
GPTBot Disallow	269	13,94
Yahoo Slurp Disallow	264	1,78
Baidu Bot Disallow	260	1,75
Common Crawl Bot Disallow	253	13,11
Microsoft MSNBot Disallow	250	1,69
Anthropic Claude Bot Disallow	218	11,3
Sogou Bot Disallow	209	1,41
Google Bard Disallow	201	10,41
Webz Omgili Disallow	189	9,79
Soso Bot Disallow	186	1,26
Google AdsBot Disallow	93	0,63
Google AdsBot Mobile Disallow	74	0,5
Google Image Bot Disallow	70	0,47
Exalead Bot Disallow	69	0,47
AppleBot Disallow	36	0,24
Facebook Bot Disallow	29	1,5
Apple AI Bot Disallow	23	1,19
ByteDance Bytespider Disallow	22	1,14
Google AdsBot Mobile Apps Disallow	21	0,14
Cohere AI Disallow	20	1,04
Perplexity	20	1,04
360 Bot Disallow	18	0,12
DuckDuckGo Bot Disallow	16	0,11
Google News Bot Disallow	14	0,09
Google Video Bot Disallow	13	0,09
Google API Bot Disallow	11	0,07
Google Favicon Disallow	10	0,07
Google DuplexWeb Disallow	10	0,07
Diffbot Disallow	8	0,41
BuiltWithBot Disallow	7	0,05
YOU Disallow	6	0,31
Pipl Disallow	5	0,03
Google FeedFetcher Disallow	2	0,01