Robots.txt Répartition de l'utilisation dans Ukraine

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Yandex Bot Disallow	30 637	28,95
Robots Disallow	14 233	13,45
Googlebot Disallow	13 658	12,91
Google AdsBot Disallow	4 953	4,68
Google AdsBot Mobile Disallow	4 142	3,91
Ahrefs Bot Disallow	3 482	3,29
Alexa Bot Disallow	2 909	2,75
Baidu Bot Disallow	2 813	2,66
Microsoft BingBot Disallow	2 788	2,63
Majestic Bot Disallow	2 759	2,61
Google Media Partners Disallow	2 756 ^*	2,6
Yahoo Slurp Disallow	2 498	2,36
Sogou Bot Disallow	2 448	2,31
Semrush Bot Disallow	1 980	1,87
Soso Bot Disallow	1 909	1,8
Some Robots Disallow	1 850	1,75
GPTBot Disallow	1 152	1,09
Common Crawl Bot Disallow	1 114	1,05
Microsoft MSNBot Disallow	946	0,89
Exalead Bot Disallow	745	0,7
Google Image Bot Disallow	607	0,57
Anthropic Claude Bot Disallow	600	0,57
360 Bot Disallow	548	0,52
Google Bard Disallow	503	0,48
AppleBot Disallow	457	0,43
Facebook Bot Disallow	396	0,37
Apple AI Bot Disallow	339	0,32
Youdao Bot Disallow	326	0,31
Cohere AI Disallow	314	0,3
Perplexity	305	0,29
ByteDance Bytespider Disallow	278	0,26
Google AdsBot Mobile Apps Disallow	268	0,25
Webz Omgili Disallow	216	0,2
YOU Disallow	200	0,19
BuiltWithBot Disallow	149	0,14
DuckDuckGo Bot Disallow	105	0,1
Diffbot Disallow	94	0,09
Google News Bot Disallow	92	0,09
Google Video Bot Disallow	72	0,07
Google API Bot Disallow	55	0,05
Google Favicon Disallow	51	0,05
Google DuplexWeb Disallow	50	0,05
Pipl Disallow	16	0,02
Google FeedFetcher Disallow	2	0
Google Read Aloud Disallow	1	0