Robots.txt Usage Distribution in Armenia

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	1,183	16.26
Robots Disallow	851	11.7
Baidu Bot Disallow	735	10.1
Sogou Bot Disallow	723	9.94
Some Robots Disallow	616	8.47
Google AdsBot Disallow	442	6.08
Google AdsBot Mobile Disallow	377	5.18
Yandex Bot Disallow	345	4.74
Google Media Partners Disallow	226 ^*	3.11
Alexa Bot Disallow	192	22.8
Microsoft BingBot Disallow	183	2.52
Yahoo Slurp Disallow	172	2.36
Soso Bot Disallow	159	2.19
Ahrefs Bot Disallow	133	1.83
Common Crawl Bot Disallow	86	10.21
GPTBot Disallow	84	9.98
Semrush Bot Disallow	81	1.11
Google Bard Disallow	78	9.26
Anthropic Claude Bot Disallow	76	9.03
AppleBot Disallow	72	0.99
Majestic Bot Disallow	69	0.95
Facebook Bot Disallow	67	7.96
Apple AI Bot Disallow	67	7.96
Cohere AI Disallow	49	5.82
Perplexity	48	5.7
Google AdsBot Mobile Apps Disallow	43	0.59
Webz Omgili Disallow	27	3.21
Microsoft MSNBot Disallow	26	0.36
ByteDance Bytespider Disallow	13	1.54
YOU Disallow	13	1.54
Exalead Bot Disallow	10	0.14
360 Bot Disallow	6	0.08
Diffbot Disallow	4	0.48
Pipl Disallow	4	0.05
Google Image Bot Disallow	3	0.04
BuiltWithBot Disallow	3	0.04
Google News Bot Disallow	2	0.03
Google Video Bot Disallow	1	0.01
DuckDuckGo Bot Disallow	1	0.01
Google Favicon Disallow	1	0.01
Youdao Bot Disallow	1	0.01
Google API Bot Disallow	1	0.01
Google DuplexWeb Disallow	1	0.01
Google FeedFetcher Disallow	1	0.01