Robots.txt Usage Distribution in Albania

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	712	9.09
Google AdsBot Disallow	656	8.37
Robots Disallow	519	6.62
Google AdsBot Mobile Disallow	456	5.82
Baidu Bot Disallow	421	5.37
Ahrefs Bot Disallow	404	5.16
Sogou Bot Disallow	368	4.7
Yandex Bot Disallow	337	4.3
Anthropic Claude Bot Disallow	277	3.53
Some Robots Disallow	275	3.51
Alexa Bot Disallow	229	2.92
Semrush Bot Disallow	224	2.86
Microsoft BingBot Disallow	220	2.81
Yahoo Slurp Disallow	218	2.78
Soso Bot Disallow	215	2.74
GPTBot Disallow	215	2.74
Common Crawl Bot Disallow	208	2.65
Google Media Partners Disallow	190 ^*	2.42
AppleBot Disallow	172	2.19
Google Bard Disallow	172	2.19
Google AdsBot Mobile Apps Disallow	168	2.14
Facebook Bot Disallow	167	2.13
Apple AI Bot Disallow	164	2.09
Cohere AI Disallow	161	2.05
Perplexity	159	2.03
Majestic Bot Disallow	135	1.72
Exalead Bot Disallow	65	0.83
Microsoft MSNBot Disallow	48	0.61
360 Bot Disallow	38	0.48
Webz Omgili Disallow	36	0.46
BuiltWithBot Disallow	30	0.38
Google Image Bot Disallow	24	0.31
YOU Disallow	23	0.29
ByteDance Bytespider Disallow	17	0.22
Google News Bot Disallow	17	0.22
Google Video Bot Disallow	17	0.22
DuckDuckGo Bot Disallow	17	0.22
Google Favicon Disallow	17	0.22
Google API Bot Disallow	17	0.22
Google DuplexWeb Disallow	17	0.22
Diffbot Disallow	9	0.11
Pipl Disallow	3	0.04