Robots.txt Usage Distribution in Greece

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	9,107	10.27
Google AdsBot Disallow	8,163	9.21
Google AdsBot Mobile Disallow	6,390	7.21
Robots Disallow	4,635	5.23
Ahrefs Bot Disallow	4,168	4.7
Baidu Bot Disallow	3,728	4.21
Sogou Bot Disallow	3,576	4.03
Yandex Bot Disallow	3,525	3.98
Microsoft BingBot Disallow	2,847	3.21
Soso Bot Disallow	2,719	3.07
Common Crawl Bot Disallow	2,709	11.41
Semrush Bot Disallow	2,601	2.93
GPTBot Disallow	2,596	10.93
Yahoo Slurp Disallow	2,572	2.9
Majestic Bot Disallow	2,485	2.8
Alexa Bot Disallow	2,473	10.41
Anthropic Claude Bot Disallow	2,165	9.12
Some Robots Disallow	1,911	2.16
Google Bard Disallow	1,776	7.48
Facebook Bot Disallow	1,752	7.38
AppleBot Disallow	1,719	1.94
Apple AI Bot Disallow	1,643	6.92
Cohere AI Disallow	1,610	6.78
Perplexity	1,590	6.7
ByteDance Bytespider Disallow	1,191	5.01
Exalead Bot Disallow	1,125	1.27
YOU Disallow	1,085	4.57
360 Bot Disallow	956	1.08
Google AdsBot Mobile Apps Disallow	937	1.06
Webz Omgili Disallow	903	3.8
Microsoft MSNBot Disallow	865	0.98
Google Media Partners Disallow	862 ^*	0.97
Diffbot Disallow	733	3.09
BuiltWithBot Disallow	593	0.67
Google Image Bot Disallow	332	0.37
DuckDuckGo Bot Disallow	217	0.24
Google News Bot Disallow	80	0.09
Google Video Bot Disallow	78	0.09
Google Favicon Disallow	69	0.08
Google API Bot Disallow	66	0.07
Google DuplexWeb Disallow	61	0.07
Pipl Disallow	22	0.02
Youdao Bot Disallow	13	0.01
Google FeedFetcher Disallow	3	0
LLMS Text	1	0