Robots.txt Usage Distribution in Maldives

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Google AdsBot Disallow	207	15.58
Robots Disallow	149	11.21
Google AdsBot Mobile Disallow	142	10.68
Googlebot Disallow	138	10.38
Ahrefs Bot Disallow	74	5.57
Common Crawl Bot Disallow	45	12.4
Some Robots Disallow	43	3.24
GPTBot Disallow	43	11.85
Anthropic Claude Bot Disallow	43	11.85
AppleBot Disallow	42	3.16
Google Bard Disallow	42	11.57
Facebook Bot Disallow	42	11.57
Cohere AI Disallow	42	11.57
Apple AI Bot Disallow	42	11.57
Perplexity	41	11.29
Google AdsBot Mobile Apps Disallow	40	3.01
Baidu Bot Disallow	32	2.41
Yandex Bot Disallow	17	1.28
Microsoft BingBot Disallow	15	1.13
Google Media Partners Disallow	13 ^*	0.98
Yahoo Slurp Disallow	13	0.98
Alexa Bot Disallow	11	3.03
Soso Bot Disallow	11	0.83
Majestic Bot Disallow	9	0.68
Semrush Bot Disallow	9	0.68
Microsoft MSNBot Disallow	5	0.38
ByteDance Bytespider Disallow	4	1.1
Webz Omgili Disallow	3	0.83
Google Image Bot Disallow	3	0.23
Diffbot Disallow	3	0.83
Exalead Bot Disallow	2	0.15
360 Bot Disallow	2	0.15
BuiltWithBot Disallow	1	0.08
YOU Disallow	1	0.28