Robots.txt Usage Distribution in Sierra Leone

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	85	22.43
Robots Disallow	51	13.46
GPTBot Disallow	21	5.54
Anthropic Claude Bot Disallow	17	4.49
Google Bard Disallow	17	4.49
Cohere AI Disallow	16	4.22
Common Crawl Bot Disallow	15	3.96
AppleBot Disallow	15	3.96
Facebook Bot Disallow	15	3.96
Apple AI Bot Disallow	15	3.96
Google Media Partners Disallow	15 ^*	3.96
Perplexity	14	3.69
Google AdsBot Mobile Apps Disallow	14	3.69
Yandex Bot Disallow	13	3.43
Ahrefs Bot Disallow	10	2.64
Majestic Bot Disallow	7	1.85
Semrush Bot Disallow	7	1.85
Microsoft BingBot Disallow	6	1.58
Alexa Bot Disallow	6	1.58
Sogou Bot Disallow	5	1.32
Yahoo Slurp Disallow	5	1.32
Soso Bot Disallow	5	1.32
ByteDance Bytespider Disallow	2	0.53
Microsoft MSNBot Disallow	1	0.26
Webz Omgili Disallow	1	0.26
Diffbot Disallow	1	0.26