Robots.txt Usage Distribution in Mali

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Robots Disallow	78	14.29
Googlebot Disallow	65	11.9
Google Media Partners Disallow	59 ^*	10.81
Google AdsBot Disallow	52	9.52
Baidu Bot Disallow	28	5.13
Sogou Bot Disallow	22	4.03
Yandex Bot Disallow	20	3.66
Alexa Bot Disallow	18	3.3
Microsoft BingBot Disallow	16	2.93
Yahoo Slurp Disallow	16	2.93
Soso Bot Disallow	15	2.75
GPTBot Disallow	14	2.56
Some Robots Disallow	13	2.38
Common Crawl Bot Disallow	13	2.38
Ahrefs Bot Disallow	11	2.01
Anthropic Claude Bot Disallow	11	2.01
Google Bard Disallow	11	2.01
Facebook Bot Disallow	11	2.01
Perplexity	11	2.01
AppleBot Disallow	10	1.83
Cohere AI Disallow	10	1.83
Apple AI Bot Disallow	9	1.65
Google AdsBot Mobile Apps Disallow	9	1.65
Majestic Bot Disallow	7	1.28
Semrush Bot Disallow	6	1.1
ByteDance Bytespider Disallow	4	0.73
Exalead Bot Disallow	1	0.18
Microsoft MSNBot Disallow	1	0.18
Webz Omgili Disallow	1	0.18
Google Image Bot Disallow	1	0.18
Diffbot Disallow	1	0.18
YOU Disallow	1	0.18
Youdao Bot Disallow	1	0.18