Robots.txt Usage Distribution in Congo

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Robots Disallow	270	25.54
Googlebot Disallow	179	16.93
Yandex Bot Disallow	56	5.3
Yahoo Slurp Disallow	56	5.3
Microsoft BingBot Disallow	53	5.01
Soso Bot Disallow	53	5.01
Alexa Bot Disallow	52	19.48
GPTBot Disallow	34	12.73
Ahrefs Bot Disallow	28	2.65
Common Crawl Bot Disallow	28	10.49
Google Bard Disallow	25	9.36
AppleBot Disallow	24	2.27
Anthropic Claude Bot Disallow	24	8.99
Facebook Bot Disallow	24	8.99
Apple AI Bot Disallow	23	8.61
Perplexity	22	8.24
Cohere AI Disallow	20	7.49
Google AdsBot Mobile Apps Disallow	20	1.89
Semrush Bot Disallow	18	1.7
Majestic Bot Disallow	16	1.51
ByteDance Bytespider Disallow	5	1.87
Microsoft MSNBot Disallow	5	0.47
Webz Omgili Disallow	4	1.5
Exalead Bot Disallow	3	0.28
360 Bot Disallow	3	0.28
Diffbot Disallow	2	0.75
BuiltWithBot Disallow	2	0.19
Google Image Bot Disallow	1	0.09
Google News Bot Disallow	1	0.09
Google Video Bot Disallow	1	0.09
DuckDuckGo Bot Disallow	1	0.09
Google Favicon Disallow	1	0.09
Youdao Bot Disallow	1	0.09
Google API Bot Disallow	1	0.09
Google DuplexWeb Disallow	1	0.09