Robots.txt Usage Distribution in Zimbabwe

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Google AdsBot Disallow	392	15.63
Googlebot Disallow	374	14.91
Google AdsBot Mobile Disallow	356	14.19
Robots Disallow	302	12.04
GPTBot Disallow	66	2.63
Baidu Bot Disallow	65	2.59
Ahrefs Bot Disallow	63	2.51
Common Crawl Bot Disallow	63	2.51
Google Bard Disallow	62	2.47
AppleBot Disallow	61	2.43
Facebook Bot Disallow	61	2.43
Anthropic Claude Bot Disallow	60	2.39
Apple AI Bot Disallow	60	2.39
Perplexity	60	2.39
Google AdsBot Mobile Apps Disallow	59	2.35
Cohere AI Disallow	58	2.31
Yandex Bot Disallow	52	2.07
Google Media Partners Disallow	45 ^*	1.79
Yahoo Slurp Disallow	34	1.36
Microsoft BingBot Disallow	31	1.24
Semrush Bot Disallow	30	1.2
Majestic Bot Disallow	29	1.16
Sogou Bot Disallow	26	1.04
Alexa Bot Disallow	26	1.04
Soso Bot Disallow	22	0.88
Microsoft MSNBot Disallow	13	0.52
Google Image Bot Disallow	9	0.36
ByteDance Bytespider Disallow	4	0.16
Webz Omgili Disallow	4	0.16
Exalead Bot Disallow	3	0.12
Google News Bot Disallow	3	0.12
360 Bot Disallow	2	0.08
Google Video Bot Disallow	2	0.08
DuckDuckGo Bot Disallow	2	0.08
Google Favicon Disallow	2	0.08
Google API Bot Disallow	2	0.08
Google DuplexWeb Disallow	2	0.08
BuiltWithBot Disallow	1	0.04
Diffbot Disallow	1	0.04
YOU Disallow	1	0.04