Robots.txt Répartition des utilisations en Mauritanie

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Robots Disallow	34	21,79
Yandex Bot Disallow	13	8,33
Alexa Bot Disallow	9	5,77
Ahrefs Bot Disallow	9	5,77
Microsoft BingBot Disallow	8	5,13
Yahoo Slurp Disallow	8	5,13
Soso Bot Disallow	7	4,49
GPTBot Disallow	6	3,85
Majestic Bot Disallow	6	3,85
Common Crawl Bot Disallow	5	3,21
Anthropic Claude Bot Disallow	5	3,21
Google Bard Disallow	5	3,21
Facebook Bot Disallow	5	3,21
Perplexity	5	3,21
Google AdsBot Mobile Apps Disallow	5	3,21
Semrush Bot Disallow	5	3,21
AppleBot Disallow	4	2,56
Cohere AI Disallow	4	2,56
Apple AI Bot Disallow	4	2,56
ByteDance Bytespider Disallow	1	0,64
Microsoft MSNBot Disallow	1	0,64
Google Image Bot Disallow	1	0,64
Google News Bot Disallow	1	0,64
Google Video Bot Disallow	1	0,64
DuckDuckGo Bot Disallow	1	0,64
Google Favicon Disallow	1	0,64
Google API Bot Disallow	1	0,64
Google DuplexWeb Disallow	1	0,64