Robots.txt Usage Distribution in Liberia

Top In Robots.txt Usage Distribution in Liberia

Technology	Websites	%
GPTBot Disallow	26	8.75
Google Bard Disallow	25	8.42
Robots Disallow	23	7.74
Common Crawl Bot Disallow	23	7.74
Anthropic Claude Bot Disallow	23	7.74
Facebook Bot Disallow	23	7.74
Cohere AI Disallow	23	7.74
AppleBot Disallow	22	7.41
Apple AI Bot Disallow	22	7.41
Perplexity	22	7.41
Google AdsBot Mobile Apps Disallow	22	7.41
Ahrefs Bot Disallow	6	2.02
Sogou Bot Disallow	5	1.68
Yandex Bot Disallow	5	1.68
Semrush Bot Disallow	5	1.68
Microsoft BingBot Disallow	4	1.35
Alexa Bot Disallow	4	1.35
Yahoo Slurp Disallow	4	1.35
Soso Bot Disallow	4	1.35
Majestic Bot Disallow	3	1.01
ByteDance Bytespider Disallow	1	0.34
Webz Omgili Disallow	1	0.34
Diffbot Disallow	1	0.34

297 Detections

of Robots.txt in Liberia. Last updated 18th Dec 2024.

GPTBot Disallow is currently the most popular technology in this category.