Robots.txt Usage Distribution in Svalbard and Jan Mayen

Top In Robots.txt Usage Distribution in Svalbard and Jan Mayen

Technology	Websites	%
Google AdsBot Disallow	23	20.18
Google AdsBot Mobile Disallow	23	20.18
Googlebot Disallow	14	12.28
GPTBot Disallow	5	4.39
Common Crawl Bot Disallow	5	4.39
AppleBot Disallow	5	4.39
Anthropic Claude Bot Disallow	5	4.39
Google Bard Disallow	5	4.39
Facebook Bot Disallow	5	4.39
Cohere AI Disallow	5	4.39
Apple AI Bot Disallow	5	4.39
Perplexity	5	4.39
Google AdsBot Mobile Apps Disallow	5	4.39
Robots Disallow	3	2.63
Some Robots Disallow	1	0.88

114 Detections

of Robots.txt in Svalbard and Jan Mayen. Last updated 07th Mar 2025.

Google AdsBot Disallow is currently the most popular technology in this category.