Robots.txt Nutzungsverteilung In Französisch Polynesien

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Robots Disallow	55	20,75
Yandex Bot Disallow	16	6,04
Ahrefs Bot Disallow	16	6,04
GPTBot Disallow	13	4,91
Semrush Bot Disallow	13	4,91
Common Crawl Bot Disallow	12	4,53
Anthropic Claude Bot Disallow	12	4,53
Google Bard Disallow	12	4,53
Google AdsBot Mobile Apps Disallow	12	4,53
AppleBot Disallow	11	4,15
Facebook Bot Disallow	11	4,15
Apple AI Bot Disallow	11	4,15
Cohere AI Disallow	11	4,15
Perplexity	11	4,15
Google Image Bot Disallow	8	3,02
Majestic Bot Disallow	7	2,64
Some Robots Disallow	5	1,89
Microsoft BingBot Disallow	3	1,13
Yahoo Slurp Disallow	3	1,13
Sogou Bot Disallow	2	0,75
Alexa Bot Disallow	2	0,75
Soso Bot Disallow	2	0,75
ByteDance Bytespider Disallow	2	0,75
Exalead Bot Disallow	2	0,75
Microsoft MSNBot Disallow	2	0,75
Google News Bot Disallow	2	0,75
Google Video Bot Disallow	2	0,75
Google Favicon Disallow	2	0,75
360 Bot Disallow	1	0,38
BuiltWithBot Disallow	1	0,38
DuckDuckGo Bot Disallow	1	0,38
Google API Bot Disallow	1	0,38
Google DuplexWeb Disallow	1	0,38