Robots.txt Répartition des utilisations à Saint-Pierre-et-Miquelon

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Googlebot Disallow	986	11,99
Baidu Bot Disallow	938	11,41
Microsoft BingBot Disallow	933	11,35
Sogou Bot Disallow	927	11,27
Yandex Bot Disallow	927	11,27
Yahoo Slurp Disallow	927	11,27
Alexa Bot Disallow	923	88,33
Soso Bot Disallow	917	11,15
Robots Disallow	291	3,54
Some Robots Disallow	101	1,23
Google Media Partners Disallow	80 ^*	0,97
GPTBot Disallow	20	1,91
Google AdsBot Disallow	17	0,21
Ahrefs Bot Disallow	16	0,19
Google AdsBot Mobile Disallow	15	0,18
Anthropic Claude Bot Disallow	13	1,24
AppleBot Disallow	12	0,15
Facebook Bot Disallow	12	1,15
Semrush Bot Disallow	12	0,15
ByteDance Bytespider Disallow	12	1,15
Common Crawl Bot Disallow	11	1,05
Google Bard Disallow	11	1,05
Google AdsBot Mobile Apps Disallow	11	0,13
Microsoft MSNBot Disallow	10	0,12
Google Image Bot Disallow	10	0,12
Apple AI Bot Disallow	9	0,86
Majestic Bot Disallow	9	0,11
Perplexity	8	0,77
Cohere AI Disallow	7	0,67
Google News Bot Disallow	7	0,09
Google Video Bot Disallow	7	0,09
DuckDuckGo Bot Disallow	7	0,09
Google Favicon Disallow	7	0,09
Google API Bot Disallow	7	0,09
Google DuplexWeb Disallow	7	0,09
Webz Omgili Disallow	6	0,57
Diffbot Disallow	3	0,29
YOU Disallow	3	0,29
Exalead Bot Disallow	1	0,01
Youdao Bot Disallow	1	0,01
Google FeedFetcher Disallow	1	0,01
Google BIG Bench Disallow	1	0,01