Robots.txt Distribuzione dell'utilizzo In Senegal

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Googlebot Disallow	229	17,88
Robots Disallow	183	14,29
Ahrefs Bot Disallow	63	4,92
GPTBot Disallow	55	4,29
Yandex Bot Disallow	53	4,14
Common Crawl Bot Disallow	53	4,14
AppleBot Disallow	48	3,75
Google Bard Disallow	48	3,75
Anthropic Claude Bot Disallow	46	3,59
Facebook Bot Disallow	46	3,59
Apple AI Bot Disallow	46	3,59
Cohere AI Disallow	46	3,59
Perplexity	46	3,59
Google AdsBot Mobile Apps Disallow	46	3,59
Sogou Bot Disallow	36	2,81
Microsoft BingBot Disallow	35	2,73
Alexa Bot Disallow	35	2,73
Soso Bot Disallow	34	2,65
Yahoo Slurp Disallow	32	2,5
Majestic Bot Disallow	26	2,03
Semrush Bot Disallow	23	1,8
Google Media Partners Disallow	15 ^*	1,17
Exalead Bot Disallow	7	0,55
Microsoft MSNBot Disallow	5	0,39
Google Image Bot Disallow	5	0,39
360 Bot Disallow	5	0,39
BuiltWithBot Disallow	3	0,23
Google News Bot Disallow	3	0,23
Webz Omgili Disallow	2	0,16
YOU Disallow	2	0,16
ByteDance Bytespider Disallow	1	0,08
Diffbot Disallow	1	0,08
DuckDuckGo Bot Disallow	1	0,08
Youdao Bot Disallow	1	0,08
Pipl Disallow	1	0,08