Robot.txt Distribuzione dell'utilizzo in Arabia Saudita

Distribuzione per siti web che utilizzano Robot.txt tecnologie

Tecnologia	Siti web	%
Ahrefs Bot Disallow	3.384	9,72
Sogou Bot Disallow	3.151	9,05
Common Crawl Bot Disallow	3.108	60,6
Soso Bot Disallow	3.069	8,82
Majestic Bot Disallow	2.958	8,5
Exalead Bot Disallow	2.874	8,26
360 Bot Disallow	2.865	8,23
BuiltWithBot Disallow	2.861	8,22
Google AdsBot Disallow	1.729	4,97
Robots Disallow	1.615	4,64
Googlebot Disallow	1.576	4,53
Google AdsBot Mobile Disallow	1.298	3,73
Baidu Bot Disallow	393	1,13
Some Robots Disallow	342	0,98
Yandex Bot Disallow	287	0,82
GPTBot Disallow	272	5,3
Google Bard Disallow	249	4,85
Anthropic Claude Bot Disallow	247	4,82
AppleBot Disallow	246	0,71
Microsoft BingBot Disallow	243	0,7
Alexa Bot Disallow	242	4,72
Yahoo Slurp Disallow	239	0,69
Facebook Bot Disallow	239	4,66
Perplexity	235	4,58
Apple AI Bot Disallow	233	4,54
Cohere AI Disallow	232	4,52
Google AdsBot Mobile Apps Disallow	231	0,66
Semrush Bot Disallow	136	0,39
Google Media Partners Disallow	80 ^*	0,23
Microsoft MSNBot Disallow	32	0,09
Google Image Bot Disallow	32	0,09
ByteDance Bytespider Disallow	22	0,43
Google News Bot Disallow	14	0,04
DuckDuckGo Bot Disallow	13	0,04
Google Video Bot Disallow	12	0,03
Webz Omgili Disallow	11	0,21
Diffbot Disallow	7	0,14
Google Favicon Disallow	7	0,02
Google API Bot Disallow	7	0,02
Google DuplexWeb Disallow	7	0,02
YOU Disallow	4	0,08
Youdao Bot Disallow	4	0,01
Pipl Disallow	1	0