Robots.txt Distribuzione dell'utilizzo In Libano

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	1.219	13,39
Ahrefs Bot Disallow	746	8,19
Google AdsBot Mobile Disallow	560	6,15
Common Crawl Bot Disallow	477	5,24
GPTBot Disallow	470	5,16
AppleBot Disallow	421	4,62
Anthropic Claude Bot Disallow	415	4,56
Google Bard Disallow	414	4,55
Facebook Bot Disallow	414	4,55
Apple AI Bot Disallow	412	4,53
Perplexity	410	4,5
Cohere AI Disallow	406	4,46
Google AdsBot Mobile Apps Disallow	406	4,46
Googlebot Disallow	347	3,81
Robots Disallow	346	3,8
Some Robots Disallow	168	1,85
Sogou Bot Disallow	150	1,65
Baidu Bot Disallow	139	1,53
Yandex Bot Disallow	122	1,34
Soso Bot Disallow	113	1,24
Semrush Bot Disallow	113	1,24
Microsoft BingBot Disallow	104	1,14
Yahoo Slurp Disallow	102	1,12
Majestic Bot Disallow	100	1,1
Alexa Bot Disallow	98	1,08
ByteDance Bytespider Disallow	64	0,7
Exalead Bot Disallow	64	0,7
360 Bot Disallow	59	0,65
BuiltWithBot Disallow	56	0,62
Microsoft MSNBot Disallow	48	0,53
Google Image Bot Disallow	38	0,42
Google Media Partners Disallow	34 ^*	0,37
Google News Bot Disallow	27	0,3
Google Video Bot Disallow	26	0,29
Webz Omgili Disallow	9	0,1
Youdao Bot Disallow	4	0,04
Diffbot Disallow	2	0,02
YOU Disallow	1	0,01