Robots.txt Nutzungsverteilung In Philippinen

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Some Robots Disallow	81.695	46,78
Googlebot Disallow	11.177	6,4
Baidu Bot Disallow	8.296	4,75
Sogou Bot Disallow	7.211	4,13
Yandex Bot Disallow	6.783	3,88
Alexa Bot Disallow	6.643	3,8
Google AdsBot Disallow	6.633	3,8
Microsoft BingBot Disallow	6.590	3,77
Yahoo Slurp Disallow	6.557	3,75
Soso Bot Disallow	6.498	3,72
Robots Disallow	5.774	3,31
Google AdsBot Mobile Disallow	4.020	2,3
Ahrefs Bot Disallow	2.962	1,7
GPTBot Disallow	1.193	0,68
Common Crawl Bot Disallow	1.181	0,68
Anthropic Claude Bot Disallow	1.135	0,65
Google Bard Disallow	1.130	0,65
AppleBot Disallow	1.126	0,64
Facebook Bot Disallow	1.102	0,63
Apple AI Bot Disallow	1.084	0,62
Perplexity	1.040	0,6
Cohere AI Disallow	1.024	0,59
Google AdsBot Mobile Apps Disallow	1.003	0,57
Google Media Partners Disallow	845 ^*	0,48
Semrush Bot Disallow	440	0,25
Majestic Bot Disallow	408	0,23
360 Bot Disallow	380	0,22
Microsoft MSNBot Disallow	119	0,07
Exalead Bot Disallow	115	0,07
Webz Omgili Disallow	112	0,06
ByteDance Bytespider Disallow	108	0,06
Google Image Bot Disallow	95	0,05
Diffbot Disallow	30	0,02
BuiltWithBot Disallow	25	0,01
DuckDuckGo Bot Disallow	21	0,01
Google News Bot Disallow	17	0,01
Google Video Bot Disallow	14	0,01
YOU Disallow	12	0,01
Google Favicon Disallow	11	0,01
Google API Bot Disallow	11	0,01
Youdao Bot Disallow	10	0,01
Pipl Disallow	10	0,01
Google DuplexWeb Disallow	9	0,01
Google FeedFetcher Disallow	3	0