User-agent: SemrushBot User-agent: SemrushBot-SA User-agent: AhrefsBot User-agent: BUbiNG User-agent: MJ12bot Disallow: / User-agent: * Disallow: /admin/ Disallow: /administrator/ Disallow: /content/ Disallow: /core/ Disallow: /error/ Disallow: /public/ Disallow: /stats/ Disallow: /upload/