User-agent: * Disallow: /signalr/ Disallow: /signalr/* Disallow: /ajax/ Disallow: /ajax/* User-Agent: MJ12bot User-agent: AhrefsBot User-agent: SurdotlyBot User-agent: SemrushBot User-agent: SeekportBot User-Agent: trendictionbot Disallow: /