How can I prevent the Baidu spider from crawling my web pages in httpd
Issue
I see the Baidu spider is crawling my web pages but I want it to stop. Lines like this keep showing up in my access_log:
192.168.0.12 - - [1/Apr/2016:03:20:52 -0500] "GET /mywebpage HTTP/1.1" 200 1234 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" "-"
Environment
- Red Hat Enterprise Linux
- Red Hat JBoss Web Server
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
