How can I prevent the Baidu spider from crawling my web pages in httpd

Solution In Progress - Updated -

Issue

I see the Baidu spider is crawling my web pages but I want it to stop. Lines like this keep showing up in my access_log:

192.168.0.12 - - [1/Apr/2016:03:20:52 -0500] "GET /mywebpage HTTP/1.1" 200 1234 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" "-"

Environment

  • Red Hat Enterprise Linux
  • Red Hat JBoss Web Server

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content