Slower disk access time when going through cache - Red Hat Customer Portal

Log in to Your Red Hat Account

Your Red Hat account gives you access to your profile, preferences, and services, depending on your status.

If you are a new customer, register now for access to product evaluations and purchasing capabilities.

Need access to an account?

If your company has an existing Red Hat account, your organization administrator can grant you access.

If you have any questions, please contact customer service.

Red Hat Account Number:

Red Hat Account

Customer Portal

For your security, if you’re on a public computer and have finished using your Red Hat services, please be sure to log out.

Select Your Language

Issue

We noticed an odd behavior on one node of an Oracle RAC cluster. If we run 'dd' to read 1mb of data from a SAN disk we get 75-80mb/sec, run after run. But if we add the "iflag=direct" parameter (bypass cache) our performance goes UP to ~250mb/sec.
Why is bypassing cache giving us much better performance?
We don't see this same behavior on the other two nodes in this cluster.

Environment

RHEL 6.5.z, kernel version 2.6.32-573.1.1.el6.x86_64
HP ProLiant DL380p Gen8, BIOS P70
EMC SYMMETRIX SAN Storage
Emulex lpfc HBAs
3 node Oracle RAC cluster with several EMC SAN LUNs shared between them.

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.

Here are the common uses of Markdown.

Code blocks

~~~
Code surrounded in tildes is easier to read
~~~

Links/URLs

[Red Hat Customer Portal](https://access.redhat.com)