qla2xxx "No room on outstanding cmd array"

Latest response

Hi,
We have problem with 16Gb HBA cards on our environment and the below message will be generated before hanging the OS:

No room on outstanding cmd array

Please explain it for me, if anyone has same experience.

BR

Responses

Hi Davoud Teimouri,

Honestly curious here, What version of using Red Hat are you using. While I suspect most probably you are using RHEL 7, we can't be 100% sure. (please include subversion too, like RHEL 7.4, or RHEL 7.5) That looks like a qlogic HBA.

Use rpm -qa | grep redhat-release and cat /etc/redhat-release (there are rare cases where someone manually adjusts the /etc/redhat-release file for 3rd party software issues)

Does the hang you speak of occur upon reboot, or later after the system is up? I would suspect the former. Which Qlogic card is it, and again what version of RHEL?

Did this work recently, and some change was made and it ceased to work? Or did this never work for this system?

RJ

Hi, We are using RHEL 6.10 and HPE SN1000Q (Qlogic adapter actually). Storage is EMC VNX5800 and SAN Switch is Brocade 6520. The servers are HPE DL580 G9 and Oracle DB 11.2.0.4 is installed as DB. After 1 hour, OS generating qla2xxx abort command issued and then it will be not responding.

I don't understand the storage layer or storage hardware very well, but it looks like this message is printed when the driver is trying to give more commands to the storage controller, but there is no room left for more commands on the storage controller.

This implies either the workload is too large, or the storage controller has some problem and has stopped processing commands. I don't know how to debug that specific problem sorry.

We don't have a knowledgebase solution about this message, so it might be worth opening a support case about this.

The servers will be worked fine when those servers have 8Gb HBA card.

Probably, you could enable verbose logging on the controller and see if that gives any hint. Please check this article regarding enabling verbose logging feature https://access.redhat.com/articles/337813 You may have to turn it off later, otherwise, it may eat up all space under /var.

Have you checked the qlogic for updates related to the firmware/drivers? You mention that it works fine on the 8Gb HBA card. You could also try hard coding you SAN switch port for the server to 8Gb to see if the problem goes away. There are quite a few factors (SAN switch, Storage settings, HBA settings, firmware, etc.) involved so any additional information could be helpful.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.