shm_area_write returning "NOSPACE" errors.

Solution Unverified - Updated -

Environment

  • Red Hat Enterprise Linux 5
  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 7

Issue

  • Server is returning memory errors from shm_area_write
[root@localhost tmp]# dmesg|grep -i nospace
shm_area_write NOSPACE2 rem=7376 len=9598 stop=ffffc90008ef6c80 wr=ffffc90008ef4fb0 writes_since_mark=875
shm_area_write NOSPACE2 rem=6936 len=9786 stop=ffffc900096b2ca8 wr=ffffc900096b1190 writes_since_mark=1762
shm_area_write NOSPACE2 rem=8544 len=8688 stop=ffffc90008fa4dc0 wr=ffffc90008fa2c60 writes_since_mark=1784
shm_area_write NOSPACE2 rem=2008 len=9490 stop=ffffc90009564990 wr=ffffc900095641b8 writes_since_mark=5

Resolution

  • Contact the module vendor responsible for creating the shm_area_write application.

Root Cause

  • These errors are being logged by the application shm_area_write and do not appear to be provided by Red Hat.

Diagnostic Steps

Note: This is one example where microsoft_dependency_agent was generating the logs and below are the steps to analyze the issue.

Step - 1: Collect the SOS report to examine errors.

Step - 2: These logs are recorded in different locations and can be determined by the below command.

$ grep -iR "shm_area_write" sos_commands/*
sos_commands/kernel/dmesg:[12531.866141] shm_area_write NOSPACE2 rem=448 len=16384 stop=ffffa6ac4aa89b70 wr=ffffa6ac4aa899b0 writes_since_mark=536
sos_commands/kernel/dmesg:[99486.908484] shm_area_write NOSPACE2 rem=3312 len=16384 stop=ffffa6ac4a96b800 wr=ffffa6ac4a96ab10 writes_since_mark=538
             [- – >  logs are timmed. <<- - ]
sos_commands/kernel/dmesg:[3212451.957548] shm_area_write NOSPACE2 rem=1216 len=16384 stop=ffffa6ac4a71b4d0 wr=ffffa6ac4a71b010 writes_since_mark=541
sos_commands/logs/journalctl_--no-pager_--catalog_--boot:Jul 04 17:03:00 localhost kernel: shm_area_write NOSPACE2 rem=12640 len=16384 stop=ffffa6ac4a91ae30 wr=ffffa6ac4a917cd0 writes_since_mark=1327
sos_commands/logs/journalctl_--no-pager_--catalog_--boot:Jul 05 17:00:46 localhost kernel: shm_area_write NOSPACE2 rem=2160 len=16384 stop=ffffa6ac4a46f880 wr=ffffa6ac4a46f010 writes_since_mark=529
            [- – >  logs are timmed. <<- - ]
sos_commands/logs/journalctl_--no-pager_--catalog_--boot:Jul 21 01:53:36 localhost kernel: shm_area_write NOSPACE2 rem=1216 len=16384 stop=ffffa6ac4a71b4d0 wr=ffffa6ac4a71b010 writes_since_mark=541
sos_commands/pacemaker/crm_report/localhost/journal.log:Jul 21 01:53:36 localhost kernel: shm_area_write NOSPACE2 rem=1216 len=16384 stop=ffffa6ac4a71b4d0 wr=ffffa6ac4a71b010 writes_since_mark=541

Step - 3: In this example, the shm_area_write function was originated by the Microsoft Dependency Agent and hence this needs to be checked with Microsoft support.

$ grep -iR "shm_area_write" proc/*
proc/kallsyms:ffffffffc065bb60 t shm_area_write [microsoft_dependency_agent]  <<===

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments