Oracle database halted with I/O errors on file system

Solution Unverified - Updated -

Issue

  • Oracle database halted with I/O errors on file system
ORACLE LOG:
Thread 1 cannot allocate new log, sequence 56248
Checkpoint not complete
  Current log# 3 seq# 56247 mem# 0: /oracle/GRP/origlogA/log_g13m1.dbf
  Current log# 3 seq# 56247 mem# 1: /oracle/GRP/mirrlogA/log_g13m2.dbf
Errors in file /oracle/GRP/saptrace/diag/rdbms/grp/GRP/trace/GRP_arc3_29803.trc:
ORA-27052: unable to flush file data
Linux-x86_64 Error: 28: No space left on device
Additional information: 1
ORA-19502: write error on file "/oracle/GRP/oraarch/GRParch1_56246_763230703.dbf", block number 98304 (block size=512)
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 28: No space left on device
Additional information: -1
Additional information: 1048576
ARC3: Error 19502 Closing archive log file '/oracle/GRP/oraarch/GRParch1_56246_763230703.dbf'
ARCH: Archival stopped, error occurred. Will continue retrying
ORACLE Instance GRP - Archival Error
ORA-16038: log 2 sequence# 56246 cannot be archived
ORA-19502: write error on file "", block number  (block size=)
ORA-00312: online log 2 thread 1: '/oracle/GRP/origlogB/log_g12m1.dbf'
ORA-00312: online log 2 thread 1: '/oracle/GRP/mirrlogB/log_g12m2.dbf'
ARCH: Archival stopped, error occurred. Will continue retrying
ORACLE Instance GRP - Archival Error
ORA-16014: log 2 sequence# 56246 not archived, no available destinations
ORA-00312: online log 2 thread 1: '/oracle/GRP/origlogB/log_g12m1.dbf'
ORA-00312: online log 2 thread 1: '/oracle/GRP/mirrlogB/log_g12m2.dbf'
Completed checkpoint up to RBA [0xdbb5.2.10], SCN: 408208172
Beginning log switch checkpoint up to RBA [0xdbb8.2.10], SCN: 408208294
Thread 1 advanced to log sequence 56248 (LGWR switch)
  Current log# 4 seq# 56248 mem# 0: /oracle/GRP/origlogB/log_g14m1.dbf
  Current log# 4 seq# 56248 mem# 1: /oracle/GRP/mirrlogB/log_g14m2.dbf
Archiver process freed from errors. No longer stopped
Thread 1 cannot allocate new log, sequence 56249

The Oracle DB encountered ENOSPC errors even though the filesystem was not full:

nznsvfnux224:root:SAP HANA FACT Prod DB/ASCS:/oracle/GRP/oraarch#df -hP /oracle/GRP/oraarch
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vgGRP-lvol2   50G   15G   35G  31% /oracle/GRP/oraarch

Environment

  • Redhat Enterprise Linux 6.3
  • Oracle DB
  • ext3 filesystem

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content