Oracle database halted with I/O errors on file system
Issue
- Oracle database halted with I/O errors on file system
ORACLE LOG:
Thread 1 cannot allocate new log, sequence 56248
Checkpoint not complete
Current log# 3 seq# 56247 mem# 0: /oracle/GRP/origlogA/log_g13m1.dbf
Current log# 3 seq# 56247 mem# 1: /oracle/GRP/mirrlogA/log_g13m2.dbf
Errors in file /oracle/GRP/saptrace/diag/rdbms/grp/GRP/trace/GRP_arc3_29803.trc:
ORA-27052: unable to flush file data
Linux-x86_64 Error: 28: No space left on device
Additional information: 1
ORA-19502: write error on file "/oracle/GRP/oraarch/GRParch1_56246_763230703.dbf", block number 98304 (block size=512)
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 28: No space left on device
Additional information: -1
Additional information: 1048576
ARC3: Error 19502 Closing archive log file '/oracle/GRP/oraarch/GRParch1_56246_763230703.dbf'
ARCH: Archival stopped, error occurred. Will continue retrying
ORACLE Instance GRP - Archival Error
ORA-16038: log 2 sequence# 56246 cannot be archived
ORA-19502: write error on file "", block number (block size=)
ORA-00312: online log 2 thread 1: '/oracle/GRP/origlogB/log_g12m1.dbf'
ORA-00312: online log 2 thread 1: '/oracle/GRP/mirrlogB/log_g12m2.dbf'
ARCH: Archival stopped, error occurred. Will continue retrying
ORACLE Instance GRP - Archival Error
ORA-16014: log 2 sequence# 56246 not archived, no available destinations
ORA-00312: online log 2 thread 1: '/oracle/GRP/origlogB/log_g12m1.dbf'
ORA-00312: online log 2 thread 1: '/oracle/GRP/mirrlogB/log_g12m2.dbf'
Completed checkpoint up to RBA [0xdbb5.2.10], SCN: 408208172
Beginning log switch checkpoint up to RBA [0xdbb8.2.10], SCN: 408208294
Thread 1 advanced to log sequence 56248 (LGWR switch)
Current log# 4 seq# 56248 mem# 0: /oracle/GRP/origlogB/log_g14m1.dbf
Current log# 4 seq# 56248 mem# 1: /oracle/GRP/mirrlogB/log_g14m2.dbf
Archiver process freed from errors. No longer stopped
Thread 1 cannot allocate new log, sequence 56249
The Oracle DB encountered ENOSPC errors even though the filesystem was not full:
nznsvfnux224:root:SAP HANA FACT Prod DB/ASCS:/oracle/GRP/oraarch#df -hP /oracle/GRP/oraarch
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vgGRP-lvol2 50G 15G 35G 31% /oracle/GRP/oraarch
Environment
- Redhat Enterprise Linux 6.3
- Oracle DB
- ext3 filesystem
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.