rhel8/9: data corruption due to stale cached iomaps

Solution Verified - Updated -

Issue

We found that some of files on xfs were partially filled by zero unexpectedly. We created files by filling them from multiple threads without overwrite/overlap on each writes using pwrite(2):

    thread1  thread2  thread3
       |     |        |
       |     |        +-----+--------------------------+----------------- ...
       |     |              |                          |
       |     +-----+--------------------------+--------------------------+---- ...
       |           |        |                 |        |                 |
       +-----+-----------------------+--------------------------+----------- ...
             |     |        |        |        |        |        |        |
     write   v     v        v        v        v        v        v        v
    file [   |     |        |        |        |        |        |        |        | ...

    (written by:
             11111122 22222233 33333311 11111122 22222233 33333311 11111122 222222 ...)

     With this write pattern, corruption was observed, for example:
             11111122 22222233 33333311 11111122 00000033 33333311 11111122 222222 ...)
                                                 ^^^^^^

Environment

  • Red Hat Enterprise Linux (RHEL)
    • RHEL8: minor releases 8.4 and later
    • RHEL9: minor releases 9.1 and earlier
  • XFS filesystem
  • various architectures affected, issue seen on aarch64 and x86_64

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content