Tomcat experiences random session drops and EOFException when using PersistentValve

Issue

• We've configured Tomcat to use a org.apache.catalina.valves.PersistentValve with a org.apache.catalina.session.FileStore:

    <Valve className="org.apache.catalina.valves.PersistentValve" />
    <Manager className="org.apache.catalina.session.PersistentManager" processExpiresFrequency="1">
        <Store className="org.apache.catalina.session.FileStore" />
    </Manager>

• With such a configuration, we see random EOFExceptions like below and the session state is lost and replaced with a new session:

 SEVERE [Catalina-utility-1] org.apache.catalina.session.StoreBase.processExpires Error processing session expiration for key [3DFF110B8415939F736A5751783ECDA9]
 java.io.EOFException
 at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2933)
 at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3428)
 at java.base/java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:985)
 at java.base/java.io.ObjectInputStream.<init>(ObjectInputStream.java:416)
 at org.apache.catalina.util.CustomObjectInputStream.<init>(CustomObjectInputStream.java:87)
 at org.apache.catalina.session.StoreBase.getObjectInputStream(StoreBase.java:204)
 at org.apache.catalina.session.FileStore.load(FileStore.java:200)
 at org.apache.catalina.session.StoreBase.processExpires(StoreBase.java:138)
 at org.apache.catalina.session.PersistentManagerBase.processExpires(PersistentManagerBase.java:409)
 at org.apache.catalina.session.ManagerBase.backgroundProcess(ManagerBase.java:587)
 at org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:4837)
 at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1172)
 at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1176)
 at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1176)
 at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1154)
 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
 at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
 at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
 at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
 at java.base/java.lang.Thread.run(Thread.java:1583)