Redis Command Timeout with JBoss EAP caused 15 minutes pause
Issue
- JBoss EAP server hanged for 15 minutes before resuming operation, and the
server.logshowed:
ERROR
UT005023: Exception handling request to /fivu/sse-listening: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.springframework.dao.QueryTimeoutException: Redis command timed out; nested exception is io.lettuce.core.RedisCommandTimeoutException: Command timed out after 3 second(s)
...
Caused by: org.springframework.dao.QueryTimeoutException: Redis command timed out; nested exception is io.lettuce.core.RedisCommandTimeoutException: Command timed out after 3 second(s)
...
Caused by: io.lettuce.core.RedisCommandTimeoutException: Command timed out after 3 second(s)
-
During this period, no abnormalities in
CPUormemory usagewere observed. -
A TCPDump analysis revealed many
missing packets, particularly related to Redis server running onport 7000.
- 179 0.024916 10.244.118.186 10.244.116.167 TCP 66 [TCP Previous segment not captured] 7000 → 38632 [FIN, ACK] Seq=9898 Ack=151 Win=29184 Len=0 TSval=1794549799 TSecr=897634296
2895 25.212968 10.244.118.188 10.244.116.165 TCP 66 [TCP ACKed unseen segment] 7000 → 46848 [ACK] Seq=1 Ack=2 Win=57 Len=0 TSval=2196452070 TSecr=1301108505
Note: 7000 port was redis server service port
Previous segment not captured indicated a missing packet (or packets) for data immediately earlier
in the data sequence for this direction of this TCP connection, or it can also be due to capture-fllow loss.
Environment
- JBoss Enterprise Application Platform
- 7.x
- 8.x
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.