Fuse applications on Netty unable to process TCP incoming requests
Issue
-
We recently noted that end users are occasionally experiencing long latencies for end to end that frustrate users, and it ultimately results in negatively affecting end users that prematurely abandon our applications. After an in depth investigation, we noted that the Fuse application running on netty at IP:PORT are failing to process incoming requests. We see that the TCP client repeatedly attempts to send an HTTP GET request to the fuse application; however, the fuse application does not start processing the message. It appears that while the client send requests to the netty module at IP:PORT, the netty module occasionally fails to even start the processing. It ultimately fails entirely after 60 seconds; when the failure happens the Load Balancer opts to re-route the given message to a different peer in the cluster at IP:PORT and then this will result in a successful interaction. Customers will not receive errors, however they experience more than 60 seconds delay for the HTTP synchronus invocation. This is unacceptable, and unusual.
-
We're interested to understand how to possibly avoid this long delays.
-
This interaction was processed in 60 seconds in total, and it ultimately failed with the server IP:PORT returning an RST signal. As a result our users experienced very long latencies.We noticed that the 60 seconds were consumed while negotiating with Netty who eventually refused to process the message; Our Fuse logs do not show any errors; however, the application categorically rejected to process the incoming request.
Environment
- Red Hat JBoss Fuse
- 6.1.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
