SwitchYard's Camel SFTP binding thread goes to waiting state if it is not able to poll files due to connection failure
Issue
- There is one
SwitchYard
application whoseSFTP binding
polls one file per day from anSFTP
server. The thread dump collected from theFSW
server where the application is deployed shows that the thread responsible for this job sometimes moves to waiting state and theSwitchYard
is not able to poll the file from theSFTP
server. This thread moves to waiting state only when the connection to theSFTP
server has failed once. Here is a snippet of the thread.
Camel (camel-100) thread #1242 - sftp://abc:******@xxx.xx.xxx.xx:xx/filelocation" daemon prio=10 tid=0x00007fe5c8749800 nid=0x24fc in Object.wait() [0x00007fe6056cf000]
- The thread stays in the same waiting state as observed from multiple thread dumps that were taken for this particular
SwitchYard
application's issue. - The files which were placed in the
SFTP
location were never consumed once after the connection failure occurred. - There are no logs in server logs for polling the inbound
SFTP
folder location after the connection failure occurs once. Though, once users redploy the interface the retry logs (polling logs) come after every 10 mins. - Users want to know why this polling thread enters waiting state and what can be done to rectify this problem.
- Meanwhile, sometime when the
SwitchYard
applicationSFTP binding
suffers from the same issue the following stack traces are observed inserver.log
file which indicates to certain connection failure occurring between theFSW
and theSFTP
servers. What does the below error in server logs that occured during the connection failure issue mean?
09-07-2015 11:08:59,209 INFO (org.apache.camel.component.file.remote.SftpOperations) [Connect thread xxx.xx.xxx.xx session] JSCH -> Disconnecting from xxx.xx.xxx.xx port 22
09-07-2015 11:08:59,210 WARN (org.apache.camel.component.file.remote.RemoteFilePollingConsumerPollStrategy) [Camel (camel-97) thread #960 - sftp://*****:******@xxx.xx.xxx.xx:22/filelocation] Trying to recover by disconnecting from remote server forcing a re-connect at next poll: sftp://*****@xxx.xx.xxx.xx:22
09-07-2015 11:08:59,211 ERROR (org.apache.camel.processor.DefaultErrorHandler) [Camel (camel-97) thread #960 - sftp://*****:******@xxx.xx.xxx.xx:22/filelocation] Failed delivery for (MessageId: ID-xxxxxx-goep-**-com-47734-1436280154289-96-79 on ExchangeId: ID-xxxxxx-goep-**-com-47734-1436280154289-96-80). Exhausted after delivery attempt: 1 caught: org.apache.camel.component.file.GenericFileOperationFailedException: Cannot list directory: filelocation: org.apache.camel.component.file.GenericFileOperationFailedException: Cannot list directory: filelocation
at org.apache.camel.component.file.remote.SftpOperations.listFiles(SftpOperations.java:534) [camel-ftp-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.component.file.remote.SftpConsumer.doPollDirectory(SftpConsumer.java:88) [camel-ftp-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.component.file.remote.SftpConsumer.pollDirectory(SftpConsumer.java:50) [camel-ftp-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:107) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:142) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:92) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_67]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [rt.jar:1.7.0_67]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [rt.jar:1.7.0_67]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [rt.jar:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_67]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_67]
Caused by: 4:
at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1600)
at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1406)
at org.apache.camel.component.file.remote.SftpOperations.listFiles(SftpOperations.java:525) [camel-ftp-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
... 12 more
Caused by: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2607)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2631)
at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1496)
... 14 more
Environment
- Red Hat JBoss Fuse Service Works (FSW)
- 6.0.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.