Encountering GC overhead and memory issues while processing huge file in camel.

Solution Unverified - Updated -

Issue

  • There is one use case in which user need to process a huge file .
  • The route is as follows
<route>            
 <from uri="file:d:\\batchInput?readLock=true"/>   
 <log message="MESSAGE Checking" loggingLevel="WARN" logName="message" />    
   <unmarshal ref="bindyDataformat" /> 
 <log message="MESSAGE ${body}" loggingLevel="WARN" logName="message" />     

<!--  <to uri="file:d:\\batchOutput"/>    -->
 <to uri="activemq:queue:BatchDatatoXML" />
</route>
  • The input csv file is of 180 MB.
  • User is encountering GC over head issues while processing the file .
  • How can user overcome these issues in Camel?
  • Can spring-batch a viable solution in this case?
  • Or is there any other alternative to tackle huge file processing scenarios.

Environment

  • Red Hat JBoss Fuse
    • 6.1.0

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.