Camel HDFS component fails with ClassNotFound Exception

Solution Verified - Updated -

Issue

  • While running Camel HDFS component it fails with the following ClassNotFoundException:
INFO  | Test-thread-2 | DefaultTypeConverter             | 148 - org.apache.camel.camel-core - 2.10.0.fuse-71-047 | TypeConverterRegistry utilization[attempts=0, hits=0, misses=0, failures=0] mappings[total=209, misses=0]
INFO  | Test-thread-2 | BlueprintCamelContext            | 148 - org.apache.camel.camel-core - 2.10.0.fuse-71-047 | Apache Camel 2.10.0.fuse-71-047 (CamelContext: 265-camel-6) is shutdown in 0.029 seconds. Uptime 0.317 seconds.
WARN  | Test-thread-2 | BlueprintCamelContext            | 151 - org.apache.camel.camel-blueprint - 2.10.0.fuse-71-047 | Error occurred during starting CamelContext(265-camel-6)
org.apache.camel.FailedToCreateProducerException: Failed to create Producer for endpoint: Endpoint[hdfs://testHost:8020/test-archive/TOPIC.test.testData.1?splitStrategy=MESSAGES%3A100000]. Reason: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.security.ShellBasedUnixGroupsMapping from bundle 265 (test-archiver)
        at org.apache.camel.impl.ProducerCache.doGetProducer(ProducerCache.java:395)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        at org.apache.camel.impl.ProducerCache.acquireProducer(ProducerCache.java:114)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        at org.apache.camel.impl.ProducerCache.startProducer(ProducerCache.java:145)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        at org.apache.camel.processor.SendProcessor.doStart(SendProcessor.java:175)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:60)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:62)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:52)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        at org.apache.camel.util.ServiceHelper.startServices(ServiceHelper.java:73)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        . 
        . 
        .
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.security.ShellBasedUnixGroupsMapping from bundle 265 (test-archiver)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:898)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.security.Groups.<init>(Groups.java:48)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:137)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:180)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:159)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:216)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:413)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:399)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1418)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1319)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.camel.component.hdfs.HdfsInfo.<init>(HdfsInfo.java:36)[269:org.apache.camel.camel-hdfs:2.10.0.fuse-71-047]
        at org.apache.camel.component.hdfs.HdfsInfoFactory.newHdfsInfo(HdfsInfoFactory.java:20)[269:org.apache.camel.camel-hdfs:2.10.0.fuse-71-047]
        at org.apache.camel.component.hdfs.HdfsOutputStream.createOutputStream(HdfsOutputStream.java:49)[269:org.apache.camel.camel-hdfs:2.10.0.fuse-71-047]
        at org.apache.camel.component.hdfs.HdfsProducer.setupHdfs(HdfsProducer.java:147)[269:org.apache.camel.camel-hdfs:2.10.0.fuse-71-047]
        at org.apache.camel.component.hdfs.HdfsProducer.doStart(HdfsProducer.java:105)[269:org.apache.camel.camel-hdfs:2.10.0.fuse-71-047]
        at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:60)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:62)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        at org.apache.camel.impl.ProducerCache.doGetProducer(ProducerCache.java:393)[148:org.apache.camel.camel-core:2.10.0.fuse-71-047]
        ... 93 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.security.ShellBasedUnixGroupsMapping from bundle 265 (test-archiver)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:866)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:890)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        ... 111 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.security.ShellBasedUnixGroupsMapping from bundle 265 (test-archiver)
        at org.apache.camel.core.osgi.utils.BundleDelegatingClassLoader.loadClass(BundleDelegatingClassLoader.java:78)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)[:1.6.0_31]
        at java.lang.Class.forName0(Native Method)[:1.6.0_31]
        at java.lang.Class.forName(Class.java:247)[:1.6.0_31]
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:864)[267:org.apache.servicemix.bundles.hadoop-core:0.20.203.0_2]
        ... 112 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.security.ShellBasedUnixGroupsMapping not found by test-archiver [265]
        at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1499)
        at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75)
        at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1882)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)[:1.6.0_31]
        at org.apache.felix.framework.Felix.loadBundleClass(Felix.java:1814)[org.apache.felix.framework-4.0.3.fuse-71-047.jar:]
        at org.apache.felix.framework.BundleImpl.loadClass(BundleImpl.java:929)
        at org.apache.camel.core.osgi.utils.BundleDelegatingClassLoader.findClass(BundleDelegatingClassLoader.java:47)
        at org.apache.camel.core.osgi.utils.BundleDelegatingClassLoader.loadClass(BundleDelegatingClassLoader.java:69)
        ... 117 more
  • Tried "dev:dynamic-import" on "camel-hdfs" bundle from the karaf shell and that did not work.
  • Checked "osgi:headers" for "camel-hdfs" and found that it already has "dynamic-import" enabled.

Environment

  • Red Hat Fuse ESB
    • 7.1

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content