All applications are marked as undeployed after deployment scanner failed due to OOME or "Too many open files" in JBoss EAP 6

Solution Verified - Updated -

Issue

  • Sometimes when hitting an OOME. We see that our applications are undeployed. We have to manually redeploy them when restarting. We notice deployment scanner messages like the following:
18:13:38,962 ERROR [org.jboss.as.controller.management-operation] (DeploymentScanner-threads - 1) JBAS014612: Operation ("read-resource") failed - address: ([("deployment" => "helloworld.war")]): java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.util.Properties$LineReader.<init>(Properties.java:405) [rt.jar:1.7.0_85]
    at java.util.Properties.load(Properties.java:341) [rt.jar:1.7.0_85]
    at java.util.PropertyResourceBundle.<init>(PropertyResourceBundle.java:130) [rt.jar:1.7.0_85]
    at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2614) [rt.jar:1.7.0_85]
    at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1436) [rt.jar:1.7.0_85]
    at java.util.ResourceBundle.findBundle(ResourceBundle.java:1400) [rt.jar:1.7.0_85]
    at java.util.ResourceBundle.findBundle(ResourceBundle.java:1354) [rt.jar:1.7.0_85]
    at java.util.ResourceBundle.findBundle(ResourceBundle.java:1354) [rt.jar:1.7.0_85]
    at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1296) [rt.jar:1.7.0_85]
    at java.util.ResourceBundle.getBundle(ResourceBundle.java:1028) [rt.jar:1.7.0_85]
    at org.jboss.as.controller.descriptions.StandardResourceDescriptionResolver.getResourceBundle(StandardResourceDescriptionResolver.java:160) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.descriptions.DefaultResourceDescriptionProvider.getModelDescription(DefaultResourceDescriptionProvider.java:74) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.OperationContextImpl$CachingDescriptionProvider.getModelDescription(OperationContextImpl.java:2030) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.operations.global.ReadResourceHandler.doExecuteInternal(ReadResourceHandler.java:321) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.operations.global.ReadResourceHandler.doExecute(ReadResourceHandler.java:147) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.operations.global.GlobalOperationHandlers$AbstractMultiTargetHandler.execute(GlobalOperationHandlers.java:249) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:702) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:537) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.AbstractOperationContext.completeStepInternal(AbstractOperationContext.java:338) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:314) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1144) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:331) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:188) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.ModelControllerImpl$3.execute(ModelControllerImpl.java:563) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.controller.ModelControllerImpl$3.execute(ModelControllerImpl.java:553) [jboss-as-controller-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.server.deployment.scanner.DefaultDeploymentOperations.getDeploymentsStatus(DefaultDeploymentOperations.java:70) [jboss-as-deployment-scanner-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$ScanContext.<init>(FileSystemDeploymentService.java:1387) [jboss-as-deployment-scanner-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$ScanContext.<init>(FileSystemDeploymentService.java:1336) [jboss-as-deployment-scanner-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.scan(FileSystemDeploymentService.java:358) [jboss-as-deployment-scanner-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentScanRunnable.run(FileSystemDeploymentService.java:149) [jboss-as-deployment-scanner-7.5.3.Final-redhat-2.jar:7.5.3.Final-redhat-2]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_85]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [rt.jar:1.7.0_85]
18:13:46,800 ERROR [org.jboss.as.controller.management-operation] (DeploymentScanner-threads - 1) JBAS014612: Operation ("read-children-resources") failed - address: ([]): java.lang.OutOfMemoryError: GC overhead limit exceeded
18:13:48,914 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) JBAS015018: Deployment helloworld.war was previously deployed by this scanner but has been removed from the server deployment list by another management tool. Marker file helloworld.war.undeployed is being added to record this fact.

Environment

  • Red Hat JBoss Enterprise Application Platform (EAP) 6.4.4 and earlier

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.