Issue

For two of our nodes, we received the following error when attempting to create an application post upgrade:

broker

2016-01-15 17:18:29.778 [INFO ] Completed 500 Internal Server Error in 378375ms (Views: 0.8ms) (pid:3380)

node

January 15 17:12:19 INFO oo_spawn running /sbin/runuser -s /bin/sh 56996ebb8b2142740e000015 -c "exec /usr/bin/runcon 'unconfined_u:system_r:openshift_t:s0:c35,c541' /bin/sh -c \"/usr/bin/oo-erb -S 2 -- /var/lib/openshift/56996ebb8b2142740e000015/jbossews/env/OPENSHIFT_JBOSSEWS_LOG_DIR.erb > /var/lib/openshift/56996ebb8b2142740e000015/jbossews/env/OPENSHIFT_JBOSSEWS_LOG_DIR\"": {:unsetenv_others=>true, :close_others=>true, :in=>"/dev/null", :chdir=>"/var/lib/openshift/56996ebb8b2142740e000015/", :out=>#<IO:fd 12>, :err=>#<IO:fd 8>}
January 15 17:12:19 INFO oo_spawn running /sbin/runuser -s /bin/sh 56996ebb8b2142740e000015 -c "exec /usr/bin/runcon 'unconfined_u:system_r:openshift_t:s0:c35,c541' /bin/sh -c \"/var/lib/openshift/56996ebb8b2142740e000015/jbossews/bin/setup --version 2.0\"": {:unsetenv_others=>true, :close_others=>true, :in=>"/dev/null", :chdir=>"/var/lib/openshift/56996ebb8b2142740e000015/jbossews", :out=>#<IO:fd 12>, :err=>#<IO:fd 8>}
January 15 17:12:19 INFO oo_spawn running /sbin/runuser -s /bin/sh 56996ebb8b2142740e000015 -c "exec /usr/bin/runcon 'unconfined_u:system_r:openshift_t:s0:c35,c541' /bin/sh -c \"/usr/bin/oo-erb -S 2 -- /var/lib/openshift/56996ebb8b2142740e000015/jbossews/metadata/jenkins_artifacts_glob.erb > /var/lib/openshift/56996ebb8b2142740e000015/jbossews/metadata/jenkins_artifacts_glob\"": {:unsetenv_others=>true, :close_others=>true, :in=>"/dev/null", :chdir=>"/var/lib/openshift/56996ebb8b2142740e000015/", :out=>#<IO:fd 12>, :err=>#<IO:fd 8>}
January 15 17:12:19 INFO oo_spawn running /sbin/runuser -s /bin/sh 56996ebb8b2142740e000015 -c "exec /usr/bin/runcon 'unconfined_u:system_r:openshift_t:s0:c35,c541' /bin/sh -c \"/var/lib/openshift/56996ebb8b2142740e000015/jbossews/bin/install --version 2.0\"": {:unsetenv_others=>true, :close_others=>true, :in=>"/dev/null", :chdir=>"/var/lib/openshift/56996ebb8b2142740e000015/jbossews", :out=>#<IO:fd 12>, :err=>#<IO:fd 8>}
January 15 17:15:24 INFO oo_spawn running /usr/bin/pkill -9 -u 35716: {:unsetenv_others=>false, :close_others=>true, :in=>"/dev/null", :out=>#<IO:fd 15>, :err=>#<IO:fd 13>}
January 15 17:15:24 INFO oo_spawn buffer(7/) Killed (core dumped)

All diagnostic commands were successful on these nodes / brokers. We removed the nodes from the district to get past this problem and attempt to complete the upgrade.