Emmanuel Touzery
2018-07-16 11:54:57 UTC
Hello,
   we have a TOMEE 7.0.3 application, which uses also JMS, which works
without issues. We've recently made a new version of the application
(one of the notable changes is a migration from using the JMS1.1 to the
JMS2 API).
   With the new version, we have an issue, but only on our PROD
environment, where we get no errors in the catalina.out log file (but
the application doesn't work), but an error in the localhost log file,
the error being as in the title of the email, I put the info from the
localhost log file at the end of this email.
   I can't put the source on github or something like that,
unfortunately. What is strange is that this error does not occur on our
staging or UAT environments, but it does occur every time (well, we
tried multiple times to restart the application and it failed
consistently) on the PROD environment. We copied the entire PROD tomee
folder on another machine and failed to reproduce the issue. We also
tried to simulate incoming traffic when restarting the application, and
run it against a backup of the PROD database... Somehow we can't
reproduce the error. We are guessing that maybe the PROD traffic is
higher than what we can simulate and some incoming message comes at just
the wrong time? But we don't know for sure.
   Now, studying the issue, we realized that we used to have this in
our src/main/webapp/WEB-INF/faces-config.xml:
<?xml version="1.0" encoding="UTF-8"?><faces-config
xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-facesconfig_2_1.xsd"
version="2.1"><managed-bean><managed-bean-name>login</managed-bean-name><managed-bean-class>com.lecip.tms.tms_core.application.Login</managed-bean-class><managed-bean-scope>view</managed-bean-scope></managed-bean></faces-config>
   We believe the JSF version is not correct for JEE7 anymore, in
addition the managed-bean annotation is not necessary, so we changed the
file to now contain:
<?xml version="1.0" encoding="UTF-8"?>
<faces-config xmlns="http://xmlns.jcp.org/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
http://xmlns.jcp.org/xml/ns/javaee/web-facesconfig_2_2.xsd"
             version="2.2">
</faces-config>
   Could this be the cause for that error that we saw in our PROD
environment? Or you would think the root cause would be something else?
The problem is that it's pretty difficult for us to experiment with
changes since any deployment on PROD must be first validated on staging
and UAT... Any hints? Is there maybe some logging configuration flag we
could activate to get more information about the error? Or any other
place to look to get more information about the issue?
   The irony is that we use JSF only for the login screen and nowhere
else...
   Regards,
Emmanuel
12-Jul-2018 17:44:04.335 SEVERE [localhost-startStop-1]
org.apache.tomcat.util.descriptor.web.SecurityConstraint.findUncoveredHttpMethods
For security constraints with URL pattern [/*] only the HTTP methods
[TRACE HEAD DELETE POST GET OPTIONS PUT] are covered. All other methods
are uncovered.
12-Jul-2018 17:44:22.078 SEVERE [localhost-startStop-1]
org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable
 java.lang.IllegalStateException: No Factories configured for this
Application. This happens if the faces-initialization does not work at
all - make sure that you properly include all configuration settings
necessary for a basic faces application and that all the necessary libs
are included. Also check the logging output of your web application and
your container for any exceptions!
If you did that and find nothing, the mistake might be due to the fact
that you use some special web-containers which do not support
registering context-listeners via TLD files and a context listener is
not setup in your web.xml.
A typical config looks like this;
<listener>
<listener-class>org.apache.myfaces.webapp.StartupServletContextListener</listener-class>
</listener>
       at javax.faces.FactoryFinder._getFactory(FactoryFinder.java:305)
       at javax.faces.FactoryFinder.getFactory(FactoryFinder.java:225)
       at javax.faces.webapp.FacesServlet.init(FacesServlet.java:115)
       at
org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1183)
       at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1099)
       at
org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:989)
       at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4913)
       at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5223)
       at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
       at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:752)
       at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:728)
       at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
       at
org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:596)
       at
org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1805)
       at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
and later:
12-Jul-2018 17:44:23.083 SEVERE [http-nio-8080-exec-3]
org.apache.catalina.core.StandardHostValve.invoke Exception Processing
/tomee_tmsp/rest/transmitter/heartbeat
 java.lang.NullPointerException
       at
org.apache.openejb.util.classloader.URLClassLoaderFirst.shouldSkipJsf(URLClassLoaderFirst.java:530)
       at
org.apache.openejb.util.classloader.URLClassLoaderFirst.shouldDelegateToTheContainer(URLClassLoaderFirst.java:146)
       at
org.apache.tomee.catalina.TomEEWebappClassLoader.loadClass(TomEEWebappClassLoader.java:163)
       at
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)
       at
org.apache.catalina.core.StandardWrapper.servletSecurityAnnotationScan(StandardWrapper.java:1128)
       at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:458)
       at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
       at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
       at
org.apache.tomee.catalina.OpenEJBSecurityListener$RequestCapturer.invoke(OpenEJBSecurityListener.java:97)
       at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624)
       at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
       at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
       at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783)
       at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
       at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:798)
       at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1434)
       at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
       at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
       at java.lang.Thread.run(Thread.java:748)
   we have a TOMEE 7.0.3 application, which uses also JMS, which works
without issues. We've recently made a new version of the application
(one of the notable changes is a migration from using the JMS1.1 to the
JMS2 API).
   With the new version, we have an issue, but only on our PROD
environment, where we get no errors in the catalina.out log file (but
the application doesn't work), but an error in the localhost log file,
the error being as in the title of the email, I put the info from the
localhost log file at the end of this email.
   I can't put the source on github or something like that,
unfortunately. What is strange is that this error does not occur on our
staging or UAT environments, but it does occur every time (well, we
tried multiple times to restart the application and it failed
consistently) on the PROD environment. We copied the entire PROD tomee
folder on another machine and failed to reproduce the issue. We also
tried to simulate incoming traffic when restarting the application, and
run it against a backup of the PROD database... Somehow we can't
reproduce the error. We are guessing that maybe the PROD traffic is
higher than what we can simulate and some incoming message comes at just
the wrong time? But we don't know for sure.
   Now, studying the issue, we realized that we used to have this in
our src/main/webapp/WEB-INF/faces-config.xml:
<?xml version="1.0" encoding="UTF-8"?><faces-config
xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-facesconfig_2_1.xsd"
version="2.1"><managed-bean><managed-bean-name>login</managed-bean-name><managed-bean-class>com.lecip.tms.tms_core.application.Login</managed-bean-class><managed-bean-scope>view</managed-bean-scope></managed-bean></faces-config>
   We believe the JSF version is not correct for JEE7 anymore, in
addition the managed-bean annotation is not necessary, so we changed the
file to now contain:
<?xml version="1.0" encoding="UTF-8"?>
<faces-config xmlns="http://xmlns.jcp.org/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
http://xmlns.jcp.org/xml/ns/javaee/web-facesconfig_2_2.xsd"
             version="2.2">
</faces-config>
   Could this be the cause for that error that we saw in our PROD
environment? Or you would think the root cause would be something else?
The problem is that it's pretty difficult for us to experiment with
changes since any deployment on PROD must be first validated on staging
and UAT... Any hints? Is there maybe some logging configuration flag we
could activate to get more information about the error? Or any other
place to look to get more information about the issue?
   The irony is that we use JSF only for the login screen and nowhere
else...
   Regards,
Emmanuel
12-Jul-2018 17:44:04.335 SEVERE [localhost-startStop-1]
org.apache.tomcat.util.descriptor.web.SecurityConstraint.findUncoveredHttpMethods
For security constraints with URL pattern [/*] only the HTTP methods
[TRACE HEAD DELETE POST GET OPTIONS PUT] are covered. All other methods
are uncovered.
12-Jul-2018 17:44:22.078 SEVERE [localhost-startStop-1]
org.apache.catalina.core.ApplicationContext.log StandardWrapper.Throwable
 java.lang.IllegalStateException: No Factories configured for this
Application. This happens if the faces-initialization does not work at
all - make sure that you properly include all configuration settings
necessary for a basic faces application and that all the necessary libs
are included. Also check the logging output of your web application and
your container for any exceptions!
If you did that and find nothing, the mistake might be due to the fact
that you use some special web-containers which do not support
registering context-listeners via TLD files and a context listener is
not setup in your web.xml.
A typical config looks like this;
<listener>
<listener-class>org.apache.myfaces.webapp.StartupServletContextListener</listener-class>
</listener>
       at javax.faces.FactoryFinder._getFactory(FactoryFinder.java:305)
       at javax.faces.FactoryFinder.getFactory(FactoryFinder.java:225)
       at javax.faces.webapp.FacesServlet.init(FacesServlet.java:115)
       at
org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1183)
       at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1099)
       at
org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:989)
       at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4913)
       at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5223)
       at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
       at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:752)
       at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:728)
       at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
       at
org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:596)
       at
org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1805)
       at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
and later:
12-Jul-2018 17:44:23.083 SEVERE [http-nio-8080-exec-3]
org.apache.catalina.core.StandardHostValve.invoke Exception Processing
/tomee_tmsp/rest/transmitter/heartbeat
 java.lang.NullPointerException
       at
org.apache.openejb.util.classloader.URLClassLoaderFirst.shouldSkipJsf(URLClassLoaderFirst.java:530)
       at
org.apache.openejb.util.classloader.URLClassLoaderFirst.shouldDelegateToTheContainer(URLClassLoaderFirst.java:146)
       at
org.apache.tomee.catalina.TomEEWebappClassLoader.loadClass(TomEEWebappClassLoader.java:163)
       at
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)
       at
org.apache.catalina.core.StandardWrapper.servletSecurityAnnotationScan(StandardWrapper.java:1128)
       at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:458)
       at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
       at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
       at
org.apache.tomee.catalina.OpenEJBSecurityListener$RequestCapturer.invoke(OpenEJBSecurityListener.java:97)
       at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624)
       at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
       at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
       at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783)
       at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
       at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:798)
       at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1434)
       at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
       at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
       at java.lang.Thread.run(Thread.java:748)