java线程死锁导致tomcat无响应

327238349

服务器在使用率比较高的情况下，cloudX5流程发布到正式服务，偶尔会出现线程死锁现象（在justep.biz.log中分析时发现卡死的时间前都会有卸载模块与批量的加载模块），
然后造成线程数量会增长并且tomcat不能访问，只能重启解决；
用jdk工具抓到了以下内容。希望尽快解决此问题！！


名称: http-8080-8715

状态: java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@91265b1上的WAITING, 拥有者: http-8080-57

总阻止数: 0, 总等待数: 1

堆栈跟踪:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
com.justep.utils.Utils.queryOnlineUser(Native Method)
com.justep.business.server.BusinessServerServlet.queryOnlineUsers(Unknown Source)
com.justep.business.server.BusinessServerServlet.execService(Unknown Source)
com.justep.business.server.BusinessServerServlet.service(Unknown Source)
javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
com.justep.x.bs.BusinessServerServlet.service(Unknown Source)
javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
com.googlecode.psiprobe.Tomcat60AgentValve.invoke(Tomcat60AgentValve.java:40)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
java.lang.Thread.run(Thread.java:745)



名称: http-8080-25
状态: java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@91265b1上的WAITING, 拥有者: http-8080-57

总阻止数: 24, 总等待数: 7

堆栈跟踪:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
com.justep.model.impl.ModelLoader.handleDepends(Native Method)
com.justep.model.impl.ModelLoader.loadModel(Unknown Source)
   - 已锁定 java.lang.Class@5230d3f0
com.justep.model.impl.ModelImpl.getModel(Unknown Source)
com.justep.model.impl.ModelLoader.unLoadModel(Unknown Source)
   - 已锁定 java.lang.Class@5230d3f0
com.justep.business.server.BusinessServer.clearCache(Unknown Source)
com.justep.business.server.BusinessServerServlet.execService(Unknown Source)
com.justep.business.server.BusinessServerServlet.service(Unknown Source)
javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
com.justep.x.bs.BusinessServerServlet.service(Unknown Source)
javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
com.googlecode.psiprobe.Tomcat60AgentValve.invoke(Tomcat60AgentValve.java:40)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
java.lang.Thread.run(Thread.java:745)

jishuang

tomcat无响应有错误信息抛出吗？

tomcat中配置的线程数是多少？可以安装一下tomcat的检测工具，监控一下线程的占用情况
http://docs.wex5.com/wex5-deploy-question-list-0004/

tiger_lixy

jishuang 发表于 2018-3-2 16:48
tomcat无响应有错误信息抛出吗？

tomcat中配置的线程数是多少？可以安装一下tomcat的检测工具，监控一下线 ...

在出问题后，probe工具是没办法访问到的。tomcat配置线程数2.1w。
线程开始是不多的，死锁后，后续响应无法处理堆积，逐渐增大到系统承受极限。

jishuang

相关人员已经在处理

nikcol

请问解决了吗？我这里也碰到这样的问题
每次发生，都得重启tomcat服务才行

jishuang

楼主的问题不是Bex5的，死锁看具体什么集成锁的