Web サーバーに http 要求を送信し続けると、応答が停止し、「xxx を待機しています」という白い画面が表示されたままになります。無反応になります。
また、スタックトレースに例外はありません。catalina.out に例外はありません。
Tomcatがソケット接続を閉じていないことに関係があると思います。これは次の場合にのみ発生します。
$ cd /proc/{processID}/fd
$ ls -l | wc -l
1000 を超える値を返します。ほとんどの場合、'ls -l | wc -l' が上がります！
そして、このコマンドを発行すると
$ ll
みたいな記録をよく見かけます
lrwx------ 1 root root 64 5월 22 17:49 989 -> socket:[1018368]
lr-x------ 1 root root 64 5월 22 17:41 99 -> socket:[1017430]
lrwx------ 1 root root 64 5월 22 17:49 990 -> socket:[1018369]
lrwx------ 1 root root 64 5월 22 17:49 991 -> socket:[1018370]
lrwx------ 1 root root 64 5월 22 17:49 992 -> socket:[1018371]
lrwx------ 1 root root 64 5월 22 17:49 993 -> socket:[1018384]
lrwx------ 1 root root 64 5월 22 17:49 994 -> socket:[1018373]
lrwx------ 1 root root 64 5월 22 17:49 995 -> socket:[1018374]
lrwx------ 1 root root 64 5월 22 17:49 996 -> socket:[1018375]
lrwx------ 1 root root 64 5월 22 17:49 997 -> socket:[1018377]
lrwx------ 1 root root 64 5월 22 17:49 998 -> socket:[1018378]
lrwx------ 1 root root 64 5월 22 17:49 999 -> socket:[1018379]
私のWebアプリケーションがデータベースへの接続を返さない（または閉じない）ことではないと確信しています。
MySQL を使用しており、my.cfg を構成しました
[mysqld]
wait-timeout=3
interactive-timeout=60
私はmysqlでそれをチェックすることができます
mysql> select @@session.wait_timeout, @@global.wait_timeout;
+------------------------+-----------------------+
| @@session.wait_timeout | @@global.wait_timeout |
+------------------------+-----------------------+
| 30 | 3 |
+------------------------+-----------------------+
1 row in set (0.00 sec)
また、私が発行した場合
これは DNS の問題ではありません。WebサーバーでWebサイトを問題なく開くことができます。
netstat -tulpn
tcp 0 0 ::ffff:127.0.0.1:8005 :::* LISTEN 21395/java
tcp 0 0 :::8009 :::* LISTEN 21395/java
tcp 0 0 :::80 :::* LISTEN 21395/java
また
lsof -p 21395
たくさん見せる
java 21395 root 359u sock 0,5 0t0 1379694 can't identify protocol
java 21395 root 360u sock 0,5 0t0 1379696 can't identify protocol
java 21395 root 361u sock 0,5 0t0 1379697 can't identify protocol
java 21395 root 362u sock 0,5 0t0 1379700 can't identify protocol
java 21395 root 363u sock 0,5 0t0 1379702 can't identify protocol
java 21395 root 364u sock 0,5 0t0 1379703 can't identify protocol
java 21395 root 365u sock 0,5 0t0 1379961 can't identify protocol
TomcatのMySQLへの接続を見ることができる場所
java 21395 root 390u IPv6 1380717 0t0 TCP cooingbee.co.kr:40026->cooingbee.co.kr:mysql (CLOSE_WAIT)
java 21395 root 391u IPv6 1380718 0t0 TCP cooingbee.co.kr:40025->cooingbee.co.kr:mysql (CLOSE_WAIT)
java 21395 root 392u IPv6 1380721 0t0 TCP cooingbee.co.kr:40028->cooingbee.co.kr:mysql (CLOSE_WAIT)
ulimitも設定しました
]# ulimit -n
4096
]#ulimit -Hn
65536
私のsysctl.confに、これらを追加しましたが、まだ運がありません。
kernel.shmmni = 4096
kernel.sem = 256 32000 100 128
fs.aio-max-nr = 1048576
fs.file-max = 6815744
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
私のサーバー環境に関する追加情報
CentOS 5.x
Java 7
Tomcat6 / 7 (I've tested my webapp on both tomcat 6 and 7)
MySQL 5.5
web application uses Hibernate, c3p0, Spring DI, MVC
を使用して、接続がリークしていないことをDBMS側から確認しました
show processlist
質問
それらの「靴下がプロトコルを識別できない」とは何ですか?
/proc/{processID}/fd に 1000 を超えるレコードがあると、Tomcat が応答しなくなるのはなぜですか?
スムーズに動作させるにはどうすればよいですか？
EDIT 1
コードを表示します。参考までに、私は Hibernate と Spring をこのように設計したり、使用するつもりはありませんでした。なぜ私が今これを言うのかわかると思います;)。
私のhibernate.cfg.xmlの一部
<property name="connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="connection.url">jdbc:mysql://***.***.***.***:3306/****</property>
<property name="connection.username">****</property>
<property name="connection.password">****</property>
<property name="hibernate.c3p0.acquire_increment">5</property>
<property name="hibernate.c3p0.idle_test_period">100</property>
<property name="hibernate.c3p0.max_size">10000</property>
<property name="hibernate.c3p0.max_statements">0</property>
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.testConnectionOnCheckout">true</property>
<property name="hibernate.c3p0.unreturnedConnectionTimeout">70</property>
<property name="hibernate.c3p0.timeout">1</property>
<property name="current_session_context_class">thread</property>
<property name="cache.provider_class">org.hibernate.cache.NoCacheProvider</property>
私のspring-servlet.xmlの一部
<bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<!--
<property name="dataSource"><ref bean="dataSource"/></property>
-->
<property name="configLocation">
<value>classpath:hibernate.cfg.xml</value>
</property>
最後に、いくつかの Java コード
public class ItemDAOImpl extends HibernateDaoSupport implements ItemDAO {
public ItemDetail itemDetail(Integer itemSeq) {
ItemDetail itemDetail = null;
try {
itemDetail = (ItemDetail) this.getHibernateTemplate().get("kr.co.sgis.services.web.mobile.item.form.ItemDetail", itemSeq);
System.out.println("Item Detail~ :: " + itemDetail.getId());
return itemDetail;
} catch (Exception ex) {
System.err.println(ex.getMessage() + "\nThe Stack Trace is:");
ex.printStackTrace();
return itemDetail;
}
}
public Item itemView(Integer itemSeq, Integer mo_id) {
Item itemView = null;
Session session = null;
try {
session = getSession();
Criteria item = session.createCriteria(Item.class)
.add(Restrictions.eq("id", itemSeq))
.add(Restrictions.eq("mobile_id", mo_id));
itemView = (Item) item.uniqueResult();
//itemView = (ItemHead) this.getHibernateTemplate().get("kr.co.sgis.services.web.mobile.item.form.Item", itemSeq);
System.out.println("Item View~ :: " + itemView.getId());
session.flush();
return itemView;
} catch (Exception ex) {
System.err.println(ex.getMessage() + "\nThe Stack Trace is:");
ex.printStackTrace();
return itemView;
}finally{
HibernateUtil.tryCloseAll(session, null);
}
}
...
}
インターネットで検索した限りでは、getHibernateTemplate() と HibernateDAOSupport は推奨されません。また、getHibernateTemplate() を使用する場合、手動で明示的にセッションを閉じる必要はありませんが、絶望的になったため、「HibernateUtil.tryCloseAll()」を追加しました。
ここ
public static void tryCloseAll(Session s, Transaction t){
try{
if(t!=null){
if(t.isActive()){
//?
t.commit();
}
}
if(s!=null){
if(s.isOpen()){
try{
s.flush();
}catch(Exception e){
System.out.println("Tried to close session and failed");
}
}
}
}catch(Exception e){
}
}
EDIT 2
以下は kill -QUIT PID の出力です
2013-05-23 17:26:10
Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode):
"http-80-6" daemon prio=10 tid=0x0000000002ad3800 nid=0x357f in Object.wait() [0x00002b75db57d000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fab5c158> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at java.lang.Object.wait(Object.java:503)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
- locked <0x00000000fab5c158> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
at java.lang.Thread.run(Thread.java:722)
"http-80-5" daemon prio=10 tid=0x0000000002505800 nid=0x3577 in Object.wait() [0x00002b75db47c000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fab1a4a0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at java.lang.Object.wait(Object.java:503)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
- locked <0x00000000fab1a4a0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
at java.lang.Thread.run(Thread.java:722)
"http-80-4" daemon prio=10 tid=0x0000000002f6a800 nid=0x3576 in Object.wait() [0x00002b75db37b000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fab60f68> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at java.lang.Object.wait(Object.java:503)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
- locked <0x00000000fab60f68> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
at java.lang.Thread.run(Thread.java:722)
"http-80-3" daemon prio=10 tid=0x00000000036fd800 nid=0x3575 in Object.wait() [0x00002b75db27a000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fab61d68> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at java.lang.Object.wait(Object.java:503)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
- locked <0x00000000fab61d68> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
at java.lang.Thread.run(Thread.java:722)
"http-80-2" daemon prio=10 tid=0x0000000002ac2000 nid=0x3574 in Object.wait() [0x00002b75db179000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fab1f2f0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at java.lang.Object.wait(Object.java:503)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
- locked <0x00000000fab1f2f0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
at java.lang.Thread.run(Thread.java:722)
"http-80-1" daemon prio=10 tid=0x00000000028eb800 nid=0x3571 in Object.wait() [0x00002b75dae6e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fab62188> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at java.lang.Object.wait(Object.java:503)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
- locked <0x00000000fab62188> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
at java.lang.Thread.run(Thread.java:722)
"TP-Monitor" daemon prio=10 tid=0x000000000260d800 nid=0x356f in Object.wait() [0x00002b75dad6d000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fab19420> (a org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
at org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable.run(ThreadPool.java:565)
- locked <0x00000000fab19420> (a org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
at java.lang.Thread.run(Thread.java:722)
"TP-Processor4" daemon prio=10 tid=0x0000000002add000 nid=0x356e runnable [0x00002b75dac6c000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
at java.net.ServerSocket.implAccept(ServerSocket.java:522)
at java.net.ServerSocket.accept(ServerSocket.java:490)
at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:311)
at org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:668)
at org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(ChannelSocket.java:879)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
at java.lang.Thread.run(Thread.java:722)
"TP-Processor3" daemon prio=10 tid=0x0000000002adb800 nid=0x356d in Object.wait() [0x00002b75dab6b000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fab18d30> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Object.wait(Object.java:503)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:662)
- locked <0x00000000fab18d30> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Thread.run(Thread.java:722)
"TP-Processor2" daemon prio=10 tid=0x00000000025d9000 nid=0x356c in Object.wait() [0x00002b75daa6a000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fab18f80> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Object.wait(Object.java:503)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:662)
- locked <0x00000000fab18f80> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Thread.run(Thread.java:722)
"TP-Processor1" daemon prio=10 tid=0x00000000025d8800 nid=0x356b in Object.wait() [0x00002b75da969000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000fab191d0> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Object.wait(Object.java:503)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:662)
- locked <0x00000000fab191d0> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Thread.run(Thread.java:722)
"http-80-Acceptor-0" daemon prio=10 tid=0x00002b75dc769000 nid=0x356a runnable [0x00002b75da622000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
at java.net.ServerSocket.implAccept(ServerSocket.java:522)
at java.net.ServerSocket.accept(ServerSocket.java:490)
at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:352)
at java.lang.Thread.run(Thread.java:722)
"ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon prio=10 tid=0x000000000424d000 nid=0x3569 waiting on condition [0x00002b75da521000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1631)
at java.lang.Thread.run(Thread.java:722)
"Timer-1" daemon prio=10 tid=0x000000000267b000 nid=0x3564 in Object.wait() [0x00002b75da82c000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000c3046fb8> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:552)
- locked <0x00000000c3046fb8> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)
"Timer-0" prio=10 tid=0x00000000026c0800 nid=0x3563 in Object.wait() [0x00002b75da72b000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000c30906a0> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:552)
- locked <0x00000000c30906a0> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)
"GC Daemon" daemon prio=10 tid=0x00002b75dc44d800 nid=0x3562 in Object.wait() [0x00002b75d9d98000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000c29e3d38> (a sun.misc.GC$LatencyLock)
at sun.misc.GC$Daemon.run(GC.java:117)
- locked <0x00000000c29e3d38> (a sun.misc.GC$LatencyLock)
"Service Thread" daemon prio=10 tid=0x0000000002382000 nid=0x3560 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" daemon prio=10 tid=0x0000000002377000 nid=0x355f waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" daemon prio=10 tid=0x0000000002374000 nid=0x355e waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x0000000002371800 nid=0x355d waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0x0000000002324800 nid=0x355c in Object.wait() [0x00002b75d5832000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000c287dbd8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
- locked <0x00000000c287dbd8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
"Reference Handler" daemon prio=10 tid=0x000000000231d000 nid=0x355b in Object.wait() [0x00002b75d5731000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000c287d7f8> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked <0x00000000c287d7f8> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0x0000000002284000 nid=0x3551 runnable [0x00002b75cc81a000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
at java.net.ServerSocket.implAccept(ServerSocket.java:522)
at java.net.ServerSocket.accept(ServerSocket.java:490)
at org.apache.catalina.core.StandardServer.await(StandardServer.java:431)
at org.apache.catalina.startup.Catalina.await(Catalina.java:676)
at org.apache.catalina.startup.Catalina.start(Catalina.java:628)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
"VM Thread" prio=10 tid=0x0000000002315800 nid=0x355a runnable
"GC task thread#0 (ParallelGC)" prio=10 tid=0x0000000002291800 nid=0x3552 runnable
"GC task thread#1 (ParallelGC)" prio=10 tid=0x0000000002293800 nid=0x3553 runnable
"GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000002295000 nid=0x3554 runnable
"GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000002297000 nid=0x3555 runnable
"GC task thread#4 (ParallelGC)" prio=10 tid=0x0000000002299000 nid=0x3556 runnable
"GC task thread#5 (ParallelGC)" prio=10 tid=0x000000000229a800 nid=0x3557 runnable
"GC task thread#6 (ParallelGC)" prio=10 tid=0x000000000229c800 nid=0x3558 runnable
"GC task thread#7 (ParallelGC)" prio=10 tid=0x000000000229e800 nid=0x3559 runnable
"VM Periodic Task Thread" prio=10 tid=0x000000000238c800 nid=0x3561 waiting on condition
JNI global references: 232
Heap
PSYoungGen total 197248K, used 65894K [0x00000000eb760000, 0x00000000fbb30000, 0x0000000100000000)
eden space 180608K, 27% used [0x00000000eb760000,0x00000000ee781540,0x00000000f67c0000)
from space 16640K, 99% used [0x00000000faaf0000,0x00000000fbb28510,0x00000000fbb30000)
to space 26368K, 0% used [0x00000000f87b0000,0x00000000f87b0000,0x00000000fa170000)
ParOldGen total 76864K, used 45414K [0x00000000c2600000, 0x00000000c7110000, 0x00000000eb760000)
object space 76864K, 59% used [0x00000000c2600000,0x00000000c5259a10,0x00000000c7110000)
PSPermGen total 67200K, used 44249K [0x00000000bd400000, 0x00000000c15a0000, 0x00000000c2600000)
object space 67200K, 65% used [0x00000000bd400000,0x00000000bff36788,0x00000000c15a0000)