このページをダウンロードしようとしています: http://ekw.ms.gov.pl/pdcbdkw/pdcbdkw.html HtmlUnit 2.11 を使用します。このページで GET を実行した後、JS スクリプトで計算して Cookie を設定し、フォームを送信します。FF 18 とまったく同じようにヘッダーを作成しました。確認したところ、HtmlUnit によって JavaScript で計算されたヘッダーと Cookie は正しいです。スクリプトを使用してページをダウンロードし、フォームのアクションを変更して独自のスクリプトを POST に変更し、リクエストのヘッダーと本文を印刷して確認しました。次のコードを使用します。
WebClient client = new WebClient();
client.setWebConnection(new HttpWebConnection(client) {
public WebResponse getResponse(WebRequest request) throws IOException {
if(request.getHttpMethod() == HttpMethod.POST) {
request.removeAdditionalHeader("Cache-Control");
}
else {
request.setAdditionalHeader("Cache-Control","max-age=0");
}
WebResponse response = super.getResponse(request);
return response;
}
});
client.addRequestHeader("User-Agent","Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0");
client.addRequestHeader("Host","ekw.ms.gov.pl");
client.addRequestHeader("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
client.addRequestHeader("Accept-Language","en-gb,en;q=0.5");
client.addRequestHeader("Accept-Encoding","gzip, deflate");
client.addRequestHeader("Connection", "keep-alive");
HtmlPage html = client.getPage("http://ekw.ms.gov.pl/pdcbdkw/pdcbdkw.html");
コードを実行すると、次の例外が発生します (ショートカットでは、接続のリセットが原因です)。
Exception in thread "main" ======= EXCEPTION START ========
Exception class=[java.lang.RuntimeException]
com.gargoylesoftware.htmlunit.ScriptException: Exception invoking submit
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:663)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:559)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:525)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:594)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:569)
at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptFunctionIfPossible(HtmlPage.java:996)
at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeEventHandler(EventListenersContainer.java:208)
at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeBubblingListeners(EventListenersContainer.java:227)
at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:813)
at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:737)
at com.gargoylesoftware.htmlunit.html.HtmlElement$1.run(HtmlElement.java:853)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:559)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:525)
at com.gargoylesoftware.htmlunit.html.HtmlElement.fireEvent(HtmlElement.java:858)
at com.gargoylesoftware.htmlunit.html.HtmlPage.executeEventHandlersIfNeeded(HtmlPage.java:1259)
at com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:227)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:440)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:308)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:373)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:358)
at cenatorium.kwtest.App.main(App.java:49)
Caused by: java.lang.RuntimeException: Exception invoking submit
at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:163)
at net.sourceforge.htmlunit.corejs.javascript.FunctionObject.call(FunctionObject.java:452)
at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1473)
at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:815)
at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:109)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:415)
at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:274)
at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3132)
at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:107)
at com.gargoylesoftware.htmlunit.javascript.host.EventHandler.call(EventHandler.java:80)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$4.doRun(JavaScriptEngine.java:587)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:651)
... 20 more
Caused by: java.lang.RuntimeException: java.net.SocketException: Connection reset
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2233)
at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLFormElement.submit(HTMLFormElement.java:310)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:137)
... 31 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:712)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:517)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:171)
at cenatorium.kwtest.App$1.getResponse(App.java:38)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1484)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1402)
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2226)
... 37 more
Enclosed exception:
java.lang.RuntimeException: Exception invoking submit
at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:163)
at net.sourceforge.htmlunit.corejs.javascript.FunctionObject.call(FunctionObject.java:452)
at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1473)
at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:815)
at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:109)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:415)
at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:274)
at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3132)
at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:107)
at com.gargoylesoftware.htmlunit.javascript.host.EventHandler.call(EventHandler.java:80)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$4.doRun(JavaScriptEngine.java:587)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:651)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:559)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:525)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:594)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:569)
at com.gargoylesoftware.htmlunit.html.HtmlPage.executeJavaScriptFunctionIfPossible(HtmlPage.java:996)
at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeEventHandler(EventListenersContainer.java:208)
at com.gargoylesoftware.htmlunit.javascript.host.EventListenersContainer.executeBubblingListeners(EventListenersContainer.java:227)
at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:813)
at com.gargoylesoftware.htmlunit.javascript.host.Node.fireEvent(Node.java:737)
at com.gargoylesoftware.htmlunit.html.HtmlElement$1.run(HtmlElement.java:853)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:559)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:525)
at com.gargoylesoftware.htmlunit.html.HtmlElement.fireEvent(HtmlElement.java:858)
at com.gargoylesoftware.htmlunit.html.HtmlPage.executeEventHandlersIfNeeded(HtmlPage.java:1259)
at com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:227)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:440)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:308)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:373)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:358)
at cenatorium.kwtest.App.main(App.java:49)
Caused by: java.lang.RuntimeException: java.net.SocketException: Connection reset
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2233)
at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLFormElement.submit(HTMLFormElement.java:310)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:137)
... 31 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:712)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:517)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:171)
at cenatorium.kwtest.App$1.getResponse(App.java:38)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1484)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1402)
at com.gargoylesoftware.htmlunit.WebClient.download(WebClient.java:2226)
... 37 more
== CALLING JAVASCRIPT ==
function () {
[native code, arity=0]
}
======= EXCEPTION END ========
POST で送信されるヘッダーは次のとおりです (FF18 と同じですが、順序が異なりますが、重要ではありません)。
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0
Accept-Language: en-gb,en;q=0.5
Host: localhost
Referer: http://localhost/test.php
Accept-Encoding: gzip, deflate
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Content-Length: 69
Content-Type: application/x-www-form-urlencoded
Cookie: TS37535a_75=a538065ddccf975d9ae56e368a94eea0:hklh:781te27c:861731662
----------------- POST content
TS37535a_id=3&TS37535a_md=1&TS37535a_rf=0&TS37535a_ct=0&TS37535a_pd=0
Cookie の値は正しいです。確認しました。
それで、何が問題なのですか?? このページを取得する方法 -ブラウザーにhttp://ekw.ms.gov.pl/pdcbdkw/pdcbdkw.htmlと入力すると、html フォームが返されます...