java - javascript でページをロードしようとすると、HtmlUnit が 40 以上の例外をスローして失敗する原因は何ですか?

Question

履歴書のウェブサイトから練習用クイズをかき集めて、それらすべてを 1 つの大きなテストにまとめて学習できるようにしようとしています。

ページはこちらです。

それはすべてによって駆動されるため、ページをスクレイピングするためjavascriptに使用しようとしています。HtmlUnit

私はどちらかというと Python 派なので、HtmlUnit の Getting Started セクションをかなり厳密に反映するように初期コードをセットアップしました。

import com.gargoylesoftware.htmlunit.*;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;

public class HelloWorld {
    public static void main(String[] args) throws Exception {
        homePage();
        System.out.println("Done.");
    }
    
    public static void homePage() throws Exception {
        final WebClient webClient = new WebClient();
    
        String url = "http://www.wwnorton.com/college/polisci/we-the-people8/shorter/ch/15/quiz.aspx";
        
        final HtmlPage page = webClient.getPage(url);

        System.out.println(page.asText());
        webClient.closeAllWindows();
    }
}

実行すると、次のような出力が得られます: Apr 27, 2013 12:50:16 PM

com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.javascript.host.ActiveXObject jsConstructor
WARNING: Automation server can't create object for 'ShockwaveFlash.ShockwaveFlash'.
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter runtimeError
SEVERE: runtimeError: message=[Automation server can't create object for 'ShockwaveFlash.ShockwaveFlash'.] sourceName=[http://www.wwnorton.com/common/js/shadowbox/shadowbox.js] line=[8] lineSource=[null] lineOffset=[0]
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.javascript.host.ActiveXObject jsConstructor
WARNING: Automation server can't create object for 'QuickTime.QuickTime'.
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter runtimeError
SEVERE: runtimeError: message=[Automation server can't create object for 'QuickTime.QuickTime'.] sourceName=[http://www.wwnorton.com/common/js/shadowbox/shadowbox.js] line=[8] lineSource=[null] lineOffset=[0]
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.javascript.host.ActiveXObject jsConstructor
WARNING: Automation server can't create object for 'wmplayer.ocx'.
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter runtimeError
SEVERE: runtimeError: message=[Automation server can't create object for 'wmplayer.ocx'.] sourceName=[http://www.wwnorton.com/common/js/shadowbox/shadowbox.js] line=[8] lineSource=[null] lineOffset=[0]
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter runtimeError
SEVERE: runtimeError: message=[The data necessary to complete this operation is not yet available.] sourceName=[http://www.wwnorton.com/common/js/shadowbox/shadowbox.js] line=[8] lineSource=[null] lineOffset=[0]
Apr 27, 2013 12:50:16 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2013 12:50:17 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2013 12:50:17 PM com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter runtimeError
SEVERE: runtimeError: message=[The data necessary to complete this operation is not yet available.] sourceName=[http://www.wwnorton.com/college/polisci/we-the-people8/shorter/ScriptResource.axd?d=_NFxbFNZD5BQbaNY82unni-tpvKHIpx_DFI8m05N9H4ZnCF8k_zg2bVneHOOjAQ58itL8--3tACMpiC67WkR4iVWW6J5oqm5iyilArcp1bA4Jl6UUf2tHGjuNfP4BmYCDciCRxCM3FV_f5qrDZM2IQ2&t=fffffffff9cbe881] line=[164] lineSource=[null] lineOffset=[0]
Apr 27, 2013 12:50:17 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2013 12:50:17 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'text/javascript'.
Apr 27, 2013 12:50:17 PM com.gargoylesoftware.htmlunit.javascript.host.html.HTMLDocument execCommand
WARNING: Nothing done for execCommand(BackgroundImageCache, ...) (feature not implemented)
Apr 27, 2013 12:50:17 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler error
WARNING: CSS error: 'http://www.wwnorton.com/common/css/ss2.0/base.css' [848:2] Error in style rule. (Invalid token "!important". Was expecting one of: <EOF>, <S>, <IDENT>, "}", ";".)
Apr 27, 2013 12:50:17 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler warning
WARNING: CSS warning: 'http://www.wwnorton.com/common/css/ss2.0/base.css' [848:2] Ignoring the following declarations in this rule.
Apr 27, 2013 12:50:17 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler error
WARNING: CSS error: 'http://www.wwnorton.com/common/css/min/reset-min.css' [57:1] Error in style rule. (Invalid token "*". Was expecting one of: <EOF>, <S>, <IDENT>, "}", ";".)
Apr 27, 2013 12:50:17 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler warning
WARNING: CSS warning: 'http://www.wwnorton.com/common/css/min/reset-min.css' [57:1] Ignoring the following declarations in this rule.
Apr 27, 2013 12:50:18 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler error
WARNING: CSS error: 'http://www.wwnorton.com/common/css/min/base-min.css' [8:580] Error in style rule. (Invalid token "*". Was expecting one of: <EOF>, <S>, <IDENT>, "}", ";".)
Apr 27, 2013 12:50:18 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler warning
WARNING: CSS warning: 'http://www.wwnorton.com/common/css/min/base-min.css' [8:580] Ignoring the following declarations in this rule.
Apr 27, 2013 12:50:18 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler error
WARNING: CSS error: 'http://www.wwnorton.com/common/css/min/fonts-min.css' [8:55] Error in style rule. (Invalid token "*". Was expecting one of: <EOF>, <S>, <IDENT>, "}", ";".)
Apr 27, 2013 12:50:18 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler warning
WARNING: CSS warning: 'http://www.wwnorton.com/common/css/min/fonts-min.css' [8:55] Ignoring the following declarations in this rule.
Apr 27, 2013 12:50:18 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler error
WARNING: CSS error: 'http://www.wwnorton.com/common/css/min/fonts-min.css' [8:237] Error in style rule. (Invalid token "*". Was expecting one of: <EOF>, <S>, <IDENT>, "}", ";".)
Apr 27, 2013 12:50:18 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler warning
WARNING: CSS warning: 'http://www.wwnorton.com/common/css/min/fonts-min.css' [8:237] Ignoring the following declarations in this rule.
Apr 27, 2013 12:50:18 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler error
WARNING: CSS error: 'http://www.wwnorton.com/college/polisci/we-the-people8/shorter/css/custom.css' [23:1] Error in style rule. (Invalid token "body". Was expecting one of: <S>, <LBRACE>, <COMMA>.)
Apr 27, 2013 12:50:18 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler warning
WARNING: CSS warning: 'http://www.wwnorton.com/college/polisci/we-the-people8/shorter/css/custom.css' [23:1] Ignoring the following declarations in this rule.
Apr 27, 2013 12:50:18 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler error
WARNING: CSS error: 'http://www.wwnorton.com/common/colorblindcss/ss2.0/style.css' [1:51] Error in style rule. (Invalid token "<EOF>". Was expecting one of: <S>, <LBRACE>, <COMMA>, <HASH>, ".", ":", "[", <S>.)
Apr 27, 2013 12:50:18 PM com.gargoylesoftware.htmlunit.DefaultCssErrorHandler warning
WARNING: CSS warning: 'http://www.wwnorton.com/common/colorblindcss/ss2.0/style.css' [1:51] Ignoring the following declarations in this rule.
Exception in thread "main" ======= EXCEPTION START ========
Exception class=[java.lang.RuntimeException]
com.gargoylesoftware.htmlunit.ScriptException: Exception invoking setOuterHTML
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:669)
    at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:601)
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:507)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:555)
    at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:1082)
    at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:399)
    at com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:260)
    at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:276)
    at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:676)
    at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
    at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:635)
    at org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1170)
    at org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1072)
    at org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
    at org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:330)
    at org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3074)
    at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2041)
    at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:918)
    at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
    at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:892)
    at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:241)
    at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:187)
    at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:268)
    at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:156)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:434)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:309)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:374)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:359)
    at HelloWorld.homePage(HelloWorld.java:16)
    at HelloWorld.main(HelloWorld.java:7)
Caused by: java.lang.RuntimeException: Exception invoking setOuterHTML
    at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:148)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject$GetterSlot.setValue(ScriptableObject.java:295)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject$RelinkedSlot.setValue(ScriptableObject.java:368)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.putImpl(ScriptableObject.java:2796)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.put(ScriptableObject.java:521)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.putProperty(ScriptableObject.java:2479)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.setObjectProp(ScriptRuntime.java:1569)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.setObjectProp(ScriptRuntime.java:1564)
    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1253)
    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)
    at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:405)
    at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:275)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3031)
    at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:115)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$3.doRun(JavaScriptEngine.java:546)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:654)
    ... 31 more
Caused by: java.lang.IllegalStateException: Previous sibling for HtmlDivision[<div style="height: 0px; overflow: hidden; border-top: solid black; border-top-width: thick;">] is null.
    at com.gargoylesoftware.htmlunit.html.DomNode.insertBefore(DomNode.java:1036)
    at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLElement.setOuterHTML(HTMLElement.java:1067)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:120)
    ... 47 more
Enclosed exception: 
java.lang.RuntimeException: Exception invoking setOuterHTML
    at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:148)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject$GetterSlot.setValue(ScriptableObject.java:295)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject$RelinkedSlot.setValue(ScriptableObject.java:368)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.putImpl(ScriptableObject.java:2796)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.put(ScriptableObject.java:521)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptableObject.putProperty(ScriptableObject.java:2479)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.setObjectProp(ScriptRuntime.java:1569)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.setObjectProp(ScriptRuntime.java:1564)
    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1253)
    at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:798)
    at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:405)
    at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:275)
    at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3031)
    at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.exec(InterpretedFunction.java:115)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$3.doRun(JavaScriptEngine.java:546)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:654)
    at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:601)
    at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:507)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.execute(JavaScriptEngine.java:555)
    at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:1082)
    at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:399)
    at com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:260)
    at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:276)
    at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:676)
    at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
    at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:635)
    at org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1170)
    at org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1072)
    at org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
    at org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:330)
    at org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3074)
    at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2041)
    at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:918)
    at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
    at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:892)
    at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:241)
    at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:187)
    at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:268)
    at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:156)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:434)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:309)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:374)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:359)
    at HelloWorld.homePage(HelloWorld.java:16)
    at HelloWorld.main(HelloWorld.java:7)
Caused by: java.lang.IllegalStateException: Previous sibling for HtmlDivision[<div style="height: 0px; overflow: hidden; border-top: solid black; border-top-width: thick;">] is null.
    at com.gargoylesoftware.htmlunit.html.DomNode.insertBefore(DomNode.java:1036)
    at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLElement.setOuterHTML(HTMLElement.java:1067)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at net.sourceforge.htmlunit.corejs.javascript.MemberBox.invoke(MemberBox.java:120)
    ... 47 more
======= EXCEPTION END ========

これらすべての例外の原因は何ですか? ページを開くことができないのはなぜですか?

編集

メニューを駆動する別のページを読み込んでみましたが、同様の出力が得られました。javascript

次に、単純に Yahoo.com を読み込んでみました。

String url = "http://www.yahoo.com";
        
final HtmlPage page = webClient.getPage(url);
        
System.out.println(page.asText());

今回はエラーはありませんが、page.asText()何も返されません..

score 7 · Accepted Answer

リンク先のページは、HtmlUnit でバグを引き起こす (未解決の問題は見つかりませんでした) か、Webkit 以外のブラウザーでは単純に壊れている Javascript を使用しています (ページは IE10 で正常に動作しますが)。

このスクリプトのどこかで、ブラウザーが「Safari 以外」(*) でない限り、一部の要素のouterHTMLプロパティが次のように設定されます。null

if (!$telerik.isSafari) {
  c.outerHTML = null;
}
if (!$telerik.isSafari) {
  a.outerHTML = null;
}

(*) 「Safari」はさらに下に表示され、実際には「Webkit ベース」を意味します。

$telerik.isChrome = Sys.Browser.agent == Sys.Browser.Chrome;
$telerik.isSafari4 = Sys.Browser.agent == Sys.Browser.WebKit && Sys.Browser.version >= 526;
$telerik.isSafari3 = Sys.Browser.agent == Sys.Browser.WebKit && Sys.Browser.version < 526 && Sys.Browser.version > 500;
$telerik.isSafari2 = Sys.Browser.agent == Sys.Browser.Safari;
$telerik.isSafari = $telerik.isSafari2 || $telerik.isSafari3 || $telerik.isSafari4 || $telerik.isChrome;

難読化された JS コードが何をしたいのかouterHTMLはわかりませんが、スクリプトの実行時に HtmlUnit の DOM 操作コードの奥深くで状態チェックに失敗します。

java.lang.RuntimeException: Exception invoking setOuterHTML
  at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:669)
  ...
Caused by: java.lang.IllegalStateException: Previous sibling for
    HtmlDivision[<div style="...">] is null.
  at com.gargoylesoftware.htmlunit.html.DomNode.insertBefore(DomNode.java:1036)
  at com.gargoylesoftware.htmlunit.javascript.host.html.HTMLElement.setOuterHTML(HTMLElement.java:1067)
  at ...

HtmlUnit に Chrome として識別するように指示すると、両方のouterHTML = null割り当てが回避されます

WebClient client = new WebClient(BrowserVersion.CHROME);
...

賢明な出力を生成します。

Chapter 15: The Federal Courts | We the People, 8e: W. W. Norton StudySpace
W.W. Norton & Company
Colorblind Mode: On | Off W. W. NORTON HOME | HELP | CREDITS
We the People, 8e: A W. W. Norton StudySpace
...

これは追跡するのが楽しいものでした。

java - javascript でページをロードしようとすると、HtmlUnit が 40 以上の例外をスローして失敗する原因は何ですか?

編集

2 に答える 2

Related

Reference