时间戳

首页 > 解决方案 > 使用JSoup获取被onclick按钮javascript隐藏的表的内容

javascript - 使用JSoup获取被onclick按钮javascript隐藏的表的内容

问题描述

我正在创建一个网络抓取供个人在游戏中使用。这是我要抓取的网站：http: //forum.toribash.com/clan_war.php? clanid=139

我想计算出现在“显示详细信息”上的名称的频率。

我已阅读此从 javascript onClick 超链接获取内容，但不知道这是否真的是我正在搜索的内容。我怀疑这不是我要寻找的东西，但无论如何我都没有尝试过这些问题的答案，因为我不知道如何使这个https://stackoverflow.com/a/12268561/10467473适合我想要的是。

        BufferedReader month = new BufferedReader(new InputStreamReader(System.in));
        String mth = month.readLine();
        //Accessing the website
        Document docs = Jsoup.connect("http://forum.toribash.com/clan_war.php?clanid=139").get();

        //Taking every entry of war history
        Elements collection = docs.getElementsByClass("war_history_entry");
        //Itterate every collection
        for(Element e : collection){
            //if the info is on the exact month that are being searched we will use the e
            if(e.getElementsByClass("war_info").text().split(" ")[1].equalsIgnoreCase(mth)){
                //supposedly it holds every element that has player as it class inside of the button onclick
                //But it doesn't work
                Elements cek = e.getElementsByClass("player");
                for(Element c : cek){
                    System.out.println(c.text());
                }
            }

现在我希望至少能在显示详细信息表上获得名称

Kaito
Chax
Draku

等等

标签： javascriptjavajsoup

解决方案

此页面不包含您要抓取的信息。单击按钮后，结果由 AJAX (Javascript) 加载。您可以使用 Web 浏览器的调试器查看“网络”选项卡，以查看单击该按钮时会发生什么。单击一个按钮

<button id="buttonwarid19557"  ... >

从 URL 加载表：

http://forum.toribash.com/clan_war_ajax.php?warid=19557&clanid=139

注意相同的身份证号码。

您要做的是从每个按钮获取 id，然后为每个按钮获取另一个文档并逐个解析它。无论如何，这就是您的网络浏览器所做的事情。

        BufferedReader month = new BufferedReader(new InputStreamReader(System.in));
        String mth = month.readLine();
        //Accessing the website
        Document docs = Jsoup.connect("http://forum.toribash.com/clan_war.php?clanid=139").get();

        //Taking every entry of war history
        Elements collection = docs.getElementsByClass("war_history_entry");
        //Itterate every collection
        for(Element e : collection){
            //if the info is on the exact month that are being searched we will use the e
            if(e.getElementsByClass("war_info").text().split(" ")[1].equalsIgnoreCase(mth)){
                // selecting button
                Element button = e.selectFirst("button");
                // getting warid from button id
                String buttonId = button.attr("id");
                // removing text because we need only number
                String warId = buttonId.replace("buttonwarid", "");

                System.out.println("downloading results for " + e.getElementsByClass("war_info").text());
                // downloading and parsing subpage containing table with info about single war
                // adding referrer to make the request look more like it comes from the real web browser to avoid possible hotlinking protection
                Document table = Jsoup.connect("http://forum.toribash.com/clan_war_ajax.php?warid=" + warId + "&clanid=139").referrer("http://forum.toribash.com/clan_war.php?clanid=139").get();
                // get every <td class="player"> ... </td>
                Elements players = table.select(".player");
                for(Element player : players){
                    System.out.println(player.text());
                }
            }
        }

推荐阅读