时间戳

首页 > 解决方案 > ping 数千个 url 的 Java 程序

java - ping 数千个 url 的 Java 程序

问题描述

我想编写一个程序来 ping 至少 10000 个 url。我写了一个小程序，发现并没有我想象的那么快。

ping 100 个网址需要 3-4 分钟。

有人有什么建议可以更好地做到这一点。

private static Map<String, String> findUnreachableUrls(Set<String> urls) {
        Map<String, String> badUrls = new TreeMap<>();
        for (String url : urls) {
            HttpURLConnection connection;
            try {
                connection = (HttpURLConnection) new URL(url).openConnection();
                connection.setRequestMethod("HEAD");
                connection.connect();
                int responseCode = connection.getResponseCode();
                if (responseCode != 200 && responseCode != 302) {
                    badUrls.put(url, Integer.toString(responseCode));
                }
            } catch (IOException e) {
                badUrls.put(url, e.getMessage());
            }

        }
        return badUrls;
    }

标签： javahttpclient

解决方案

您应该使用并行线程，例如 5 个线程对 20 个 URL 执行相同的过程并最终聚合结果。这将使结果更快。最简单的解决方案是使用 Java 8 Streams 并行处理 URL。下面是一个相同的示例程序：

import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.TreeMap;

public class Main {

    public static void main(String[] args) {
        Set<String> urlSet = new HashSet<>();
        //Populate your set with the Strings
        findUnreachableUrls(urlSet);
    }
    private static Map<String, String> findUnreachableUrls(Set<String> urls) {
        Map<String, String> badUrls = new TreeMap<>();
        urls.parallelStream().forEach(
                url->{
                    badUrls.put(url,checkUrl(url));
                }
        );
        return badUrls;
    }

    private static String checkUrl(String url)
    {
        HttpURLConnection connection;
        String returnCode="";
        try {
            connection = (HttpURLConnection) new URL(url).openConnection();
            connection.setRequestMethod("HEAD");
            connection.connect();
            int responseCode = connection.getResponseCode();
            if (responseCode != 200 || responseCode != 302) {
                returnCode=responseCode+"";
            }
        }catch(IOException e)
        {
            returnCode=e.getMessage();
        }
        return returnCode;
    }

}

推荐阅读