java - 获取远程数据失败（被动模式、TLS隐式、客户端证书+登录/密码认证）
问题描述
我尝试使用 apache-commons-net-3.7.2（隐式 TLS，使用客户端证书 + 登录名/密码的双因素身份验证）连接 FTP 服务器。
我可以验证自己，进入被动模式，但是客户端无法成功连接到服务器以通过数据套接字获取数据。
我可以在同一台计算机上使用 WinSCP（相同的设置）连接自己，我已经激活了 WinSCP 日志以查看协议详细信息，并且我已经使用相同的选项调整了我的源代码。我可以使用 ProtocolCommandListener 验证我的协议是否正常。
WinSCP 日志如下
连接并浏览根目录
Chiffrement : Chiffrement SSL/TLSimplicite
Version min TLS/SSL : TLS 1.0
Version max TLS/SSL : TLS 1.2
Réutiliser l'ID de session TLS/SL pour le connexions de données activé
(reuse ssl session id for data连接已激活）
我使用与我的 Java 相同的证书（带有 WinSCP 的 PFX 格式，用于 Java 的 JCEKS，即 Java 8）
. 2021-01-07 15:36:03.031 --------------------------------------------------------------------------
. 2021-01-07 15:36:03.032 WinSCP Version 5.17.9 (Compilation 10905) (OS 10.0.19041 - Windows 10 Enterprise)
. 2021-01-07 15:36:03.032 Configuration: HKCU\Software\Martin Prikryl\WinSCP 2\
. 2021-01-07 15:36:03.032 Log level: Normal
. 2021-01-07 15:36:03.032 Local account: FANLESS\damie
. 2021-01-07 15:36:03.032 Working directory: C:\Users\damie\Desktop
. 2021-01-07 15:36:03.032 Process ID: 22344
. 2021-01-07 15:36:03.033 Command-line: "C:\Program Files (x86)\WinSCP\WinSCP.exe" "xxxxxx" /Desktop /UploadIfAny /log="d:\temp\debugftp.log" /loglevel=0
. 2021-01-07 15:36:03.033 Time zone: Current: GMT+1, Standard: GMT+1 (Paris, Madrid), DST: GMT+2 (Paris, Madrid (heure d’été)), DST Start: 28/03/2021, DST End: 31/10/2021
. 2021-01-07 15:36:03.033 Login time: jeudi 7 janvier 2021 15:36:03
. 2021-01-07 15:36:03.033 --------------------------------------------------------------------------
. 2021-01-07 15:36:03.033 Session name: XXX@XXX (Site)
. 2021-01-07 15:36:03.034 Host name: XXX (Port: 999)
. 2021-01-07 15:36:03.034 User name: XXX (Password: Yes, Key file: No, Passphrase: No)
. 2021-01-07 15:36:03.034 Transfer Protocol: FTP
. 2021-01-07 15:36:03.034 Ping type: Dummy, Ping interval: 30 sec; Timeout: 15 sec
. 2021-01-07 15:36:03.034 Disable Nagle: No
. 2021-01-07 15:36:03.034 Proxy: None
. 2021-01-07 15:36:03.034 Send buffer: 262144
. 2021-01-07 15:36:03.034 UTF: Auto
. 2021-01-07 15:36:03.034 FTPS: Implicit TLS/SSL [Client certificate: Yes]
. 2021-01-07 15:36:03.034 FTP: Passive: Yes [Force IP: Auto]; MLSD: Auto [List all: Auto]; HOST: Auto
. 2021-01-07 15:36:03.034 Session reuse: Yes
. 2021-01-07 15:36:03.034 TLS/SSL versions: TLSv1.0-TLSv1.2
. 2021-01-07 15:36:03.034 Local directory: D:\Local\Documents, Remote directory: /, Update: Yes, Cache: Yes
. 2021-01-07 15:36:03.034 Cache directory changes: Yes, Permanent: Yes
. 2021-01-07 15:36:03.034 Recycle bin: Delete to: No, Overwritten to: No, Bin path:
. 2021-01-07 15:36:03.034 Timezone offset: 0h 0m
. 2021-01-07 15:36:03.034 --------------------------------------------------------------------------
. 2021-01-07 15:36:03.381 Connexion à XXX:999...
. 2021-01-07 15:36:03.382 Connecté à XXX:999, négociation de la connexion SSL...
. 2021-01-07 15:36:03.382 Server asks for authentication with a client certificate.
. 2021-01-07 15:36:03.382 Verifying certificate for "RESEAU GDS" with fingerprint 58:12:03:0e:b3:dd:cc:d9:c4:a1:ce:a0:4a:d8:83:77:46:0b:9e:4c and 20 failures
. 2021-01-07 15:36:03.382 Certificate common name "XXX" matches hostname
. 2021-01-07 15:36:03.463 Certificate verified against Windows certificate store
. 2021-01-07 15:36:03.463 Using TLSv1.2, cipher TLSv1.2: AES256-GCM-SHA384, 2048 bit RSA, AES256-GCM-SHA384 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(256) Mac=AEAD
. 2021-01-07 15:36:03.479 Connexion SSL établie. En attente du message de bienvenue...
< 2021-01-07 15:36:03.479 220 w2003-ftp X2 WS_FTP Server 7.7(73371983)
> 2021-01-07 15:36:03.479 USER XXX
< 2021-01-07 15:36:03.483 331 Enter password
> 2021-01-07 15:36:03.483 PASS ********************
< 2021-01-07 15:36:03.533 230-User logged in
< 2021-01-07 15:36:03.533 Bonjour Serveur FTP de ReseauGDS
< 2021-01-07 15:36:03.533 230 User logged in
> 2021-01-07 15:36:03.533 SYST
< 2021-01-07 15:36:03.548 215 UNIX
> 2021-01-07 15:36:03.548 FEAT
< 2021-01-07 15:36:03.567 211-Extensions supported
< 2021-01-07 15:36:03.567 SIZE
< 2021-01-07 15:36:03.567 XMD5
< 2021-01-07 15:36:03.567 XSHA1
< 2021-01-07 15:36:03.567 XSHA256
< 2021-01-07 15:36:03.567 XSHA512
< 2021-01-07 15:36:03.567 XQUOTA
< 2021-01-07 15:36:03.567 LANG EN, ES, FR, GE
< 2021-01-07 15:36:03.567 MDTM
< 2021-01-07 15:36:03.567 MLST size*;type*;perm*;create*;modify*;
< 2021-01-07 15:36:03.567 REST STREAM
< 2021-01-07 15:36:03.567 TVFS
< 2021-01-07 15:36:03.567 UTF8
< 2021-01-07 15:36:03.568 AUTH SSL;TLS-P;
< 2021-01-07 15:36:03.568 PBSZ
< 2021-01-07 15:36:03.568 PROT C;P;
< 2021-01-07 15:36:03.568 211 end
> 2021-01-07 15:36:03.568 OPTS UTF8 ON
< 2021-01-07 15:36:03.584 200 Command OPTS succeed
> 2021-01-07 15:36:03.584 PBSZ 0
< 2021-01-07 15:36:03.598 200 PBSZ=0
> 2021-01-07 15:36:03.598 PROT P
< 2021-01-07 15:36:03.615 200 PRIVATE data channel protection level set
. 2021-01-07 15:36:03.664 Connecté
. 2021-01-07 15:36:03.664 --------------------------------------------------------------------------
. 2021-01-07 15:36:03.664 Using FTP protocol.
. 2021-01-07 15:36:03.664 Doing startup conversation with host.
> 2021-01-07 15:36:03.679 PWD
< 2021-01-07 15:36:03.694 257 "/" is current directory
. 2021-01-07 15:36:03.694 Changing directory to "/".
> 2021-01-07 15:36:03.695 CWD /
< 2021-01-07 15:36:03.723 250 Command CWD succeed
. 2021-01-07 15:36:03.723 Getting current directory name.
> 2021-01-07 15:36:03.723 PWD
< 2021-01-07 15:36:03.740 257 "/" is current directory
. 2021-01-07 15:36:03.787 Listage du répertoire en cours...
> 2021-01-07 15:36:03.787 TYPE A
< 2021-01-07 15:36:03.802 200 Transfer mode set to ASCII
> 2021-01-07 15:36:03.802 PASV
< 2021-01-07 15:36:03.817 227 Entering Passive Mode (192,168,4,122,235,142).
. 2021-01-07 15:36:03.817 Server sent passive reply with unroutable address 192.168.4.122, using host address instead.
> 2021-01-07 15:36:03.817 MLSD
. 2021-01-07 15:36:03.817 Connexion à 83.XXX.XXX.XXX:60302...
< 2021-01-07 15:36:03.845 150 Transferring directory
. 2021-01-07 15:36:03.846 Session ID reused
. 2021-01-07 15:36:03.847 Using TLSv1.2, cipher TLSv1.2: AES256-GCM-SHA384, 2048 bit RSA, AES256-GCM-SHA384 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(256) Mac=AEAD
. 2021-01-07 15:36:03.847 Connexion SSL établie
< 2021-01-07 15:36:03.869 226 Transfer completed
. 2021-01-07 15:36:03.869 Data connection closed
. 2021-01-07 15:36:03.869 size=0;type=cdir;create=20200623122630;modify=20201020142437; .
. 2021-01-07 15:36:03.869 size=0;type=pdir;create=20160308132406;modify=20200812101755; ..
. 2021-01-07 15:36:03.869 size=0;type=dir;create=20201020141336;modify=20201208104858; Mensuel
. 2021-01-07 15:36:03.869 size=0;type=dir;create=20201020141315;modify=20210107105026; Quotidien
. 2021-01-07 15:36:03.869 Listage du répertoire réussi
. 2021-01-07 15:36:03.870 ..;D;0;1899-12-30T01:00:00.000Z;0;"" [0];"" [0];---------;0
. 2021-01-07 15:36:03.870 Mensuel;D;0;2020-12-08T10:48:58.000Z;3;"" [0];"" [0];---------;0
. 2021-01-07 15:36:03.870 Quotidien;D;0;2021-01-07T10:50:26.000Z;3;"" [0];"" [0];---------;0
. 2021-01-07 15:36:03.893 Startup conversation with host finished.
Java 代码摘录如下。您可以看到我使用 ProtocolCommandListener 来输出协议详细信息
public static void main(String[] args) throws SocketException, IOException, GeneralSecurityException {
String hostname = args[0];
int port = Integer.parseInt(args[1]);
String login = args[2];
String pwd = args[3];
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
//CompliantFTPSClient client = new CompliantFTPSClient("TLS", true);
FTPSClient client = new FTPSClient("TLS", true);
File clientCertStore = new File(
"path to store");
KeyManager keyManager = KeyManagerUtils.createClientKeyManager("JCEKS", clientCertStore, "",
"alias", "");
client.setKeyManager(keyManager);
client.connect(hostname, port);
int reply = client.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
client.disconnect();
System.err.println("FTP server refused connection.");
System.exit(1);
} else {
if (client.login(login, pwd)) {
client.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.err), true));
client.sendCommand("OPTS", "UTF8 ON");
client.sendCommand("PBSZ", "0");
client.sendCommand("PROT", "P");
client.enterLocalPassiveMode();
System.out.println("Current directory : "+client.printWorkingDirectory());
System.out.println(client.listHelp());
FTPFile[] remoteFiles = client.mlistDir();
reply = client.getReplyCode();
System.out.println("fichiers : "+reply);
System.out.println("nombre : "+remoteFiles.length);
System.out.println("LOGOUT");
client.logout();
} else {
System.out.println("echec login");
}
}
}
}
Java 详细日志如下（system.err 和 System.out）：
janv. 07, 2021 8:31:50 PM sun.security.ssl.SSLLogger log
AVERTISSEMENT: Signature algorithm, ed25519, is not supported by the underlying providers
janv. 07, 2021 8:31:50 PM sun.security.ssl.SSLLogger log
AVERTISSEMENT: Signature algorithm, ed448, is not supported by the underlying providers
janv. 07, 2021 8:31:50 PM sun.security.ssl.SSLLogger log
INFOS: No available application protocols
janv. 07, 2021 8:31:50 PM sun.security.ssl.SSLLogger log
AVERTISSEMENT: Ignore impact of unsupported extension: renegotiation_info
OPTS UTF8 ON
200 Command OPTS succeed
PBSZ 0
200 PBSZ=0
PROT P
200 PRIVATE data channel protection level set
PWD
257 "/" is current directory
Current directory : /
HELP
214-The following commands are implemented
ABOR ACCT ALLO* APPE CDUP CWD DELE FEAT+ HELP
HOST+ LANG+ LIST MDTM+ MLST+ MKD MODE NLST NOOP
OPTS+ PASS PASV PORT PWD QUIT REIN REST RETR
RMD RNFR RNTO SITE SIZE SMNT STAT STOR STOU
STRU* SYST TYPE USER XCUP XCRC XCWD XMD5 XMKD
XPWD XRMD XSIGN XSHA1 XSHA256 XSHA512 XQUOTA
214 Help complete
214-The following commands are implemented
ABOR ACCT ALLO* APPE CDUP CWD DELE FEAT+ HELP
HOST+ LANG+ LIST MDTM+ MLST+ MKD MODE NLST NOOP
OPTS+ PASS PASV PORT PWD QUIT REIN REST RETR
RMD RNFR RNTO SITE SIZE SMNT STAT STOR STOU
STRU* SYST TYPE USER XCUP XCRC XCWD XMD5 XMKD
XPWD XRMD XSIGN XSHA1 XSHA256 XSHA512 XQUOTA
214 Help complete
PASV
227 Entering Passive Mode (192,168,4,122,236,242).
[Replacing PASV mode reply address 192.168.4.122 with 83.206.162.234]
MLSD
150 Transferring directory
MLSD 命令从不回复。
也许问题在于 ssl 会话 ID 重用？这似乎是 apache-commons-net 的一个已知问题，参见https://www.javaer101.com/en/article/970693.html
我问过 FTP 管理员，这是否在他这边被激活。他说“是”，这是一个“ipswitch ftp 服务器”。我们计划在 11 号星期一进行一次调试会话。
我还尝试了 Martin Prikryl 向我指出的解决方法，对方法进行子类FTPSClient化和重载prepareDataSocket，但它不起作用。传递给该方法的套接字不是 SSLSocket，因此没有任何反应（第一次测试）。
编码：
public class CompliantFTPSClient extends FTPSClient {
public CompliantFTPSClient(String protocol, boolean isImplicit) {
super(protocol, isImplicit);
}
@Override
protected void _prepareDataSocket_(Socket socket) throws IOException {
if (socket instanceof SSLSocket) {
// Control socket is SSL
final SSLSession session = ((SSLSocket) _socket_).getSession();
if (session.isValid()) {
final SSLSessionContext context = session.getSessionContext();
context.setSessionCacheSize(0);
try {
final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
method.setAccessible(true);
method.invoke(cache, String
.format("%s:%s", socket.getInetAddress().getHostName(), String.valueOf(socket.getPort()))
.toLowerCase(Locale.ROOT), session);
method.invoke(cache, String
.format("%s:%s", socket.getInetAddress().getHostAddress(), String.valueOf(socket.getPort()))
.toLowerCase(Locale.ROOT), session);
} catch (NoSuchFieldException e) {
System.err.println("WARN : No field sessionHostPortCache in SSLSessionContext");
e.printStackTrace(System.err);
} catch (Exception e) {
System.err.println("WARN : " + e.getMessage());
}
} else {
System.err.println(String.format("SSL session %s for socket %s is not rejoinable", session, socket));
}
}
}
}
我正在尝试做的事情：只需递归浏览服务器上的目录、列出文件并下载一些文件，这些事情通常我可以在 SFTP 或 SCP 中几分钟内完成。但是在带有这个 API 的 FTPS 中，这只是痛苦的。
解决方案
您通过向PROT P服务器发送命令导致数据连接加密不匹配。
由于您仅使用 将命令发送到服务器FTP.sendCommand，因此该命令仅影响服务器。但不是客户端（Apache Common FTPClient）。因此，当开始数据连接时，服务器等待 TLS/SSL 交换，但客户端没有响应，因为它只期望纯数据。这就是连接挂起的原因。
你必须打电话FTPClient.execPROT("P")。它在内部调用FTP.sendCommand("PROT", "P")，但它也在客户端启用加密。
同样，您应该使用FTPSClient.execPBSZ而不是发送 raw PBSZ 0。尽管目前，除了FTPSClient.execPBSZ调用之外，它并没有做任何事情FTP.sendCommand("PBSZ", ...)（除了检查结果）。
WinSCP 还重用会话 ID。如果您的服务器需要，请参阅：
如何使用相同的 TLS 会话通过数据连接连接到 FTPS 服务器？
推荐阅读
- javascript - Banana sprite js 使用 json 数据定制前进和后退动画
- python - 如何根据多个条件填写 Nan？
- php - 从 php Mysqli_query 运行自定义 Mysql 函数返回未定义索引错误
- python - 将 GridSearchCV 结果传递给 Imbalanced-Learn 的 Pipeline 对象
- ruby - 为什么我在 ruby ruport 中出现“未初始化的常量”错误
- python - 使用 python yum API 进行通配符搜索
- javascript - 如何使用 react 和 node.js 在网站上制作动态内容
- laravel - 将变量从 Controller 传递到 View Laravel 5.8 时出错
- python - 虽然循环运行太多次？
- javascript - 为什么排序/过滤不适用于此实时数据库查询？