c# - HttpClient 无法访问登录页面后面的页面
问题描述
[编辑-从提琴手添加分析,添加更多代码以复制身份验证标头] [编辑-现在使用 FormUrlEncodedContent]
我在这里有一个页面:https ://www.cdc.co.nz/products/list.html?cat=5201 ,通过此处登录受密码保护:https ://www.cdc.co.nz/login/
下面的代码允许我成功登录。但是,尽管使用相同的客户端,但我无法调用上述页面(401 Unauthorized)
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
var baseAddress = new Uri("https://www.cdc.co.nz");
var cookieContainer = new CookieContainer();
using (var handler = new HttpClientHandler() { CookieContainer = cookieContainer, UseCookies = true })
using (HttpClient client = new HttpClient(handler) { BaseAddress = baseAddress })
{
HttpResponseMessage response = null;
//Let's visit the homepage to set initial cookie values
Task.Run(async () => response = await client.GetAsync("/")).GetAwaiter().GetResult(); //200
string urlToPost = "/login/";
var postData = new List<KeyValuePair<string, string>>();
postData.Add(new KeyValuePair<string, string>("username", "username"));
postData.Add(new KeyValuePair<string, string>("password", "password"));
HttpContent stringContent = new FormUrlEncodedContent(postData);
client.DefaultRequestHeaders.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
client.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate, br");
client.DefaultRequestHeaders.Add("Accept-Language", "en-GB,en-US;q=0.9,en;q=0.8");
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36");
client.DefaultRequestHeaders.Add("Origin", "https://www.cdc.co.nz");
client.DefaultRequestHeaders.Add("Upgrade-Insecure-Requests", "1");
client.DefaultRequestHeaders.Add("Connection", "keep-alive");
client.DefaultRequestHeaders.Add("Host", "www.cdc.co.nz");
client.DefaultRequestHeaders.Add("Referer", "https://www.cdc.co.nz/login/");
cookieContainer.Add(baseAddress, new Cookie("_ga", "GA1.3.720299450.1533761418"));
cookieContainer.Add(baseAddress, new Cookie("_gat_oldTracker", "1"));
cookieContainer.Add(baseAddress, new Cookie("_gat", "1"));
cookieContainer.Add(baseAddress, new Cookie("_gid", "GA1.3.1011102476.1533761418"));
//Tyler's suggestion here works!
//cookieContainer.Add(baseAddress, new Cookie("PHPSESSID", "value from browser login response header"));
//Receiving 200 response for the nextline, though it returns a 302 in a browser environment
Task.Run(async () => response = await client.PostAsync(urlToPost, stringContent)).GetAwaiter().GetResult();
//401 response for the next line
Task.Run(async () => response = await client.GetAsync("/products/list.html?cat=5201")).GetAwaiter().GetResult();
}
浏览器环境的提琴手:结果:302 协议:HTTPS 主机:www.cdc.co.nz URL:/login/
原始请求标头浏览器环境:
POST /login/ HTTP/1.1
Host: www.cdc.co.nz
Connection: keep-alive
Content-Length: 69
Cache-Control: max-age=0
Origin: https://www.cdc.co.nz
Upgrade-Insecure-Requests: 1
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Referer: https://www.cdc.co.nz/login/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: _ga=GA1.3.720299450.1533761418; _gid=GA1.3.1011102476.1533761418; PHPSESSID=p3jn5qqhcul59blum597mp2o41; _gat=1; _gat_oldTracker=1
浏览器环境中的响应原始标头(Set-Cookie:PHPSESSID=oh7in7n5pjbkrkng4qwwwn22uaq951 是我感兴趣的):
HTTP/1.1 302 Found
Date: Thu, 09 Aug 2018 00:51:11 GMT
Server: Apache/2.4.7 (Ubuntu)
X-Powered-By: PHP/5.5.9-1ubuntu4.25
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=oh7in7n5pjbkrkng4qwwwn22uaq951 <-------- Needed in subsequent Request headers to not 401.
Location: https://www.cdc.co.nz/home/news.html
Content-Length: 0
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html
HttpClient 的提琴手:结果:200 协议:HTTPS 主机:www.cdc.co.nz URL:/login/
HttpClient 环境中的原始标头:
GET /login/ HTTP/1.1
Host: www.cdc.co.nz
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Referer: https://www.cdc.co.nz/home/my-account/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: _ga=GA1.3.720299450.1533761418; _gid=GA1.3.1011102476.1533761418; _gat=1; _gat_oldTracker=1; PHPSESSID=sdjm7r2jge751jo39mkesqnfl6
HttpClient 环境中的原始响应标头(注意这里没有 Set-Cookie 标头/值吗?):
HTTP/1.1 200 OK
Date: Thu, 09 Aug 2018 01:11:14 GMT
Server: Apache/2.4.7 (Ubuntu)
X-Powered-By: PHP/5.5.9-1ubuntu4.25
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8
Content-Length: 5668
回答
添加额外的 KV 对(甚至没有指定其他不必要的细节)现在可以使代码正常工作:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
var baseAddress = new Uri("https://www.cdc.co.nz");
using (HttpClient client = new HttpClient() { BaseAddress = baseAddress })
{
HttpResponseMessage response = null;
//Let's visit the homepage to set initial cookie values
Task.Run(async () => response = await client.GetAsync("/")).GetAwaiter().GetResult(); //200
string urlToPost = "/login/";
var postData = new List<KeyValuePair<string, string>>();
postData.Add(new KeyValuePair<string, string>("username", "username"));
postData.Add(new KeyValuePair<string, string>("password", "password"));
postData.Add(new KeyValuePair<string, string>("returnUrl", "/login/")); <----- To simulate the browser
postData.Add(new KeyValuePair<string, string>("service", "login")); <----- To simulate the browser
HttpContent stringContent = new FormUrlEncodedContent(postData);
//Receiving 200 response for the nextline, though it returns a 302 in a browser environment
Task.Run(async () => response = await client.PostAsync(urlToPost, stringContent)).GetAwaiter().GetResult();
//200 response now
Task.Run(async () => response = await client.GetAsync("/products/list.html?cat=5201")).GetAwaiter().GetResult();
}
解决方案
尝试像浏览器一样添加隐藏的表单值
var postData = new List<KeyValuePair<string, string>>();
postData.Add(new KeyValuePair<string, string>("username", "username"));
postData.Add(new KeyValuePair<string, string>("password", "password"));
postData.Add(new KeyValuePair<string, string>("returnUrl", "/login/"));
postData.Add(new KeyValuePair<string, string>("service", "login"));
推荐阅读
- sql-server - 从另一个数据库中的存储过程调用链接服务器上的视图
- clearcase - 是否存在使用“rmname”而不是“rmelem”作为其调用一部分的 rmver 版本?
- python - 有没有人能够让子弹在 Colab 中工作?
- swift - Swift 不允许 if 语句,因为它无法识别 if 语句中定义的变量
- google-cloud-platform - Google Dataprep/Trifacta - 加入三个数据集,重复数据删除但维护不匹配的记录
- r - Tidyverse - rsample。nested_cv + group_vfold_cv = 错误。插入符号上的嵌套交叉验证(trainControl)
- vb.net - 在 Linq 中排序,其中一些日期是炒作
- python - 重命名实例而不重复
- windows - Server 2012 MS 更新导致引导循环 - 如何停止或诊断?
- ios - 将数据从 AppDelegate 传递到 ViewController