python - 通过 Python Requests 和 Postman 重新创建 XMLHttpRequest 的问题
问题描述
我正在尝试使用 Python 或 Postman 缩短 bit.do 上的链接。
在 Chrome 中一切正常。但不适用于 Python/Postman。我得到了该页面,但只有错误,但是来自 Chrome 的请求和我尝试制作的请求看起来相同。使用 Chrome 开发工具捕获两个 POST 请求,均名为url-shortener.pl
. 以下是这些请求(所以强迫我不要使用 URL 缩短器中的链接,所以我不得不转义它们。):
General:
Request URL: https://bit\.do/mod_perl/url-shortener.pl
Request Method: POST
Status Code: 200
Remote Address: 54.83.52.76:443
Referrer Policy: no-referrer-when-downgrade
Response Headers:
content-type: application/json
date: Sat, 20 Apr 2019 20:12:06 GMT
server: nginx/1.14.1
status: 200
Request Headers:
:authority: bit\.do
:method: POST
:path: /mod_perl/url-shortener.pl
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9,ru-RU;q=0.8,ru;q=0.7
content-length: 112
content-type: application/x-www-form-urlencoded; charset=UTF-8
cookie: permasession=1554914974|phkaoymp1b; __utmc=60667454; __utma=60667454.372171702.1554914974.1555785612.1555789898.5; __utmz=60667454.1555789898.5.4.utmcsr=dynomapper.com|utmccn=(referral)|utmcmd=referral|utmcct=/blog/21-sitemaps-and-seo/495-top-14-url-shorteners; __utmt=1; __utmb=60667454.3.10.1555789898
origin: https://bit\.do
referer: https://bit\.do/
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
x-requested-with: XMLHttpRequest
Form Data:
action: shorten
url: google.com
url2: site2
url_hash:
url_stats_is_private: 0
permasession: 1554914974|phkaoymp1b
General:
Request URL: https://bit\.do/mod_perl/url-shortener.pl
Request Method: POST
Status Code: 200
Response Headers:
Remote Address: 54.83.52.76:443
Referrer Policy: no-referrer-when-downgrade
content-type: application/json
date: Sat, 20 Apr 2019 20:12:06 GMT
server: nginx/1.14.1
status: 200
Request Headers:
:authority: bit\.do
:method: POST
:path: /mod_perl/url-shortener.pl
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9,ru-RU;q=0.8,ru;q=0.7
content-length: 32
content-type: application/x-www-form-urlencoded; charset=UTF-8
cookie: permasession=1554914974|phkaoymp1b; __utmc=60667454; __utma=60667454.372171702.1554914974.1555785612.1555789898.5; __utmz=60667454.1555789898.5.4.utmcsr=dynomapper.com|utmccn=(referral)|utmcmd=referral|utmcct=/blog/21-sitemaps-and-seo/495-top-14-url-shorteners; __utmt=1; __utmb=60667454.3.10.1555789898
origin: https://bit\.do
referer: https://bit\.do/
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
x-requested-with: XMLHttpRequest
Form Data:
action: get_title
url_id: 49444432
首先,我尝试使用 Postman。参数: 标题: 响应预览: 结果:错误:无效的 url。请输入有效的 url 地址。然后我切换到 Python 并尝试使用 Requests。
def bitdo():
headers = {
'accept': '*/*',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'en-US,en;q=0.9,ru-RU;q=0.8,ru;q=0.7',
'content-length': '112',
'content-type': 'application/x-www-form-urlencoded; charset=UTF-8',
'cookie': 'permasession=1554914974|phkaoymp1b; __utmc=60667454; __utma=60667454.372171702.1554914974.1555785612.1555789898.5; __utmz=60667454.1555789898.5.4.utmcsr=dynomapper.com|utmccn=(referral)|utmcmd=referral|utmcct=/blog/21-sitemaps-and-seo/495-top-14-url-shorteners; __utmt=1; __utmb=60667454.3.10.1555789898',
'origin': 'https://bit\.do',
'referer': 'https://bit\.do/',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36'
}
params = {
'action': 'shorten',
'url': 'ya.ru',
'url2': 'site2',
'url_hash': '',
'url_stats_is_private': 0,
'permasessions': '1554914974|phkaoymp1b'
}
r = requests.post('https://bit\.do/mod_perl/url-shortener.pl', params, headers=headers)
print(r.text)
回应是:
<title>Warning</title>
<!-- head.thtml -->
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<style type="text/css">
a { color: #338; text-decoration: none; }
html {
height: 100%;
}
body {
font-family: Arial;
margin: 0;
height: 100%;
color: #404040;
}
.url {
background-color: #f9f9f9;
width: 100%;
height: 16px;
border: 1px solid #654;
padding: 3px;
}
.round {
background-color: white;
border: 1px solid #bbb;
margin-bottom: 10px;
-moz-border-radius: 15px;
-webkit-border-radius: 15px;
border-radius: 15px;
width: 100%;
max-width: 840px;
}
.input-placeholder {
position: absolute;
font-style: italic;
color: #aaa;
margin: 0.3em 0 0 0.5em;
}
.orange_logo {
color: #fab035;
}
.orange_text {
color: #D04000;
}
.orange_dot {
color: #ff8800;
margin-left: 10px;
}
.table-striped > tbody > tr:nth-child(odd) {
background-color: #efefef;
}
.table1 {
border-spacing: 0;
border-collapse: collapse;
}
.table1 th {
text-align: center;
}
.table1 td, .table1 th {
border: 1px solid #a0a0a0;
padding: 5px;
}
.bitbox {
width: 100%;
}
/* first td should have minimal width */
.bitbox td:first-child {
width: 12px;
padding-right: 12px;
white-space: nowrap;
}
</style>
<body>
<div style="
height: 30px;
line-height: 30px;
background-color: #fab035;
xtext-align: right;
color: black;
padding-right: 10px;
font-size: 0.8em;
">
<a href="http://bit.do/"><img src="/images/bit-do-url-shortener-logo-66x66.png"
alt="URL Shortener - bit.do"
style="
width: 31px;
height: 31px;
margin-left: 80px;
margin-right: 10px;
vertical-align: middle;
"
></a>
<!--
Create short link:
<input placeholder="http://...">
-->
<!--
<b>
<a href="http://bit.do/" style="color: black">bit.do - URL Shortener</a>
</b>
-->
<!--
<a href="/admin" style="color: black">Login to manage your links</a>
-->
<table border=0 cellpadding=0 cellspacing=0 width=100%>
<tr>
<td class=top_left> </td>
<td class=top_middle><b>Warning</b></td>
<td class=top_right> </td>
</tr>
<tr>
<td class=middle_left></td>
<td class=middle_middle>
<ul>
<br>
<pre class="warning_message">ERROR site2: Can not create short link. Contact us for API usage.
</pre>
<p>
<br>
<br>
<br>
<br>
<br>
</ul>
<p>
<a href="http://bit.do/" class="button">↩ Back to bit.do (url shortener)</a>
<p>
</td>
</tr>
</table>
<!-- /table all -->
<p style="float: right; text-align: right; font-size: 0.8em; color: #808080; margin: 0; margin-right: 10px;">
Follow us on Twitter: <b><a href="https://twitter.com/bitdo" target=_blank class="orange_logo" style="border: 0;">@bitdo</a> </b>
</p>
<br style="clear: both;">
<hr style="
border: 0px;
height: 1px;
background-color: #e0e0e0;
xbackground-color: #fab035;
">
<div style="
margin: 0 auto;
text-align: center;
xbackground-color: #c8c8c8;
bottom: 0px;
"
>
<span style="font-size: 0.7em; line-height: 35px;">
Shorten and personalize long web addresses. Get real-time traffic statistics for your links. Free service.<br>
</span>
<span style="font-size: 15px; font-family: arial;">
<a href="http://bit.do/">bit.do - home</a>
<span style="color:#bbbbbb">|</span>
<a href="/best-url-shortener.php">why bit.do is better</a>
<span style="color:#bbbbbb">|</span>
<a href="/about-us.php">about us</a>
<span style="color:#bbbbbb">|</span>
<a href="/what-is-url-shortener.php">about url shortener</a>
<span style="color:#bbbbbb">|</span>
<a href="/contact.php">contact</a>
<!--
<a href="">FAQ</a> |
<a href="">terms</a> |
-->
</span>
<p style="margin: 7px; color: #909090; font-size: 0.5em;">
Copyright © 2019 - Insite</p>
</div> <!-- /foot -->
</div>
<!-- /height 100% -->
<!-- TODO: ALREADY LOADED ? -->
<script src="/js/jquery/jquery.min.js"></script>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-756399-13']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</html>
<!-- END OF erro.thtml -->
错误site2:无法创建短链接。联系我们了解 API 使用情况。
我希望在 HTML 响应中获得缩短的链接。有人可以解释我所做的请求有什么问题吗?
解决方案
它似乎是最小的工作代码。
它需要标头'X-Requested-With'
,因为它是 AXAJ/XHR 请求。
它需要permasession
但首先GET
不发送它,因此它可能是使用 JavaScript 在页面上生成的。但它一直对我permasession
有用。
也许以后它会需要新的/新鲜的permasession
里面有空格" site2 "
import requests
headers={
'X-Requested-With': 'XMLHttpRequest', # need it
}
data = {
'action': 'shorten',
'url': 'https://onet.pl',
'url2': ' site2 ', # need spaces
'url_hash': None,
'url_stats_is_private': 0,
'permasession': '1555801674|ole2ky65f9', # need it
}
r = requests.post('http://bit\.do/mod_perl/url-shortener.pl', headers=headers, data=data)
print(r.status_code)
print(r.json())
它在开始时不需要requests.Session()
也不需要请求。User-Agent
GET
编辑:值1555801674
是'permasession': '1555801674|ole2ky65f9'
当前日期和时间的时间戳。
import datetime
datetime.datetime.fromtimestamp(1555801674)
datetime.datetime(2019, 4, 21, 1, 7, 54)
也许ole2ky65f9
也是时间戳,但作为缩短值。
推荐阅读
- jmeter - JMETER 登录测试因会话而失败
- git - 远程:对 XXX/.git 的权限拒绝 YYY。致命:无法访问“https://github.com/XXX//”:请求的 URL 返回错误:403
- database - 如何读取数据库的某些行,对某些列求和并存储在集合中?
- javascript - 在 xslt 中将日期值转换为字符串日期格式
- angular - 为什么我在这个 HttpHeader 上出现 Angular 错误?
- html - about:blank#blocked on Link (Markdown Page)
- javascript - 单击图像图标时,请参阅 REACT.js 中的输入类型文件功能
- python-3.x - Pandas:智能创建新列
- android - 为什么我使用 SmallWidth 不适用于支持 1080x2244 像素的设备?
- javascript - 存储为 .env 变量与系统变量时字符串值的差异