python - python-请求特殊字符编码

Question

我有以下 curl 命令，它给了我 200 状态。

curl 'https://www.upwork.com/ab/proposals/api/v2/application/new' -H 'sec-fetch-mode: cors' -H 'origin: https://www.upwork.com' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'x-requested-with: XMLHttpRequest' -H 'cookie: whatevercookie' -H 'pragma: no-cache' -H 'x-odesk-user-agent: oDesk LM' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36' -H 'content-type: application/json;charset=UTF-8' -H 'accept: application/json, text/plain, */*' -H 'cache-control: no-cache' -H 'authority: www.upwork.com' -H 'referer: https://www.upwork.com/ab/proposals/job/~01bcda049b4a63f946/apply/' -H 'sec-fetch-site: same-origin' -H 'x-odesk-csrf-token: 367665096e96498719bbb6f93173f40d' -H 'dnt: 1' --data-binary $'{"jobReference":"1169738129700466688","chargedAmount":45,"earnedAmount":null,"coverLetter":"Greetings\u0021\\n\\nAfter reading your stated job requirements,I can definitely help you with scraping data from multiple sources.It would be really great if you can share more detailed requirement in your next message so as to get the scope more clear.\\n\\nI am a COMPUTER SCIENCE graduate with over 13+ YEARS of professional experience in Web development. I have solid programming skills in PHP, MySQL, JavaScript, HTML5, CSS3, JQuery and WordPress. I\'ve depth experience with Responsive web application development, API integration, database development, modern UI & UX design as well as ongoing maintenance, troubleshooting, enhancement.\\n\\nI have developed a wide range of web applications earlier with high level of my client\'s SATISFACTION. Hence this project most appealing to me and based on my experience, So I am willing to do this job\\n\\nIt would be really appreciated if you provide me the Detailed specification document, Design Files, etc of the project. So, I can review it and provide you my BEST SUGGESTION along with my EXACT ESTIMATION for time and quote ASAP.\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nFor your reference, please find below few links that demonstrate my work:\\n\\n=> Web Projects : \\n=============\\nhttps://www.sleeprate.com/ [SleepRate]\\nhttps://www.timecamp.com/ [Business]\\nhttp://www.indochino.com/ [Shopping]\\nhttp://www.liquor.com [Community, Blog]\\nhttp://www.creativeadawards.com [Community, Advertise]\\nhttp://www.atfirstsightfilms.com/ [Photography]\\n\\nIn addition, I have developed ample number of websites for my clients which are equally unique and good but because of NDA with them I cannot disclose all such.\\n\\nPlease find below some of the technologies and frameworks that lie down my expertise belt.\\n\\n-> JavaScript, JQuery, AJAX, NodeJS. AngularJs, ReactJS, Python\\n-> HTML5, CSS3, Bootstrap, SASS\\n-> MySQL, MongoDB.\\n-> Adobe Photoshop, Illustrator.\\n-> REST, SOAP, JSON, XML.\\n\\nIf you feel I\'m an appropriate fit to get this job done for you then please get in touch with me. \\nI can fluently speak in English and I have extensively worked with international clients. I am available during your time zone for project discussion. I can communicate with you via  and E-Mail or any-other Medium as per your preference and would be happy to set up a convenient time to discuss more about the Project\\n\\nAwaiting an affirmative response from your end.\\n\\nThanks & Regards.\\n\nCould we schedule  MEETING for quick conversation of your project?","questions":[],"connectsForApply":{"connects":138,"canBuyConnects":true,"canApply":true,"jobsPrice":6,"connectsV2Applicable":true},"estimatedDuration":null,"selectedContractor":null,"agency":null,"attachment":null,"attachments":[],"readyToStartDate":null,"occupationUID":null}' --compressed

我将相同的 curl 命令转换为 python-requests。

import requests

headers = {
    'sec-fetch-mode': 'cors',
    'origin': 'https://www.upwork.com',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-US,en;q=0.9',
    'x-requested-with': 'XMLHttpRequest',
    'cookie': 'test',
    'pragma': 'no-cache',
    'x-odesk-user-agent': 'oDesk LM',
    'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36',
    'content-type': 'application/json;charset=UTF-8',
    'accept': 'application/json, text/plain, */*',
    'cache-control': 'no-cache',
    'authority': 'www.upwork.com',
    'referer': 'https://www.upwork.com/ab/proposals/job/~01bcda049b4a63f946/apply/',
    'sec-fetch-site': 'same-origin',
    'x-odesk-csrf-token': '367665096e96498719bbb6f93173f40d',
    'dnt': '1',
}

data = '${"jobReference":"1169738129700466688","chargedAmount":45,"earnedAmount":null,"coverLetter":"Greetings\\u0021\\\\n\\\\nAfter reading your stated job requirements,I can definitely help you with scraping data from multiple sources.It would be really great if you can share more detailed requirement in your next message so as to get the scope more clear.\\\\n\\\\nI am a COMPUTER SCIENCE graduate with over 13+ YEARS of professional experience in Web development. I have solid programming skills in PHP, MySQL, JavaScript, HTML5, CSS3, JQuery and WordPress. I\'ve depth experience with Responsive web application development, API integration, database development, modern UI & UX design as well as ongoing maintenance, troubleshooting, enhancement.\\\\n\\\\nI have developed a wide range of web applications earlier with high level of my client\'s SATISFACTION. Hence this project most appealing to me and based on my experience, So I am willing to do this job\\\\n\\\\nIt would be really appreciated if you provide me the Detailed specification document, Design Files, etc of the project. So, I can review it and provide you my BEST SUGGESTION along with my EXACT ESTIMATION for time and quote ASAP.\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\nFor your reference, please find below few links that demonstrate my work:\\\\n\\\\n=> Web Projects : \\\\n=============\\\\nhttps://www.sleeprate.com/ [SleepRate]\\\\nhttps://www.timecamp.com/ [Business]\\\\nhttp://www.indochino.com/ [Shopping]\\\\nhttp://www.liquor.com [Community, Blog]\\\\nhttp://www.creativeadawards.com [Community, Advertise]\\\\nhttp://www.atfirstsightfilms.com/ [Photography]\\\\n\\\\nIn addition, I have developed ample number of websites for my clients which are equally unique and good but because of NDA with them I cannot disclose all such.\\\\n\\\\nPlease find below some of the technologies and frameworks that lie down my expertise belt.\\\\n\\\\n-> JavaScript, JQuery, AJAX, NodeJS. AngularJs, ReactJS, Python\\\\n-> HTML5, CSS3, Bootstrap, SASS\\\\n-> MySQL, MongoDB.\\\\n-> Adobe Photoshop, Illustrator.\\\\n-> REST, SOAP, JSON, XML.\\\\n\\\\nIf you feel I\'m an appropriate fit to get this job done for you then please get in touch with me. \\\\nI can fluently speak in English and I have extensively worked with international clients. I am available during your time zone for project discussion. I can communicate with you via  and E-Mail or any-other Medium as per your preference and would be happy to set up a convenient time to discuss more about the Project\\\\n\\\\nAwaiting an affirmative response from your end.\\\\n\\\\nThanks & Regards.\\\\n\\\\nBiplov\\\\n\\\\nCould we schedule  MEETING for quick conversation of your project?","questions":[],"connectsForApply":{"connects":138,"canBuyConnects":true,"canApply":true,"jobsPrice":6,"connectsV2Applicable":true},"estimatedDuration":null,"selectedContractor":null,"agency":null,"attachment":null,"attachments":[],"readyToStartDate":null,"occupationUID":null}'

response = requests.post('https://www.upwork.com/ab/proposals/api/v2/application/new', headers=headers, data=data)

这给了我 500 的回报，但是如果我将有效负载封面更改为简单的字符串，如“测试”，它会给我 200 的回报。我认为这与包含特殊字符或太长的coverLetter有关，但对于为什么它在curl中工作但在python请求中失败并没有意义。有任何想法吗？

标签： pythonpython-requests

Answer 1

我建议用data以下字典替换您当前的字符串：

data = {"jobReference":"1169738129700466688","chargedAmount":45,"earnedAmount":"null","coverLetter":"Greetings\\u0021\\\\n\\\\nAfter reading your stated job requirements,I can definitely help you with scraping data from multiple sources.It would be really great if you can share more detailed requirement in your next message so as to get the scope more clear.\\\\n\\\\nI am a COMPUTER SCIENCE graduate with over 13+ YEARS of professional experience in Web development. I have solid programming skills in PHP, MySQL, JavaScript, HTML5, CSS3, JQuery and WordPress. I\'ve depth experience with Responsive web application development, API integration, database development, modern UI & UX design as well as ongoing maintenance, troubleshooting, enhancement.\\\\n\\\\nI have developed a wide range of web applications earlier with high level of my client\'s SATISFACTION. Hence this project most appealing to me and based on my experience, So I am willing to do this job\\\\n\\\\nIt would be really appreciated if you provide me the Detailed specification document, Design Files, etc of the project. So, I can review it and provide you my BEST SUGGESTION along with my EXACT ESTIMATION for time and quote ASAP.\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\nFor your reference, please find below few links that demonstrate my work:\\\\n\\\\n=> Web Projects : \\\\n=============\\\\nhttps://www.sleeprate.com/ [SleepRate]\\\\nhttps://www.timecamp.com/ [Business]\\\\nhttp://www.indochino.com/ [Shopping]\\\\nhttp://www.liquor.com [Community, Blog]\\\\nhttp://www.creativeadawards.com [Community, Advertise]\\\\nhttp://www.atfirstsightfilms.com/ [Photography]\\\\n\\\\nIn addition, I have developed ample number of websites for my clients which are equally unique and good but because of NDA with them I cannot disclose all such.\\\\n\\\\nPlease find below some of the technologies and frameworks that lie down my expertise belt.\\\\n\\\\n-> JavaScript, JQuery, AJAX, NodeJS. AngularJs, ReactJS, Python\\\\n-> HTML5, CSS3, Bootstrap, SASS\\\\n-> MySQL, MongoDB.\\\\n-> Adobe Photoshop, Illustrator.\\\\n-> REST, SOAP, JSON, XML.\\\\n\\\\nIf you feel I\'m an appropriate fit to get this job done for you then please get in touch with me. \\\\nI can fluently speak in English and I have extensively worked with international clients. I am available during your time zone for project discussion. I can communicate with you via  and E-Mail or any-other Medium as per your preference and would be happy to set up a convenient time to discuss more about the Project\\\\n\\\\nAwaiting an affirmative response from your end.\\\\n\\\\nThanks & Regards.\\\\n\\\\nBiplov\\\\n\\\\nCould we schedule  MEETING for quick conversation of your project?","questions":[],"connectsForApply":{"connects":138,"canBuyConnects":"true","canApply":"true","jobsPrice":6,"connectsV2Applicable":"true"},"estimatedDuration":"null","selectedContractor":"null","agency":"null","attachment":"null","attachments":[],"readyToStartDate":"null","occupationUID":"null"}

也就是说，您还应该替换这一行：

response = requests.post('https://www.upwork.com/ab/proposals/api/v2/application/new', headers=headers, data=data)

有了这个：

response = requests.post('https://www.upwork.com/ab/proposals/api/v2/application/new', headers=headers, json=data)

希望这可以帮助