首页 > 解决方案 > 是否可以在 Selenium 和 LWP::UserAgent 之间共享会话?

问题描述

我正在尝试从需要登录过程的网站下载文件。此过程可以手动完成,如下所示:

  1. 打开浏览器并登录页面
  2. 如果您有 url,只需输入它,您将访问该文件。

我的问题比那个更复杂,我必须做一些处理才能用 selenium 获取那个 url。但我无法使用常用方法下载文件:

  1. 获取 url(我得到一个无法用 try/catch 逃脱的呱呱叫)。
  2. 生成 javascript 元素并单击它会在网站中生成异常。

所以我的新方法是:

  1. 用硒做登录的东西。
  2. 获取在请求中完成的 cookie 和标头。
  3. 用硒提出请求。

我就是这样做的:

use Selenium::Firefox;
use strict;
use warnings;
use utf8;
use Selenium::Waiter qw/wait_until/;

use LWP::UserAgent;
use File::Temp;
use HTTP::Cookies;
use Data::Dumper qw/Dumper/;


my $driver = Selenium::Firefox->new(marionette_enabled => 1 );
$driver->set_implicit_wait_timeout(3000);

$driver->get('https://example.com');

# processing and login

my $uri = 'https://example.com/myfile.tcx';

print "Downloading the following file: $url\n";

my $cookies = $driver->get_all_cookies();

# Create temporary file to download
my $tmp = File::Temp->new( TEMPLATE => 'myTemplateXXXX', SUFFIX=>'.tcx', UNLINK=>0 );

my $cookie_jar_obj = HTTP::Cookies->new(ignore_discard=>1,hide_cookie2=>1);
my $ua = LWP::UserAgent->new();
$ua->cookie_jar( $cookie_jar_obj );

foreach my $cookie_ref (@{$cookies}){
  my %cookie = %{$cookie_ref};
  my $name = $cookie{'name'};# - STRING
  my $value = $cookie{'value'};# - STRING
  my $path = $cookie{'path'};# - STRING
  my $domain = $cookie{'domain'};# - STRING
  my $secure = $cookie{'secure'};# - BOOLEAN

  print "Name: $name ";
  print "Value: $value ";
  print "Path: $path ";
  print "Domain: $domain ";
  print "Secure: $secure \n";

  $cookie_jar_obj->set_cookie(0, $name, $value
                             , $path, $domain, 443
                             , '', $secure, 14400000, 0
                    );
}



print "Cookies to be sent:\n";
print Dumper($cookie_jar_obj->get_cookies( ".myhost.com" ));

$ua->default_header('Accept' => "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
$ua->default_header('Accept-Encoding' => "gzip, deflate, br");
$ua->default_header('Accept-Language' => "en-US,en;q=0.5");
$ua->default_header('Connection' => "keep-alive");
$ua->default_header('Host' => "www.example.com");
$ua->default_header('Upgrade-Insecure-Requests' => "1");
$ua->default_header('User-Agent' => "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:64.0) Gecko/20100101 Firefox/64.0");

$ua->show_progress( 1 );

my $response = $ua->get($uri, ':content_file' => $tmp->filename);

print $response->status_line,"\n" if !$response->is_success;
print $tmp->filename . "\n";

<>;
$driver->get('https://example.com');

但我仍然得到:

401未经授权(9s)

标签: perlseleniumcookieslwp-useragent

解决方案


推荐阅读