时间戳

首页 > 解决方案 > 使用 Selenium 获取 HTML 的某些元素

python - 使用 Selenium 获取 HTML 的某些元素

问题描述

我使用 selenium 的原因是，我导航到填写几个表格的 URL，然后进入一个页面，该页面为我提供了基于某些信息计算的保险价格的概述。

因此，我还需要继续使用 Selenium 来抓取网站。

<div class="list-item-pricing">
    <div class="table">
        <div class="tableCell">
            <div class="main-price" basicprice="3459">
                <h2 class="nmt nmb"><small>kr./år</small> <span ng-show="insuranceMatch.basicprice < 999999" class="ng-binding">3,459​&lt;/span><span ng-show="insuranceMatch.basicprice >= 999999" class="ng-hide">-</span></h2>
                <div class="discount-price" discountedprice="2958">
                    <h4 class="nmt nmb ng-binding"><small>Med samlerabat kr./år</small> 2,958​&lt;/h4>
                </div>
                <div class="deductible-price" selvrisiko="6324">
                    <h4 class="nmt nmb ng-binding"><small>Selvrisiko</small> 6,324​&lt;/h4>
                </div>
            </div>
        </div>
    </div>
</div>

这是我的代码：

resultatside = driver.get("https://forsikringsguiden.dk/#!/bilforsikring/resultatside")
def get_car_insurance(url):          
    wait = WebDriverWait(driver,10)
    wait
    res_element = driver.find_elements_by_xpath('//*[@id="listInsurances"]/div[10]/div/div/div[4]/div/div/div')

    res = res_element.get_attribute("ng-binding")
    #res = [x.text for x  in res_element]

    return res        

print(get_car_insurance(resultatside))

在我的代码中，我尝试获得第一个基本价格，但没有任何运气。我不能在这里使用 BS4。

我希望输出为： 基本价格：3,459 折扣价：2,958 selvrisiko：6,324

编辑：

如果需要，这里是一行元素的整个 HTML。我有17个。

<div ng-repeat="insuranceMatch in insuranceMatches" isotope-item="" class="kindling ng-scope has-no-membership isotope-item" ng-class="(insuranceMatch.hasmemberrequirements) ? 'has-membership' : 'has-no-membership'" style="position: absolute; left: 0px; top: 0px; transform: translate3d(0px, 0px, 0px);">
    <div class="list-item enlarge-main">
        <div class="panel default small">
            <!--<div class="loading-indicator" ng-show="insuranceMatch.isLoading">
                <img src="app/img/spinner.gif" />
            </div>-->
            <div class="containerLoaderProduct ng-hide" ng-show="insuranceMatch.isLoading"><div class="loader"></div></div>
            <!--<div class="list-item-row line1">-->
            <div class="list-item-company ng-hide" ng-click="openCompanyPopup($index)" ng-hide="insuranceMatch.gotoCompanyWithoutData">
                <div class="list-item-brand" ng-style="{'background-image':'url(/api/img/companies/'+insuranceMatch.companyId+'.svg)'}" style="background-image: url(&quot;/api/img/companies/18.svg&quot;);">
                    <!-- <img ng-src="/api/img/companies/{{insuranceMatch.companyId}}.svg">-->
                </div>
                <div class="go-to-company">Gå til selskab</div>
            </div>
            <form id="form9" name="form9" action="/" method="post" target="_blank" ng-click="gotoCompanyWithoutData($index,insuranceMatch.companycontacturl)" ng-show="insuranceMatch.gotoCompanyWithoutData" class="ng-pristine ng-valid">
                <div class="list-item-company">
                    <div class="list-item-brand" ng-style="{'background-image':'url(/api/img/companies/'+insuranceMatch.companyId+'.svg)'}" style="background-image: url(&quot;/api/img/companies/18.svg&quot;);">
                    </div>
                    <div class="go-to-company">Gå til selskab</div>
                </div>
            </form>
            <div class="list-item-information">

                <div class="list-item-coverage">
                    <div class="containerInner">
                        <div class="yourCoverageText">Din dækning</div><div class="star-ratings ng-isolate-scope" star-rating="" stars="insuranceMatch.stars" coveragequality="9303.697368421055">
                            <!-- ngRepeat: star in stars --><span ng-repeat="star in stars" class="rating-icon ng-scope">
                                <i class="fg-icon-int_star_c rating-icon-bg invisible" ng-class="{'invisible': star.half || star.full}"></i>
                                <i class="fg-icon-int_star_b rating-icon-half" ng-class="{visible: star.half}"></i>
                                <i class="fg-icon-int_star_a rating-icon-full visible" ng-class="{visible: star.full}"></i>
                            </span><!-- end ngRepeat: star in stars --><span ng-repeat="star in stars" class="rating-icon ng-scope">
                                <i class="fg-icon-int_star_c rating-icon-bg invisible" ng-class="{'invisible': star.half || star.full}"></i>
                                <i class="fg-icon-int_star_b rating-icon-half" ng-class="{visible: star.half}"></i>
                                <i class="fg-icon-int_star_a rating-icon-full visible" ng-class="{visible: star.full}"></i>
                            </span><!-- end ngRepeat: star in stars --><span ng-repeat="star in stars" class="rating-icon ng-scope">
                                <i class="fg-icon-int_star_c rating-icon-bg invisible" ng-class="{'invisible': star.half || star.full}"></i>
                                <i class="fg-icon-int_star_b rating-icon-half" ng-class="{visible: star.half}"></i>
                                <i class="fg-icon-int_star_a rating-icon-full visible" ng-class="{visible: star.full}"></i>
                            </span><!-- end ngRepeat: star in stars --><span ng-repeat="star in stars" class="rating-icon ng-scope">
                                <i class="fg-icon-int_star_c rating-icon-bg invisible" ng-class="{'invisible': star.half || star.full}"></i>
                                <i class="fg-icon-int_star_b rating-icon-half" ng-class="{visible: star.half}"></i>
                                <i class="fg-icon-int_star_a rating-icon-full visible" ng-class="{visible: star.full}"></i>
                            </span><!-- end ngRepeat: star in stars --><span ng-repeat="star in stars" class="rating-icon ng-scope">
                                <i class="fg-icon-int_star_c rating-icon-bg invisible" ng-class="{'invisible': star.half || star.full}"></i>
                                <i class="fg-icon-int_star_b rating-icon-half" ng-class="{visible: star.half}"></i>
                                <i class="fg-icon-int_star_a rating-icon-full visible" ng-class="{visible: star.full}"></i>
                            </span><!-- end ngRepeat: star in stars -->
                        </div>
                    </div>
                </div>
                <div class="list-item-insurance">
                    <div class="containerInner">
                        <ul class="inline-list list-item-addons">
                            <li class="addon-icon active" ng-class="{true: 'active'}[insuranceMatch.addtionalOptions[0].chosen]"><i class="fg fg-icon-icon_bil_kasko" tooltipnew="Kasko dækker: skader på bilen,&nbsp;skader på bilens tilbehør, tyveri og&nbsp;retshjælp" tooltip-placement="right" tooltip-append-to-body="true"></i></li>
                            <li class="addon-icon" ng-class="{true: 'active'}[insuranceMatch.addtionalOptions[1].chosen]"><i class="fg fg-icon-icon_bil_friskade" tooltipnew="Friskade betyder, at prisen på forsikringen ikke stiger ved skader, og der ikke skal betales selvrisiko ved fx tyveri, brand, nedstyrtning af ting på bilen og seriehærværk. Læs om den konkrete dækning hos det enkelte selskab på selskabets detaljeside eller på sammenligningssiden." tooltip-placement="right" tooltip-append-to-body="true"></i></li>
                            <li class="addon-icon" ng-class="{true: 'active'}[insuranceMatch.addtionalOptions[2].chosen]"><i class="fg fg-icon-icon_bil_udvidetglas" tooltipnew="Udvidet glas betyder, at prisen på forsikringen ikke stiger, og der ikke skal betales fuld selvrisiko ved skader på bilens glas.&nbsp;Læs om den konkrete dækning hos det enkelte selskab på selskabets detaljeside eller på sammenligningssiden." tooltip-placement="right" tooltip-append-to-body="true" tooltipnew-active="true"></i></li>
                            <li class="addon-icon" ng-class="{true: 'active'}[insuranceMatch.addtionalOptions[3].chosen]"><i class="fg fg-icon-icon_bil_forerdaekning" tooltipnew="Førerdækning dækker personskader, hvis føreren af bilen kommer til skade i forbindelse med et færdselsuheld.&nbsp;Læs om den konkrete dækning hos det enkelte selskab på selskabets detaljeside eller på sammenligningssiden." tooltip-placement="right" tooltip-append-to-body="true"></i></li>
                            <li class="addon-icon" ng-class="{true: 'active'}[insuranceMatch.addtionalOptions[4].chosen]"><i class="fg fg-icon-icon_bil_vejhjaelp" tooltipnew="Vejhjælp dækker som udgangspunkt følgende:&nbsp;bugsering, starthjælp, fritrækning, o.lign. Læs om den konkrete dækning hos det enkelte selskab på selskabets detaljeside eller på sammenligningssiden." tooltip-placement="right" tooltip-append-to-body="true"></i></li>
                            <li class="addon-icon" ng-class="{true: 'active'}[insuranceMatch.addtionalOptions[5].chosen]"><i class="fg fg-icon-icon_bil_fastpraemie" tooltipnew="Fastpris betyder, at forsikringens pris ikke automatisk stiger ved en skade." tooltip-placement="right" tooltip-append-to-body="true"></i></li>
                        </ul>
                    </div>
                </div>
            </div>
            <div class="list-item-pricing">
                <div class="table">
                    <div class="tableCell">
                        <div class="main-price" basicprice="3459">
                            <h2 class="nmt nmb"><small>kr./år</small> <span ng-show="insuranceMatch.basicprice < 999999" class="ng-binding">3,459​&lt;/span><span ng-show="insuranceMatch.basicprice >= 999999" class="ng-hide">-</span></h2>
                            <div class="discount-price" discountedprice="2958">
                                <h4 class="nmt nmb ng-binding"><small>Med samlerabat kr./år</small> 2,958​&lt;/h4>
                            </div>
                            <div class="deductible-price" selvrisiko="6324">
                                <h4 class="nmt nmb ng-binding"><small>Selvrisiko</small> 6,324​&lt;/h4>
                            </div>
                        </div>
                    </div>
                </div>
            </div>

            <div class="list-item-actions">
                <div class="containerButtonDetails">
                    <button type="submit" class="button primary inverse expand" ng-click="goToDetails(insuranceMatch.companyId, insuranceMatch.produktId, insuranceMatch.resultid)" ng-class="{'disabled': insurancematch.isloading}">Se detaljer</button>
                </div>
                <div class="containerButtonCompare">
                    <button ng-hide="insuranceMatch.addToComparison" class="button compare expand" ng-click="addToComparison($index)" ng-class="{'active': insuranceMatch.addToComparison}">
                        <span ng-hide="insuranceMatch.addToComparison" class="">Sammenlign</span>
                    </button>
                    <div ng-show="insuranceMatch.addToComparison" class="button compare expand ng-hide" ng-class="{'active': insuranceMatch.addToComparison}">
                        <span class="closeIconSmallWhite" ng-click="addToComparison($index)"></span>
                        <span ng-show="addedToComparison > 1" ng-click="comparisonQuery()" class="button-text ng-hide">Gå til sammenlign</span>
                        <span ng-hide="addedToComparison > 1" class="button-text">Vælg min. 2 selskaber</span>
                    </div>
                </div>
            </div>
        </div>
        <!-- ngIf: insuranceMatch.hasmemberrequirements -->
        <!-- ngIf: insuranceMatch.discountItemCount >0 -->
    </div>
</div>

标签： pythonselenium

解决方案

您可以使用简单的 css 选择器获得所有信息：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

#...

wait = WebDriverWait(driver, 10)
resultatside = driver.get("https://forsikringsguiden.dk/#!/bilforsikring/resultatside")
prices = get_car_insurance()
basic_price = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "[basicprice]"))).get_attribute(
        "basicprice")
discounted_price = driver.find_element_by_css_selector("[discountedprice]").get_attribute("discountedprice")
selvrisiko = driver.find_element_by_css_selector("[selvrisiko]").get_attribute("selvrisiko")

print(basic_price, discounted_price, selvrisiko)

你的代码已经完全应该被修改了。
下面的代码什么都不做：

wait = WebDriverWait(driver,10)
wait

在下面的代码中resultatside是 None - 什么都没有，driver.get返回 None：

resultatside = driver.get("https://forsikringsguiden.dk/#!/bilforsikring/resultatside")

推荐阅读