如何通过PhantomJS驱动从网站的HTML中提取数据[英] How to extract data from the HTML of the website through PhantomJS Driver

问题描述

我正在尝试解析以下网页 https://shop.sprouts.com/shop/shop/shop/shop/flyer 使用.net,硒,phantomjs.我在元素文本中看到的数据与我在屏幕上看到的数据完全不同.有没有更好的方法来解析网页?

using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
using OpenQA.Selenium.PhantomJS;
[TestClass]
  public class UnitTest1
  {
    const string PhantomDirectory = @"..\..\..\packages\PhantomJS.2.1.1\tools\phantomjs";

[TestMethod]
    public void GetSproutsWeeklyAdDetails()
    {
      using (IWebDriver phantomDriver = new PhantomJSDriver(PhantomDirectory))
      {
        phantomDriver.Navigate().GoToUrl("https://shop.sprouts.com/shop/flyer");
        var elements = phantomDriver.FindElements(By.ClassName("cell-title-text"));
      }
    }
}

推荐答案

根据网站 https://shop.sprouts.com/shop/flyer来解析您在元素文本中看到的数据,您需要诱导 webdriverwait 可见性在所有所需元素中,您可以使用以下解决方案:

  • 解决方案:

    IList<IWebElement> elements = new WebDriverWait(driver, TimeSpan.FromSeconds(3)).Until(ExpectedConditions.VisibilityOfAllElementsLocatedBy(By.XPath("//span[@class='cell-title-text' and @ng-bind-html='productTitle()']")));
    foreach (IWebElement element in elements)
    {
        Console.WriteLine(element.GetAttribute("innerHTML"));
    }
    
  • 等效python exmaple:

    driver.get('https://shop.sprouts.com/shop/flyer')
    myList = WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH, "//span[@class='cell-title-text' and @ng-bind-html='productTitle()']")))
    for item in myList:
        print(item.text)
    
  • 控制台输出:

    Sweet Corn, 1 EA
    Cantaloupe Melons, 1 LB
    Red Cherries
    Half Chicken Breast
    Roma Tomatoes
    100% Grass Fed Ground Beef Value Pack
    Colby Jack Rbst Free
    Walnut Halves & Pieces
    

本文地址:https://www.itbaoku.cn/post/1740152.html