httplib.badstatusline:''在Linux上,而不是Mac[英] httplib.BadStatusLine: '' on Linux but not Mac

问题描述

现在,这个错误已经存在了几个小时.我决定对一个单独的项目进行编码,只是为了查看是否可以复制它,但只能在服务器上.这在我的Mac上起作用.

  • Mac:OSX El Capitan 10.11.6

  • 服务器:CENTOS 7.2.1511

  • 两者都有phantomjs版本:2.1.1

  • Python Mac:Python 2.7.11

  • Python服务器:2.7.5

  • 两者都有硒版本:2.53.0

同一代码在这两个上运行:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.common.exceptions import NoSuchElementException
import time

dcap = dict(DesiredCapabilities.PHANTOMJS)
dcap["phantomjs.page.settings.userAgent"] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36"
dcap["phantomjs.page.customHeaders.accept"] = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
dcap["phantomjs.page.customHeaders.Accept-Language"] = "en-US,en;q=0.8"
dcap["phantomjs.page.customHeaders.connection"] = "keep-alive"

driver = webdriver.PhantomJS(desired_capabilities=dcap)
driver.set_window_size(1120, 700)
driver.get("https://www.instagram.com/espn/")

while True:
    print len(driver.find_elements_by_css_selector("a[href*='/p/']"))
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    try:
        loadMore = driver.find_element_by_link_text("Load more")
        loadMore.click()
    except NoSuchElementException:
        print "No such"
        driver.save_screenshot('none.png')

Mac输出:

12
24
No such
24
No such
36
No such
48
No such
48
No such
60
No such
72
No such
84
# This goes until I end it

服务器输出:

12
24
No such
Traceback (most recent call last):
  File "junk.py", line 27, in <module>
    driver.save_screenshot('none.png')
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 790, in get_screenshot_as_file
    png = self.get_screenshot_as_png()
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 809, in get_screenshot_as_png
    return base64.b64decode(self.get_screenshot_as_base64().encode('ascii'))
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 819, in get_screenshot_as_base64
    return self.execute(Command.SCREENSHOT)['value']
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 231, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 395, in execute
    return self._request(command_info[0], url, body=data)
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 463, in _request
    resp = opener.open(request, timeout=self._timeout)
  File "/usr/lib64/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 1244, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib64/python2.7/urllib2.py", line 1217, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib64/python2.7/httplib.py", line 1089, in getresponse
    response.begin()
  File "/usr/lib64/python2.7/httplib.py", line 444, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.7/httplib.py", line 408, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine: ''

删除屏幕截图线后的服务器输出:

12
24
No such
24
Traceback (most recent call last):
  File "junk.py", line 23, in <module>
    loadMore = driver.find_element_by_link_text("Load more")
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 314, in find_element_by_link_text
    return self.find_element(by=By.LINK_TEXT, value=link_text)
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 744, in find_element
    {'using': by, 'value': value})['value']
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 231, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 395, in execute
    return self._request(command_info[0], url, body=data)
  File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 463, in _request
    resp = opener.open(request, timeout=self._timeout)
  File "/usr/lib64/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 1244, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib64/python2.7/urllib2.py", line 1217, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib64/python2.7/httplib.py", line 1089, in getresponse
    response.begin()
  File "/usr/lib64/python2.7/httplib.py", line 444, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.7/httplib.py", line 408, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine: ''

我发现的一个相关答案在这里:不能在Python中运行Phantomjs通过硒

所以我安装了硒2.37,并给出了相同的错误.

我阅读这个答案问题可能与更改标头有关,因此我通过将驱动程序更改为driver = webdriver.PhantomJS()并仍然遇到相同的错误来删除标题.

我还在服务器上安装了2.7.12,以查看是否存在差异.输出为:

# python2.7 junk.py
12
24
No such
24
Traceback (most recent call last):
  File "junk.py", line 29, in <module>
    loadMore = driver.find_element_by_link_text("Load more")
  File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 314, in find_element_by_link_text
    return self.find_element(by=By.LINK_TEXT, value=link_text)
  File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 744, in find_element
    {'using': by, 'value': value})['value']
  File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 231, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 395, in execute
    return self._request(command_info[0], url, body=data)
  File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 463, in _request
    resp = opener.open(request, timeout=self._timeout)
  File "/usr/local/lib/python2.7/urllib2.py", line 429, in open
    response = self._open(req, data)
  File "/usr/local/lib/python2.7/urllib2.py", line 447, in _open
    '_open', req)
  File "/usr/local/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python2.7/urllib2.py", line 1228, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/local/lib/python2.7/urllib2.py", line 1201, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/local/lib/python2.7/httplib.py", line 1136, in getresponse
    response.begin()
  File "/usr/local/lib/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python2.7/httplib.py", line 417, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine: ''

检查系统上的空间.这是一个全新的副总裁,但仍然可以确认:

在此处输入图像说明

推荐答案

编辑3

添加以下内容:

except httplib.BadStatusLine:
    pass

编辑2

python webdriver和phantomjs在 keep_alive "> keep_alive "> keep_alive 中.这可能是您的问题.因此,添加keep_alive = false如下:

driver = webdriver.PhantomJS(desired_capabilities=dcap,keep_alive=False)

结束编辑


添加以下

import httplib
import socket

from selenium.webdriver.remote.command import Command

def get_status(driver):
    try:
        driver.execute(Command.STATUS)
        return "Alive"
    except (socket.error, httplib.CannotSendRequest):
        return "Dead"

在save_screenshot语句之前调用get_status(驱动程序)并打印结果.这将告诉我们驾驶员是否过早关闭.

编辑

在驱动程序= webdriver.phantomjs(desired_capabilities = dcap)之后添加以下内容

driver.implicitly_wait(10) #wait 10 seconds when doing a find_element before carrying on

本文地址:https://www.itbaoku.cn/post/1740036.html