不可靠/薄片辣椒粉/Angularjs集成测试与时机问题[英] Unreliable/Flakey Capybara/AngularJS Integration Tests With Timing Issues

问题描述

如何可靠地通过?

目前这些测试是薄弱的.
有时他们通过.有时他们失败了.
以下是演示此问题的设置,代码和输出.
要克服这个问题的建议将不胜感激,我敢肯定会帮助许多其他问题,所以请评论

测试代码环境

  1. Rails 3.2
  2. RSPEC 2.x
  3. 辣椒粉
  4. Poltergeist
  5. phantomjs
  6. angularjs
  7. Google Chrome版本47.0.2526.106(64位)

从gemfile.lock

测试宝石
capybara (2.1.0)
database_cleaner (0.7.1)
debug_inspector (0.0.2)
guard-bundler (0.1.3)
guard-livereload (1.2.0)
guard-rspec (2.1.2)
jasminerice (0.0.10)
pg (0.17.1)
phantomjs (2.1.1.0)
poltergeist (1.4.1)
protractor-rails (0.0.17)
pry (0.9.12)
rack (1.4.7)
rack-test (0.6.3)
rails (3.2.21)
rails-assets-angular (1.3.20)
rspec-rails (2.11.4)
simplecov (0.8.2)
sprockets (2.2.3)
zeus (0.13.3)
zeus-parallel_tests (0.2.1)

我尝试过的东西

  1. 确保我使用Capybara的等待DSL匹配器
  2. 确保正确设置我的数据库清洁器
  3. 测试每个页面,假设它可能不在页面上,并且仍然可以加载
  4. 缩小不一致的测试
  5. 单独进行不一致的测试
  6. 识别代码Capybara DSL,这是触发测试结果不一致的触发因素.

    • 即.创建新记录并假设页面已重定向,并且该记录在页面上click_on

    • .单击不始终如一地"工作"
  7. 升级Capybara到最新版本(在单独的分支中)
  8. 升级的Poltergeist和RSPEC到最新版本(在单独的分支中,仍在处理此问题)

我使用的资源

[1] capybara dsl

[2] 提示
还有更多...

如何运行测试

rspec spec/integration/costings/show_costing_spec.rb --format documentation

测试代码

show_costing_spec.rb
require "spec_helper"

RSpec.describe "Show a new costing in the listing," do

  before :each do
    admin_sign_in
    create_costing("test1")
  end

  it "shows the costing after creation" do
    within "#costings_table" do
      expect(page).to have_css("#name", text: "test1")
    end
  end

  it "shows the details of the new costing after creation" do
    expect(page).to have_content("Costings")
    within "#costings_table" do
      expect(page).to have_content("test1")
      all("#show").last.click
    end

    expect(page).to have_content("Costing Details")
    expect(page).to have_css("#name", text: "test1")
  end
end  
spec_helper.rb
# This file is copied to spec/ when you run 'rails generate r spec:install'  
ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)
# Add library functions here so we can test them.
require File.expand_path(File.dirname(__FILE__) + "/../lib/general")
require 'rspec/rails'
require 'rspec/autorun'

# Integration Testing
require 'capybara/poltergeist'
Capybara.register_driver :poltergeist_debug do |app|
 Capybara::Poltergeist::Driver.new(app, :inspector => true)  
end
Capybara.javascript_driver = :poltergeist_debug
Capybara.default_driver = :poltergeist_debug

# Capybara Integration Test Helpers
def admin_sign_in
  visit "/login"
  #Create staff member in database
  Staff.make!(:admin)
  #Log In
  fill_in "staff_username", with: "adminstaff"
  fill_in "staff_password", with: "password"
  click_button "login"
end

def create_costing(item)
  visit "/api#/costings"
  click_on "new_btn"
  within "#form_costing" do
    find("#name", match: :first).set("#{item}")
    find("#description", match: :first).set("test description")    
    find("#from_date", match: :first).set("15/02/2016")
    find("#cost_hourly_cents", match: :first).set("1.00")
    click_on "create_btn"
  end
end

RSpec.configure do |config|
  config.before(:suite) do
    # Requires supporting ruby files with custom matchers and macros, etc,
    # in spec/support/ and its subdirectories.
    require File.expand_path(File.dirname(__FILE__) + "/support/blueprints")
    Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}
  end

  # Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
  config.fixture_path = "#{::Rails.root}/spec/fixtures"

  # Allow a 'focus' tag so that we can run just a few tests which we are currently working on
  config.treat_symbols_as_metadata_keys_with_true_values = true
  config.filter_run focus: true
  config.run_all_when_everything_filtered = true
  config.filter_run_excluding :slow unless ENV["SLOW_SPECS"]

  # Defer Garbage Collection
  config.before(:all) { DeferredGarbageCollection.start }
  config.after(:all)  { DeferredGarbageCollection.reconsider }

  # If you're not using ActiveRecord, or you'd prefer not to run each of your
  # examples within a transaction, remove the following line or assign false
  # instead of true.
  config.use_transactional_fixtures = false
  # config.infer_spec_type_from_file_location!

  # Configure Database Cleaner
  config.include Capybara::DSL
  config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
  end

  config.before(:each) do
    DatabaseCleaner.strategy = :transaction
  end

  config.before(:each, :js => true) do
    DatabaseCleaner.strategy = :truncation
  end

  config.before(:each) do
    DatabaseCleaner.start
  end

  config.after(:each) do
    DatabaseCleaner.clean
  end
end

测试结果

Test Run 1: Failing

运行选项: 包括{:focus => true} 排除{:slow => true}

所有示例都被过滤了;忽略{:focus => true}

在清单中显示一个新成本, 显示创建后的成本 显示创建后新成本的细节(失败-1)

失败:

1)在列表中显示一个新成本,
显示创建后新成本的细节
故障/错误:期望(页).to has_content(" test1")
预期的#has_content?(" test1")要返回true,得到false
#./spec/integration/costings/show_costing_spec.rb:20: in block(3级)
in
#./spec/integration/costings/show_costing_spec.rb:19:in Block(2级)

in

在5.46秒内完成 2个例子,1个失败

Test Run 2: Passing

运行选项: 包括{:focus => true} 排除{:slow => true}

所有示例都被过滤了;忽略{:focus => true}

在列表中显示一个新成本,
显示创建后的成本
显示创建后新成本的细节

在3.57秒内完成 2个例子,0失败

更新1

将测试宝石升级到以下版本:
(2.1.0) Capybara(2.6.2) database_cleaner(1.5.1)来自(0.7.1)
debug_inspector(0.0.2)
后卫捆绑(0.1.3)
后卫 - 利弗洛德(1.2.0)
后卫规格(2.1.2)
Jasminerice(0.0.10)
pg(0.17.1)
phantomjs(2.1.1.0)
Poltergeist(1.9.0)来自(1.4.1)
量角轨(0.0.17)
pry(0.10.3)来自(0.9.12)
架子(1.4.7)
机架测试(0.6.3)
铁轨(3.2.21)
Rails-Assets-angular(1.4.9)来自(1.3.20)
(2.11.4) rspec-rails(3.4.2) Simplecov(0.8.2)
链轮(2.2.3)
宙斯(0.13.3)
Zeus-Paralleal_tests(0.2.1)

Result 1

不幸的是,升级这些宝石似乎并没有有所作为,我的测试仍然很薄弱.

更新2

我实施了汤姆·沃尔波尔的建议.确保我的admin_sign_in等待sign_in完成.

还按照汤姆的建议更新了我的数据库_Cleaner设置.

Result 2

对于我的堆栈,这些变化似乎没有效果.

注意:如果不使用AngularJ,我认为这些更改会有所作为.所以谢谢汤姆的建议.

更新3

我需要获取有关测试过程中发生的事情的更多信息.网上有一些建议来记录,使用屏幕截图节省宝石等,但我觉得这不会是最有效的时间.我想指定我希望测试在何处暂停并查看变量和表单字段的内容.在浏览器中是理想的.

我使用的
我正在使用" save_and_open_page"和"打印page.html"进行调试.

我搬到
当我运行RSPEC 3.4.2时,我添加了调试辅助方法:

rails_helper.rb

def debugit
  puts current_url
  require 'pry'
  binding.pry
end

Result 3

将在控制台中打印一个URL,并暂停测试.在此阶段,我将能够导航到URL,登录,导航到测试页面并查看Capybara测试所做的工作.

这使我能够确定测试使用Capybara的Fill_in DSL时出现的问题来源. 在某些测试运行中,字段将正确填充,并将提交表格. 在其他情况下,表格将正确填充,但提交按钮将被击中太快.这里的结果是创建了记录,但名称和描述的输入字段并未持续存在.

更新4

我发现,由于我使用的是AngularJS输入形式和表,因此AngularJS需要一小部分时间与输入字段结合.如果不允许这次,将无法保存输入数据.

Capybara提供等待方法,例如"内部"和"查找".我使用了这些,但它们没有帮助解决Angularjs绑定时间问题. 我发现ng-if可以用来创建一个if语句,以等待特定的项目,该项目表示符合AngularJS绑定到表单字段的完整.

所以我使用Capybara等待方法等待我想填充的字段,然后我使用了Angularjs的NG-如果不显示该字段,则直到准备就绪.

实施
index.html.erb

<div  ng-if="tableParams.data">
  <table id="costings_table ng-table="tableParams" class="table">
    <td id="field1">{{table.field1}}</td>
    <td id="field2">{{table.field2}}</td>
  </table>
</div>

Result 4

测试终于通过了!但是,我有所有这些找到具有XPATH的方法,以确保特定且难以定位的项目等待...

更新5

即使在我的Gemfile中,我正在运行GEM Phantomjs版本2.1.1我的命令行版本仅为1.x.事实证明这很重要.

我将命令行Phantomjs版本更新为2.1.1.同时,我确保所有输入框,按钮,桌子,标题具有唯一的ID.然后,我能够删除所有发现(:XPATH)的情况,而无需打破测试.

Result 5

这套测试套件现在一直可靠地通过!正是我想要的!是!

推荐答案

问题

测试使用Capybara的Fill_in DSL时会出现问题.在某些测试运行中,字段将正确填充,并将提交表格.在其他情况下,表格将正确填充,但提交按钮将被击中太快.这里的结果是创建了记录,但名称和描述的输入字段并未持续存在.

1.填写表格时,请确保AngularJS绑定完成,并且使用Capybara等待方法

angularjs的ng-if语句需要使用不显示形式字段才能准备就绪.
这需要与使用Capybara等待方法结合使用,以确保仅在Form Load完成后才提交Fill_in字段.

index.html.erb或等效:

<div  ng-if="tableParams.data">
  <table id="costings_table ng-table="tableParams" class="table">
    <td id="field1">{{table.field1}}</td>
    <td id="field2">{{table.field2}}</td>
   </table>
</div>

2.将phantomjs的命令行版本更新为最新(2.1.1)

这似乎使测试能够在没有那么多的辣椒等候方法的情况下进行运行,以实现可靠的测试.

更新的测试代码
show_costing_spec.rb

require "rails_helper"

RSpec.describe "Show a new costing in the listing,", :type => :feature do

  before :each do
    admin_sign_in
    create_costing("test1")
  end

  it "shows the costing after creation" do
    within "#costings_table" do
      expect(page.find("#code2")).to have_content("2")
      expect(page.find("#name2")).to have_content("test1")
    end
  end

  it "shows the details of the new costing after creation" do
    within "#costings_table" do
      click_on "show2"
    end

    expect(page.find("#page_title")).to have_content("Costing Details")
    expect(page.find("#code")).to have_content("2")
    expect(page.find("#name")).to have_content("test1") 
    expect(page.find("#description")).to have_content("test description")
  end
end

rails_helper.rb

# This file is copied to spec/ when you run 'rails generate rspec:install'
ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)

# Add library functions here so we can test them.
require File.expand_path(File.dirname(__FILE__) + "/../lib/general")

require 'rspec/rails'
require 'devise'

RSpec.configure do |config|
  config.before(:suite) do
    # Requires supporting ruby files with custom matchers and macros, etc,
    # in spec/support/ and its subdirectories.
    require File.expand_path(File.dirname(__FILE__) + "/support/blueprints")
    Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}

    # Setup Devise before it is used in rails_helper
    config.include Devise::TestHelpers, :type => :controller
    Devise.stretches = 1 # Improves speed.
end

config.include Capybara::DSL, :type => :feature
  config.mock_with :rspec

# Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
config.fixture_path = "#{::Rails.root}/spec/fixtures"

# Allow a 'focus' tag so that we can run just a few tests which we are currently working on
config.filter_run focus: true
config.run_all_when_everything_filtered = true
config.filter_run_excluding :slow unless ENV["SLOW_SPECS"]

# Defer Garbage Collection
config.before(:all) { DeferredGarbageCollection.start }
config.after(:all)  { DeferredGarbageCollection.reconsider }

# Integration Testing
require 'capybara/rspec'
require 'capybara/poltergeist'

Capybara.register_driver :poltergeist_debug do |app|
  Capybara::Poltergeist::Driver.new(app, {:inspector => true, js_errors: false })  
end

Capybara.javascript_driver = :poltergeist_debug
Capybara.default_driver = :poltergeist_debug

# Debugging tools
def debugit
  puts current_url
  require 'pry'
  binding.pry
end

# If you're not using ActiveRecord, or you'd prefer not to run each of your
# examples within a transaction, remove the following line or assign false
# instead of true.
config.use_transactional_fixtures = false

#Show Deprications As Errors with full backtracing
config.raise_errors_for_deprecations!

#rest of the file....
# Final part of Configure Database Cleaner

Capybara.default_max_wait_time = 5
config.use_transactional_fixtures = false

config.before(:suite) do
  if config.use_transactional_fixtures?
    raise(<<-MSG)
      Delete line `config.use_transactional_fixtures = true` from
      rails_helper.rb (or set it to false) to prevent uncommitted
      transactions being used in JavaScript-dependent specs. During 
      testing, the app-under-test that the browser driver connects to 
      uses a different database connection to the database connection 
      used by the spec. The app's database connection would not be 
      able to access uncommitted transaction data setup over the 
      spec's database connection.
     MSG
  end
  DatabaseCleaner.clean_with(:truncation)
end  

config.before(:each) do
  DatabaseCleaner.strategy = :transaction
end

config.before(:each, type: :feature) do
  # :rack_test driver's Rack app under test shares database connection
  # with the specs, so continue to use transaction strategy for speed.
  driver_shares_db_connection_with_specs = Capybara.current_driver == :rack_test

    if !driver_shares_db_connection_with_specs
      # Driver is probably for an external browser with an app
      # under test that does *not* share a database connection with the
      # specs, so use truncation strategy.
      DatabaseCleaner.strategy = :truncation
    end
  end

  config.before(:each) do
    DatabaseCleaner.start
  end

  config.append_after(:each) do
    DatabaseCleaner.clean
  end
end


def admin_sign_in
  visit "/login"

  #Create staff member in database
  Staff.make!(:admin)

  #Log In
  fill_in "staff_username", with: "adminstaff"
  fill_in "staff_password", with: "password"
  click_button "login"

  expect(page).to have_text('Logout')
end

def create_costing(item)
  @item = item
  visit "/api#/costings"

  expect(page).to have_selector("#new_btn")
  click_on "new_btn"

  expect(page).to have_text("New Costing")
  within "#form_costing" do
    fill_in "name", with: "#{@item}"
    fill_in "description", with: "test description"
    fill_in "from_date1", with: "15/02/2015" 
    fill_in "cost_hourly_cents1", with: "12.00"

    expect(page).to have_selector("#create_btn")
    click_on "create_btn"
  end
  expect(page.find("#page_title")).to have_content("Costings")
end

其他推荐答案

跳出来的直接事情是,您的admin_sign_in实际上没有等待sign_in完成.这意味着您可以在没有浏览器中设置的会话cookie的情况下发生create_costing. admin_sign_in方法中的最后一行应该是

expect(page).to have_text('You are signed in') # whatever message is shown upon sign in

expect(page).to have_current_path('/') # whatever path an admin is redirected to upon signing in

将确保登录实际完成,因此会话cookie已在您的浏览器中设置.

此外,您的数据库清洁器配置应使用一个append_after块,而不是之后 - 参见 httpps "> httpps "> httpps ://github.com/databasecleaner/database_cleaner#rspec-with-capybara-example

本文地址:https://www.itbaoku.cn/post/1739945.html