Module: Mutineer::TestRunners::RSpec
- Defined in:
- lib/mutineer/test_runners/rspec.rb
Overview
Child-process-only RSpec runner.
Mirrors MinitestIntegration's contract: run the given spec files and return 0 (all passed), 1 (an example failed) or 2 (no verdict). Exit 2 is an error outside examples, such as a spec file that raised while loading, or a run whose first failed example failed with an Active Record connection or lock error (#347).
Defined Under Namespace
Modules: FileOrder Classes: KillFormatter
Class Method Summary collapse
-
.failure_code ⇒ Integer
private
The exit status of a failed run: 2 when no example failed and RSpec recorded an error outside examples, or when the first failed example's own exception is an Active Record connection or lock error (#347).
-
.first_failed_database_error ⇒ Exception?
private
The database error of the first failed example, in run order, or nil when that example failed for another reason.
-
.keep_file_order(spec_files) ⇒ void
private
Runs the top-level example groups in the order of
spec_files(#203), after RSpec's own ordering, which then decides the order within a file. -
.last_output ⇒ String?
The sink text from the most recent RSpec.run (#360 clean-failure detail).
-
.outside_examples? ⇒ Boolean
private
True when RSpec recorded an error outside examples and no example failed.
-
.primary_exception(error) ⇒ Exception?
private
The exception that ended the example, before a later hook error.
-
.require_rspec! ⇒ Object
private
Requires rspec-core from the project under test.
-
.run(spec_files, stop_at_first_failure: false, record_to: nil) ⇒ Integer
Runs the given RSpec files.
-
.with_fail_fast_off(record_to) ⇒ Integer
private
Runs the block with fail-fast forced off when
record_tois set: a matrix run must run every example.
Class Method Details
.failure_code ⇒ Integer
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
The exit status of a failed run: 2 when no example failed and RSpec recorded an error outside examples, or when the first failed example's own exception is an Active Record connection or lock error (#347). Otherwise 1.
A later after hook is not the example's own exception. RSpec keeps
the first one and wraps a later hook error around it, the same way
Minitest records a teardown error after the setup or body error.
103 104 105 106 107 108 109 110 111 112 |
# File 'lib/mutineer/test_runners/rspec.rb', line 103 def self.failure_code return 2 if outside_examples? if (error = first_failed_database_error) MinitestIntegration::InTestDatabaseError.warn_database_error(error) return 2 end 1 end |
.first_failed_database_error ⇒ Exception?
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
The database error of the first failed example, in run order, or nil when that example failed for another reason.
135 136 137 138 139 140 141 142 143 |
# File 'lib/mutineer/test_runners/rspec.rb', line 135 def self.first_failed_database_error example = ::RSpec.configuration.reporter.failed_examples.first return unless example error = primary_exception(example.exception) error if MinitestIntegration::InTestDatabaseError.database_exception?(error) rescue StandardError nil end |
.keep_file_order(spec_files) ⇒ void
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
This method returns an undefined value.
Runs the top-level example groups in the order of spec_files (#203),
after RSpec's own ordering, which then decides the order within a file.
Every run uses it, including the clean check, so a mutant run and the
clean check of the same files share that order (#282).
169 170 171 172 |
# File 'lib/mutineer/test_runners/rspec.rb', line 169 def self.keep_file_order(spec_files) FileOrder.rank = Array(spec_files).each_with_index.to_h { |f, i| [File.(f), i] } ::RSpec::Core::World.prepend(FileOrder) unless ::RSpec::Core::World <= FileOrder end |
.last_output ⇒ String?
The sink text from the most recent run (#360 clean-failure detail).
27 28 29 |
# File 'lib/mutineer/test_runners/rspec.rb', line 27 def self.last_output @last_output end |
.outside_examples? ⇒ Boolean
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
True when RSpec recorded an error outside examples and no example failed.
Exit status 2 is also an in-example database error (failure_code). A cause line for the outside-examples case has to test this predicate, not a status of 2. Pull request #443 writes that cause when the status is 2; after this change that condition would label a database error as an error outside examples (#347, #417).
124 125 126 127 128 |
# File 'lib/mutineer/test_runners/rspec.rb', line 124 def self.outside_examples? world = ::RSpec.world outside = world.respond_to?(:non_example_failure) && world.non_example_failure !!(outside && ::RSpec.configuration.reporter.failed_examples.empty?) end |
.primary_exception(error) ⇒ Exception?
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
The exception that ended the example, before a later hook error.
RSpec::Core::MultipleExceptionError keeps that first exception at
the front of all_exceptions and appends each later hook error.
152 153 154 155 156 157 158 159 |
# File 'lib/mutineer/test_runners/rspec.rb', line 152 def self.primary_exception(error) nested = ::RSpec::Core::MultipleExceptionError return error.all_exceptions.first if error.is_a?(nested) && error.respond_to?(:all_exceptions) error rescue StandardError error end |
.require_rspec! ⇒ Object
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
Requires rspec-core from the project under test.
441 442 443 444 445 446 447 |
# File 'lib/mutineer/test_runners/rspec.rb', line 441 def self.require_rspec! require "rspec/core" rescue LoadError raise Mutineer::FrameworkUnavailable, "framework 'rspec' requested but rspec is not available; " \ "add rspec to the project under test (its bundle), then retry" end |
.run(spec_files, stop_at_first_failure: false, record_to: nil) ⇒ Integer
Runs the given RSpec files.
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
# File 'lib/mutineer/test_runners/rspec.rb', line 40 def self.run(spec_files, stop_at_first_failure: false, record_to: nil) if stop_at_first_failure && record_to raise ArgumentError, "stop_at_first_failure and record_to cannot be combined: a matrix run runs every example" end require_rspec! RSpecSeed.install! ::RSpec::Core::Runner.disable_autorun! # Not RSpec.reset: it also drops the settings that gems added when Bundler # required them, such as rspec-retry's verbose_retry. ::RSpec.clear_examples # Added before the run reads its options, which leaves the reporter # unbuilt, so the run's output stream still applies. With a formatter # present, RSpec adds no default one; its output went to the sink anyway. KillFormatter.last = nil ::RSpec.configuration.add_formatter(KillFormatter.registered, record_to) if record_to # The sink takes RSpec's own formatter output. Spec output is not # silenced here: the fork boundary that calls this method has already # pointed stdout at File::NULL (see ChildStdout). sink = StringIO.new # RSpec matches the spec path with a UTF-8 regexp. A binary path # under LC_ALL=C raises before the example runs (#346). files = Array(spec_files).map { |file| FsPath.read(file) } # Clean checks pass no hook. They still use this file order, so they # share it with the mutant run (#282). keep_file_order(files) args = ["--no-color"] args << "--fail-fast" if stop_at_first_failure status = with_fail_fast_off(record_to) do ::RSpec::Core::Runner.run([*args, *files], sink, sink) end @last_output = sink.string # Reached only when the suite returned: an example that exits the # process skips it, and the parent then keeps the row incomplete. A # return alone proves nothing: fail-fast turned on mid-run stops RSpec # early, so `end` needs a reported outcome for every planned example. KillChannel.write_end(record_to) if record_to && KillFormatter.last&.saw_every_example? return 0 if status.zero? code = failure_code # Fixed cause when RSpec failed outside examples (#417 / KTD5). Status 2 # is also an in-example database error (#347), so test the predicate, # not the status. Write here so failure_code stays a pure status helper. ChildCause.write_text(ChildCause::RSPEC_OUTSIDE_EXAMPLES) if outside_examples? code ensure FileOrder.rank = nil end |
.with_fail_fast_off(record_to) ⇒ Integer
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
Runs the block with fail-fast forced off when record_to is set: a
matrix run must run every example. RSpec reads options from .rspec, then
the command line, then SPEC_OPTS, and the last one wins, so the option
goes at the end of SPEC_OPTS (restored afterwards). A forced option
also beats config.fail_fast = true in a spec helper.
209 210 211 212 213 214 215 216 217 218 219 |
# File 'lib/mutineer/test_runners/rspec.rb', line 209 def self.with_fail_fast_off(record_to) return yield unless record_to saved = ENV.fetch("SPEC_OPTS", nil) ENV["SPEC_OPTS"] = [saved, "--no-fail-fast"].compact.join(" ") begin yield ensure ENV["SPEC_OPTS"] = saved end end |