Module: Mutineer::MinitestIntegration::KillRecorder

Extended by:
OuterReporter::Client
Defined in:
lib/mutineer/minitest_integration/kill_recorder.rb

Overview

Sends the outcome of each test to the parent of a --matrix mutant run, through KillChannel. It never stops the run: every covering test runs, so the parent learns exactly which tests kill the mutant. Child-process only, like MinitestIntegration.

Only the outer reporter counts (see OuterReporter): a test can build and record on its own reporter, and those results are not the run's.

Defined Under Namespace

Modules: MarkSkippedClass5, MarkSkippedClass6, MarkSkippedTest, Record

Constant Summary collapse

LOCK =

Serializes the counting and the writing of results, which Minitest records from several threads in a parallel class.

Mutex.new

Class Attribute Summary collapse

Attributes included from OuterReporter::Client

#armed_pid, #armed_reporter

Class Method Summary collapse

Methods included from OuterReporter::Client

armed_here?, outer_reporter?

Class Attribute Details

.channel ⇒ IO?

The write end of the channel of the armed run.

Returns:

  • (IO, nil)


101
102
103
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 101

def channel
  @channel
end

.parallel_marked ⇒ Boolean?

True once the parallel line has gone out.

Returns:

  • (Boolean, nil)


106
107
108
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 106

def parallel_marked
  @parallel_marked
end

.runnables ⇒ Array<Class>?

The loaded test classes of the armed run.

Returns:

  • (Array<Class>, nil)


111
112
113
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 111

def runnables
  @runnables
end

.seen ⇒ Integer?

How many results the outer reporter has recorded, skips included.

Returns:

  • (Integer, nil)


116
117
118
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 116

def seen
  @seen
end

.serial_failed ⇒ Boolean?

True once a serial test failed, including a database failure: from then on, a run without --matrix skips each later test and class.

Returns:

  • (Boolean, nil)


122
123
124
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 122

def serial_failed
  @serial_failed
end

Class Method Details

.arm!(channel, runnables) ⇒ Boolean

Installs the hooks, arms the recorder for this process, and writes the channel's start line. Call it after the test files load, so the recorder knows the test classes. An unknown Minitest shape arms nothing and writes nothing, so the parent sees no start and keeps the row incomplete.

Parameters:

  • channel (IO) —

    the write end of the channel.

  • runnables (Array<Class>) —

    the loaded test classes.

Returns:

  • (Boolean) —

    false when the Minitest shape is unknown.



133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 133

def arm!(channel, runnables)
  outer = OuterReporter.hook_for_loaded_minitest
  return false unless outer

  OuterReporter.prepend_once(::Minitest::CompositeReporter, Record)
  OuterReporter.prepend_once(::Minitest.singleton_class, outer)
  class_hook = outer.equal?(OuterReporter::Hook6) ? MarkSkippedClass6 : MarkSkippedClass5
  runnables.each do |klass|
    OuterReporter.prepend_once(klass, MarkSkippedTest)
    OuterReporter.prepend_once(klass.singleton_class, class_hook)
  end
  OuterReporter.register(self)
  self.armed_pid = Process.pid
  self.armed_reporter = nil
  self.channel = channel
  self.parallel_marked = false
  # A copy: Minitest's list is live, and a class a test defines during
  # the run joins it without running, which would break the count.
  self.runnables = runnables.dup
  self.seen = 0
  self.serial_failed = false
  KillChannel.write_start(channel)
  true
end

.class_starting(klass, reporter) ⇒ void

This method returns an undefined value.

Writes the parallel line when the first parallel class starts on the outer reporter. The class object is known here, so an anonymous class (which Minitest records with no name) counts too. Minitest runs every serial class first, so a kill before this line came from a serial test. A test that runs a class on its own reporter is not the run's parallel phase, so it marks nothing.

Parameters:

  • klass (Class) —

    the test class starting its run.

  • reporter (Minitest::CompositeReporter) —

    the reporter it runs on.



203
204
205
206
207
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 203

def class_starting(klass, reporter)
  return unless channel && outer_reporter?(reporter) && parallel?(klass)

  LOCK.synchronize { mark_parallel }
end

.disarm! ⇒ void

This method returns an undefined value.

Clears the armed state.



185
186
187
188
189
190
191
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 185

def disarm!
  self.armed_pid = nil
  self.armed_reporter = nil
  self.channel = nil
  self.runnables = nil
  self.serial_failed = false
end

.finish! ⇒ void

This method returns an undefined value.

Writes the channel's end line once the suite has returned and the outer reporter has seen every test the loaded classes run. A return alone proves nothing: Minitest catches an Interrupt in a test and returns after the tests so far. Does nothing unless the recorder is armed in this process.



165
166
167
168
169
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 165

def finish!
  return unless channel && armed_here?

  KillChannel.write_end(channel) if seen == planned_tests
end

.mark_parallel ⇒ void (private)

This method returns an undefined value.

Writes the parallel line, once.



289
290
291
292
293
294
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 289

def mark_parallel
  return if parallel_marked

  self.parallel_marked = true
  KillChannel.write_parallel(channel)
end

.name_and_id(result, line) ⇒ Array(String, String) (private)

The test's name and id. Class#method is unique within a run, so the name is also the id. An anonymous class has no name (Minitest records nil), so its tests would all merge into #method; their id adds the line that defines the method, which no mutant changes. The test's file is already part of its identity, so the id holds no path.

Parameters:

  • result (Minitest::Result) —

    the result of one test.

  • line (Integer, nil) —

    the line that defines the test method.

Returns:

  • (Array(String, String)) —

    name and id.



279
280
281
282
283
284
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 279

def name_and_id(result, line)
  return ["#{result.klass}##{result.name}"] * 2 unless result.klass.to_s.empty?

  name = "(anonymous)##{result.name}"
  [name, "#{name}@#{line}"]
end

.parallel?(klass) ⇒ Boolean

True when the test class runs its tests in parallel threads (parallelize_me!). Minitest runs these after every serial class, and the stop at the first failure cannot skip the tests it already queued. Minitest 6 names the order run_order; Minitest 5 names it test_order.

Parameters:

  • klass (Class) —

    a loaded test class.

Returns:

  • (Boolean)


178
179
180
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 178

def parallel?(klass)
  %i[run_order test_order].any? { |order| klass.respond_to?(order) && klass.public_send(order) == :parallel }
end

.planned_tests ⇒ Integer? (private)

How many tests the loaded classes ran, or nil when it cannot be told. Asked after the run, because runnable_methods shuffles with the seed that Minitest.run sets.

Returns:

  • (Integer, nil)


301
302
303
304
305
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 301

def planned_tests
  runnables.sum { |klass| klass.runnable_methods.size }
rescue StandardError
  nil
end

.record(reporter, result) ⇒ void

This method returns an undefined value.

Sends one result recorded on reporter: a pass, or a kill for a failure or an error, or an unavailable database outcome. Skips send no test line, though they count as seen, and results on other reporters count for nothing. The parallel line goes out when the first parallel class starts (see class_starting), so a kill that precedes it came from a serial test.

Parameters:

  • reporter (Minitest::CompositeReporter) —

    the recording reporter.

  • result (Minitest::Result) —

    the result of one test.



235
236
237
238
239
240
241
242
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 235

def record(reporter, result)
  return unless channel && outer_reporter?(reporter)

  LOCK.synchronize do
    self.seen += 1
    send_result(result)
  end
end

.send_result(result) ⇒ void (private)

This method returns an undefined value.

Writes one result's line. A recorder that raised would change the verdict, so a result it cannot describe (a Minitest without source_location or klass on its results) is a lost line, which keeps the row incomplete, as on the RSpec side.

Parameters:

  • result (Minitest::Result) —

    the result of one test.



253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 253

def send_result(result)
  return if result.skipped?

  event = if result.passed?
            KillChannel::PASS
          elsif InTestDatabaseError.database_error(result.failures.first)
            KillChannel::UNAVAILABLE
          else
            KillChannel::KILL
          end
  self.serial_failed = true if event != KillChannel::PASS && !parallel_marked
  file, line = Array(result.source_location)
  KillChannel.write(channel, event, file, *name_and_id(result, line))
rescue StandardError
  KillChannel.write_lost(channel)
end

.skipping ⇒ Object

Runs the block between skip and unskip when a serial test has failed in this armed process, else just runs it. No ensure: a block that never returns (an exit, a crash) must leave its region open, so the parent knows the child ended where a run without --matrix would not have been.

Yield Returns:

  • (Object)

Returns:

  • (Object) —

    the block's value.



217
218
219
220
221
222
223
# File 'lib/mutineer/minitest_integration/kill_recorder.rb', line 217

def skipping
  open = channel && serial_failed && armed_here?
  KillChannel.write_skip(channel) if open
  value = yield
  KillChannel.write_unskip(channel) if open
  value
end