Class: Mutineer::TestRunners::RSpec::KillFormatter

Inherits:
Object
  • Object
show all
Defined in:
lib/mutineer/test_runners/rspec.rb

Overview

Sends each example's outcome to the channel of a --matrix run. A pending or skipped example sends nothing. A test is its spec file, its full description (the name reports show) and its example id, which tells apart examples that share a description and stays the same when a mutant changes a generated description. Example ids need RSpec 3.3 or later; before that no id tells apart examples on one line, so an older RSpec sends no start and every row stays incomplete.

It writes only from the process that created it, so an example that runs an RSpec suite in a forked process cannot add lines. It writes start when the run begins with fail-fast off; a run that could stop early sends no start, and its rows stay incomplete. It also registers an after(:suite) hook that writes cleanup ahead of the suite's own hooks.

After the first failure, a run without --matrix (--fail-fast) skips each later example and each example group that starts later, hooks included, but it still runs the after(:all) hooks of the groups it is in. So once an example has failed, the formatter writes skip when a later example or group starts and unskip when it finishes (see KillChannel).

Class Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(channel) ⇒ KillFormatter

Returns a new instance of KillFormatter.

Parameters:

  • channel (IO) —

    the write end of the channel.



260
261
262
263
264
265
266
267
268
269
# File 'lib/mutineer/test_runners/rspec.rb', line 260

def initialize(channel)
  @channel = channel
  @pid = Process.pid
  @failed = false
  @example_skipping = false
  @groups_skipping = []
  @planned = nil
  @reported = 0
  self.class.last = self
end

Class Attribute Details

.last ⇒ KillFormatter?

The formatter of the latest run in this process.

Returns:



246
247
248
# File 'lib/mutineer/test_runners/rspec.rb', line 246

def last
  @last
end

Class Method Details

.registered ⇒ Class

Registers the formatter with RSpec for the notifications it handles. Called at run time, because rspec-core is not loaded with Mutineer.

Returns:

  • (Class) —

    this class.



253
254
255
256
257
# File 'lib/mutineer/test_runners/rspec.rb', line 253

def self.registered
  ::RSpec::Core::Formatters.register(self, :start, :example_started, :example_passed, :example_failed,
                                     :example_pending, :example_group_started, :example_group_finished)
  self
end

Instance Method Details

#cleanup_begins ⇒ void

This method returns an undefined value.

Sends cleanup: every example has run, and the suite's own hooks follow.



299
300
301
# File 'lib/mutineer/test_runners/rspec.rb', line 299

def cleanup_begins
  KillChannel.write_cleanup(@channel) if owner?
end

#example_event(example) ⇒ String (private)

KillChannel::UNAVAILABLE when the example's own exception is a database error, otherwise KillChannel::KILL. A fault here is a kill: recording nothing would hide a failure, and raising would change the verdict.

Parameters:

  • example (RSpec::Core::Example) —

    the failed example.

Returns:

  • (String)


400
401
402
403
404
405
406
407
408
409
410
# File 'lib/mutineer/test_runners/rspec.rb', line 400

def example_event(example)
  error = example.exception if example.respond_to?(:exception)
  error = Mutineer::TestRunners::RSpec.primary_exception(error)
  if error && Mutineer::MinitestIntegration::InTestDatabaseError.database_exception?(error)
    KillChannel::UNAVAILABLE
  else
    KillChannel::KILL
  end
rescue StandardError
  KillChannel::KILL
end

#example_failed(notification) ⇒ void

This method returns an undefined value.

Sends a kill.

Parameters:

  • notification (RSpec::Core::Notifications::FailedExampleNotification)


327
328
329
330
331
332
# File 'lib/mutineer/test_runners/rspec.rb', line 327

def example_failed(notification)
  example = notification.example
  send_event(example_event(example), example)
  example_finished
  @failed = true
end

#example_finished ⇒ void (private)

This method returns an undefined value.

Counts an example's outcome and closes its skip region, if any.



383
384
385
386
387
388
389
390
391
# File 'lib/mutineer/test_runners/rspec.rb', line 383

def example_finished
  return unless owner?

  @reported += 1
  return unless @example_skipping

  @example_skipping = false
  KillChannel.write_unskip(@channel)
end

#example_group_finished(_notification) ⇒ void

This method returns an undefined value.

Closes the group's skip region, if it opened one. RSpec sends this after the group's after(:all) hooks, from an ensure, so it also comes while an exit or a crash unwinds the group. Then $! holds an exception that was not in flight when the group started, and the region stays open: the run ended inside it.

Parameters:

  • _notification (RSpec::Core::Notifications::GroupNotification)


364
365
366
367
368
369
# File 'lib/mutineer/test_runners/rspec.rb', line 364

def example_group_finished(_notification)
  return unless owner?

  opened, in_flight = @groups_skipping.pop
  KillChannel.write_unskip(@channel) if opened && $!.equal?(in_flight)
end

#example_group_started(_notification) ⇒ void

This method returns an undefined value.

Opens a skip region for a group that starts after a failure.

Parameters:

  • _notification (RSpec::Core::Notifications::GroupNotification)


346
347
348
349
350
351
352
353
354
# File 'lib/mutineer/test_runners/rspec.rb', line 346

def example_group_started(_notification)
  return unless owner?

  # The exception in flight now: RSpec can start and finish a group
  # inside its own `rescue` (a failed `before(:all)` with nested
  # groups), where `$!` is already set.
  @groups_skipping.push([@failed, $!])
  KillChannel.write_skip(@channel) if @failed
end

#example_passed(notification) ⇒ void

This method returns an undefined value.

Sends a pass.

Parameters:

  • notification (RSpec::Core::Notifications::ExampleNotification)


318
319
320
321
# File 'lib/mutineer/test_runners/rspec.rb', line 318

def example_passed(notification)
  send_event(KillChannel::PASS, notification.example)
  example_finished
end

#example_pending(_notification) ⇒ void

This method returns an undefined value.

A pending or skipped example sends no test line, but it finishes.

Parameters:

  • _notification (RSpec::Core::Notifications::ExampleNotification)


338
339
340
# File 'lib/mutineer/test_runners/rspec.rb', line 338

def example_pending(_notification)
  example_finished
end

#example_started(_notification) ⇒ void

This method returns an undefined value.

Opens a skip region for an example that starts after a failure.

Parameters:

  • _notification (RSpec::Core::Notifications::ExampleNotification)


307
308
309
310
311
312
# File 'lib/mutineer/test_runners/rspec.rb', line 307

def example_started(_notification)
  return unless owner? && @failed

  @example_skipping = true
  KillChannel.write_skip(@channel)
end

#identify(example) ⇒ String (private)

What tells example apart from the others, and stays the same across mutants: its id (RSpec 3.3 and later; see #start).

Parameters:

  • example (RSpec::Core::Example) —

    the example.

Returns:

  • (String)


433
434
435
# File 'lib/mutineer/test_runners/rspec.rb', line 433

def identify(example)
  example.id
end

#owner? ⇒ Boolean (private)

True in the process that created the formatter.

Returns:

  • (Boolean)


376
377
378
# File 'lib/mutineer/test_runners/rspec.rb', line 376

def owner?
  Process.pid == @pid
end

#saw_every_example? ⇒ Boolean

True when RSpec reported an outcome for every example it planned to run.

Returns:

  • (Boolean)


274
275
276
# File 'lib/mutineer/test_runners/rspec.rb', line 274

def saw_every_example?
  !@planned.nil? && @reported == @planned
end

#send_event(event, example) ⇒ void (private)

This method returns an undefined value.

Writes one test line for example.

Parameters:



418
419
420
421
422
423
424
425
426
# File 'lib/mutineer/test_runners/rspec.rb', line 418

def send_event(event, example)
  return unless owner?

  file = example.[:absolute_file_path] || File.expand_path(example.file_path)
  KillChannel.write(@channel, event, file, example.full_description, identify(example))
rescue StandardError
  # A recorder that raised would change the verdict; the lost line keeps the row incomplete.
  KillChannel.write_lost(@channel)
end

#start(notification) ⇒ void

This method returns an undefined value.

Sends start, unless fail-fast is on or RSpec has no example ids, and registers the hook that sends cleanup. RSpec runs after(:suite) hooks last-defined first, and this runs once the spec files have loaded, so the hook goes before every hook the suite defines.

Parameters:

  • notification (RSpec::Core::Notifications::StartNotification)


285
286
287
288
289
290
291
292
293
294
# File 'lib/mutineer/test_runners/rspec.rb', line 285

def start(notification)
  return unless owner?
  return if ::RSpec.configuration.fail_fast
  return unless ::RSpec::Core::Example.method_defined?(:id)

  @planned = notification.count
  KillChannel.write_start(@channel)
  formatter = self
  ::RSpec.configuration.after(:suite) { formatter.cleanup_begins }
end