Module: Mutineer::KillChannel

Defined in:
lib/mutineer/kill_channel.rb

Overview

The line protocol a --matrix mutant child uses to tell its parent how its tests went. One JSON array per line:

["start"]                         the recorder is armed; once, first
["pass", file, name, id]          a test passed against the mutant
["kill", file, name, id]          a test failed or errored: it killed it
["unavailable", file, name, id]   a test ran but its database failed
["parallel"]                      the run reached its parallel tests; once
["skip"]                          what runs next, a run without --matrix
                                skips (see below)
["unskip"]                        the matching end of a ["skip"]
["cleanup"]                       every test ran; suite hooks follow; once
["lost"]                          a test the recorder could not describe
["end"]                           the suite returned with every test seen;
                                once, last

id tells two tests apart when [file, name] does not, and it is what identifies the test across mutants (an RSpec example id such as ./spec/calc_spec.rb[1:2]; the name can change with the mutant). For Minitest it equals the name. A test line is written when the test is recorded, so a child killed at the timeout has already sent every kill it saw. A skipped test sends nothing.

Minitest runs its serial test classes first and its parallelize_me! classes after them, and the stop at the first failure of a run without --matrix skips everything after a serial failure. parallel marks where the parallel tests begin, so the parent can tell a kill that the stop would have followed from one it could not. cleanup marks where the tests end and the suite's own hooks begin (RSpec after(:suite)). It decides no verdict (the skip regions do); it only marks a later test line as out of order.

After the first serial failure, a run without --matrix skips every later test and every later test class or example group, but it still runs the rest of the code around the failing test: the remainder of its Minitest class run wrapper, the after(:all) hooks of its RSpec groups, and the suite hooks. The recorder writes skip as it enters something that run would skip and unskip as that returns, so a child that exits, crashes or hangs inside a skipped region is one the plain run would have stopped before (see Isolation.finish). Its first failure decides the verdict: killed for a real kill, error for an unavailable database outcome. The pairs nest. An unavailable result also leaves the row incomplete.

A row is complete only with start and end, in order, and no lost line: a test that exits the process, a crash, an interrupted run or a recorder that never armed leaves one out. A stream out of order (a duplicate marker, a test line before start or after end) is invalid, and the parent trusts nothing from it.

Writing never raises: a recorder that broke the test run would change the verdict, and the exit status decides the verdict. A lost line leaves the row incomplete. After one write to a channel fails, nothing more is written to it, so a later end can never make a row with a missing line look complete.

Stdlib-only, so the app-side daemon can load the files that require it.

Defined Under Namespace

Classes: Report

Constant Summary collapse

PASS =

Event for a test that passed against the mutant.

"pass"
KILL =

Event for a test that failed or errored against the mutant: it killed it.

"kill"
UNAVAILABLE =

Event for a test whose database failed, so it provides no mutation verdict.

"unavailable"
START =

Event that opens a recorded run.

"start"
PARALLEL =

Event that marks the first parallel test of a run.

"parallel"
CLEANUP =

Event that marks the end of the tests and the start of the suite's cleanup.

"cleanup"
SKIP =

Event that opens a region a run without --matrix would skip.

"skip"
UNSKIP =

Event that closes the latest open SKIP region.

"unskip"
LOST =

Event for a test the recorder could not describe.

"lost"
FINISH =

Event that closes a run whose suite returned with every test seen.

"end"
LOCK =

Serializes writes from tests that record on several threads.

Mutex.new

Class Method Summary collapse

Class Method Details

.apply(report, fields) ⇒ void (private)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Adds one parsed line to report. A line out of order marks the report invalid; the line still counts, since what it names happened.

Parameters:

  • report (Report) —

    the report being built.

  • fields (Array<String>) —

    a valid line's fields.



196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
# File 'lib/mutineer/kill_channel.rb', line 196

def self.apply(report, fields)
  case fields[0]
  when START
    report.invalid = true if report.started || report.ran.any? || report.parallel || report.cleanup || report.finished
    report.started = true
  when PARALLEL
    report.invalid = true unless open?(report) && !report.parallel && !report.cleanup
    report.parallel = true
  when SKIP
    report.invalid = true unless open?(report) && !report.cleanup
    report.skipping += 1
  when UNSKIP
    report.invalid = true unless open?(report) && !report.cleanup && report.skipping.positive?
    report.skipping -= 1 if report.skipping.positive?
  when CLEANUP
    report.invalid = true unless open?(report) && !report.cleanup
    report.cleanup = true
  when LOST then report.lost += 1
  when FINISH
    report.invalid = true unless open?(report) && !report.finished
    report.finished = true
  else
    report.invalid = true unless open?(report) && !report.cleanup
    test = fields[1, 3]
    report.ran << test
    if fields[0] == UNAVAILABLE
      report.lost += 1
      report.serial_error = true unless report.parallel || report.serial_kill
    elsif fields[0] == KILL
      report.killed << test
      report.serial_kill = true unless report.parallel || report.serial_error
    end
  end
end

.emit(io, fields) ⇒ void (private)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Writes one line. Never raises. After a write to io fails, writes nothing more to it.

Parameters:

  • io (IO) —

    the write end of the channel.

  • fields (Array<String>) —

    the line's fields.



266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
# File 'lib/mutineer/kill_channel.rb', line 266

def self.emit(io, fields)
  line = "#{JSON.generate(fields)}\n"
  LOCK.synchronize do
    next if @broken.equal?(io)

    begin
      io.write(line)
    rescue StandardError
      @broken = io
    end
  end
  nil
rescue StandardError
  nil
end

.open?(report) ⇒ Boolean (private)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

True between start and end.

Parameters:

  • report (Report) —

    the report being built.

Returns:

  • (Boolean)


236
237
238
# File 'lib/mutineer/kill_channel.rb', line 236

def self.open?(report)
  report.started && !report.finished
end

.parse(buffer) ⇒ Report

Reads every complete line of buffer.

Parameters:

  • buffer (String) —

    bytes read from the channel.

Returns:



177
178
179
180
181
182
183
184
185
186
187
# File 'lib/mutineer/kill_channel.rb', line 177

def self.parse(buffer)
  report = Report.new(killed: [], ran: [], started: false, finished: false, parallel: false, cleanup: false,
                      serial_kill: false, serial_error: false, skipping: 0, invalid: false, lost: 0)
  buffer.dup.force_encoding(Encoding::UTF_8).each_line do |line|
    fields = line.end_with?("\n") ? parse_line(line) : nil
    fields ? apply(report, fields) : report.lost += 1
  end
  report.killed = report.killed.uniq.sort
  report.ran = report.ran.uniq.sort
  report
end

.parse_line(line) ⇒ Array<String>? (private)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Parses one line into its fields, or nil when it is not a valid line.

Parameters:

  • line (String) —

    one newline-terminated line.

Returns:

  • (Array<String>, nil)


245
246
247
248
249
250
251
252
253
254
255
256
257
# File 'lib/mutineer/kill_channel.rb', line 245

def self.parse_line(line)
  fields = JSON.parse(line)
  return unless fields.is_a?(Array) && fields.all?(String)

  valid =
    case fields[0]
    when PASS, KILL, UNAVAILABLE then fields.size == 4
    when START, PARALLEL, SKIP, UNSKIP, CLEANUP, LOST, FINISH then fields.size == 1
    end
  fields if valid
rescue JSON::ParserError
  nil
end

.utf8(value) ⇒ String (private)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

A UTF-8 copy of value, with invalid bytes replaced, so JSON can encode it.

Parameters:

  • value (Object) —

    a file path or test name.

Returns:

  • (String)


287
288
289
290
291
# File 'lib/mutineer/kill_channel.rb', line 287

def self.utf8(value)
  str = value.to_s
  str = str.encode(Encoding::UTF_8, invalid: :replace, undef: :replace) unless str.encoding == Encoding::UTF_8
  str.scrub
end

.write(io, event, file, name, id = name) ⇒ void

This method returns an undefined value.

Writes one test line.

Parameters:

  • io (IO) —

    the write end of the channel.

  • event (String) —
  • file (String) —

    the file that defines the test.

  • name (String) —

    the test's display name, e.g. CalculatorTest#test_add.

  • id (String) (defaults to: name) —

    what tells the test apart from others with its file and name; defaults to the name.



113
114
115
# File 'lib/mutineer/kill_channel.rb', line 113

def self.write(io, event, file, name, id = name)
  emit(io, [event, utf8(file), utf8(name), utf8(id)])
end

.write_cleanup(io) ⇒ void

This method returns an undefined value.

Writes the cleanup line.

Parameters:

  • io (IO) —

    the write end of the channel.



153
154
155
# File 'lib/mutineer/kill_channel.rb', line 153

def self.write_cleanup(io)
  emit(io, [CLEANUP])
end

.write_end(io) ⇒ void

This method returns an undefined value.

Writes the end line.

Parameters:

  • io (IO) —

    the write end of the channel.



169
170
171
# File 'lib/mutineer/kill_channel.rb', line 169

def self.write_end(io)
  emit(io, [FINISH])
end

.write_lost(io) ⇒ void

This method returns an undefined value.

Writes the lost line, for a test the recorder could not describe.

Parameters:

  • io (IO) —

    the write end of the channel.



161
162
163
# File 'lib/mutineer/kill_channel.rb', line 161

def self.write_lost(io)
  emit(io, [LOST])
end

.write_parallel(io) ⇒ void

This method returns an undefined value.

Writes the parallel line.

Parameters:

  • io (IO) —

    the write end of the channel.



129
130
131
# File 'lib/mutineer/kill_channel.rb', line 129

def self.write_parallel(io)
  emit(io, [PARALLEL])
end

.write_skip(io) ⇒ void

This method returns an undefined value.

Writes the skip line.

Parameters:

  • io (IO) —

    the write end of the channel.



137
138
139
# File 'lib/mutineer/kill_channel.rb', line 137

def self.write_skip(io)
  emit(io, [SKIP])
end

.write_start(io) ⇒ void

This method returns an undefined value.

Writes the start line.

Parameters:

  • io (IO) —

    the write end of the channel.



121
122
123
# File 'lib/mutineer/kill_channel.rb', line 121

def self.write_start(io)
  emit(io, [START])
end

.write_unskip(io) ⇒ void

This method returns an undefined value.

Writes the unskip line.

Parameters:

  • io (IO) —

    the write end of the channel.



145
146
147
# File 'lib/mutineer/kill_channel.rb', line 145

def self.write_unskip(io)
  emit(io, [UNSKIP])
end