Module: Mutineer::DatabaseUrls

Defined in:
lib/mutineer/database_urls.rb

Overview

DATABASE_URL values stored in a project's dotenv files. The tool process does not see a URL the app loads itself during boot. Reading the files lets redaction hide that password (#341). Stdlib only, so the app-side daemon can load this file.

Constant Summary collapse

FILE_NAMES =

Files dotenv-rails reads. Later files override at runtime. Every value is kept: hiding a password the running app did not use is safe.

%w[
  .env
  .env.local
  .env.development
  .env.development.local
  .env.test
  .env.test.local
].freeze

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.closing_quote(raw, quote) ⇒ Integer?

Index of the closing quote. A backslash escapes the next character inside double quotes, so \" does not end the value.

Parameters:

  • raw (String)
  • quote (String)

Returns:

  • (Integer, nil)


166
167
168
169
170
171
172
173
174
175
176
177
178
# File 'lib/mutineer/database_urls.rb', line 166

def closing_quote(raw, quote)
  index = 1
  while index < raw.length
    if quote == '"' && raw[index] == "\\"
      index += 2
    elsif raw[index] == quote
      return index
    else
      index += 1
    end
  end
  nil
end

.decode_double(text) ⇒ String

Double-quoted dotenv escapes this reader understands. Other backslashes stay as written.

Parameters:

  • text (String)

Returns:

  • (String)


185
186
187
# File 'lib/mutineer/database_urls.rb', line 185

def decode_double(text)
  text.gsub(/\\[nrt"\\]/, "\\n" => "\n", "\\r" => "\r", "\\t" => "\t", "\\\"" => '"', "\\\\" => "\\")
end

.expand(value, vars) ⇒ String

Replace ${NAME} and $NAME. A backslash before $ stays a dollar and is not a reference. The environment wins over an earlier assignment in the file. Anything else is left as written.

Parameters:

  • value (String)
  • vars (Hash{String => String})

Returns:

  • (String)


133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
# File 'lib/mutineer/database_urls.rb', line 133

def expand(value, vars)
  value.gsub(/\\\$|\$\{([A-Za-z_][A-Za-z0-9_]*)\}|\$([A-Za-z_][A-Za-z0-9_]*)/) do
    match = Regexp.last_match
    next "$" if match[0] == "\\$"

    name = match[1] || match[2]
    if ENV.key?(name)
      ENV[name].to_s
    elsif vars.key?(name)
      vars[name]
    else
      match[0]
    end
  end
end

.from_file(path) ⇒ Array<String>

DATABASE_URL values in one file. Every line is read. A double-quoted value may continue on later lines until its closing quote. A quote that never closes is a plain value on its own line, as in dotenv. ${NAME} and $NAME expand from an earlier assignment in the file, or from the environment when that variable is already set. A backslash before $ is a literal dollar. A single-quoted value is not expanded. An invalid byte is replaced and does not stop the read. A directory, a missing path, and an unreadable file add nothing. An unknown $NAME is left as written, so that reference does not become a password.

Parameters:

  • path (String)

Returns:

  • (Array<String>)


46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
# File 'lib/mutineer/database_urls.rb', line 46

def from_file(path)
  return [] unless File.file?(path)

  vars = {}
  urls = []
  lines = File.open(path, "rb") { |io| io.each_line.map { |line| utf8_line(line) } }
  index = 0
  while index < lines.size
    stop = index
    chunk = lines[index]
    parsed = parse_assignment(chunk)
    while parsed == :open && stop + 1 < lines.size
      stop += 1
      chunk += lines[stop]
      parsed = parse_assignment(chunk)
    end
    if parsed == :open
      # A quote that never closes: dotenv reads that line as a plain
      # value, and the lines after it as their own assignments.
      stop = index
      parsed = parse_assignment(lines[index], plain: true)
    end
    index = stop + 1
    next unless parsed.is_a?(Array)

    key, raw, expand_value = parsed
    value = expand_value ? expand(raw, vars) : raw
    vars[key] = value
    urls << value if key == "DATABASE_URL" && !value.empty?
  end
  urls
rescue SystemCallError
  []
end

.from_files(root) ⇒ Array<String>

DATABASE_URL values from the dotenv files under root. Missing files add nothing. An unreadable file adds nothing: a permission error must not stop a run. Estimate: a project dotenv file is readable by the user who starts the run.

Parameters:

  • root (String, nil) —

    project directory.

Returns:

  • (Array<String>)


28
29
30
31
32
# File 'lib/mutineer/database_urls.rb', line 28

def from_files(root)
  return [] if root.nil? || root.empty?

  FILE_NAMES.flat_map { |name| from_file(File.join(root, name)) }.uniq
end

.line_url(line) ⇒ String?

The DATABASE_URL value on one dotenv line, or nil when the line is a comment, another key, or an empty value.

Parameters:

  • line (String)

Returns:

  • (String, nil)


86
87
88
89
90
91
# File 'lib/mutineer/database_urls.rb', line 86

def line_url(line)
  parsed = parse_assignment(line)
  return nil unless parsed.is_a?(Array) && parsed[0] == "DATABASE_URL"

  parsed[1].empty? ? nil : parsed[1]
end

.parse_assignment(line, plain: false) ⇒ Array(String, String, Boolean), ...

One KEY=value assignment. The third element is false for a single-quoted value, which dotenv does not expand. :open means a quoted value has not reached its closing quote yet. plain reads a leading quote as part of an unquoted value.

Parameters:

  • line (String)
  • plain (Boolean) (defaults to: false)

Returns:

  • (Array(String, String, Boolean), Symbol, nil)


101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
# File 'lib/mutineer/database_urls.rb', line 101

def parse_assignment(line, plain: false)
  text = utf8_line(line).sub(/\A\uFEFF/, "").strip
  return nil if text.empty? || text.start_with?("#")

  text = text.sub(/\Aexport\s+/, "")
  match = text.match(/\A([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)\z/m)
  return nil unless match

  key = match[1]
  raw = match[2]
  if !plain && raw.start_with?("'")
    stop = closing_quote(raw, "'")
    return :open unless stop

    [key, raw[1...stop], false]
  elsif !plain && raw.start_with?('"')
    stop = closing_quote(raw, '"')
    return :open unless stop

    [key, decode_double(raw[1...stop]), true]
  else
    [key, raw.sub(/\s+#.*\z/, "").strip, true]
  end
end

.utf8_line(line) ⇒ String

One line as UTF-8. An invalid byte becomes U+FFFD so a binary dotenv line cannot raise before redaction starts.

Parameters:

  • line (String)

Returns:

  • (String)


154
155
156
157
158
# File 'lib/mutineer/database_urls.rb', line 154

def utf8_line(line)
  text = line.to_s.dup
  text.force_encoding(Encoding::UTF_8)
  text.scrub
end

Instance Method Details

#closing_quote(raw, quote) ⇒ Integer? (private)

Index of the closing quote. A backslash escapes the next character inside double quotes, so \" does not end the value.

Parameters:

  • raw (String)
  • quote (String)

Returns:

  • (Integer, nil)


166
167
168
169
170
171
172
173
174
175
176
177
178
# File 'lib/mutineer/database_urls.rb', line 166

def closing_quote(raw, quote)
  index = 1
  while index < raw.length
    if quote == '"' && raw[index] == "\\"
      index += 2
    elsif raw[index] == quote
      return index
    else
      index += 1
    end
  end
  nil
end

#decode_double(text) ⇒ String (private)

Double-quoted dotenv escapes this reader understands. Other backslashes stay as written.

Parameters:

  • text (String)

Returns:

  • (String)


185
186
187
# File 'lib/mutineer/database_urls.rb', line 185

def decode_double(text)
  text.gsub(/\\[nrt"\\]/, "\\n" => "\n", "\\r" => "\r", "\\t" => "\t", "\\\"" => '"', "\\\\" => "\\")
end

#expand(value, vars) ⇒ String (private)

Replace ${NAME} and $NAME. A backslash before $ stays a dollar and is not a reference. The environment wins over an earlier assignment in the file. Anything else is left as written.

Parameters:

  • value (String)
  • vars (Hash{String => String})

Returns:

  • (String)


133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
# File 'lib/mutineer/database_urls.rb', line 133

def expand(value, vars)
  value.gsub(/\\\$|\$\{([A-Za-z_][A-Za-z0-9_]*)\}|\$([A-Za-z_][A-Za-z0-9_]*)/) do
    match = Regexp.last_match
    next "$" if match[0] == "\\$"

    name = match[1] || match[2]
    if ENV.key?(name)
      ENV[name].to_s
    elsif vars.key?(name)
      vars[name]
    else
      match[0]
    end
  end
end

#from_file(path) ⇒ Array<String> (private)

DATABASE_URL values in one file. Every line is read. A double-quoted value may continue on later lines until its closing quote. A quote that never closes is a plain value on its own line, as in dotenv. ${NAME} and $NAME expand from an earlier assignment in the file, or from the environment when that variable is already set. A backslash before $ is a literal dollar. A single-quoted value is not expanded. An invalid byte is replaced and does not stop the read. A directory, a missing path, and an unreadable file add nothing. An unknown $NAME is left as written, so that reference does not become a password.

Parameters:

  • path (String)

Returns:

  • (Array<String>)


46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
# File 'lib/mutineer/database_urls.rb', line 46

def from_file(path)
  return [] unless File.file?(path)

  vars = {}
  urls = []
  lines = File.open(path, "rb") { |io| io.each_line.map { |line| utf8_line(line) } }
  index = 0
  while index < lines.size
    stop = index
    chunk = lines[index]
    parsed = parse_assignment(chunk)
    while parsed == :open && stop + 1 < lines.size
      stop += 1
      chunk += lines[stop]
      parsed = parse_assignment(chunk)
    end
    if parsed == :open
      # A quote that never closes: dotenv reads that line as a plain
      # value, and the lines after it as their own assignments.
      stop = index
      parsed = parse_assignment(lines[index], plain: true)
    end
    index = stop + 1
    next unless parsed.is_a?(Array)

    key, raw, expand_value = parsed
    value = expand_value ? expand(raw, vars) : raw
    vars[key] = value
    urls << value if key == "DATABASE_URL" && !value.empty?
  end
  urls
rescue SystemCallError
  []
end

#from_files(root) ⇒ Array<String> (private)

DATABASE_URL values from the dotenv files under root. Missing files add nothing. An unreadable file adds nothing: a permission error must not stop a run. Estimate: a project dotenv file is readable by the user who starts the run.

Parameters:

  • root (String, nil) —

    project directory.

Returns:

  • (Array<String>)


28
29
30
31
32
# File 'lib/mutineer/database_urls.rb', line 28

def from_files(root)
  return [] if root.nil? || root.empty?

  FILE_NAMES.flat_map { |name| from_file(File.join(root, name)) }.uniq
end

#line_url(line) ⇒ String? (private)

The DATABASE_URL value on one dotenv line, or nil when the line is a comment, another key, or an empty value.

Parameters:

  • line (String)

Returns:

  • (String, nil)


86
87
88
89
90
91
# File 'lib/mutineer/database_urls.rb', line 86

def line_url(line)
  parsed = parse_assignment(line)
  return nil unless parsed.is_a?(Array) && parsed[0] == "DATABASE_URL"

  parsed[1].empty? ? nil : parsed[1]
end

#parse_assignment(line, plain: false) ⇒ Array(String, String, Boolean), ... (private)

One KEY=value assignment. The third element is false for a single-quoted value, which dotenv does not expand. :open means a quoted value has not reached its closing quote yet. plain reads a leading quote as part of an unquoted value.

Parameters:

  • line (String)
  • plain (Boolean) (defaults to: false)

Returns:

  • (Array(String, String, Boolean), Symbol, nil)


101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
# File 'lib/mutineer/database_urls.rb', line 101

def parse_assignment(line, plain: false)
  text = utf8_line(line).sub(/\A\uFEFF/, "").strip
  return nil if text.empty? || text.start_with?("#")

  text = text.sub(/\Aexport\s+/, "")
  match = text.match(/\A([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)\z/m)
  return nil unless match

  key = match[1]
  raw = match[2]
  if !plain && raw.start_with?("'")
    stop = closing_quote(raw, "'")
    return :open unless stop

    [key, raw[1...stop], false]
  elsif !plain && raw.start_with?('"')
    stop = closing_quote(raw, '"')
    return :open unless stop

    [key, decode_double(raw[1...stop]), true]
  else
    [key, raw.sub(/\s+#.*\z/, "").strip, true]
  end
end

#utf8_line(line) ⇒ String (private)

One line as UTF-8. An invalid byte becomes U+FFFD so a binary dotenv line cannot raise before redaction starts.

Parameters:

  • line (String)

Returns:

  • (String)


154
155
156
157
158
# File 'lib/mutineer/database_urls.rb', line 154

def utf8_line(line)
  text = line.to_s.dup
  text.force_encoding(Encoding::UTF_8)
  text.scrub
end