Recent commits to derivepassphrase.git (b685fdd3a683a781d48c35832818f9c4e0f6790a)

Add hypothesis-based tests (and bugfix) for the vault derivation scheme

2024-09-27T19:05:16+02:00

There are currently three such property-based tests: a test for general
adherence to constraints (with small numbers), and two tests
specifically for length and repeat constraints (with larger numbers).
Combining these into a single test caused the test runs to vary wildly
in execution time, which is undesirable from a configuration point of
view (and also causes hypothesis to complain).

The hypothesis tests actually uncovered a bug in the API specification
of the vault derivation scheme: during generation, we may run out of
possible characters for the next letter due to repeat constraints, but
only realize this now.  This already generated a `ValueError`, but with
a confusing error message, and the documentation seemed to suggest that
all unsatisfiable constraints are already detected at construction time.
So now explicitly mention that constraint unsatisfiability may be
detected both in the `Vault` constructor and in `Vault.generate`, and
handle lower-lever `ValueError`s in `Vault.generate` explicitly too.
</pre>

Set up the "hypothesis" testing library

2024-09-27T17:32:46+02:00

"hypothesis" is a Python library for property-based testing, i.e. for
asserting that the output of the tested code satisfies some property by
generating valid test inputs and testing these.  For `derivepassphrase`
in particular, this means generating different passphrase constraint
settings and testing that the generated passphrases match these
constraints, no matter what constraints exactly are chosen (within
reason).  The hypothesis library provides methods to generate test input
(test function parameter values, or custom walks for state machine-like
systems) and, for failing examples, shrink those to shorter/simpler ones
exhibiting the same error.

By its nature, hypothesis runs unit test functions multiple times with
different inputs, and so takes much longer than normal unit tests.  (In
the default configuration, 200 examples per test are run.)  It is
therefore sensible to set up alternate testing profiles with more or
less examples per test, and to include a mechanism in the test runner to
switch to these profiles.  The hypothesis documentation provides an
explicit example for profile setup for pytest use, plus three sensible
configurations "ci", "dev" and "debug", which I copied.  The pytest
command-line as called by `hatch test` is also straightforward to adapt:
add `--hypothesis-profile=...` to all pytest invocations.  However, the
shape of the `hatch test` environment matrix no longer matches the
reality of scenarios we'd like to test (because we added a new
dimension: slow, thorough testing, or fast testing).  So use this
opportunity to also clean up the matrix.
</pre>

Merge topic branch 'issue12-ssh-agent-spawning' into master

2024-09-22T19:23:07+02:00

* t/issue12-ssh-agent-spawning:
  Remove debugging-only code and add missing docstring in pytest fixtures
</pre>

Remove debugging-only code and add missing docstring in pytest fixtures

2024-09-22T19:19:38+02:00

</pre>

Merge topic branch 'issue12-ssh-agent-spawning' into master

2024-09-22T18:40:33+02:00

* t/issue12-ssh-agent-spawning:
  Add Changelog entry for the test suite SSH agent spawner
  Add test fixture for manually spawning known SSH agents
  Retire non-repeatability check for unsuitable SSH keys in the tests
  Add principal support for uploading SSH keys to the agent
  Simplify some SSH agent key uploading tests
  Support passing expected SSH agent response codes

GitHub: Closes #12.
</pre>

Add Changelog entry for the test suite SSH agent spawner

2024-09-22T18:36:19+02:00

</pre>

Add test fixture for manually spawning known SSH agents

2024-09-22T16:37:58+02:00

Include pytest fixtures to spawn known SSH agents and interface with
the running SSH agent, in an agent-agnostic way.  Move the key loading
parts from the test functions into the test fixtures.

This generally makes the actual test functions somewhat cleaner and
easier to read, but because the monkeypatch fixture interferes with
these new fixtures, the net improvements to readibility are only
moderate.  The test functions do however profit directly from
reduced copy-and-paste in the key loading part.

Pageant is one of the supported agents, and it behaves markedly
differently than OpenSSH's agent.  In particular, Pageant does not
support adding keys with constraints or re-submitting a key it is
already holding, and *every* key type Pageant currently offers yields
a deterministic signature.  Furthermore, a bug concerning output
buffering in Pageant 0.81 and lower currently makes it impossible to use
Pageant as a subprocess properly without correctly guessing the socket
address.  (This has already been reported upstream.) On the other hand,
Pageant supports ed448 keys, which OpenSSH doesn't.  So, implementing
support for Pageant was very valuable to highlight areas where the code
made unreasonable assumptions about SSH agent behavior, in particular
the availability and behavior of the system SSH agent service.

The new fixtures live in `tests/conftest.py`, following a relevant
pytest convention.  The fixtures themselves are necessarily platform-
and runtime-dependent, so even though they are test code that should be
included in test coverage, all parts dealing with querying the system,
spawning programs, error handling related to the former and ensuring
a certain functionality is available (or skipping the test otherwise)
are excluded from test coverage.  In particular, this includes the
entire fixture to ensure a running agent, and the cleanup part of the
agent with loaded keys fixture; I have however tried various
constellations by hand to ensure the code works if certain agents are
available or unavailable.
</pre>

Retire non-repeatability check for unsuitable SSH keys in the tests

2024-09-21T12:17:48+02:00

DSA and ECDSA keys use a nonce during signing, and it is well-known that
reusing the nonce for another signature allows the private key to be
derived directly from those two signatures.  Because of this, many
implementations choose the nonce via a high-quality random number
generator.  This leads to DSA and ECDSA signatures being non-repeatable,
i.e. signing the same document twice leads to two different
signatures/binary strings.  OpenSSH's agent behaves this way.

However, various implementations of DSA or DSA variants have attempted
to find a way to avoid the random number generator by choosing the nonce
deterministically (but still unpredictably, for an attacker): EdDSA
mandates a specific nonce as part of the specification, and RFC 6979
outlines a different deterministic nonce scheme for all (other) DSA
variants.  All versions of PuTTY/Pageant use deterministic nonce
generation (a homegrown system in 0.80 and lower, RFC 6979 afterwards),
so DSA and ECDSA signatures by Pageant *are* repeatable.  And there is
no reason why OpenSSH couldn't adopt RFC 6979 in the future.

Therefore, remove the check for repeatability in the tests.  The `Vault`
class check for key suitability remains unchanged, because while
DSA/ECDSA keys *can* use repeatable signatures, such use is not
*guaranteed*.
</pre>

Add principal support for uploading SSH keys to the agent

2024-09-21T12:10:20+02:00

Add the necessary protocol numbers to the `SSH_AGENT` and `SSH_AGENTC`
enums, and the necessary `private_key_blob` data to the
`tests.SUPPORTED_KEYS` and `tests.UNSUITABLE_KEYS` objects.

This suffices for key uploads without constraints, but adding a key
*with* constraints still requires knowledge about the encoding of the
constraints in the agent protocol.
</pre>

Simplify some SSH agent key uploading tests

2024-09-21T11:58:57+02:00

Remove unused test parameters, which were used solely for display
purposes to generate a readable test ID.  Instead of this, use the `ids`
parameter of `pytest.mark.parametrize` directly.
</pre>