Recent commits to derivepassphrase.git (b21188f464eb047ac53f122b2ec64eb8a8a383ac)

Reformat everything with ruff

2026-07-05T21:47:45+02:00

Linting and type checking passes, so nothing to do there.
</pre>

Update wishlist

2026-07-05T21:46:51+02:00

</pre>

Set the Python path correctly during pytest invocation

2026-07-05T21:31:22+02:00

Some of our tests spawn subprocesses, and without this, the subprocesses
fail to import code from the `tests` hierarchy (e.g., `tests.data`.)

We have so far sidestepped this issue by requiring `$PYTHONPATH` to be
set accordingly, but it turns out that setting this in the pytest
settings works even more reliably.

In the long run, it might be smarter to use a similar source layout
in `tests` as we already do in `src`: a top-level
`test_derivepassphrase` package, and the `data`, `machinery` and other
tests modules/packages as submodules/subpackages of
`test_derivepassphrase`.
</pre>

Merge branch 'economic-test-suite'

2026-07-05T16:57:37+02:00

This topic branch implements steps to make the test suite more
economical (i.e., to ensure a high value-to-code ratio in the test
suite) on the grounds that changes to the main codebase usually cause
changes to the test suite, and thus the test suite needs similar
management to the main codebase regarding technical debt.

 1. As the first big step, we divide our tests semantically into
    "regression tests" and "correctness tests", serving [the two main
    purposes of a test suite][TWO_KINDS]: fast feedback ("regression
    tests") vs. high coverage/confidence ("correctness tests").
    Specifically:

      - During regression testing, random input generation is turned
        down severely, currently to three examples.  In the long run,
        property-based tests without explicit examples will be skipped,
        and thus every non-correctness-only hypothesis test should have
        at least one explicit example.

      - Regression tests that "should" be correctness tests shall be
        implemented as property-based tests with explicit examples.
        Conversely, property-based tests that implement an exhaustive
        testing matrix over some parameter set shall be refactored such
        that the testing matrix is expressed with
        `pytest.mark.parametrize`.  (It is possible for a test to be
        a "combined" test using both `pytest.mark.parametrize` and
        `hypothesis.given`.)

 2. As a second step, avoid using "heavyweight" machinery if more
    "lightweight" machinery will do.  Specifically, avoid spawning or
    interfacing with an external SSH agent if the internal stub SSH
    agent works just as well, e.g., because the communication is being
    stubbed out anyway.

Beyond these two steps, add several quality-of-life improvements to the
test suite:

  - To convert `parametrize`'d tests to hypothesis tests, add
    a converter function that takes the same arguments as the
    `parametrize` mark and decorates the respective function with the
    equivalent explicit examples (and, if not yet a hypothesis test,
    also synthesizes a `hypothesis.given`).

  - Fix the stub SSH agent to behave more like a real communication
    socket (incomplete messages, EBADF after closing), thus also making
    it more suitable to model actual connection problems.

  - Clean up multiple "magic strings" in the test suite by introducing
    appropriate enums.

  - Move some common setup code or parametrization into the tests
    package.

  - Update the tooling: update `ruff`, and use `pyrefly` in addition to
    `mypy`.

[TWO_KINDS]: https://blog.nelhage.com/post/two-kinds-of-testing/ "Nelson Elhage: Two kinds of testing"

* economic-test-suite: (28 commits)
  Restore the `test_hash_length_expansion`/`test_wrong_initial_hash_length` vault test
  Fix some test coverage slipups
  Define `use_stub_agent`/`use_stub_agent_with_address` in conftest
  Add a heavy-duty test for Sequin internals
  Implement missing agent system support via tests.data.SystemSupportAction
  Move some setup "action" classes into tests.data
  Turn some parametrized unit tests into nominal property-based tests
  Rewrite the vault CLI tests for config management to use the stub agent
  Fix tiny mistakes across the test suite
  Update tooling (mypy/pyrefly, mkdocstrings overrides)
  Run the vault utility `key_to_phrase` tests against the stub SSH agent
  Add a pytest marker for external SSH agent use
  Run vault CLI config management tests against the stub agent where possible
  Use the stub agent for SSH agent client communications faking
  Harmonize tests for missing SSH agent communication support
  Reformat recent test changes for consistent style
  Type generators explicitly as generators, not iterators
  Refactor the (SSH agent) testing machinery tests for modularity
  Re-align the stubbed SSH agent socket with "real world" behavior
  Serialize the SSH agent protocol numbers as request/response codes
  ...
</pre>

Restore the `test_hash_length_expansion`/`test_wrong_initial_hash_length` vault test

2026-07-05T16:52:19+02:00

This test exercises the "automatically enlarging the sequin input" path
of passphrase derivation in vault.

We originally removed this test because its original name and
description suggested a somewhat different purpose, and necessity.
</pre>

Merge branch 'click-8.4.2-compatibility'

2026-07-05T16:38:19+02:00

This topic branch restores compatibility with click 8.4.2 (and probably
with all versions from 8.2.0 onwards) by working around the changes to
the test runner since 8.2.0.

This is a stopgap measure.  In the long run, we intend to move away from
click, because the API is both unstable and, at times, asinine.  See the
"remove-click" wish in the 0.6 wishlist.

* click-8.4.2-compatibility:
  Work around test runner changes in click > 8.1.8
</pre>

Fix some test coverage slipups

2026-07-05T16:21:55+02:00

</pre>

Work around test runner changes in click > 8.1.8

2026-07-05T14:28:33+02:00

More modern versions of click – I believe 8.2.0 and higher, and
definitely in 8.4.0 and higher – implement stdout and stderr capture in
a completely broken way when prompts are involved and the streams are
mixed: mixed streams are not truly interleaved, and prompts cause extra
typed-in whitespace from stdin to appear in stderr as well.  As
a result, the "correct" captured output in a "mixed stdout/stderr" test
runner looks neither like an interactive shell session with TTY I/O nor
like a non-interactive shell session with redirected I/O; it's
a hobgobbled mess of a bit of both.

Work around this for now by supplying the correct broken expected output
for test runners from newer click versions.

But if anything, this reinforces my desire to drop click as a dependency
as soon as possible.  (See the wishlist item "remove-click" in the 0.6
wishlist.  That wish also details *other* API breakage that users have
to wrangle with.)
</pre>

Define `use_stub_agent`/`use_stub_agent_with_address` in conftest

2026-07-05T11:13:19+02:00

This avoids redefining the same fixture in multiple test modules.
</pre>

Add a heavy-duty test for Sequin internals

2026-07-04T23:22:58+02:00

Add a heavy-duty hypothesis test (rule-based state machine) that
compares the state of our Sequin class against an as-literal-as-possible
port of the original JavaScript sequin module, included inline in the
test class.

(The heavy-duty marker and the state machine machinery are probably
overkill for this application, given that there is only one rule, and
that the test runs very fast.  Nevertheless, phrasing this as a state
machine makes the workings of the test very explicit: what are the state
transitions, what are the invariants, etc.)
</pre>