Recent commits to derivepassphrase.git (unstable/modularize-and-refactor-test-machinery)

Refactor the testing machinery tests

2025-08-22T19:50:46+02:00

(This is part 6 of a series of refactorings for the test suite.)

Assign all tests to a common class, then elide common parts in the test
names and docstrings.

Also factor out the OpenSSH cipher blocksize for the "none" cipher into
a separate constant, and use that constant in lieu of magic numbers
throughout the helper functions.  We correct one instance of where
double the block size is used, which (if memory serves correctly) was
due to a stylistic difference between OpenSSH- and Pageant-generated
keys, but both conforming to the specification.  These differences are
removed in the canonicalization step anyway, but the double-sized block
size is probably confusing to read if you don't already anticipate the
canonicalization step.
</pre>

Refactor the SSH agent tests

2025-08-17T18:00:57+02:00

(This is part 5 of a series of refactorings for the test suite.)

In the basic tests, for the stubbed SSH agent socket and the agent error
response tests, factor out the common environment setup for the
respective group (in particular, the multiple copies of the fake
`request` function from the SSH agent client).  Combine the SSH agent
data signing tests for suitable and unsuitable keys into a single, more
heavily parametrized test.  (The combined test is branchier, but more
straightforward for discerning the differences between the two types of
keys.)  Also fix some miscellaneous errors: the IDs in the
`RESOLVE_CHAINS` parametrization, and the order of contexts in the
`test_request_error_responses` method.

Further concerning the basic tests, add an explicit test for the
`list-extended@putty.projects.tartarus.org` extension (which the
extended stubbed SSH agent implements to signal support for
deterministic DSA signatures, in a lackluster attempt to masquerade as
Pageant).  Also address a TODO in the test for SSH key selection
concerning degenerate key lists.

In the test data, add an extra comment on how suitable and unsuitable
SSH test keys are determined, and add a function signature for the
SSH agent client `request` function.

In the heavy-duty tests, merely add missing `draw` labels.
</pre>

Refactor the `exporter` tests

2025-08-17T16:42:24+02:00

(This is part 4 of a series of refactorings for the test suite.)

For the `export vault` command-line interface tests, split the tests
into tests for command-line argument support, tests for various
command-line or format-related errors, and the already existing groups
for "storeroom" and "vault v0.2"/"vault v0.3" format tests.  For the
former two groups, factor out the common test operation, which in both
cases is the whole test content, though with differing call conventions.
For the latter two groups, factor out the common environment setup
instead.

For the `exporter` subpackage tests, collect the data generation
strategies in a common class `Strategies` (similar to the `Parametrize`
class).  Split the existing `TestUtilities` class into
a `TestCLIUtilities` and a `TestExportVaultConfigDataHandlerRegistry`
class, and factor out the common environment setup for each respective
group.  Rename the `TestCLI` class into `TestGenericVaultCLIErrors`, and
factor out the common environment setup and the CLI call function.
</pre>

Refactor the `vault` and `sequin` tests

2025-08-17T16:21:27+02:00

(This is part 3 of a series of refactorings for the test suite.)

For the `sequin` tests, merge the two tests for bit extraction from
a big-endian number.  They both test testing infrastructure, and they
already both had the same test parametrization.

For the `vault` tests, factor out common test operations for the
`TestPhraseDependence` and `TestInterchangablePhrases` classes (which in
both cases amounts to the whole test body).  Rewrite the
`TestStringAndBinaryExchangability` tests to a more uniform style,
parametrizing over the specific binary class.  Also, for
`test_binary_service_name` (now `test_binary_service_name_and_phrase`),
cross-check that all combinations of phrase and service name classes
lead to identical results.

Finally, fix the explicit forbidden patterns in
`test_only_numbers_and_very_high_repetition_limit`, which were one
character too short, as well as a typo in the
`test_arbitrary_repetition_limit` docstring.
</pre>

Refactor the types tests

2025-08-17T16:01:53+02:00

(This is part 2 of a series of refactorings for the test suite.)

For the basic tests, collect all test data in a common class
`Strategies` (similar to the `Parametrize` class).  Split them into
groups for validity testing and validation/data cleaning.  For each
group, factor out the common test operation... which in both cases, is
the whole test content, save for the different input data set they run
on.

For the heavy-duty tests, there is only one data generation strategy,
which needn't artifically be wrapped in a class (yet).  There is also
only one test, so there is no need for further grouping either.
Instead, add some missing commentary on the one explicit example in that
test set.
</pre>

Refactor the localization machinery tests

2025-08-17T15:58:47+02:00

(This is part 1 of a series of refactorings for the test suite.)

Collect all test data generation strategies in a common class
`Strategies` (similar to the `Parametrize` class).  Split the tests into
groups for debug translations, operations on translatable strings
(currently only the hashability check) and suppression of interpolation.
Within each group, attempt to factor out common operations, though at
the moment, only the debug translations group has code factored out.
</pre>

Remove the remaining ordinals from the test names

2025-08-15T18:26:34+02:00

The existing system of ordinals – loosely modeled on HTTP status codes,
but subverted repeatedly because of ad hoc additions or `hypothesis`
reimplementations in the test suite – is hopelessly out of sync, and
quite an obstacle while refactoring the test suite.  Remove them, at
least until the test suite is more stable again.

On the other hand, there is some value in ordering tests so that –
absent any randomization or parallelization of the run order –, the
"simpler" or "more fundamental" tests run first: this sometimes provides
the needed insight to track down a problem that only manifests in a more
cryptic way in the other, more complicated tests.  So in the long run,
some level of ordering will likely be introduced.  But whether this is
an ordinal in the test name, or perhaps something more complicated such
as a signalling test fixture, or mark, or a custom collection function,
or whatever, is still open.
</pre>

Annotate all boolean parametrization with sensible test IDs

2025-08-15T17:59:56+02:00

Because the boolean value would otherwise become part of the `pytest`
test ID, and because the value is devoid of any context, give each
boolean parametrization an explicit, hopefully sensible test ID.
</pre>

Refactor the hypothesis strategies for `vault` tests

2025-08-15T06:54:29+02:00

Collect, reorganize and reimplement the `hypothesis` strategies for
generating phrases and service names in the `vault` module tests.
Phrases come in three possible size ranges, are usually binary but
sometimes textual (if short), and at times, we explicitly want pairs
of phrases that are not interchangable under `vault`, and at other
times, we want a second interchangable phrase given a first one.  We
reimplement the (size-dependent) binary phrase and pair of phrases
strategies with `hypothesis.strategies.composite`, and add them to a new
namespace of stategies, similar to the `Parametrize` class.

I find the result very pleasing to read, and also much more amenable to
adjusting the strategies than when the definitions are always included
inline.
</pre>

Implement the TODO tests for the `vault` command-line interface

2025-08-15T06:51:10+02:00

Implement alluded to, but missing, tests for `derivepassphrase vault`:
passphrase usage based on stored configuration, passphrase usage based
on the command-line, and exporting configurations that were originally
smudged upon import.
</pre>