https://git.schokokeks.org/derivepassphrase.git/tree/unstable/modularize-and-refactor-test-machinery Recent commits feed provided by GitList. (This is part 6 of a series of refactorings for the test suite.) Assign all tests to a common class, then elide common parts in the test names and docstrings. Also factor out the OpenSSH cipher blocksize for the "none" cipher into a separate constant, and use that constant in lieu of magic numbers throughout the helper functions. We correct one instance of where double the block size is used, which (if memory serves correctly) was due to a stylistic difference between OpenSSH- and Pageant-generated keys, but both conforming to the specification. These differences are removed in the canonicalization step anyway, but the double-sized block size is probably confusing to read if you don't already anticipate the canonicalization step. https://git.schokokeks.org/derivepassphrase.git/commit/2184c76b9bbde9cb90374e285ca79f8a2ba8f8a5 software@the13thletter.info (Marco Ricci) Fri, 22 Aug 2025 19:50:46 +0200 2184c76b9bbde9cb90374e285ca79f8a2ba8f8a5 (This is part 5 of a series of refactorings for the test suite.) In the basic tests, for the stubbed SSH agent socket and the agent error response tests, factor out the common environment setup for the respective group (in particular, the multiple copies of the fake `request` function from the SSH agent client). Combine the SSH agent data signing tests for suitable and unsuitable keys into a single, more heavily parametrized test. (The combined test is branchier, but more straightforward for discerning the differences between the two types of keys.) Also fix some miscellaneous errors: the IDs in the `RESOLVE_CHAINS` parametrization, and the order of contexts in the `test_request_error_responses` method. Further concerning the basic tests, add an explicit test for the `list-extended@putty.projects.tartarus.org` extension (which the extended stubbed SSH agent implements to signal support for deterministic DSA signatures, in a lackluster attempt to masquerade as Pageant). Also address a TODO in the test for SSH key selection concerning degenerate key lists. In the test data, add an extra comment on how suitable and unsuitable SSH test keys are determined, and add a function signature for the SSH agent client `request` function. In the heavy-duty tests, merely add missing `draw` labels. https://git.schokokeks.org/derivepassphrase.git/commit/ff2c5f5099c8216f92aa87113dcabee1768a851d software@the13thletter.info (Marco Ricci) Sun, 17 Aug 2025 18:00:57 +0200 ff2c5f5099c8216f92aa87113dcabee1768a851d (This is part 4 of a series of refactorings for the test suite.) For the `export vault` command-line interface tests, split the tests into tests for command-line argument support, tests for various command-line or format-related errors, and the already existing groups for "storeroom" and "vault v0.2"/"vault v0.3" format tests. For the former two groups, factor out the common test operation, which in both cases is the whole test content, though with differing call conventions. For the latter two groups, factor out the common environment setup instead. For the `exporter` subpackage tests, collect the data generation strategies in a common class `Strategies` (similar to the `Parametrize` class). Split the existing `TestUtilities` class into a `TestCLIUtilities` and a `TestExportVaultConfigDataHandlerRegistry` class, and factor out the common environment setup for each respective group. Rename the `TestCLI` class into `TestGenericVaultCLIErrors`, and factor out the common environment setup and the CLI call function. https://git.schokokeks.org/derivepassphrase.git/commit/f805c904589ffba6e0828c4aa5f596c56aba7e00 software@the13thletter.info (Marco Ricci) Sun, 17 Aug 2025 16:42:24 +0200 f805c904589ffba6e0828c4aa5f596c56aba7e00 (This is part 3 of a series of refactorings for the test suite.) For the `sequin` tests, merge the two tests for bit extraction from a big-endian number. They both test testing infrastructure, and they already both had the same test parametrization. For the `vault` tests, factor out common test operations for the `TestPhraseDependence` and `TestInterchangablePhrases` classes (which in both cases amounts to the whole test body). Rewrite the `TestStringAndBinaryExchangability` tests to a more uniform style, parametrizing over the specific binary class. Also, for `test_binary_service_name` (now `test_binary_service_name_and_phrase`), cross-check that all combinations of phrase and service name classes lead to identical results. Finally, fix the explicit forbidden patterns in `test_only_numbers_and_very_high_repetition_limit`, which were one character too short, as well as a typo in the `test_arbitrary_repetition_limit` docstring. https://git.schokokeks.org/derivepassphrase.git/commit/33ad3a10b7c1f4aa17a6d32a696136b8b00a4e0e software@the13thletter.info (Marco Ricci) Sun, 17 Aug 2025 16:21:27 +0200 33ad3a10b7c1f4aa17a6d32a696136b8b00a4e0e (This is part 2 of a series of refactorings for the test suite.) For the basic tests, collect all test data in a common class `Strategies` (similar to the `Parametrize` class). Split them into groups for validity testing and validation/data cleaning. For each group, factor out the common test operation... which in both cases, is the whole test content, save for the different input data set they run on. For the heavy-duty tests, there is only one data generation strategy, which needn't artifically be wrapped in a class (yet). There is also only one test, so there is no need for further grouping either. Instead, add some missing commentary on the one explicit example in that test set. https://git.schokokeks.org/derivepassphrase.git/commit/facb2eebb96672abd53f2fdf25cd39f89035dd03 software@the13thletter.info (Marco Ricci) Sun, 17 Aug 2025 16:01:53 +0200 facb2eebb96672abd53f2fdf25cd39f89035dd03 (This is part 1 of a series of refactorings for the test suite.) Collect all test data generation strategies in a common class `Strategies` (similar to the `Parametrize` class). Split the tests into groups for debug translations, operations on translatable strings (currently only the hashability check) and suppression of interpolation. Within each group, attempt to factor out common operations, though at the moment, only the debug translations group has code factored out. https://git.schokokeks.org/derivepassphrase.git/commit/4496ed5d9d2b816bb8b13d4b7650248584270574 software@the13thletter.info (Marco Ricci) Sun, 17 Aug 2025 15:58:47 +0200 4496ed5d9d2b816bb8b13d4b7650248584270574 The existing system of ordinals – loosely modeled on HTTP status codes, but subverted repeatedly because of ad hoc additions or `hypothesis` reimplementations in the test suite – is hopelessly out of sync, and quite an obstacle while refactoring the test suite. Remove them, at least until the test suite is more stable again. On the other hand, there is some value in ordering tests so that – absent any randomization or parallelization of the run order –, the "simpler" or "more fundamental" tests run first: this sometimes provides the needed insight to track down a problem that only manifests in a more cryptic way in the other, more complicated tests. So in the long run, some level of ordering will likely be introduced. But whether this is an ordinal in the test name, or perhaps something more complicated such as a signalling test fixture, or mark, or a custom collection function, or whatever, is still open. https://git.schokokeks.org/derivepassphrase.git/commit/2ca71b02fe5c2ebdbdda6ffaf2fd5c6c26c5ff1d software@the13thletter.info (Marco Ricci) Fri, 15 Aug 2025 18:26:34 +0200 2ca71b02fe5c2ebdbdda6ffaf2fd5c6c26c5ff1d Because the boolean value would otherwise become part of the `pytest` test ID, and because the value is devoid of any context, give each boolean parametrization an explicit, hopefully sensible test ID. https://git.schokokeks.org/derivepassphrase.git/commit/80e0ae7cb7fdeeaabcb1fab8a857419dc61f4bb4 software@the13thletter.info (Marco Ricci) Fri, 15 Aug 2025 17:59:56 +0200 80e0ae7cb7fdeeaabcb1fab8a857419dc61f4bb4 Collect, reorganize and reimplement the `hypothesis` strategies for generating phrases and service names in the `vault` module tests. Phrases come in three possible size ranges, are usually binary but sometimes textual (if short), and at times, we explicitly want pairs of phrases that are not interchangable under `vault`, and at other times, we want a second interchangable phrase given a first one. We reimplement the (size-dependent) binary phrase and pair of phrases strategies with `hypothesis.strategies.composite`, and add them to a new namespace of stategies, similar to the `Parametrize` class. I find the result very pleasing to read, and also much more amenable to adjusting the strategies than when the definitions are always included inline. https://git.schokokeks.org/derivepassphrase.git/commit/ad9d846f6a380009f73fe69e479886c99ee6070e software@the13thletter.info (Marco Ricci) Fri, 15 Aug 2025 06:54:29 +0200 ad9d846f6a380009f73fe69e479886c99ee6070e Implement alluded to, but missing, tests for `derivepassphrase vault`: passphrase usage based on stored configuration, passphrase usage based on the command-line, and exporting configurations that were originally smudged upon import. https://git.schokokeks.org/derivepassphrase.git/commit/791a882431f75bd28fc11b9fada74aa7c3fbe621 software@the13thletter.info (Marco Ricci) Fri, 15 Aug 2025 06:51:10 +0200 791a882431f75bd28fc11b9fada74aa7c3fbe621