https://git.schokokeks.org/derivepassphrase.git/tree/4496ed5d9d2b816bb8b13d4b7650248584270574 Recent commits feed provided by GitList. (This is part 1 of a series of refactorings for the test suite.) Collect all test data generation strategies in a common class `Strategies` (similar to the `Parametrize` class). Split the tests into groups for debug translations, operations on translatable strings (currently only the hashability check) and suppression of interpolation. Within each group, attempt to factor out common operations, though at the moment, only the debug translations group has code factored out. https://git.schokokeks.org/derivepassphrase.git/commit/4496ed5d9d2b816bb8b13d4b7650248584270574 software@the13thletter.info (Marco Ricci) Sun, 17 Aug 2025 15:58:47 +0200 4496ed5d9d2b816bb8b13d4b7650248584270574 The existing system of ordinals – loosely modeled on HTTP status codes, but subverted repeatedly because of ad hoc additions or `hypothesis` reimplementations in the test suite – is hopelessly out of sync, and quite an obstacle while refactoring the test suite. Remove them, at least until the test suite is more stable again. On the other hand, there is some value in ordering tests so that – absent any randomization or parallelization of the run order –, the "simpler" or "more fundamental" tests run first: this sometimes provides the needed insight to track down a problem that only manifests in a more cryptic way in the other, more complicated tests. So in the long run, some level of ordering will likely be introduced. But whether this is an ordinal in the test name, or perhaps something more complicated such as a signalling test fixture, or mark, or a custom collection function, or whatever, is still open. https://git.schokokeks.org/derivepassphrase.git/commit/2ca71b02fe5c2ebdbdda6ffaf2fd5c6c26c5ff1d software@the13thletter.info (Marco Ricci) Fri, 15 Aug 2025 18:26:34 +0200 2ca71b02fe5c2ebdbdda6ffaf2fd5c6c26c5ff1d Because the boolean value would otherwise become part of the `pytest` test ID, and because the value is devoid of any context, give each boolean parametrization an explicit, hopefully sensible test ID. https://git.schokokeks.org/derivepassphrase.git/commit/80e0ae7cb7fdeeaabcb1fab8a857419dc61f4bb4 software@the13thletter.info (Marco Ricci) Fri, 15 Aug 2025 17:59:56 +0200 80e0ae7cb7fdeeaabcb1fab8a857419dc61f4bb4 Collect, reorganize and reimplement the `hypothesis` strategies for generating phrases and service names in the `vault` module tests. Phrases come in three possible size ranges, are usually binary but sometimes textual (if short), and at times, we explicitly want pairs of phrases that are not interchangable under `vault`, and at other times, we want a second interchangable phrase given a first one. We reimplement the (size-dependent) binary phrase and pair of phrases strategies with `hypothesis.strategies.composite`, and add them to a new namespace of stategies, similar to the `Parametrize` class. I find the result very pleasing to read, and also much more amenable to adjusting the strategies than when the definitions are always included inline. https://git.schokokeks.org/derivepassphrase.git/commit/ad9d846f6a380009f73fe69e479886c99ee6070e software@the13thletter.info (Marco Ricci) Fri, 15 Aug 2025 06:54:29 +0200 ad9d846f6a380009f73fe69e479886c99ee6070e Implement alluded to, but missing, tests for `derivepassphrase vault`: passphrase usage based on stored configuration, passphrase usage based on the command-line, and exporting configurations that were originally smudged upon import. https://git.schokokeks.org/derivepassphrase.git/commit/791a882431f75bd28fc11b9fada74aa7c3fbe621 software@the13thletter.info (Marco Ricci) Fri, 15 Aug 2025 06:51:10 +0200 791a882431f75bd28fc11b9fada74aa7c3fbe621 For asserting the correctness of `vault`'s repetition limitation setting, we used to extract all size `r` substrings of the derived passphrase (where `r` is the repetition count that is *not* allowed anymore) and tested whether they contained more than one different character (by building a set over the characters). That works, but it repeatedly builds sets, and scales badly with increased repetition count. Instead, we adopt the faster approach that examines the derived passphrase once, character by character, keeping track of the longest seen run of identical characters, and asserting that that run is within the permitted repetition limit. Although it consists of more instructions, these are "simpler" instructions that do not involve set object construction, and in particular, they are independent of the repetition limit, leading to better scalability. Sample runs with Python's `timeit` module also indicate that for length-200 strings and repetition limit 100, the set-building version takes 2-5 times as long as the direct run counting version. Given the nature of this code – it runs in `hypothesis`, so is executed repeatedly and cannot afford to be *too* slow –, I posit that the speed gain is worth the slightly indirect measurement style. https://git.schokokeks.org/derivepassphrase.git/commit/e0dcb95d74e152744bcda119c632ee79ab3dddfa software@the13thletter.info (Marco Ricci) Thu, 14 Aug 2025 22:58:08 +0200 e0dcb95d74e152744bcda119c632ee79ab3dddfa Sadly, there exist configurations and pairs of master passphrases (and presumably, pairs of services names as well) that lead to the same derived passphrase. (These are typically short-length derived passphrases with strongly restricted character sets.) Once `hypothesis` has found such a set of inputs, its example database will cause it to keep rediscovering that example. Ideally, we want to express that given enough entropy through the configuration, the chance of deriving the same passphrases with two different master passphrases or two different service names becomes very small. However, this is a statement about the function's state space, and I do not know how to sensibly express this statement in a unit-test or `hypothesis`-test-compatible way, short of perhaps enumerating the whole state space (which is computationally infeasible). So, we remove these tests of config dependence; they are clearly non-functional and misleading. https://git.schokokeks.org/derivepassphrase.git/commit/f2ec081dd4fc85960a80fad12993ffe8f074658f software@the13thletter.info (Marco Ricci) Thu, 14 Aug 2025 22:47:35 +0200 f2ec081dd4fc85960a80fad12993ffe8f074658f Also resolve some TODOs, usually by removing duplicate or quasi-duplicate tests. https://git.schokokeks.org/derivepassphrase.git/commit/d4ff1362953e1acdf74dfa1b9529bb97df552604 software@the13thletter.info (Marco Ricci) Thu, 14 Aug 2025 20:58:10 +0200 d4ff1362953e1acdf74dfa1b9529bb97df552604 As in a2c0a0b3b6f000d824787dabc011041549bda206, introduce further classes/groupings for the `vault` derivation scheme tests, trimming test names appropriately. https://git.schokokeks.org/derivepassphrase.git/commit/b2fb0785d0d6b62077171e797424b2f30db4760c software@the13thletter.info (Marco Ricci) Thu, 14 Aug 2025 19:10:17 +0200 b2fb0785d0d6b62077171e797424b2f30db4760c Beyond the five main groups that reflect the original five test classes for the command-line interface, introduce further classes/groupings for related tests, and trim the test names appropriately. In particular, remove the ordinal in the test name until the new order (due to the new class names and trimmed test names) sufficiently settles. Do the same for the SSH agent tests. (This could have been done for all tests, but with diminishing returns.) The grouping currently (mostly) reflects thematic similarity, but in the long run, we intend for it to reflect both thematic similarity and common test environments, with respect to setup and teardown: tests of the same group should look similar to each other too, and common code among tests of the same group should be factored out if possible. The grouping is inserted, syntactically, between the existing test functions. In particular, to keep the diff readable, we do not relocate or reorder any test functions yet in this commit. We add some TODOs to indicate where further reordering or rewriting is necessary, beyond just introducing new groups. The intended (but not yet implemented) sharing of test setup and teardown code will probably entail using fixtures, pytest's preferred mechanism for shared setup and teardown. However, as of this writing, it is still an open question how to deal with `hypothesis`-based tests in this scenario: they do not work well together with per-function test fixtures, but we also do *not* want to have to (re-)implement all setup/teardown pairs once as a fixture and once as a context manager. https://git.schokokeks.org/derivepassphrase.git/commit/79e5c37b8eac357c6b5afb6fdcd5e70c0ee3dd43 software@the13thletter.info (Marco Ricci) Mon, 11 Aug 2025 21:50:42 +0200 79e5c37b8eac357c6b5afb6fdcd5e70c0ee3dd43