https://git.schokokeks.org/derivepassphrase.git/tree/02e015e8ba265b4f82bac3815db93be2188a6bbc Recent commits feed provided by GitList. When importing a configuration, the original vault(1) does not replace its configuration as a whole with the supplied one. Instead, each "section" of the new configuration – the global settings, and each named service's settings – replaces the corresponding section in the old configuration, if any. Any sections not mentioned in the new configuration are left untouched; in particular, existing service settings are *kept* if they are not mentioned in the new configuration. `derivepassphrase` uses a simpler, database-like dump/restore model, overwriting the whole old configuration with the new one. As a consequence, old services settings are *dropped* if they are not mentioned in the new configuration. This behavior is a visible deviation from vault(1), and shall thus be removed. This commit contains the necessary changes to the import machinery to correctly calculate the new, merged configuration. Surprisingly, this already passes all tests, which is more a sign that our tests are incomplete rather than that the code is robust against this failure type. So, in upcoming commits we will introduce functional tests for the config import/merging machinery second, and bugfixes for other things we uncovered while writing these functional tests first. https://git.schokokeks.org/derivepassphrase.git/commit/02e015e8ba265b4f82bac3815db93be2188a6bbc software@the13thletter.info (Marco Ricci) Sun, 13 Oct 2024 14:39:43 +0200 02e015e8ba265b4f82bac3815db93be2188a6bbc So far, we haven't been explicitly testing for whether the master and session keys decrytion routines correctly reject payloads with invalid MAC values, and with the correct error type. Surely the `cryptography` primitive is implemented correctly, but there was no test whether *we* were using the primitive correctly. So, add one. https://git.schokokeks.org/derivepassphrase.git/commit/3ad62c4d59c6d54fb86b2774919b2f71713eecae software@the13thletter.info (Marco Ricci) Thu, 10 Oct 2024 12:36:24 +0200 3ad62c4d59c6d54fb86b2774919b2f71713eecae Before this commit, certain consistency checks within the storeroom exporter that seemed difficult to test remained untested: a payload size check in the master keys decryption routine, another payload size check in the session keys decryption routine, and object connectivity and type correctness checks in the top-level exporter routine. The master and session keys decryption routines, it turns out, don't need this explicit size check: the `struct` library, used for decoding the payload even further, already checks this automatically. (What *is* needed is a wrapper to convert the exception type, in general, for the whole decryption block.) For the connectivity and type correctness checks in the top-level exporter routine, I generated another couple of broken storeroom configurations (e.g. where directory contents, encoded as a JSON array, contain non-string elements). We now test for each of these configurations if they correctly fail to parse. Finally, it turns out that many of the docstrings reported the ciphertext sizes incorrectly, because they wrongly neglected the padding in their calculations. Fix this, of course. https://git.schokokeks.org/derivepassphrase.git/commit/837e57d2ce6c317d5ba483d88baddbdccfae03df software@the13thletter.info (Marco Ricci) Thu, 10 Oct 2024 12:18:23 +0200 837e57d2ce6c317d5ba483d88baddbdccfae03df Signal whether cleanup was actually perfomed on the requested object or not, and if yes, list the actual cleanup steps undertaken. When importing a vault configuration on the command-line, issue a warning for each cleaning step. Since the warning messages in both the cleanup steps and the check for non-normalized passphrases report on "paths" in a JSON object, implement a fully general JSONPath formatting function (single item selection from the root only). This harmonizes the warnings output, but also causes changes in the test cases and expected output. Additionally, the JSONPath function name clashes with a common local variable name, necessitating renaming, and control flow for the validation function and the vault configuration import action have changed somewhat; the former to impose a consistent validation order (global first, service-specific next), the latter to avoid extraneous else-branches. As a result of all this, this patch is somewhat larger and less concise than it should be, given the modest magnitude of changes it actually introduces. https://git.schokokeks.org/derivepassphrase.git/commit/2eaaa7f8ace52c29c83e36568417c92b358d9710 software@the13thletter.info (Marco Ricci) Wed, 09 Oct 2024 16:20:12 +0200 2eaaa7f8ace52c29c83e36568417c92b358d9710 Some of the tests that time out under coverage-based slow instrumentation time out during the data generation phase, not the actual test phase (i.e. trigger health check errors). The root cause is the same, and settings objects cannot be stacked, so amend the standard decorator for slow `hypothesis`-based tests, instead of introducing a new one. https://git.schokokeks.org/derivepassphrase.git/commit/695ac3fd833a1d50ba99f9539d668982bbd2bc00 software@the13thletter.info (Marco Ricci) Tue, 08 Oct 2024 13:57:21 +0200 695ac3fd833a1d50ba99f9539d668982bbd2bc00 Our unit tests run in multiple, very different environments, which leads to drastically different execution times, up to a slowdown factor of roughly 40 (test coverage, "timid" Python tracer). The `hypothesis` library however runs timing checks on each of its tests, indepedent of the available processing power and coverage instrumentation. As a result, some benign tests time out under these circumstances regardless. In the past, I've raised their execution deadline in an ad-hoc manner whenever this happens (or fixed the tests, if they weren't so benign). But instead of littering the test suite with one-time adjustments of deadlines, a more sensible approach is to use a test decorator that ensures a common extended deadline for tests that need it, only if they need it (i.e. run under coverage). So do that. (Sadly, because of how the settings decorator works, this must be applied function-wise, and cannot be stacked with other settings decorators.) Finally, if this deadline extension still doesn't help, then this usually means we are generating huge or expensive-to-evaluate inputs. So limit the size of some of the inputs (string length, recursion depth, size of passphrases to derive) to keep execution times better constrained. https://git.schokokeks.org/derivepassphrase.git/commit/3eabf0cd303c0e2e83a61a7c7835ee66b7fb5acf software@the13thletter.info (Marco Ricci) Tue, 08 Oct 2024 11:43:10 +0200 3eabf0cd303c0e2e83a61a7c7835ee66b7fb5acf Document the changes in 7d2f2b1bda31ead428d3c009772aaf3d2261d60c and 798ddc103c6c03835394733aeca128b970aacd06 in the changelog. https://git.schokokeks.org/derivepassphrase.git/commit/63053f40ef9487c6ede43eb863bbd9abe578e258 software@the13thletter.info (Marco Ricci) Tue, 08 Oct 2024 10:04:11 +0200 63053f40ef9487c6ede43eb863bbd9abe578e258 The original vault(1) sometimes checks only for falsy values (in the JavaScript sense) for its configuration settings. `derivepassphrase` however uses strict type and value checks, and rejects falsy values of the wrong type. This behavior is a visible deviation from vault(1), and shall thus be removed. A new function, `_types.clean_up_falsy_vault_config_values`, normalizes falsy values in a vault configuration to their correct types, in-place. Running this on a potential vault configuration and then calling `_types.is_vault_config` should return the same validity results as vault(1) does. The new handling of falsy values invalidates most of the tests for validation errors, as `None`/`null` was a common way to generate an invalid setting. Instead, keep a master list of vault configurations that is used (perhaps filtered first) for all validation tests, and test the handling of falsy values by generating vault configurations with falsy value replacements from the master list (a custom `hypothesis` strategy). On that note, the existing `_types.validate_vault_config` has proved rather difficult to keep at 100% coverage with the new example vault configurations, because some of the error conditions are triggered elsewhere. Accordingly, instead of treating global and service-specific settings separately and quasi-duplicating all validation checks, unify them into a queue of settings dicts to check, only mildly adjusting for the very few differing keys between them. GitHub: Closes #17. https://git.schokokeks.org/derivepassphrase.git/commit/7d2f2b1bda31ead428d3c009772aaf3d2261d60c software@the13thletter.info (Marco Ricci) Tue, 08 Oct 2024 09:32:00 +0200 7d2f2b1bda31ead428d3c009772aaf3d2261d60c When both a key and a passphrase are specified in the vault configuration, vault(1) would unconditionally use the key, *unless* the command-line overrides this choice. `derivepassphrase` however always gave preference to the most "specific" configuration, and would error out if both key and passphrase were specified at the same specificity. While arguably more intuitive, this behavior is a visible deviation from vault(1), and shall thus be removed. Besides two instances of the `test_200_is_vault_config` in `tests.test_derivepassphrase_types`, this also flips the result of `test_205_service_phrase_if_key_in_global_config` in `tests.test_derivepassphrase_cli`. Because that flipped version needs extra mocking infrastructure – the `sign` function – and because that mock function already exists in another test (but local to that test), promote that mock function to global and shift it into the top-level `tests` module. Since we had to update the imports in `tests` anyway, we also purged `dpp.vault...` references in `tests.test_derivepassphrase_cli` in favor of `vault...`. https://git.schokokeks.org/derivepassphrase.git/commit/798ddc103c6c03835394733aeca128b970aacd06 software@the13thletter.info (Marco Ricci) Sat, 05 Oct 2024 23:30:07 +0200 798ddc103c6c03835394733aeca128b970aacd06 The master changelog file is included, of course, but we don't want MkDocs to bother with the single snippets (rendering them, generating warnings that they're not part of the navigation tree, etc.). https://git.schokokeks.org/derivepassphrase.git/commit/59082d1f81b629c4be67bdcce2977db289d7c3af software@the13thletter.info (Marco Ricci) Fri, 04 Oct 2024 10:55:32 +0200 59082d1f81b629c4be67bdcce2977db289d7c3af