Recent commits to derivepassphrase.git (9713343438217b44f3eb408b54f0d17f8baca64a)

Correctly model vault global and service settings

2024-10-14T22:31:40+02:00

Excluding extensions due to `derivepassphrase`, in vault(1), the service
settings are a superset of the global settings… though some settings
might be weird in a global setting.  The abstract data types don't
accurately model this, though.  So fix those.

(The sole current `derivepassphrase` extension, the
`unicode_normalization_form` setting, modifies the `phrase` setting, and
so also makes sense to specify globally or locally.  So use the same
inheritance and scoping rules as `phrase`.)

Incidentally, the command-line interface did not respect this
inheritance either… in a more destructive way, asserting that some
options require a service when they don't, or don't take a service when
they do.  Redo the categorization there into three groups instead of
two: service name mandatory, forbidden, or optional (new).
</pre>

Move vault service config generation to top-level tests module

2024-10-14T22:20:29+02:00

The generation of vault service configurations is not inherently tied to
the `vault` module only; the CLI also requires its use.  So move it to
a more central location.
</pre>

Align behavior with vault concerning config imports

2024-10-13T14:39:43+02:00

When importing a configuration, the original vault(1) does not replace
its configuration as a whole with the supplied one.  Instead, each
"section" of the new configuration – the global settings, and each named
service's settings – replaces the corresponding section in the old
configuration, if any.  Any sections not mentioned in the new
configuration are left untouched; in particular, existing service
settings are *kept* if they are not mentioned in the new configuration.

`derivepassphrase` uses a simpler, database-like dump/restore model,
overwriting the whole old configuration with the new one.  As
a consequence, old services settings are *dropped* if they are not
mentioned in the new configuration.  This behavior is a visible
deviation from vault(1), and shall thus be removed.

This commit contains the necessary changes to the import machinery to
correctly calculate the new, merged configuration.  Surprisingly, this
already passes all tests, which is more a sign that our tests are
incomplete rather than that the code is robust against this failure
type.  So, in upcoming commits we will introduce functional tests for
the config import/merging machinery second, and bugfixes for other
things we uncovered while writing these functional tests first.
</pre>

Trigger InvalidSignature exceptions in storeroom exporter tests

2024-10-10T12:36:24+02:00

So far, we haven't been explicitly testing for whether the master and
session keys decrytion routines correctly reject payloads with invalid
MAC values, and with the correct error type.  Surely the `cryptography`
primitive is implemented correctly, but there was no test whether
*we* were using the primitive correctly.  So, add one.
</pre>

Add remaining tests to the storeroom exporter for 100% coverage

2024-10-10T12:18:23+02:00

Before this commit, certain consistency checks within the storeroom
exporter that seemed difficult to test remained untested: a payload size
check in the master keys decryption routine, another payload size check
in the session keys decryption routine, and object connectivity and type
correctness checks in the top-level exporter routine.

The master and session keys decryption routines, it turns out, don't
need this explicit size check: the `struct` library, used for decoding
the payload even further, already checks this automatically.  (What *is*
needed is a wrapper to convert the exception type, in general, for the
whole decryption block.)

For the connectivity and type correctness checks in the top-level
exporter routine, I generated another couple of broken storeroom
configurations (e.g. where directory contents, encoded as a JSON array,
contain non-string elements).  We now test for each of these
configurations if they correctly fail to parse.

Finally, it turns out that many of the docstrings reported the
ciphertext sizes incorrectly, because they wrongly neglected the
padding in their calculations.  Fix this, of course.
</pre>

Signal and list falsy value cleanup steps that were actually performed

2024-10-09T16:20:12+02:00

Signal whether cleanup was actually perfomed on the requested object or
not, and if yes, list the actual cleanup steps undertaken.  When
importing a vault configuration on the command-line, issue a warning for
each cleaning step.

Since the warning messages in both the cleanup steps and the check for
non-normalized passphrases report on "paths" in a JSON object, implement
a fully general JSONPath formatting function (single item selection from
the root only).  This harmonizes the warnings output, but also causes
changes in the test cases and expected output.  Additionally, the
JSONPath function name clashes with a common local variable name,
necessitating renaming, and control flow for the validation function and
the vault configuration import action have changed somewhat; the former
to impose a consistent validation order (global first, service-specific
next), the latter to avoid extraneous else-branches.

As a result of all this, this patch is somewhat larger and less concise
than it should be, given the modest magnitude of changes it actually
introduces.
</pre>

Manage health checks in centralized hypothesis settings as well

2024-10-08T13:57:21+02:00

Some of the tests that time out under coverage-based slow
instrumentation time out during the data generation phase, not the
actual test phase (i.e. trigger health check errors).  The root cause is
the same, and settings objects cannot be stacked, so amend the standard
decorator for slow `hypothesis`-based tests, instead of introducing
a new one.
</pre>

Centralize settings for hypothesis deadline management

2024-10-08T11:43:10+02:00

Our unit tests run in multiple, very different environments, which leads
to drastically different execution times, up to a slowdown factor of
roughly 40 (test coverage, "timid" Python tracer).  The `hypothesis`
library however runs timing checks on each of its tests, indepedent of
the available processing power and coverage instrumentation.  As
a result, some benign tests time out under these circumstances
regardless.

In the past, I've raised their execution deadline in an ad-hoc manner
whenever this happens (or fixed the tests, if they weren't so benign).
But instead of littering the test suite with one-time adjustments of
deadlines, a more sensible approach is to use a test decorator that
ensures a common extended deadline for tests that need it, only if they
need it (i.e. run under coverage).  So do that.  (Sadly, because of how
the settings decorator works, this must be applied function-wise, and
cannot be stacked with other settings decorators.)

Finally, if this deadline extension still doesn't help, then this
usually means we are generating huge or expensive-to-evaluate inputs.
So limit the size of some of the inputs (string length, recursion depth,
size of passphrases to derive) to keep execution times better
constrained.
</pre>

Add changelog entry for key/phrase and falsy behavior changes

2024-10-08T10:04:11+02:00

Document the changes in 7d2f2b1bda31ead428d3c009772aaf3d2261d60c and
798ddc103c6c03835394733aeca128b970aacd06 in the changelog.
</pre>

Align behavior with vault concerning falsy values in config

2024-10-08T09:32:00+02:00

The original vault(1) sometimes checks only for falsy values (in the
JavaScript sense) for its configuration settings.  `derivepassphrase`
however uses strict type and value checks, and rejects falsy values of
the wrong type.  This behavior is a visible deviation from vault(1), and
shall thus be removed.

A new function, `_types.clean_up_falsy_vault_config_values`, normalizes
falsy values in a vault configuration to their correct types, in-place.
Running this on a potential vault configuration and then calling
`_types.is_vault_config` should return the same validity results as
vault(1) does.

The new handling of falsy values invalidates most of the tests for
validation errors, as `None`/`null` was a common way to generate an
invalid setting.  Instead, keep a master list of vault configurations
that is used (perhaps filtered first) for all validation tests, and test
the handling of falsy values by generating vault configurations with
falsy value replacements from the master list (a custom `hypothesis`
strategy).

On that note, the existing `_types.validate_vault_config` has proved
rather difficult to keep at 100% coverage with the new example vault
configurations, because some of the error conditions are triggered
elsewhere.  Accordingly, instead of treating global and service-specific
settings separately and quasi-duplicating all validation checks, unify
them into a queue of settings dicts to check, only mildly adjusting for
the very few differing keys between them.

GitHub: Closes #17.
</pre>