https://git.schokokeks.org/derivepassphrase.git/tree/881e05f45103800021077ac152098af878c3295f Recent commits feed provided by GitList. Also include a changelog entry for the self-testing feature flags. https://git.schokokeks.org/derivepassphrase.git/commit/881e05f45103800021077ac152098af878c3295f software@the13thletter.info (Marco Ricci) Mon, 04 Aug 2025 21:37:03 +0200 881e05f45103800021077ac152098af878c3295f Make the enums `_types.Subcommand`, `_types.DerivationScheme`, etc. all capable of testing whether they are active/enabled/supported, or not. Specifically, make each enum definition include a name and an "is enabled?" test function. Then all the command/format/feature support information is bundled at the same site in the code, is harder to get out of sync, and is also nicer to read at the call sites. We choose to implement a single test function that dispatches to the enum value-specific test functions manually. I originally considered using a tuple of name and test function as the enum value, and then automatically dispatching on the test function value, akin to how one might implement this e.g. in Java. However, the corresponding testing code is written generically, permitting arbitrary texts as values for the categories that these enums represent. Having to explicitly convert between strings and (non-string) enums makes the testing code much more complicated (and thus harder itself to test), extremely brittle, and the sample test cases markedly less readable. Having a hand-written dispatch function for each enum class, with relatively few cases per class, seems like an acceptable price to pay if it makes for more easily and more readably testable code. https://git.schokokeks.org/derivepassphrase.git/commit/6e2216d4c205f6cf00478c193c0925b71b4f7f6e software@the13thletter.info (Marco Ricci) Sun, 03 Aug 2025 11:42:44 +0200 6e2216d4c205f6cf00478c193c0925b71b4f7f6e Document the SSH agent socket providers in the API reference documentation. Additionally, update the user-facing, checklist-style reference documentation on the state of SSH agent support for The Annoying Operating System: still unsupported, but basic infrastructure is being built. Also go into more detail on GnuPG's `gpg-agent` -- how it still is unsupported on The Annoying Operating System, how to connect to it on other systems, and how to prepare OpenPGP keys of the right type for use with SSH. Finally -- though the information will likely still relevant for quite some time -- date the SSH key type recommendations explicitly, and update the explanation of the `--version` output of `derivepassphrase vault` to mention that SSH agent construction might still fail at runtime, even if support is indicated. https://git.schokokeks.org/derivepassphrase.git/commit/cf02478d27865fe8c5ee649b364b9278b4dceb9e software@the13thletter.info (Marco Ricci) Sat, 02 Aug 2025 14:27:29 +0200 cf02478d27865fe8c5ee649b364b9278b4dceb9e Change the feature detection system to actually test whether it can successfully construct an SSH agent client, and then report that the feature is available (or not) based on the success of this test. Previously, `derivepassphrase` would determine whether it supported connecting to an SSH agent or not by checking for Python support for UNIX domain sockets. This was the same criteria as used by the actual SSH agent client constructor to attempt, well, constructing an SSH agent client. However, since the SSH agent client now delegates the construction to the socket provider mechanism, which includes fallbacks and "available, but not sensibly callable" providers, determining whether SSH agent support is available is not quite as straight-forward anymore as testing for `socket.AF_UNIX` was. I could attempt to replicate the registry entry resolving steps to see if we have a working socket provider, but even that entails actually running the provider and checking for the absence of certain kinds of exceptions (but not other kinds, which indicate a manner of success!). Plus, a separate, indepedent replication of the search logic runs the risk of becoming desynchronized with the original implementation: `derivepassphrase vault --version` thinks that support is or is not available, and the SSH agent client constructor thinks the opposite. A *far* more robust solution is to actually test the functionality we are hoping to use directly, i.e., to test constructing an SSH agent client: it obviously doesn't work if we don't have support, and if it does work, we have support. (This technique would of course generalize to other feature flags, if we had any such.) So that's what we'll do. https://git.schokokeks.org/derivepassphrase.git/commit/0b625854c198b74a15a80bb9e1fd1123326e79a8 software@the13thletter.info (Marco Ricci) Sat, 02 Aug 2025 14:26:22 +0200 0b625854c198b74a15a80bb9e1fd1123326e79a8 Support looking up a socket provider registry entry without bailing if the entry is merely registered, but not implemented. Call this operation "lookup", as opposed to "resolve". Use this operation directly whereever it makes sense (which currently is only in testing code and in the implementation of "resolve"). https://git.schokokeks.org/derivepassphrase.git/commit/e48a2b9a75d62d42cadc7f397b5e5adefbcc4f6a software@the13thletter.info (Marco Ricci) Sat, 02 Aug 2025 14:22:51 +0200 e48a2b9a75d62d42cadc7f397b5e5adefbcc4f6a We add the stub agent to the list of spawnable SSH agents, since it is always available. This necessitates a lot of cleanup to old test suite functionality that assumes SSH agents are available if and only if UNIX domain sockets are supported. The `PERMITTED_SSH_AGENTS` environment variable now controls which (external) SSH agents will be attempted to be spawned, if supported by the system. Additionally, the existing agents that can be spawned are all UNIX-only, and correctly marked as such; this avoids attempting to spawn Pageant on The Annoying Operating System using a UNIX Pageant command-line. We include a stub for the entry points list/table from `importlib.metadata`, which sadly is rather ugly due to having to reflect both an explicit API change in Python 3.12, and an undocumented and hairy compatibility hack in Python 3.10. We add a `SSHAgentSocketProviderRegistryStateMachine` class for testing socket provider registrations via a `hypothesis` state machine, and quite a few new entries in the `TestStaticFunctionality` class relating to locating and registering socket providers on handcrafted explicit examples. I have resisted the temptation to implement all registry tests as `hypothesis` tests, in favor of having at least *some* unit tests left over to test the registry, even if skipping `hypothesis`-based tests for speed reasons. Finally, we split the monolithic test `TestCLIUtils.test_400_key_to_phrase` from the CLI tests into a parametrized series of smaller tests. https://git.schokokeks.org/derivepassphrase.git/commit/da49d3bf1fe6ecf850520922fd36381cf3191c26 software@the13thletter.info (Marco Ricci) Sat, 02 Aug 2025 13:50:27 +0200 da49d3bf1fe6ecf850520922fd36381cf3191c26 Introduce a stubbed SSH agent socket that simulates a real agent's responses (for test keys only). Include tests to verify the correct workings of the agent. The stub agent comes in three versions. The basic version implements all the necessary functionality, but intentionally disables SSH agent extension support. The "address" version additionally supports generating errors upon construction by both requiring an address (in `SSH_AUTH_SOCK`) and by raising specific errors for specific addresses. The "deterministic DSA" version additionally supports the "query" and the "list-extended@putty.projects.tartarus.org" extension, and yields the recorded RFC 6979 deterministic DSA signatures for DSA and ECDSA test keys. The "address" and "deterministic DSA" versions are intended for use in the test suite, as test doubles for real SSH agents, depending on whether deterministic DSA signatures are required or not. The "basic" version is intended for verification of the stubbed SSH agent itself, whereever it can be used in place of more powerful stubbed SSH agent versions. Though tested *individually*, as a new piece of code added to the code base, the stub agent (in all its variations) is not yet *integrated* into the test suite as a "proper", spawnable SSH agent. That we leave to the following commits. https://git.schokokeks.org/derivepassphrase.git/commit/dff1cd10db7f489264f24a30c22ab166038eaf0a software@the13thletter.info (Marco Ricci) Sat, 02 Aug 2025 13:22:55 +0200 dff1cd10db7f489264f24a30c22ab166038eaf0a Introduce a new class of objects, SSH agent socket providers, that provide sockets connected to SSH agents to the SSH agent client. In turn, the clients now depend on the socket providers for establishing the connection to the agent. Upon construction, the client queries an implicit or explicit list of such provider names (from a local, in-code registry) and uses the first provider that successfully returns a connected socket. Providers may be, uh, provided by `derivepassphrase` itself, or by third-party developers, and they may depend, e.g., on a certain operating system or Python installation, or certain installed third-party software. `derivepassphrase` includes three standard providers: `posix`, `the_annoying_os` and `native`. The `posix` provider attempts to connect via UNIX domain sockets and the `SSH_AUTH_SOCK` environment variable, as `derivepassphrase` supported it before this commit. The `the_annoying_os` provider, currently a stub, is intended to eventually support connecting via Windows named pipes. The `native` provider is an alias to whichever of `posix` and `the_annoying_os` is more appropriate to this `derivepassphrase` installation. `derivepassphrase` further registers and reserves several provider names related to testing, as well as several aliases for the aforementioned three standard providers. Third-party socket providers are possible through the `derivepassphrase.ssh_agent_socket_providers` Python package entry point, which expects a `SSHAgentSocketProviderEntry` object. See the corresponding documentation in the `_types` module. The intent of the socket provider system is to decouple the SSH agent client code from the establishing of the connection to the agent. The client code can then be (mostly) operating system- and agent-agnostic, and in turn, the socket provider code can be as operating system- and third party software-specific as necessary. Third-party developers can also add socket providers that cannot or should not be included with `derivepassphrase` because they are niche, or require additional software, or have a different level of stability or development cycle length than `derivepassphrase` has. Additionally, the decoupled system can be more easily tested, by mocking the other side of the connection (which also extends to third-party socket providers). Several socket provider names are already reserved for the test suite, as mentioned above. This new functionality necessitates some large-ish code changes. The new submodule `derivepassphrase.ssh_agent.socketprovider` contains all socket provider-related functionality and data, including the provider registry. The `derivepassphrase._types` submodule gains a new type for the socket providers, and for the registry metadata used by the `derivepassphrase.ssh_agent_socket_providers` entry point. The SSH agent client naturally changes its call signature, thus necessitating typing updates with all callers, direct or indirect. Somewhat non-obviously, establishing a socket connection is now a search operation, which can yield multiple unrelated errors. We thus make use of PEP 654 Exception Groups, and require new compatibility libraries to handle this in a cross-Python manner. Exception groups also make the error handling syntactically more complex, more so if we're using a compatibility library that restricts us to old (pre-3.11) syntax: it requires us to write our error handlers as separate functions, instead of inline code blocks, thus folding much of the contents of a function `f` into inner functions, or into separate functions outside of `f`. Therefore, we use this opportunity to streamline our error handling and reporting when calling into SSH agent code. Specifically, in the `cli_helpers` module, when interactively selecting an SSH key (`select_ssh_key`) and when converting a key to a master passphrase (`key_to_phrase`), we now accept and pass through connection hints to the SSH agent client constructor (now also in `key_to_phrase`), and handle errors and emit error messages already within the function (now also in `select_ssh_key`). In turn, both `select_ssh_key` and `key_to_phrase` now require error and warning callbacks to do useful error handling. https://git.schokokeks.org/derivepassphrase.git/commit/2413d9dc10ede315c295ab7520a19b21d597a668 software@the13thletter.info (Marco Ricci) Sat, 02 Aug 2025 13:12:43 +0200 2413d9dc10ede315c295ab7520a19b21d597a668 Turn the `derivepassphrase.ssh_agent` submodule into a subpackage, in anticipation of a new, yet-to-be-committed submodule of that subpackage. https://git.schokokeks.org/derivepassphrase.git/commit/56229e3d966688f3b0672c927583f19c6b2f7e37 software@the13thletter.info (Marco Ricci) Sat, 02 Aug 2025 09:28:34 +0200 56229e3d966688f3b0672c927583f19c6b2f7e37 We retire the error message for missing UNIX domain socket support, in favor of a more general error message indicating that this Python and/or this `derivepassphrase` installation lacks certain functionality. We further delegate the UNIX domain socket-specific message to a warning message. We also introduce a warning message specific to Windows named pipes. (Both warning messages are prepared, but not yet in actual use.) Unfortunately, this entails changing the symbol name for the error message, because it would otherwise be misleading. https://git.schokokeks.org/derivepassphrase.git/commit/b57626ae8c2e75e09b0c3f937140f27f3f3c1161 software@the13thletter.info (Marco Ricci) Sat, 02 Aug 2025 09:06:07 +0200 b57626ae8c2e75e09b0c3f937140f27f3f3c1161