Recent commits to derivepassphrase.git (da49d3bf1fe6ecf850520922fd36381cf3191c26)

Add tests for the SSH agent socket provider machinery

2025-08-02T13:50:27+02:00

We add the stub agent to the list of spawnable SSH agents, since it is
always available.  This necessitates a lot of cleanup to old test suite
functionality that assumes SSH agents are available if and only if UNIX
domain sockets are supported.  The `PERMITTED_SSH_AGENTS` environment
variable now controls which (external) SSH agents will be attempted to
be spawned, if supported by the system.  Additionally, the existing
agents that can be spawned are all UNIX-only, and correctly marked as
such; this avoids attempting to spawn Pageant on The Annoying Operating
System using a UNIX Pageant command-line.

We include a stub for the entry points list/table from
`importlib.metadata`, which sadly is rather ugly due to having to
reflect both an explicit API change in Python 3.12, and an undocumented
and hairy compatibility hack in Python 3.10.

We add a `SSHAgentSocketProviderRegistryStateMachine` class for testing
socket provider registrations via a `hypothesis` state machine, and
quite a few new entries in the `TestStaticFunctionality` class relating
to locating and registering socket providers on handcrafted explicit
examples.  I have resisted the temptation to implement all registry
tests as `hypothesis` tests, in favor of having at least *some* unit
tests left over to test the registry, even if skipping
`hypothesis`-based tests for speed reasons.

Finally, we split the monolithic test
`TestCLIUtils.test_400_key_to_phrase` from the CLI tests into
a parametrized series of smaller tests.
</pre>

Introduce a stubbed SSH agent, for testing

2025-08-02T13:22:55+02:00

Introduce a stubbed SSH agent socket that simulates a real agent's
responses (for test keys only).  Include tests to verify the correct
workings of the agent.

The stub agent comes in three versions.  The basic version implements
all the necessary functionality, but intentionally disables SSH agent
extension support.  The "address" version additionally supports
generating errors upon construction by both requiring an address (in
`SSH_AUTH_SOCK`) and by raising specific errors for specific addresses.
The "deterministic DSA" version additionally supports the "query" and
the "list-extended@putty.projects.tartarus.org" extension, and yields
the recorded RFC 6979 deterministic DSA signatures for DSA and ECDSA
test keys.  The "address" and "deterministic DSA" versions are intended
for use in the test suite, as test doubles for real SSH agents,
depending on whether deterministic DSA signatures are required or not.
The "basic" version is intended for verification of the stubbed SSH
agent itself, whereever it can be used in place of more powerful stubbed
SSH agent versions.

Though tested *individually*, as a new piece of code added to the code
base, the stub agent (in all its variations) is not yet *integrated*
into the test suite as a "proper", spawnable SSH agent.  That we leave
to the following commits.
</pre>

Introduce SSH agent socket providers

2025-08-02T13:12:43+02:00

Introduce a new class of objects, SSH agent socket providers, that
provide sockets connected to SSH agents to the SSH agent client.  In
turn, the clients now depend on the socket providers for establishing
the connection to the agent.  Upon construction, the client queries an
implicit or explicit list of such provider names (from a local, in-code
registry) and uses the first provider that successfully returns
a connected socket.  Providers may be, uh, provided by
`derivepassphrase` itself, or by third-party developers, and they may
depend, e.g., on a certain operating system or Python installation, or
certain installed third-party software.

`derivepassphrase` includes three standard providers: `posix`,
`the_annoying_os` and `native`.  The `posix` provider attempts to
connect via UNIX domain sockets and the `SSH_AUTH_SOCK` environment
variable, as `derivepassphrase` supported it before this commit.  The
`the_annoying_os` provider, currently a stub, is intended to eventually
support connecting via Windows named pipes.  The `native` provider is an
alias to whichever of `posix` and `the_annoying_os` is more appropriate
to this `derivepassphrase` installation.  `derivepassphrase` further
registers and reserves several provider names related to testing, as
well as several aliases for the aforementioned three standard providers.

Third-party socket providers are possible through the
`derivepassphrase.ssh_agent_socket_providers` Python package entry
point, which expects a `SSHAgentSocketProviderEntry` object.  See the
corresponding documentation in the `_types` module.

The intent of the socket provider system is to decouple the SSH agent
client code from the establishing of the connection to the agent.  The
client code can then be (mostly) operating system- and agent-agnostic,
and in turn, the socket provider code can be as operating system- and
third party software-specific as necessary.  Third-party developers can
also add socket providers that cannot or should not be included
with `derivepassphrase` because they are niche, or require additional
software, or have a different level of stability or development cycle
length than `derivepassphrase` has.  Additionally, the decoupled
system can be more easily tested, by mocking the other side of the
connection (which also extends to third-party socket providers).
Several socket provider names are already reserved for the test suite,
as mentioned above.

This new functionality necessitates some large-ish code changes.

The new submodule `derivepassphrase.ssh_agent.socketprovider` contains
all socket provider-related functionality and data, including the
provider registry.  The `derivepassphrase._types` submodule gains a new
type for the socket providers, and for the registry metadata used by the
`derivepassphrase.ssh_agent_socket_providers` entry point.  The SSH
agent client naturally changes its call signature, thus necessitating
typing updates with all callers, direct or indirect.

Somewhat non-obviously, establishing a socket connection is now a search
operation, which can yield multiple unrelated errors.  We thus make use
of PEP 654 Exception Groups, and require new compatibility libraries to
handle this in a cross-Python manner.

Exception groups also make the error handling syntactically more
complex, more so if we're using a compatibility library that restricts
us to old (pre-3.11) syntax: it requires us to write our error handlers
as separate functions, instead of inline code blocks, thus folding much
of the contents of a function `f` into inner functions, or into separate
functions outside of `f`.  Therefore, we use this opportunity to
streamline our error handling and reporting when calling into SSH agent
code.  Specifically, in the `cli_helpers` module, when interactively
selecting an SSH key (`select_ssh_key`) and when converting a key to
a master passphrase (`key_to_phrase`), we now accept and pass through
connection hints to the SSH agent client constructor (now also in
`key_to_phrase`), and handle errors and emit error messages already
within the function (now also in `select_ssh_key`).  In turn, both
`select_ssh_key` and `key_to_phrase` now require error and warning
callbacks to do useful error handling.
</pre>

Turn the `ssh_agent` submodule into a subpackage

2025-08-02T09:28:34+02:00

Turn the `derivepassphrase.ssh_agent` submodule into a subpackage, in
anticipation of a new, yet-to-be-committed submodule of that subpackage.
</pre>

Generalize the error message for missing SSH agent support

2025-08-02T09:06:07+02:00

We retire the error message for missing UNIX domain socket support, in
favor of a more general error message indicating that this Python and/or
this `derivepassphrase` installation lacks certain functionality.

We further delegate the UNIX domain socket-specific message to a warning
message.  We also introduce a warning message specific to Windows named
pipes.  (Both warning messages are prepared, but not yet in actual use.)

Unfortunately, this entails changing the symbol name for the error
message, because it would otherwise be misleading.
</pre>

Merge topic branch 'test-suite-cleanups' into master

2025-07-27T23:12:36+02:00

* test-suite-cleanups:
  Comment on coverage exclusions, in shorthand
  Provide more deterministic signatures for the test keys
  Fix, and test for, the consistency of the SSH test keys
  Fix some test suite setup mistakes
</pre>

Comment on coverage exclusions, in shorthand

2025-07-26T17:35:21+02:00

Add shorthand codes for different types of coverage exclusions,
explained in `pyproject.toml` in the `tool.coverage` section, because
reasons for excluding code branches from coverage tend to fall into
a handful of common categories.

Also add explicit coverage exclusion patterns for dummy classes which
are never intended to actually be called, such as the `_types._Omitted`
class (for styling function signatures in the generated documentation)
and the two `_DummyModule` classes (which stub out missing dependencies
to appease the type checker).

A very few coverage exclusions were actually unnecessary or nonsensical,
and have been rectified.
</pre>

Provide more deterministic signatures for the test keys

2025-07-26T16:45:41+02:00

Record and provide more deterministic signatures for the test keys,
where feasible.  In particular, for DSA and ECDSA keys, support
recording the RFC 6979 deterministic DSA signatures, as implemented by
Pageant.  In particular, this entails that the `expected_signature` and
`derived_passphrase` fields of the test keys need to change to
accomodate the new shape of the data.

For the existing DSA and ECDSA keys, RFC 6979 signatures and expected
master passphrases are also provided.  Providing the Pageant 0.68–0.80
deterministic signatures and expected master passphrases however is not
quite so straight-forward: because the signature class is the subject of
CVE-2024-31497, Pageant < 0.81 is considered to have a security hole,
and thus it is hard to obtain pre-compiled, unpatched installations of
Pageant to compute the signatures/master passphrases against.
</pre>

Fix, and test for, the consistency of the SSH test keys

2025-07-26T11:45:19+02:00

A typo existed in the ECDSA NIST P-521 raw public key data, making it
incompatible with the copy of the public key included in the raw private
key blob. This caused UNIX Pageant (and likely any other SSH agent
supporting deterministic DSA signatures, if they exist) to reject
unloading and signing with that key, despite actually supporting the key
format. So fix the typo, and add tests and machinery to ensure
rudimentary consistency of all test key data.
</pre>

Fix some test suite setup mistakes

2025-07-23T20:13:06+02:00

These mistakes were carried over from v0.5, and cause woe on The
Annoying OS even before actually running any tests.
</pre>