Consolidate shell completion, add tests and fix Zsh output
Marco Ricci

Marco Ricci commited on 2025-01-07 12:10:27
Zeige 6 geänderte Dateien mit 916 Einfügungen und 52 Löschungen.


We move the shell completion code in `derivepassphrase/cli.py` to a new
section and retire the specific handling of `VAULT_PATH` for the two
path completion functions.  Having actually tried this out interactively
in Bash, it does not work that well if we complete filenames, partial
filenames, and fixed strings in the same completion function: you would
have to set conflicting completion options for this function.  It *also*
does not work at all with `click`'s stock Bash completion script, which
would then discard either the fixed strings and partial filenames or the
complete filenames, depending on the order they are emitted in by our
shell completion code.  So abandon the completion handling of
`VAULT_PATH` in favor of standard filename completion, for paths.

Every supported shell has further limitations on which inputs it can
properly deserialize: Bash strips NUL characters from command
substitutions, and Fish (v3) breaks in most situations involving
newlines.  Furthermore, the stock shell completion scripts add their own
additional limitations: all shells use newline-terminated messages, so
embedded newlines in the completion item (type, value, help text) cause
(generally silent) parsing failures; and the Zsh completion functions in
particular parse completions as "name:description" pairs, and thus need
colons in the name to be escaped.

We fix the colon handling of the Zsh completion script by providing
a fixed serialization handler for Zsh.  We avoid all of the other
aforementioned issues by not returning any service names containing
ASCII control characters as completion items.  We also warn the user
upon importing or configuring such a service that its service name will
not be available for shell completion.

Finally, we document this new warning in the manpage and the completion
behavior in the changelog, and add tests for (the Python side of) the
completion machinery, dependent on the current serialization format.

References:
[fish-shell#10874](https://github.com/fish-shell/fish-shell/issues/10874),
[fish-shell#9693](https://github.com/fish-shell/fish-shell/issues/9693),
[fish-shell#751](https://github.com/fish-shell/fish-shell/issues/751),
[fish-shell#10651](https://github.com/fish-shell/fish-shell/issues/10651),
[fish-shell#9847](https://github.com/fish-shell/fish-shell/issues/9847),
[click#2703](https://github.com/pallets/click/issues/2703).
... ...
@@ -1,5 +1,11 @@
1 1
 ### Added
2 2
 
3 3
   - `derivepassphrase` now explicitly supports shell completion, in
4
-    particular including filename and service name completion in the `export
5
-    vault` and `vault` subcommands.
4
+    particular filename and service name completion in the `export vault`
5
+    and `vault` subcommands.
6
+
7
+    However, because of restrictions regarding the exchange of data between
8
+    `derivepassphrase` and the shell, `derivepassphrase` will not offer any
9
+    service names containing ASCII control characters for completion, and
10
+    a warning will be issued when importing or configuring such a service.
11
+    They may still otherwise be used normally.
... ...
@@ -1,4 +1,4 @@
1
-.Dd 2024-12-25
1
+.Dd 2025-01-07
2 2
 .Dt DERIVEPASSPHRASE-VAULT 1
3 3
 .Os derivepassphrase 0.4.0
4 4
 .
... ...
@@ -802,6 +802,15 @@ When importing a configuration, the indicated ineffective setting has been
802 802
 removed.
803 803
 .Pq The Do interpretation Dc of the configuration doesn't change .
804 804
 .
805
+.It "The service name %s" "contains an ASCII control character," "which is not supported" "by our shell completion code."
806
+Because of limitations in the shell completion code, this specific service name
807
+will not be available as a suggestion in tab completion.
808
+.Po
809
+This
810
+.Em only
811
+affects tab completion, not other functionality.
812
+.Pc
813
+.
805 814
 .It Setting a %s passphrase is ineffective because a key is also set.
806 815
 The configuration (global or key-specific) contains both a stored master
807 816
 passphrase and an SSH key.
... ...
@@ -390,6 +390,11 @@ The <b>derivepassphrase vault</b> utility exits 0 on success, and >0 if an error
390 390
     When importing a configuration, the indicated ineffective setting has been removed.
391 391
     (The "interpretation" of the configuration doesn’t change).
392 392
 
393
+??? warning "`The service name %s contains an ASCII control character, which is not supported by our shell completion code.`"
394
+
395
+    Because of limitations in the shell completion code, this specific service name will not be available as a suggestion in tab completion.
396
+    (This *only* affects tab completion, not other functionality.)
397
+
393 398
 ??? warning "`Setting a %s passphrase is ineffective because a key is also set.`"
394 399
 
395 400
     The configuration (global or key-specific) contains both a stored master passphrase and an SSH key.
... ...
@@ -816,6 +816,19 @@ class WarnMsgTemplate(enum.Enum):
816 816
         context='warning message',
817 817
         flags='python-brace-format',
818 818
     )
819
+    SERVICE_NAME_INCOMPLETABLE = _prepare_translatable(
820
+        msg="""
821
+        The service name {service!r} contains an ASCII control
822
+        character, which is not supported by our shell completion code.
823
+        This service name will therefore not be available for completion
824
+        on the command-line.  You may of course still type it in
825
+        manually in whatever format your shell accepts, but we highly
826
+        recommend choosing a different service name instead.
827
+        """,
828
+        comments='',
829
+        context='warning message',
830
+        flags='python-brace-format',
831
+    )
819 832
     SERVICE_PASSPHRASE_INEFFECTIVE = _prepare_translatable(
820 833
         comments=r"""
821 834
         TRANSLATORS: The key that is set need not necessarily be set at
... ...
@@ -33,11 +33,13 @@ from typing import (
33 33
 )
34 34
 
35 35
 import click
36
+import click.shell_completion
36 37
 from typing_extensions import (
37 38
     Any,
38 39
     ParamSpec,
39 40
     Self,
40 41
     assert_never,
42
+    override,
41 43
 )
42 44
 
43 45
 import derivepassphrase as dpp
... ...
@@ -1102,6 +1104,135 @@ def standard_logging_options(f: Callable[P, R]) -> Callable[P, R]:
1102 1104
     """
1103 1105
     return debug_option(verbose_option(quiet_option(f)))
1104 1106
 
1107
+# Shell completion
1108
+# ================
1109
+
1110
+# Use naive filename completion for the `path` argument of
1111
+# `derivepassphrase vault`'s `--import` and `--export` options, as well
1112
+# as the `path` argument of `derivepassphrase export vault`.  The latter
1113
+# treats the pseudo-filename `VAULT_PATH` specially, but this is awkward
1114
+# to combine with standard filename completion, particularly in bash, so
1115
+# we would probably have to implement *all* completion (`VAULT_PATH` and
1116
+# filename completion) ourselves, lacking some niceties of bash's
1117
+# built-in completion (e.g., adding spaces or slashes depending on
1118
+# whether the completion is a directory or a complete filename).
1119
+
1120
+
1121
+def _shell_complete_path(
1122
+    ctx: click.Context,
1123
+    parameter: click.Parameter,
1124
+    value: str,
1125
+) -> list[str | click.shell_completion.CompletionItem]:
1126
+    """Request standard path completion for the `path` argument."""
1127
+    del ctx, parameter, value
1128
+    return [click.shell_completion.CompletionItem('', type='file')]  # noqa: DOC201
1129
+
1130
+
1131
+# The standard `click` shell completion scripts serialize the completion
1132
+# items as newline-separated one-line entries, which get silently
1133
+# corrupted if the value contains newlines.  Each shell imposes
1134
+# additional restrictions: Fish uses newlines in all internal completion
1135
+# helper scripts, so it is difficult, if not impossible, to register
1136
+# completion entries containing newlines if completion comes from within
1137
+# a Fish completion function (instead of a Fish builtin).  Zsh's
1138
+# completion system supports descriptions for each completion item, and
1139
+# the completion helper functions parse every entry as a colon-separated
1140
+# 2-tuple of item and description, meaning any colon in the item value
1141
+# must be escaped.  Finally, Bash requires the result array to be
1142
+# populated at the completion function's top-level scope, but for/while
1143
+# loops within pipelines do not run at top-level scope, and Bash *also*
1144
+# strips NUL characters from command substitution output, making it
1145
+# difficult to read in external data into an array in a cross-platform
1146
+# manner from entirely within Bash.
1147
+#
1148
+# We capitulate in front of these problems---most egregiously because of
1149
+# Fish---and ensure that completion items (in this case: service names)
1150
+# never contain ASCII control characters by refusing to offer such
1151
+# items as valid completions.  On the other side, `derivepassphrase`
1152
+# will warn the user when configuring or importing a service with such
1153
+# a name that it will not be available for shell completion.
1154
+
1155
+
1156
+def _is_completable_item(obj: object) -> bool:
1157
+    """Return whether the item is completable on the command-line.
1158
+
1159
+    The item is completable if and only if it contains no ASCII control
1160
+    characters (U+0000 through U+001F, and U+007F).
1161
+
1162
+    """
1163
+    obj = str(obj)
1164
+    forbidden = frozenset(chr(i) for i in range(32)) | {'\x7F'}
1165
+    return not any(f in obj for f in forbidden)
1166
+
1167
+
1168
+def _shell_complete_service(
1169
+    ctx: click.Context,
1170
+    parameter: click.Parameter,
1171
+    value: str,
1172
+) -> list[str | click.shell_completion.CompletionItem]:
1173
+    """Return known vault service names as completion items.
1174
+
1175
+    Service names are looked up in the vault configuration file.  All
1176
+    errors will be suppressed.  Additionally, any service names deemed
1177
+    not completable as per [`_is_completable_item`][] will be silently
1178
+    skipped.
1179
+
1180
+    """
1181
+    del ctx, parameter
1182
+    try:
1183
+        config = _load_config()
1184
+        return sorted(
1185
+            sv
1186
+            for sv in config['services']
1187
+            if sv.startswith(value) and _is_completable_item(sv)
1188
+        )
1189
+    except FileNotFoundError:
1190
+        try:
1191
+            config, _exc = _migrate_and_load_old_config()
1192
+            return sorted(
1193
+                sv
1194
+                for sv in config['services']
1195
+                if sv.startswith(value) and _is_completable_item(sv)
1196
+            )
1197
+        except FileNotFoundError:
1198
+            return []
1199
+    except Exception:  # noqa: BLE001
1200
+        return []
1201
+
1202
+
1203
+class ZshComplete(click.shell_completion.ZshComplete):
1204
+    """Zsh completion class that supports colons.
1205
+
1206
+    `click`'s Zsh completion class (at least v8.1.7 and v8.1.8) uses
1207
+    completion helper functions (provided by Zsh) that parse each
1208
+    completion item into value-description pairs, separated by a colon.
1209
+    Correspondingly, any internal colons in the completion item's value
1210
+    need to be escaped.  `click` doesn't do this.  So, this subclass
1211
+    overrides those parts, and adds the missing escaping.
1212
+
1213
+    """
1214
+
1215
+    @override
1216
+    def format_completion(
1217
+        self,
1218
+        item: click.shell_completion.CompletionItem,
1219
+    ) -> str:
1220
+        """Return a suitable serialization of the CompletionItem.
1221
+
1222
+        This serialization ensures colons in the item value are properly
1223
+        escaped.
1224
+
1225
+        """
1226
+        type, value, help = (  # noqa: A001
1227
+            item.type,
1228
+            item.value.replace(':', '\\:'),
1229
+            item.help or '_',
1230
+        )
1231
+        return f'{type}\n{value}\n{help}'
1232
+
1233
+
1234
+click.shell_completion.add_completion_class(ZshComplete)
1235
+
1105 1236
 
1106 1237
 # Top-level
1107 1238
 # =========
... ...
@@ -1380,23 +1511,6 @@ def _load_data(
1380 1511
         assert_never(fmt)
1381 1512
 
1382 1513
 
1383
-def _shell_complete_vault_path(  # pragma: no cover
1384
-    ctx: click.Context,
1385
-    param: click.Parameter,
1386
-    incomplete: str,
1387
-) -> list[str | click.shell_completion.CompletionItem]:
1388
-    del ctx, param
1389
-    if incomplete and 'VAULT_PATH'.startswith(incomplete):
1390
-        ret: set[str | click.shell_completion.CompletionItem] = {'VAULT_PATH'}
1391
-        for f in os.listdir():
1392
-            if f.startswith(incomplete):
1393
-                ret.add(f + os.path.sep if os.path.isdir(f) else f)
1394
-        return sorted(ret)
1395
-    return [
1396
-        click.shell_completion.CompletionItem('', type='file'),
1397
-    ]
1398
-
1399
-
1400 1514
 @derivepassphrase_export.command(
1401 1515
     'vault',
1402 1516
     context_settings={'help_option_names': ['-h', '--help']},
... ...
@@ -1456,7 +1570,7 @@ def _shell_complete_vault_path(  # pragma: no cover
1456 1570
     'path',
1457 1571
     metavar=_msg.TranslatedString(_msg.Label.EXPORT_VAULT_METAVAR_PATH),
1458 1572
     required=True,
1459
-    shell_complete=_shell_complete_vault_path,
1573
+    shell_complete=_shell_complete_path,
1460 1574
 )
1461 1575
 @click.pass_context
1462 1576
 def derivepassphrase_export_vault(
... ...
@@ -2258,36 +2372,6 @@ def _validate_length(
2258 2372
     return int_value
2259 2373
 
2260 2374
 
2261
-def _shell_complete_path(  # pragma: no cover
2262
-    ctx: click.Context,
2263
-    parameter: click.Parameter,
2264
-    incomplete: str,
2265
-) -> list[str | click.shell_completion.CompletionItem]:
2266
-    del ctx, parameter, incomplete
2267
-    return [click.shell_completion.CompletionItem('', type='file')]
2268
-
2269
-
2270
-def _shell_complete_service(  # pragma: no cover
2271
-    ctx: click.Context,
2272
-    parameter: click.Parameter,
2273
-    incomplete: str,
2274
-) -> list[str | click.shell_completion.CompletionItem]:
2275
-    del ctx, parameter
2276
-    try:
2277
-        config = _load_config()
2278
-        return [sv for sv in config['services'] if sv.startswith(incomplete)]
2279
-    except FileNotFoundError:
2280
-        try:
2281
-            config, _exc = _migrate_and_load_old_config()
2282
-            return [
2283
-                sv for sv in config['services'] if sv.startswith(incomplete)
2284
-            ]
2285
-        except FileNotFoundError:
2286
-            return []
2287
-    except Exception:  # noqa: BLE001
2288
-        return []
2289
-
2290
-
2291 2375
 DEFAULT_NOTES_TEMPLATE = """\
2292 2376
 # Enter notes below the line with the cut mark (ASCII scissors and
2293 2377
 # dashes).  Lines above the cut mark (such as this one) will be ignored.
... ...
@@ -3070,6 +3154,15 @@ def derivepassphrase_vault(  # noqa: C901,PLR0912,PLR0913,PLR0914,PLR0915
3070 3154
                 ),
3071 3155
                 extra={'color': ctx.color},
3072 3156
             )
3157
+        for service_name in sorted(maybe_config['services'].keys()):
3158
+            if not _is_completable_item(service_name):
3159
+                logger.warning(
3160
+                    _msg.TranslatedString(
3161
+                        _msg.WarnMsgTemplate.SERVICE_NAME_INCOMPLETABLE,
3162
+                        service=service_name,
3163
+                    ),
3164
+                    extra={'color': ctx.color},
3165
+                )
3073 3166
         try:
3074 3167
             _check_for_misleading_passphrase(
3075 3168
                 ('global',),
... ...
@@ -3336,6 +3429,14 @@ def derivepassphrase_vault(  # noqa: C901,PLR0912,PLR0913,PLR0914,PLR0915
3336 3429
                         setting=setting,
3337 3430
                     )
3338 3431
                     raise click.UsageError(str(err_msg))
3432
+            if not _is_completable_item(service):
3433
+                logger.warning(
3434
+                    _msg.TranslatedString(
3435
+                        _msg.WarnMsgTemplate.SERVICE_NAME_INCOMPLETABLE,
3436
+                        service=service,
3437
+                    ),
3438
+                    extra={'color': ctx.color},
3439
+                )
3339 3440
             subtree: dict[str, Any] = (
3340 3441
                 configuration['services'].setdefault(service, {})  # type: ignore[assignment]
3341 3442
                 if service
... ...
@@ -29,9 +29,12 @@ import tests
29 29
 from derivepassphrase import _types, cli, ssh_agent, vault
30 30
 
31 31
 if TYPE_CHECKING:
32
-    from collections.abc import Callable, Iterable, Iterator
32
+    from collections.abc import Callable, Iterable, Iterator, Sequence
33
+    from collections.abc import Set as AbstractSet
33 34
     from typing import NoReturn
34 35
 
36
+    from typing_extensions import Literal
37
+
35 38
 DUMMY_SERVICE = tests.DUMMY_SERVICE
36 39
 DUMMY_PASSPHRASE = tests.DUMMY_PASSPHRASE
37 40
 DUMMY_CONFIG_SETTINGS = tests.DUMMY_CONFIG_SETTINGS
... ...
@@ -3142,6 +3145,28 @@ class TestCLITransition:
3142 3145
             'Failed to migrate to ', caplog.record_tuples
3143 3146
         ), 'expected known warning message in stderr'
3144 3147
 
3148
+    def test_400_completion_service_name_old_config_file(
3149
+        self,
3150
+        monkeypatch: pytest.MonkeyPatch,
3151
+    ) -> None:
3152
+        runner = click.testing.CliRunner(mix_stderr=False)
3153
+        config = {'services': {DUMMY_SERVICE: DUMMY_CONFIG_SETTINGS.copy()}}
3154
+        with tests.isolated_vault_config(
3155
+            monkeypatch=monkeypatch,
3156
+            runner=runner,
3157
+            vault_config=config,
3158
+        ):
3159
+            old_name = cli._config_filename(subsystem='old settings.json')
3160
+            new_name = cli._config_filename(subsystem='vault')
3161
+            with contextlib.suppress(FileNotFoundError):
3162
+                os.remove(old_name)
3163
+            os.rename(new_name, old_name)
3164
+            assert cli._shell_complete_service(
3165
+                click.Context(cli.derivepassphrase),
3166
+                click.Argument(['some_parameter']),
3167
+                '',
3168
+            ) == [DUMMY_SERVICE]
3169
+
3145 3170
 
3146 3171
 _known_services = (DUMMY_SERVICE, 'email', 'bank', 'work')
3147 3172
 _valid_properties = (
... ...
@@ -3490,3 +3515,708 @@ class ConfigManagementStateMachine(stateful.RuleBasedStateMachine):
3490 3515
 
3491 3516
 
3492 3517
 TestConfigManagement = ConfigManagementStateMachine.TestCase
3518
+
3519
+
3520
+def bash_format(item: click.shell_completion.CompletionItem) -> str:
3521
+    type, value = (  # noqa: A001
3522
+        item.type,
3523
+        item.value,
3524
+    )
3525
+    return f'{type},{value}'
3526
+
3527
+
3528
+def fish_format(item: click.shell_completion.CompletionItem) -> str:
3529
+    type, value, help = (  # noqa: A001
3530
+        item.type,
3531
+        item.value,
3532
+        item.help,
3533
+    )
3534
+    return f'{type},{value}\t{help}' if help else f'{type},{value}'
3535
+
3536
+
3537
+def zsh_format(item: click.shell_completion.CompletionItem) -> str:
3538
+    type, value, help = (  # noqa: A001
3539
+        item.type,
3540
+        item.value.replace(':', r'\:'),
3541
+        item.help or '_',
3542
+    )
3543
+    return f'{type}\n{value}\n{help}'
3544
+
3545
+
3546
+def completion_item(
3547
+    item: str | click.shell_completion.CompletionItem
3548
+) -> click.shell_completion.CompletionItem:
3549
+    return (
3550
+        click.shell_completion.CompletionItem(item, type='plain')
3551
+        if isinstance(item, str)
3552
+        else item
3553
+    )
3554
+
3555
+
3556
+def assertable_item(
3557
+    item: str | click.shell_completion.CompletionItem
3558
+) -> tuple[str, Any, str | None]:
3559
+    item = completion_item(item)
3560
+    return (item.type, item.value, item.help)
3561
+
3562
+
3563
+class TestShellCompletion:
3564
+
3565
+    class Completions:
3566
+        def __init__(
3567
+            self,
3568
+            args: Sequence[str],
3569
+            incomplete: str,
3570
+        ) -> None:
3571
+            self.args = tuple(args)
3572
+            self.incomplete = incomplete
3573
+
3574
+        def __call__(self) -> Sequence[click.shell_completion.CompletionItem]:
3575
+            args = list(self.args)
3576
+            completion = click.shell_completion.ShellComplete(
3577
+                cli=cli.derivepassphrase,
3578
+                ctx_args={},
3579
+                prog_name='derivepassphrase',
3580
+                complete_var='_DERIVEPASSPHRASE_COMPLETE',
3581
+            )
3582
+            return completion.get_completions(args, self.incomplete)
3583
+
3584
+        def get_words(self) -> Sequence[str]:
3585
+            return tuple(c.value for c in self())
3586
+
3587
+    @pytest.mark.parametrize(
3588
+        ['partial', 'is_completable'],
3589
+        [
3590
+            ('', True),
3591
+            (DUMMY_SERVICE, True),
3592
+            ('a\bn', False),
3593
+            ('\b', False),
3594
+            ('\x00', False),
3595
+            ('\x20', True),
3596
+            ('\x7f', False),
3597
+            ('service with spaces', True),
3598
+            ('service\nwith\nnewlines', False),
3599
+        ]
3600
+    )
3601
+    def test_100_is_completable_item(
3602
+        self,
3603
+        partial: str,
3604
+        is_completable: bool,
3605
+    ) -> None:
3606
+        assert cli._is_completable_item(partial) == is_completable
3607
+
3608
+    @pytest.mark.parametrize(
3609
+        ['command_prefix', 'incomplete', 'completions'],
3610
+        [
3611
+            pytest.param(
3612
+                (),
3613
+                '-',
3614
+                frozenset({
3615
+                    '--help',
3616
+                    '-h',
3617
+                    '--version',
3618
+                    '--debug',
3619
+                    '--verbose',
3620
+                    '-v',
3621
+                    '--quiet',
3622
+                    '-q',
3623
+                }),
3624
+                id='derivepassphrase',
3625
+            ),
3626
+            pytest.param(
3627
+                ('export',),
3628
+                '-',
3629
+                frozenset({
3630
+                    '--help',
3631
+                    '-h',
3632
+                    '--version',
3633
+                    '--debug',
3634
+                    '--verbose',
3635
+                    '-v',
3636
+                    '--quiet',
3637
+                    '-q',
3638
+                }),
3639
+                id='derivepassphrase-export',
3640
+            ),
3641
+            pytest.param(
3642
+                ('export', 'vault'),
3643
+                '-',
3644
+                frozenset({
3645
+                    '--help',
3646
+                    '-h',
3647
+                    '--version',
3648
+                    '--debug',
3649
+                    '--verbose',
3650
+                    '-v',
3651
+                    '--quiet',
3652
+                    '-q',
3653
+                    '--format',
3654
+                    '-f',
3655
+                    '--key',
3656
+                    '-k',
3657
+                }),
3658
+                id='derivepassphrase-export-vault',
3659
+            ),
3660
+            pytest.param(
3661
+                ('vault',),
3662
+                '-',
3663
+                frozenset({
3664
+                    '--help',
3665
+                    '-h',
3666
+                    '--version',
3667
+                    '--debug',
3668
+                    '--verbose',
3669
+                    '-v',
3670
+                    '--quiet',
3671
+                    '-q',
3672
+                    '--phrase',
3673
+                    '-p',
3674
+                    '--key',
3675
+                    '-k',
3676
+                    '--length',
3677
+                    '-l',
3678
+                    '--repeat',
3679
+                    '-r',
3680
+                    '--upper',
3681
+                    '--lower',
3682
+                    '--number',
3683
+                    '--space',
3684
+                    '--dash',
3685
+                    '--symbol',
3686
+                    '--config',
3687
+                    '-c',
3688
+                    '--notes',
3689
+                    '-n',
3690
+                    '--delete',
3691
+                    '-x',
3692
+                    '--delete-globals',
3693
+                    '--clear',
3694
+                    '-X',
3695
+                    '--export',
3696
+                    '-e',
3697
+                    '--import',
3698
+                    '-i',
3699
+                    '--overwrite-existing',
3700
+                    '--merge-existing',
3701
+                    '--unset',
3702
+                    '--export-as',
3703
+                }),
3704
+                id='derivepassphrase-vault',
3705
+            ),
3706
+        ],
3707
+    )
3708
+    def test_200_options(
3709
+        self,
3710
+        command_prefix: Sequence[str],
3711
+        incomplete: str,
3712
+        completions: AbstractSet[str],
3713
+    ) -> None:
3714
+        comp = self.Completions(command_prefix, incomplete)
3715
+        assert frozenset(comp.get_words()) == completions
3716
+
3717
+    @pytest.mark.parametrize(
3718
+        ['command_prefix', 'incomplete', 'completions'],
3719
+        [
3720
+            pytest.param(
3721
+                (),
3722
+                '',
3723
+                frozenset({'export', 'vault'}),
3724
+                id='derivepassphrase',
3725
+            ),
3726
+            pytest.param(
3727
+                ('export',),
3728
+                '',
3729
+                frozenset({'vault'}),
3730
+                id='derivepassphrase-export',
3731
+            ),
3732
+        ],
3733
+    )
3734
+    def test_201_subcommands(
3735
+        self,
3736
+        command_prefix: Sequence[str],
3737
+        incomplete: str,
3738
+        completions: AbstractSet[str],
3739
+    ) -> None:
3740
+        comp = self.Completions(command_prefix, incomplete)
3741
+        assert frozenset(comp.get_words()) == completions
3742
+
3743
+    @pytest.mark.parametrize(
3744
+        'command_prefix',
3745
+        [
3746
+            pytest.param(
3747
+                ('export', 'vault'),
3748
+                id='derivepassphrase-export-vault',
3749
+            ),
3750
+            pytest.param(
3751
+                ('vault', '--export'),
3752
+                id='derivepassphrase-vault--export',
3753
+            ),
3754
+            pytest.param(
3755
+                ('vault', '--import'),
3756
+                id='derivepassphrase-vault--import',
3757
+            ),
3758
+        ],
3759
+    )
3760
+    @pytest.mark.parametrize('incomplete', ['', 'partial'])
3761
+    def test_202_paths(
3762
+        self,
3763
+        command_prefix: Sequence[str],
3764
+        incomplete: str,
3765
+    ) -> None:
3766
+        file = click.shell_completion.CompletionItem('', type='file')
3767
+        completions = frozenset({(file.type, file.value, file.help)})
3768
+        comp = self.Completions(command_prefix, incomplete)
3769
+        assert frozenset(
3770
+            (x.type, x.value, x.help) for x in comp()
3771
+        ) == completions
3772
+
3773
+    @pytest.mark.parametrize(
3774
+        ['config', 'incomplete', 'completions'],
3775
+        [
3776
+            pytest.param(
3777
+                {"services": {}},
3778
+                '',
3779
+                frozenset(),
3780
+                id='no_services',
3781
+            ),
3782
+            pytest.param(
3783
+                {"services": {}},
3784
+                'partial',
3785
+                frozenset(),
3786
+                id='no_services_partial',
3787
+            ),
3788
+            pytest.param(
3789
+                {"services": {DUMMY_SERVICE: {"length": 10}}},
3790
+                '',
3791
+                frozenset({DUMMY_SERVICE}),
3792
+                id='one_service',
3793
+            ),
3794
+            pytest.param(
3795
+                {"services": {DUMMY_SERVICE: {"length": 10}}},
3796
+                DUMMY_SERVICE[:4],
3797
+                frozenset({DUMMY_SERVICE}),
3798
+                id='one_service_partial',
3799
+            ),
3800
+            pytest.param(
3801
+                {"services": {DUMMY_SERVICE: {"length": 10}}},
3802
+                DUMMY_SERVICE[-4:],
3803
+                frozenset(),
3804
+                id='one_service_partial_miss',
3805
+            ),
3806
+        ],
3807
+    )
3808
+    def test_203_service_names(
3809
+        self,
3810
+        monkeypatch: pytest.MonkeyPatch,
3811
+        config: _types.VaultConfig,
3812
+        incomplete: str,
3813
+        completions: AbstractSet[str],
3814
+    ) -> None:
3815
+        runner = click.testing.CliRunner(mix_stderr=False)
3816
+        with tests.isolated_vault_config(
3817
+            monkeypatch=monkeypatch,
3818
+            runner=runner,
3819
+            vault_config=config,
3820
+        ):
3821
+            comp = self.Completions(['vault'], incomplete)
3822
+            assert frozenset(comp.get_words()) == completions
3823
+
3824
+    @pytest.mark.parametrize(
3825
+        ['shell', 'format_func'],
3826
+        [
3827
+            pytest.param('bash', bash_format, id='bash'),
3828
+            pytest.param('fish', fish_format, id='fish'),
3829
+            pytest.param('zsh', zsh_format, id='zsh'),
3830
+        ],
3831
+    )
3832
+    @pytest.mark.parametrize(
3833
+        ['config', 'comp_func', 'args', 'incomplete', 'results'],
3834
+        [
3835
+            pytest.param(
3836
+                {"services": {DUMMY_SERVICE: DUMMY_CONFIG_SETTINGS.copy()}},
3837
+                cli._shell_complete_service,
3838
+                ['vault'],
3839
+                '',
3840
+                [DUMMY_SERVICE],
3841
+                id='base_config-service',
3842
+            ),
3843
+            pytest.param(
3844
+                {"services": {}},
3845
+                cli._shell_complete_service,
3846
+                ['vault'],
3847
+                '',
3848
+                [],
3849
+                id='empty_config-service',
3850
+            ),
3851
+            pytest.param(
3852
+                {
3853
+                    "services": {
3854
+                        DUMMY_SERVICE: DUMMY_CONFIG_SETTINGS.copy(),
3855
+                        "newline\nin\nname": DUMMY_CONFIG_SETTINGS.copy(),
3856
+                    }
3857
+                },
3858
+                cli._shell_complete_service,
3859
+                ['vault'],
3860
+                '',
3861
+                [DUMMY_SERVICE],
3862
+                id='incompletable_newline_config-service',
3863
+            ),
3864
+            pytest.param(
3865
+                {
3866
+                    "services": {
3867
+                        DUMMY_SERVICE: DUMMY_CONFIG_SETTINGS.copy(),
3868
+                        "backspace\bin\bname": DUMMY_CONFIG_SETTINGS.copy(),
3869
+                    }
3870
+                },
3871
+                cli._shell_complete_service,
3872
+                ['vault'],
3873
+                '',
3874
+                [DUMMY_SERVICE],
3875
+                id='incompletable_backspace_config-service',
3876
+            ),
3877
+            pytest.param(
3878
+                {
3879
+                    "services": {
3880
+                        DUMMY_SERVICE: DUMMY_CONFIG_SETTINGS.copy(),
3881
+                        "colon:in:name": DUMMY_CONFIG_SETTINGS.copy(),
3882
+                    }
3883
+                },
3884
+                cli._shell_complete_service,
3885
+                ['vault'],
3886
+                '',
3887
+                sorted([DUMMY_SERVICE, 'colon:in:name']),
3888
+                id='brittle_colon_config-service',
3889
+            ),
3890
+            pytest.param(
3891
+                {
3892
+                    "services": {
3893
+                        DUMMY_SERVICE: DUMMY_CONFIG_SETTINGS.copy(),
3894
+                        "colon:in:name": DUMMY_CONFIG_SETTINGS.copy(),
3895
+                        "newline\nin\nname": DUMMY_CONFIG_SETTINGS.copy(),
3896
+                        "backspace\bin\bname": DUMMY_CONFIG_SETTINGS.copy(),
3897
+                        "nul\x00in\x00name": DUMMY_CONFIG_SETTINGS.copy(),
3898
+                        "del\x7fin\x7fname": DUMMY_CONFIG_SETTINGS.copy(),
3899
+                    }
3900
+                },
3901
+                cli._shell_complete_service,
3902
+                ['vault'],
3903
+                '',
3904
+                sorted([DUMMY_SERVICE, 'colon:in:name']),
3905
+                id='brittle_incompletable_multi_config-service',
3906
+            ),
3907
+            pytest.param(
3908
+                {"services": {DUMMY_SERVICE: DUMMY_CONFIG_SETTINGS.copy()}},
3909
+                cli._shell_complete_path,
3910
+                ['vault', '--import'],
3911
+                '',
3912
+                [click.shell_completion.CompletionItem('', type='file')],
3913
+                id='base_config-path',
3914
+            ),
3915
+            pytest.param(
3916
+                {"services": {}},
3917
+                cli._shell_complete_path,
3918
+                ['vault', '--import'],
3919
+                '',
3920
+                [click.shell_completion.CompletionItem('', type='file')],
3921
+                id='empty_config-path',
3922
+            ),
3923
+        ],
3924
+    )
3925
+    def test_300_shell_completion_formatting(
3926
+        self,
3927
+        monkeypatch: pytest.MonkeyPatch,
3928
+        shell: str,
3929
+        format_func: Callable[[click.shell_completion.CompletionItem], str],
3930
+        config: _types.VaultConfig,
3931
+        comp_func: Callable[
3932
+            [click.Context, click.Parameter, str],
3933
+            list[str | click.shell_completion.CompletionItem]
3934
+        ],
3935
+        args: list[str],
3936
+        incomplete: str,
3937
+        results: list[str | click.shell_completion.CompletionItem],
3938
+    ) -> None:
3939
+        runner = click.testing.CliRunner(mix_stderr=False)
3940
+        with tests.isolated_vault_config(
3941
+            monkeypatch=monkeypatch,
3942
+            runner=runner,
3943
+            vault_config=config,
3944
+        ):
3945
+            expected_items = [
3946
+                assertable_item(item)
3947
+                for item in results
3948
+            ]
3949
+            expected_string = '\n'.join(
3950
+                format_func(completion_item(item))
3951
+                for item in results
3952
+            )
3953
+            manual_raw_items = comp_func(
3954
+                click.Context(cli.derivepassphrase),
3955
+                click.Argument(['sample_parameter']),
3956
+                incomplete,
3957
+            )
3958
+            manual_items = [
3959
+                assertable_item(item)
3960
+                for item in manual_raw_items
3961
+            ]
3962
+            manual_string = '\n'.join(
3963
+                format_func(completion_item(item))
3964
+                for item in manual_raw_items
3965
+            )
3966
+            assert manual_items == expected_items
3967
+            assert manual_string == expected_string
3968
+            comp_class = click.shell_completion.get_completion_class(shell)
3969
+            assert comp_class is not None
3970
+            comp = comp_class(
3971
+                cli.derivepassphrase,
3972
+                {},
3973
+                'derivepassphrase',
3974
+                '_DERIVEPASSPHRASE_COMPLETE',
3975
+            )
3976
+            monkeypatch.setattr(
3977
+                comp,
3978
+                'get_completion_args',
3979
+                lambda *_a, **_kw: (args, incomplete),
3980
+            )
3981
+            actual_raw_items = comp.get_completions(
3982
+                *comp.get_completion_args()
3983
+            )
3984
+            actual_items = [
3985
+                assertable_item(item)
3986
+                for item in actual_raw_items
3987
+            ]
3988
+            actual_string = comp.complete()
3989
+            assert actual_items == expected_items
3990
+            assert actual_string == expected_string
3991
+
3992
+    @pytest.mark.parametrize('mode', ['config', 'import'])
3993
+    @pytest.mark.parametrize(
3994
+        ['config', 'key', 'incomplete', 'completions'],
3995
+        [
3996
+            pytest.param(
3997
+                {
3998
+                    "services": {
3999
+                        DUMMY_SERVICE: {"length": 10},
4000
+                        "newline\nin\nname": {"length": 10},
4001
+                    },
4002
+                },
4003
+                'newline\nin\nname',
4004
+                '',
4005
+                frozenset({DUMMY_SERVICE}),
4006
+                id='newline',
4007
+            ),
4008
+            pytest.param(
4009
+                {
4010
+                    "services": {
4011
+                        DUMMY_SERVICE: {"length": 10},
4012
+                        "newline\nin\nname": {"length": 10},
4013
+                    },
4014
+                },
4015
+                'newline\nin\nname',
4016
+                'serv',
4017
+                frozenset({DUMMY_SERVICE}),
4018
+                id='newline_partial_other',
4019
+            ),
4020
+            pytest.param(
4021
+                {
4022
+                    "services": {
4023
+                        DUMMY_SERVICE: {"length": 10},
4024
+                        "newline\nin\nname": {"length": 10},
4025
+                    },
4026
+                },
4027
+                'newline\nin\nname',
4028
+                'newline',
4029
+                frozenset({}),
4030
+                id='newline_partial_specific',
4031
+            ),
4032
+            pytest.param(
4033
+                {
4034
+                    "services": {
4035
+                        DUMMY_SERVICE: {"length": 10},
4036
+                        "nul\x00in\x00name": {"length": 10},
4037
+                    },
4038
+                },
4039
+                'nul\x00in\x00name',
4040
+                '',
4041
+                frozenset({DUMMY_SERVICE}),
4042
+                id='nul',
4043
+            ),
4044
+            pytest.param(
4045
+                {
4046
+                    "services": {
4047
+                        DUMMY_SERVICE: {"length": 10},
4048
+                        "nul\x00in\x00name": {"length": 10},
4049
+                    },
4050
+                },
4051
+                'nul\x00in\x00name',
4052
+                'serv',
4053
+                frozenset({DUMMY_SERVICE}),
4054
+                id='nul_partial_other',
4055
+            ),
4056
+            pytest.param(
4057
+                {
4058
+                    "services": {
4059
+                        DUMMY_SERVICE: {"length": 10},
4060
+                        "nul\x00in\x00name": {"length": 10},
4061
+                    },
4062
+                },
4063
+                'nul\x00in\x00name',
4064
+                'nul',
4065
+                frozenset({}),
4066
+                id='nul_partial_specific',
4067
+            ),
4068
+            pytest.param(
4069
+                {
4070
+                    "services": {
4071
+                        DUMMY_SERVICE: {"length": 10},
4072
+                        "backspace\bin\bname": {"length": 10},
4073
+                    },
4074
+                },
4075
+                'backspace\bin\bname',
4076
+                '',
4077
+                frozenset({DUMMY_SERVICE}),
4078
+                id='backspace',
4079
+            ),
4080
+            pytest.param(
4081
+                {
4082
+                    "services": {
4083
+                        DUMMY_SERVICE: {"length": 10},
4084
+                        "backspace\bin\bname": {"length": 10},
4085
+                    },
4086
+                },
4087
+                'backspace\bin\bname',
4088
+                'serv',
4089
+                frozenset({DUMMY_SERVICE}),
4090
+                id='backspace_partial_other',
4091
+            ),
4092
+            pytest.param(
4093
+                {
4094
+                    "services": {
4095
+                        DUMMY_SERVICE: {"length": 10},
4096
+                        "backspace\bin\bname": {"length": 10},
4097
+                    },
4098
+                },
4099
+                'backspace\bin\bname',
4100
+                'back',
4101
+                frozenset({}),
4102
+                id='backspace_partial_specific',
4103
+            ),
4104
+            pytest.param(
4105
+                {
4106
+                    "services": {
4107
+                        DUMMY_SERVICE: {"length": 10},
4108
+                        "del\x7fin\x7fname": {"length": 10},
4109
+                    },
4110
+                },
4111
+                'del\x7fin\x7fname',
4112
+                '',
4113
+                frozenset({DUMMY_SERVICE}),
4114
+                id='del',
4115
+            ),
4116
+            pytest.param(
4117
+                {
4118
+                    "services": {
4119
+                        DUMMY_SERVICE: {"length": 10},
4120
+                        "del\x7fin\x7fname": {"length": 10},
4121
+                    },
4122
+                },
4123
+                'del\x7fin\x7fname',
4124
+                'serv',
4125
+                frozenset({DUMMY_SERVICE}),
4126
+                id='del_partial_other',
4127
+            ),
4128
+            pytest.param(
4129
+                {
4130
+                    "services": {
4131
+                        DUMMY_SERVICE: {"length": 10},
4132
+                        "del\x7fin\x7fname": {"length": 10},
4133
+                    },
4134
+                },
4135
+                'del\x7fin\x7fname',
4136
+                'del',
4137
+                frozenset({}),
4138
+                id='del_partial_specific',
4139
+            ),
4140
+        ],
4141
+    )
4142
+    def test_400_incompletable_service_names(
4143
+        self,
4144
+        monkeypatch: pytest.MonkeyPatch,
4145
+        caplog: pytest.LogCaptureFixture,
4146
+        mode: Literal['config', 'import'],
4147
+        config: _types.VaultConfig,
4148
+        key: str,
4149
+        incomplete: str,
4150
+        completions: AbstractSet[str],
4151
+    ) -> None:
4152
+        runner = click.testing.CliRunner(mix_stderr=False)
4153
+        vault_config = config if mode == 'config' else {'services': {}}
4154
+        with tests.isolated_vault_config(
4155
+            monkeypatch=monkeypatch,
4156
+            runner=runner,
4157
+            vault_config=vault_config,
4158
+        ):
4159
+            if mode == 'config':
4160
+                _result = runner.invoke(
4161
+                    cli.derivepassphrase_vault,
4162
+                    ['--config', '--length=10', '--', key],
4163
+                    catch_exceptions=False,
4164
+                )
4165
+            else:
4166
+                _result = runner.invoke(
4167
+                    cli.derivepassphrase_vault,
4168
+                    ['--import', '-'],
4169
+                    catch_exceptions=False,
4170
+                    input=json.dumps(config),
4171
+                )
4172
+            result = tests.ReadableResult.parse(_result)
4173
+            assert result.clean_exit(), 'expected clean exit'
4174
+            assert tests.warning_emitted(
4175
+                'contains an ASCII control character', caplog.record_tuples
4176
+            ), 'expected known warning message in stderr'
4177
+            assert tests.warning_emitted(
4178
+                'not be available for completion', caplog.record_tuples
4179
+            ), 'expected known warning message in stderr'
4180
+            assert cli._load_config() == config
4181
+            comp = self.Completions(['vault'], incomplete)
4182
+            assert frozenset(comp.get_words()) == completions
4183
+
4184
+    def test_410a_service_name_exceptions_not_found(
4185
+        self,
4186
+        monkeypatch: pytest.MonkeyPatch,
4187
+    ) -> None:
4188
+        runner = click.testing.CliRunner(mix_stderr=False)
4189
+        with tests.isolated_vault_config(
4190
+            monkeypatch=monkeypatch,
4191
+            runner=runner,
4192
+            vault_config={'services': {DUMMY_SERVICE: DUMMY_CONFIG_SETTINGS}},
4193
+        ):
4194
+            os.remove(cli._config_filename(subsystem='vault'))
4195
+            assert not cli._shell_complete_service(
4196
+                click.Context(cli.derivepassphrase),
4197
+                click.Argument(['some_parameter']),
4198
+                '',
4199
+            )
4200
+
4201
+    @pytest.mark.parametrize('exc_type', [RuntimeError, KeyError, ValueError])
4202
+    def test_410b_service_name_exceptions_custom_error(
4203
+        self,
4204
+        monkeypatch: pytest.MonkeyPatch,
4205
+        exc_type: type[Exception],
4206
+    ) -> None:
4207
+        runner = click.testing.CliRunner(mix_stderr=False)
4208
+        with tests.isolated_vault_config(
4209
+            monkeypatch=monkeypatch,
4210
+            runner=runner,
4211
+            vault_config={'services': {DUMMY_SERVICE: DUMMY_CONFIG_SETTINGS}},
4212
+        ):
4213
+
4214
+            def raiser(*_a: Any, **_kw: Any) -> NoReturn:
4215
+                raise exc_type('just being difficult')  # noqa: EM101,TRY003
4216
+
4217
+            monkeypatch.setattr(cli, '_load_config', raiser)
4218
+            assert not cli._shell_complete_service(
4219
+                click.Context(cli.derivepassphrase),
4220
+                click.Argument(['some_parameter']),
4221
+                '',
4222
+            )
3493 4223