[bug-fix] Fix reinstall-overwrites-kept-config: preserve config on plain reinstall after --keep-config#3449
Open
github-actions[bot] wants to merge 24 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes extension config loss when reinstalling after remove --keep-config.
Changes:
- Rescues and restores preserved extension configuration files.
- Adds regression tests for standard and local configs.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/extensions/__init__.py |
Adds preserved-config rescue and restoration. |
tests/test_extensions.py |
Adds reinstall config-preservation tests. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 3
- Review effort level: Medium
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…rt' and 'import from'' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
| if target_fd is not None: | ||
| try: | ||
| os.close(target_fd) | ||
| except OSError: |
| from unittest.mock import MagicMock | ||
|
|
||
| from tests.conftest import strip_ansi | ||
| import specify_cli.extensions as _ext_module |
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
On Windows os.open() defaults to text mode, so os.write() of preserved config bytes containing \r\n was translated to \r\r\n, corrupting the staged backup and failing the retry-restore regression test. Add O_BINARY (0 on POSIX) to the staging file open flags. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The .extensionignore loader can raise ValidationError (invalid UTF-8) or OSError. Previously it ran after dest_dir was removed, so such a failure left the kept config only in the hidden staging directory rather than its documented location. Load/validate it before the rmtree so every post-deletion failure path restores the config. Adds a regression test. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Loading .extensionignore after the rescue staging directory was published meant a validation failure left a complete staging copy behind. A later retry (after the user fixed the ignore file and edited the kept config) would reload the stale staged bytes and silently overwrite the newer config. Move the loader ahead of reading/creating rescue staging so a failure aborts while the kept config is still authoritative on disk, and extend the regression test to prove no staging is published and a retry adopts the newer bytes. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment on lines
+1487
to
+1491
| if staging_is_complete and not self.registry.is_installed(manifest.id): | ||
| # A previous install attempt staged the configs but never | ||
| # completed cleanly. Reload from the durable backup so the | ||
| # original bytes are used on retry rather than whatever | ||
| # mixture of packaged defaults and partial restores remains |
| # staging dir is removed only after every config has been successfully | ||
| # restored. | ||
| stranded_configs: dict[str, tuple[bytes, int]] = {} | ||
| rescue_staging_dir = self.extensions_dir / f".rescue-staging-{manifest.id}" |
| with tempfile.NamedTemporaryFile( | ||
| mode="wb", | ||
| dir=target.parent, | ||
| prefix=f".{target.name}.", |
Address three review findings on the reinstall config-rescue path: - A complete .rescue-complete marker proves only that staging finished, not that dest_dir was modified. A crash after staging sync but before the rmtree leaves the live kept config intact; if the user edits it before retrying, preferring the staged bytes silently overwrote the newer config. The two copies are indistinguishable in provenance from disk, so detect divergence between a complete staging copy and the live config and abort (preserving both) instead of unconditionally choosing staging. - The staging directory embedded the full extension ID in one path component. Extension IDs are length-unbounded, so a valid long ID could install at dest_dir yet fail every reinstall-after-keep-config with ENAMETOOLONG. Derive the staging component from a fixed-length hash via a new _rescue_staging_dir() helper. - The stranded-config restore used the full config filename as a NamedTemporaryFile prefix; a name already near the component limit plus the random suffix raised ENAMETOOLONG. Use a short fixed prefix. Updates the retry regression test to the new divergence semantics and adds conflict-abort, long-ID, and fixed-prefix coverage. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug fix — reinstall-overwrites-kept-config
Proposed fix for issue #3427, applying the remediation from the bug assessment.
Verdict: Valid · Severity: medium
Summary
When
specify extension remove <ext> --keep-configis used, the extension is unregistered but its*-config.ymlfiles survive in the extension directory. A subsequent plainspecify extension add <ext>unconditionally deleted that directory before copying the fresh extension in, silently discarding the preserved config. The fix rescues those stranded config files into memory before thermtreeand writes them back aftercopytree, so user-customized values always win over the packaged defaults.Changes
src/specify_cli/extensions/__init__.pyrmtree(dest_dir), collect any*-config.yml/*-config.local.ymlfiles from an unregistereddest_dirinto memory; restore them aftercopytreetests/test_extensions.pytest_reinstall_after_keep_config_preserves_configandtest_reinstall_after_keep_config_preserves_local_configTests Added or Updated
tests/test_extensions.py::TestInstallFromDirectory::test_reinstall_after_keep_config_preserves_config— pins that a customized*-config.ymlsurvives aremove --keep-config→ plain reinstall cycletests/test_extensions.py::TestInstallFromDirectory::test_reinstall_after_keep_config_preserves_local_config— same for*-config.local.ymloverride filesLocal Verification
--forcereinstall path, applying it to the previously-unhandled "unregistered but config-bearing directory" case.Deviations from Assessment
None. The implementation follows the preferred remediation exactly as described.
Risks & Review Notes
not self.registry.is_installed(manifest.id)guard ensures we only rescue configs when the extension is genuinely unregistered, avoiding picking up stale files from a different install.shutil.copytree, so packaged defaults are always superseded by the user's values — no risk of defaults silently winning.install_from_zip()delegates toinstall_from_directory(), so it is covered without additional changes.install_from_directory().Refs #3427 · cc
@grafvonbremove --keep-config#3427