The Lexicon-based DNS plugins use a mechanism to determine which actual segment of the input domain is actually the DNS zone in which the DNS-01 challenge has to be initiated (eg. `subdomain.domain.com` or `domain.com` for input `subdomain.domain.com`): they tries recursively to configure Lexicon and initiate authentication from the most specific to most generic domain segment, and select the first segment where Lexicon stop erroring out.
This mechanism broke with #9746 because now the plugins call Lexicon client instead of the underlying providers, and the client makes guess on the actual domain requested. Typically for `subdomain.domain.com` it will actually try to authenticate against `domain.com`, and so the mechanism above does not work anymore.
This PR fixes the issue by using the `delegated` field in Lexicon config each time the plugin needs it. This field is designed for this kind of purpose: it will instruct Lexicon what is the actual DNS zone domain instead of guessing it.
I tested the change with one of my OVH account. The expected behavior is re-established and the plugin is able to test `subdomain.domain.com` then `domain.com` as before.
Fixes#9791Fixes#9818
* helpful: fix handling of abbreviated ConfigArgparse arguments (#9796)
* helpful: fix handling of abbreviated ConfigArgparse arguments
ConfigArgparse allows for "abbreviated" arguments, i.e. just the prefix
of an argument, but it doesn't set the argument sources in these cases.
This commit checks for those cases and sets the sources appropriately.
* failing to find an action raises an error instead of logging
* Update changelog
* Add handling for short arguments, fix equals sign handling
These were silently being dropped before, possibly leading to instances
of `NamespaceConfig.set_by_user()` returning false negatives.
(cherry picked from commit 11e17ef77b)
* Fix finish_release.py (#9800)
* response is value
* rename vars
(cherry picked from commit a96fb4b6ce)
* Merge pull request #9762 from certbot/docs/yaml-config
Add YAML files for Readthedocs requirements
(cherry picked from commit 44046c70c3)
* Update Lexicon requirements to stabilize certbot-dns-ovh behavior (#9802)
* Update minimum Lexicon version required for certbot-dns-ovh
* Add types
* FIx mypy
* Fix lint
* Fix BOTH lint and mypy
(cherry picked from commit 5cf5f36f19)
* simplify code (#9807)
(cherry picked from commit 6f7b5ab1cd)
* Include linting fixes from 8a95c03
---------
Co-authored-by: Will Greenberg <willg@eff.org>
Co-authored-by: Alexis <alexis@eff.org>
Co-authored-by: Adrien Ferrand <adferrand@users.noreply.github.com>
* helpful: fix handling of abbreviated ConfigArgparse arguments
ConfigArgparse allows for "abbreviated" arguments, i.e. just the prefix
of an argument, but it doesn't set the argument sources in these cases.
This commit checks for those cases and sets the sources appropriately.
* failing to find an action raises an error instead of logging
* Update changelog
* Add handling for short arguments, fix equals sign handling
These were silently being dropped before, possibly leading to instances
of `NamespaceConfig.set_by_user()` returning false negatives.
* Drop Python 3.7 support
* Fix lint and test
* Check for venv generation
* Update requirements
* Update oldest constaints and compatibility tests runtime
* Refactor Lexicon-based DNS plugins and upgrade minimal version of Lexicon
* Relax filterwarning to comply with envs where boto3 is not installed
* Update pinned dependencies
* Use our previous method to deprecate part of modules
* Safe import internally
* Add changelog
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Migrate entrypoint logic from pkg_resources to importlib.metadata
* Usage of importlib_metadata up to Python 3.9 to align API behavior to Python 3.10
---------
Co-authored-by: Adrien Ferrand <adrien.ferrand@amadeus.com>
Co-authored-by: Adrien Ferrand <adrien.ferrand@arteris.com>
* Migrate pkg_resources API related to resources to importlib_resources
* Fix lint and mypy + pin lexicon
* Update filterwarnings
* Update oldest tests requirements
* Update pinned dependencies
* Fix for modern versions of python
* Fix assets load in nginx integration tests
* Fix a warning
* Isolate static generation from importlib.resource into a private function
---------
Co-authored-by: Adrien Ferrand <adrien.ferrand@amadeus.com>
* dns-google: fix condition to don't use private dns zones
* update MD
* Fix condition
* fix condition
* update testdata
* fix identation
* update tests
* update changelog
* Update dns_google.py
* add test for split horizon dns google
* add dnsName to managed zones
* update quickstart and remove os import
* simplify theme use
* list sphinx_rtd_theme as extension
Our docs builds failed last night, presumably because #9754 updated `sphinx_rtd_theme` which changed some unknown thing.
Looking into it, our usage of this project was very unconventional. Following the code comment I deleted in this PR to https://docs.readthedocs.io/en/stable/faq.html#i-want-to-use-the-read-the-docs-theme-locally, simple instructions are given to put the following in your `conf.py` file:
```
extensions = [
...
'sphinx_rtd_theme',
]
html_theme = "sphinx_rtd_theme"
```
I did this instead of the more complicated logic we were using and all builds passed locally. I also triggered a build on readthedocs with these changes which also passed.
This takes care of the dependabot alerts those with access can see at https://github.com/certbot/certbot/security/dependabot.
Pinning back `cython` is needed because without it, our full test suite will fail when trying to build `pyyaml` on ARM systems.
* Do not call deprecated datetime.utcnow() and datetime.utcfromtimestamp()
* Ignore DeprecationWarnings from importing dependencies
$ python3 -Wdefault
Python 3.12.0b4 (main, Jul 12 2023, 00:00:00) [GCC 13.1.1 20230614 (Red Hat 13.1.1-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
/usr/lib/python3.12/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
>>> import pytz
/usr/lib/python3.12/site-packages/pytz/tzinfo.py:27: DeprecationWarning: datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.fromtimestamp(timestamp, datetime.UTC).
_epoch = datetime.utcfromtimestamp(0)
* Used pytz.UTC consistently for clarity
* repin current
* repin oldest
* csr must have version set to zero
* only set PIP_USE_PEP517 for macOS
* experiment with brew update git failure workaround
* Revert change to NamespaceConfig's constructor
NamespaceConfig's argument sources dict is now set with a method,
and raises a runtime error if one isn't set when set_by_user() is
called.
* Actually update CHANGELOG to reflect the set_by_user changes
* linter appeasement
* configuration: update docs, add test
This test ensures that calling `set_by_user` without an initialized
sources dict raises a RuntimeError.
* Rewrite helpful_test to appease the linter
* Use public interface to access argparse sources dict
* HelpfulParser builds ArgumentSources dict, stores it in NamespaceConfig
After arguments/config files/user prompted input have been parsed, we
build a mapping of Namespace options to an ArgumentSource value. These
generally come from argparse's builtin "source_to_settings" dict, but
we also add a source value representing dynamic values set at runtime.
This dict is then passed to NamespaceConfig, which can then be queried
directly or via the "set_by_user" method, which replaces the global
"set_by_cli" and "option_was_set" functions.
* Use NamespaceConfig.set_by_user instead of set_by_cli/option_was_set
This involves passing the NamespaceConfig around to more functions
than before, removes the need for most of the global state shenanigans
needed by set_by_cli and friends.
* Set runtime config values on the NamespaceConfig object
This'll correctly mark them as being "runtime" values in the
ArgumentSources dict
* Bump oldest configargparse version
We need a version that has get_source_to_settings_dict()
* Add more cli unit tests, use ArgumentSource.DEFAULT by default
One of the tests revealed that ConfigArgParse's source dict excludes
arguments it considers unimportant/irrelevant. We now mark all arguments
as having a DEFAULT source by default, and update them otherwise.
* Mark more argument sources as RUNTIME
* Removes some redundant helpful_test.py, moves one to cli_test.py
We were already testing most of these cases in cli_test.py, only
with a more complete HelpfulArgumentParser setup. And since the hsts/no-hsts
test was manually performing the kind of argument adding that cli
already does out of the box, I figured the cli tests were a more natural
place for it.
* appease the linter
* Various fixups from review
* Add windows compatability fix
* Add test ensuring relevant_values behaves properly
* Build sources dict in a more predictable manner
The dict is now built in a defined order: first defaults, then config
files, then env vars, then command line args. This way we eliminate the
possibility of undefined behavior if configargparse puts an arg's entry
in multiple source dicts.
* remove superfluous update to sources dict
* remove duplicate constant defines, resolve circular import situation
* letstest: -ubuntu18.04 +centos9stream +debian11
* letstest: username for centos 9 stream is ec2-user
This is mentioned on https://centos.org/download/aws-images/
* ensure mod_ssl is installed
in centos 9 stream, apache has to be restarted after mod_ssl is
installed, or the snakeoil certificates will not be present and
apache won't start.
this also removes nghttp2 being installed as the relevant bug
is long fixed.
* dns-rfc2136: add test coverage for PR #9672
* fix compatibility with oldest dnspython
* rename test to be more descriptive
Co-authored-by: ohemorange <ebportnoy@gmail.com>
---------
Co-authored-by: ohemorange <ebportnoy@gmail.com>
This is, to my knowledge, an entirely inconsequential PR to add support for entirely novel challenge types.
Presently in the [`challb_to_achall` function](399b932a86/certbot/certbot/_internal/auth_handler.py (L367)) if the challenge type is not of a type known to certbot an error is thrown. This check is mostly pointless as an authenticator would not request a challenge unknown to it. This check does however forbid any plugins from supporting entirely novel challenges not of the key authorisation form.
* support unknown ACME challenge types
* add to changelog
* update tests
---------
Co-authored-by: Brad Warren <bmw@eff.org>
* remove pointless paragraph about --server and wildcards
* docs: update help text for --dry-run and --staging
* docs: update "Changing the ACME Server" for --dry-run
* add note about webserver reloads
* Optionally sign initial SOA query
Added configuration file option to enable signing of the initial SOA query when determining the authoritative nameserver for the zone. Default is disabled.
* Better handling of sign_query configuration and fix lint issues
* Update str casting to match 5503d12395
* Update certbot/CHANGELOG.md
Co-authored-by: alexzorin <alex@zorin.au>
* Update certbot/CHANGELOG.md
Co-authored-by: alexzorin <alex@zorin.au>
* Update dns_rfc2136.py
Updated with feedback from certbot/certbot#9672
---------
Co-authored-by: alexzorin <alex@zorin.au>
In addition to the speed improvements in CI, the speed improvements locally with both this https://github.com/certbot/certbot/pull/9666 which this builds on is even more significant. After it's been run once so it's had a chance to set up the different virtual environments, `tox` locally now takes 39 seconds on my laptop when it used to take 137 seconds.
Fixes#6127.
* Added lineage name validity check
* Verify lineage name validity before obtaining certificate
* Added linage name limitation to cli help
* Update documentation on certificate name
* Added lineage name validation to changelog
* Use filepath seperators to determine lineagename validity
* Add unittest for private choose_lineagename method
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* add some missing types
* install pkg-config
* install pkg-config for docker too
* add pkg-config to plugins
* pkg-config when cryptography may need to be built
* deps cleanup
* more comments
* more tweaks
Fixes https://github.com/certbot/certbot/issues/7921.
In all cases when we run `pip_install.py`, we first run `pipstrap.py`. This PR combines these two steps for convenience and to make always doing that less error prone. This will also help me with some of the `tox.ini` refactoring I'm planning to do.
I ran the full test suite on everything and tested the release script changes locally.
This change shouldn't have any effect on cryptography's setup because they install `certbot[test]` which depends on pip, setuptools, and wheel.
* always pipstrap
* use pip_install.py during releases
This is a first step towards implementing the plan I described at https://github.com/certbot/certbot/issues/7909#issuecomment-1448675456 which got a +1 from both Erica and Will. Similar changes for our other packages will be made in followup PRs to try and make this easier to review.
It may be helpful to look at https://github.com/certbot/certbot/pull/7600 when reviewing this PR where we did something similar in the past.
The value of `ignore-paths` in `.pylintrc` should work on Windows based on https://pylint.readthedocs.io/en/latest/user_guide/configuration/all-options.html#ignore-paths and the fact that on macOS/linux, changing path delimiters to `\` still causes these directories to be ignored.
I started testing this for mypy as well, but mypy doesn't current pass for us on Windows so I didn't bother and took this opportunity to remove it from the default environments in `tox.ini`. I'll update https://github.com/certbot/certbot/issues/7803 to mention that the value of `exclude` in `mypy.ini` may need to be tweaked if anyone works on that issue.
* make acme tests internal
* no mypy-win
Adrien and I added this is in https://github.com/certbot/certbot/pull/6590 in response to https://github.com/certbot/certbot/issues/6582 which I wrote. I now personally think these tests are way more trouble than they're worth.
In almost all cases, the versions pinned in `tools/requirements.txt` are used. The two exceptions to this that come to mind are users using OS packages and pip. In the former, the version of our dependencies is picked by the OS and do not change much on most systems. As for pip, [we only "support it on a best effort basis"](https://eff-certbot.readthedocs.io/en/stable/install.html#alternative-2-pip).
Even for pip users, I'm not convinced this buys us much other than frequent test failures. We have our tests configured to error on all Python warnings and [we regularly update `tools/requirements.txt`](https://github.com/certbot/certbot/commits/master/tools/requirements.txt). Due to that, assuming our dependencies follow normal conventions, we should have a chance to fix things in response to planned API changes long before they make their way to our users. I do not think it is necessary for our tests to break immediately after an API is deprecated.
I think almost all other failures due to these tests are caused by upstream bugs. In my experience, almost all of them will sort themselves out pretty quickly. I think that responding to those that are not or planned API changes we somehow missed can be addressed when `tools/requirements.txt` is updated or when someone opens an issue. I personally don't think blocking releases or causing our nightly tests to fail is at all worth it here. I think removing this frequent cause of test failures makes things just a little bit easier for Certbot devs without costing us much of anything.
* Add async interface for finalization to acme.client.ClientV2
Add `begin_order_finalization()`/`poll_finalization()` to
`acme.client.ClientV2`, which are directly analogous to
`answer_challenge()`/`poll_authorizations()`. This allows us to
finalize an order and then later poll for its completion as separate
steps.
* Address code review feedback
Rename `begin_order_finalization` -> `begin_finalization` and tweak
wording of changelog entry
Right now if you to_json() an `OrderResource` and later deserialize
it, the `AuthorizationResource` objects don't come back through the
round-trip (they just get de-jsonified as frozendicts and worse, they
can't even be passed to `AuthorizationResource.from_json` because
frozendicts aren't dicts). In addition, the `csr_pem` field gets
encoded as an array of integers, which definitely does not get
de-jsonified into what we want.
Fix these by adding an encoder to `authorizations` and encoder and
decoder to `csr_pem`.
`lock_test.py` is a weird, heavily customized, standalone testing relic that's giving me trouble because the name currently conflicts with `certbot/tests/lock_test.py`. Moving `certbot/tests` inside the Certbot package as discussed at https://github.com/certbot/certbot/issues/7909#issuecomment-1448675456 would avoid this, however, this is at least somewhat blocked on getting that test code passing lint and mypy checks again because we run those checks on the entirety of the Certbot package 🙃 Since `lock_test.py` could probably stand to be rewritten/refactored anyway, I took this approach.
What I did is I rewrote something largely equivalent to `lock_test.py` inside Certbot's unit tests. I chose not to do this in `certbot-ci` because its not necessary to have an ACME server available. We're no longer explicitly testing things with the nginx plugin here like we were in `lock_test.py`, however, we are checking that `prepare` is called on the plugin at the right time and I added comments about the importance of checking that we lock the directory during the call to `prepare` in the Apache and nginx test code.
As a bonus, this fixes https://github.com/certbot/certbot/issues/8121.
2023-03-15 12:54:20 -07:00
540 changed files with 4668 additions and 3800 deletions
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
