Fixes#7110
This PR declares docker-compose as a requirement for certbot-ci. This way, a recent version of docker-compose is installed in the standard virtual environment set up by `tools/venv.py` and `tools/venv3.py`, and so is available to pytest integration tests from `tox` or in the virtual environment enabled.
* Add docker-compose as a dev dependency and declares it in certbot-ci requirements
* Update docker-compose 1.25.0
This PRs extends the installer tests on Azure Pipeline, in order to run the integration tests on a certbot instance installed with the Windows installer for several Windows versions, corresponding to the scope of supported versions on Certbot:
* Windows Server 2012 R2
* Windows Server 2016
* Windows Server 2019
One can see the result on: https://dev.azure.com/adferrand/certbot/_build/results?buildId=311
* Try specific installer-build step
* Install Python manually
* Add tests on windows 2019
Part of #7550
This PR makes appropriate corrections to run pylint on Python 3.
Why not keeping the dependencies unchanged and just run pylint on Python 3?
Because the old version of pylint breaks horribly on Python 3 because of unsupported version of astroid.
Why updating pylint + astroid to the latest version ?
Because this version only fixes some internal errors occuring during the lint of Certbot code, and is also ready to run gracefully on Python 3.8.
Why upgrading mypy ?
Because the old version does not support the new version of astroid required to run pylint correctly.
Why not upgrading mypy to its latest version ?
Because this latest version includes a new typshed version, that adds a lot of new type definitions, and brings dozens of new errors on the Certbot codebase. I would like to fix that in a future PR.
That said so, the work has been to find the correct set of new dependency versions, then configure pylint for sane configuration errors in our situation, disable irrelevant lintings errors, then fixing (or ignoring for good reason) the remaining mypy errors.
I also made PyLint and MyPy checks run correctly on Windows.
* Start configuration
* Reconfigure travis
* Suspend a check specific to python 3. Start fixing code.
* Repair call_args
* Fix return + elif lints
* Reconfigure development to run mainly on python3
* Remove incompatible Python 3.4 jobs
* Suspend pylint in some assertions
* Remove pylint in dev
* Take first mypy that supports typed-ast>=1.4.0 to limit the migration path
* Various return + else lint errors
* Find a set of deps that is working with current mypy version
* Update local oldest requirements
* Remove all current pylint errors
* Rebuild letsencrypt-auto
* Update mypy to fix pylint with new astroid version, and fix mypy issues
* Explain type: ignore
* Reconfigure tox, fix none path
* Simplify pinning
* Remove useless directive
* Remove debugging code
* Remove continue
* Update requirements
* Disable unsubscriptable-object check
* Disable one check, enabling two more
* Plug certbot dev version for oldest requirements
* Remove useless disable directives
* Remove useless no-member disable
* Remove no-else-* checks. Use elif in symetric branches.
* Add back assertion
* Add new line
* Remove unused pylint disable
* Remove other pylint disable
A lot of Certbot's files don't have API documentation which is fixed by this PR. To do this, from the top level certbot directory I ran:
```
sphinx-apidoc -Me -o docs/api certbot
```
I then merged the resulting `modules.rst` file with `docs/api.rst`.
The old plugin at https://github.com/marcan/certbot-external says it's obsolete and points people to https://github.com/EnigmaBridge/certbot-external-auth. The new plugin is also an installer.
I also removed the reference to #2782 about us adding similar functionality since that's been done for a long time. We could reference our manual plugin instead, but I think that devalues their plugin a bit which I don't think is necessary or correct as it has different features.
Current version of pywin32 used in certbot (225) does not have wheels available for Python 3.8. Installing certbot for development in this case requires to build from source. On Windows, this implies a Visual Studio C++ environment up and ready, which is absolutely not fun.
Let's upgrade to pywin32 227, that provides these wheels for all Python versions from 3.5 up to current dev status of 3.9.
This is my proposed fix for #7540. I would ideally like this to be included in our 1.0 release.
I came up with this design by adding all attributes used either in our own plugins, 3rd party plugins listed at https://certbot.eff.org/docs/using.html#third-party-plugins, or our public API code.
Despite me thinking that zope is unneeded nowadays, I initially tried to use it to define this interface since we have it and it gives us a way to define expected attributes, but it doesn't work because zope interface objects also have a method called `names` which conflict with the API.
I talked about this with Adrien out of band and did some of my own research and there are some minor benefits with this new approach of using properties:
1. It's more conventional.
2. If you also change the implementation to inherit from the class, Python will error if all properties aren't defined.
3. The PEP 526 style type annotations with mypy seem to (currently) only be used to validate code using the class, not the class implementation itself. You can add a type annotation saying the class needs to have this attribute, never define it, and mypy won't complain.
With this new approach, I had to fix `names` because pylint was complaining that the arguments differed, however, we never used the optional parameter to `names` outside of tests so I just deleted the code altogether.
* fixes#7540
* move to properties
* Move acme tests to tests/ directory outside of acme module
* Fix call to messages_test in client_test
* Move test_util.py and testdata/ into tests/
* Update manifest to package tests
* Exclude pycache and .py[cod]
* Refactor tests out of module for certbot-dns-cloudflare
* Refactor tests out of module for certbot-dns-cloudxns
* Refactor tests out of module for certbot-dns-digitalocean
* Refactor tests out of module for certbot-dns-dnsimple
* Refactor tests out of module for certbot-dns-dnsmadeeasy
* Refactor tests out of module for certbot-dns-gehirn
* Refactor tests out of module for certbot-dns-google
* Refactor tests out of module for certbot-dns-linode
* Refactor tests out of module for certbot-dns-luadns
* Refactor tests out of module for certbot-dns-nsone
* Refactor tests out of module for certbot-dns-ovh
* Refactor tests out of module for certbot-dns-rfc2136
* Refactor tests out of module for certbot-dns-sakuracloud
* Refactor tests out of module for certbot-dns-route53
* Move certbot-dns-google testdata/ under tests/
* Use pytest for dns plugins
* Exclude pycache and .py[cod]
Clean up some places missed by #7544.
Found this when running test farm tests. They were working as of 5d90544, and I will truly shocked if subsequent changes (all to the windows installer) made them stop working.
* Release script needs to target new CHANGELOG location
* Clean up various other CHANGELOG path references
* Update windows paths for new certbot location
* Add certbot to packages list for windows installer
* Update pull_request_template.md
* Remove line breaks
Github seems to be keeping the line breaks rather than ignoring them, making it be formatted weirdly, so remove them.
Part of #5775.
* Create _internal folder certbot-nginx
* Move configurator.py to _internal
* Move constants.py to _internal
* Move display_ops.py to _internal
* Move http_01.py to _internal
* Move nginxparser.py to _internal
* Move obj.py to _internal
* Move parser_obj.py to _internal
* Move parser.py to _internal
* Update location and references for tls_configs
* exclude parser_obj from coverage
Summary of changes in this PR:
- Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below).
- Make repo root README symlink to `certbot` README.
- Pull tests outside of the distributed module.
- Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery.
- Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments.
- Replace mentions of `.` with `certbot` when referring to packages to install, usually editably.
- Clean up some unused code around executing tests in a different directory.
- Create public shim around main and make that the entry point.
New directory structure summary:
```
repo root ("certbot", probably, but for clarity all files I mention are relative to here)
├── certbot
│ ├── setup.py
│ ├── certbot
│ │ ├── __init__.py
│ │ ├── achallenges.py
│ │ ├── _internal
│ │ │ ├── __init__.py
│ │ │ ├── account.py
│ │ │ ├── ...
│ │ ├── ...
│ ├── tests
│ │ ├── account_test.py
│ │ ├── display
│ │ │ ├── __init__.py
│ │ │ ├── ...
│ │ ├── ... # note no __init__.py at this level
│ ├── ...
├── acme
│ ├── ...
├── certbot-apache
│ ├── ...
├── ...
```
* refactor certbot/ and certbot/tests/ to use the same structure as the other packages
* git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g"
* git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g"
* git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g"
* Remove replacement of certbot into . in install_and_test.py
* copy license back out to main folder
* remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/
* Move README back into main folder, and make the version inside certbot/ a symlink
* symlink certbot READMEs the other way around
* move testdata into the public api certbot zone
* update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests
* certbot version has been bumped down a directory level
* make certbot tests directory not a package and import sibling as module
* Remove unused script cruft
* change . to certbot in test_sdists
* remove outdated comment referencing a command that doesn't work
* Install instructions should reference an existing file
* update file paths in Dockerfile
* some package named in tox.ini were manually specified, change those to certbot
* new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code
* remove other instance of pyargs
* fix up some references in _release.sh by searching for ' . ' and manual check
* another stray . in tox.ini
* fix paths in tools/_release.sh
* Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper
* Create public shim around main and make that the entry point
* without pyargs, tests cannot be run from an empty directory
* Remove cruft for running certbot directly from main
* Have main shim take real arg
* add docs/api file for main, and fix up main comment
* Update certbot/docs/install.rst
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Fix comments in readthedocs requirements files to refer to current package
* Update .[docs] reference in contributing.rst
* Move plugins tests to certbot tests directory
* add certbot tests to MANIFEST.in so packagers can run python setup.py test
* move examples directory inside certbot/
* Move CHANGELOG into certbot, and create a top-level symlink
* Remove unused sys and logging from main shim
* nginx http01 test no longer relies on certbot plugins common test
Part of #5775. We don't use these docs anywhere, so delete them.
Removes:
- `certbot-nginx/readthedocs.org.requirements.txt`
- `certbot-nginx/docs/` folder
- docs include in `MANIFEST.in`
- docs dependencies in `setup.py`
* Remove unused nginx docs
* Add changelog entry about the removal
Part of #5775. We don't use these docs anywhere, so delete them.
Removes:
- `certbot-compatibility-test/readthedocs.org.requirements.txt`
- `certbot-compatibility-test/docs/` folder
- docs include in `MANIFEST.in`
- docs dependencies in `setup.py`
Part of #5775. We don't use these docs anywhere, so delete them.
Removes:
- `certbot-apache/readthedocs.org.requirements.txt`
- `certbot-apache/docs/` folder
- docs include in `MANIFEST.in`
- docs dependencies in `setup.py`
Fixes#7184.
I updated #7358 to track the issue of unpinning all of these dependencies.
* pin back configargparse
* Pin back zope packages.
* update deps
* Add changelog entry.
* run build.py
When you try to run this script, it crashes with:
```
standard_init_linux.go:211: exec user process caused "exec format error"
```
This is caused by the script being written to have the contents:
```
\
#!/bin/sh
set -e
...
```
This fixes the problem by removing the slash and moving the shebang to the first line of the string.
Part of #5775. Methodology similar to #7528. Also refactors NGINX test util to use certbot.tests.util.ConfigTestCase.
* refactor nginx tests to no longer rely on certbot.configuration internals
* Move configuration.py to _internal
While coding for #7536, I ran into another issue. It appears that Certbot logs generated during the scheduled task execution have wrong permissions that make them almost unusable: they do not have an owner, and their ACL contains nonsense values (non existant accounts name).
The class `logging.handler.RotatingFileHandler` is responsible for these logs, and become mad when it is in a Python process run under a scheduled task owned by `SYSTEM`. This is precisely our case here.
This PR avoids (but not fix) the issue, by changing the owner of the scheduled task from `SYSTEM` to the `Administrators` group, that appears to work fine.
* Use Administrators group instead of SYSTEM to run the certbot renew task
Turned out that the scheduled task that runs `certbot renew` twice a day, is failing. Without any kind of log of course, otherwise it would not be fun.
It can be revealed by opening a powershell under the `NT AUTHORITY\SYSTEM` account, under which the scheduled task is run. Under theses circumstances, the bug is revealed: Certbot breaks when trying to invoke `certbot.compat.filesystem._get_current_user()`. Indeed the logic there implied to call `win32api.GetUserNameEx(win32api.NameSamCompatible)` and this function does not return always a useful value.
For normal account, it will be typically `DOMAIN_OR_MACHINE_NAME\YOUR_USER_NAME` (e.g. `My Machine\Adrien Ferrand`). But for the account `NT AUTHORITY\SYSTEM`, it will return `MACHINE_NAME\DOMAIN$`, which is a nonsense and makes fail the resolution of the actual SID of the account at the end of `_get_current_user()`.
This PR fixes this behavior by using an explicit construction of the account name that works both for normal users and `SYSTEM`.
* Use a different way to resolve current user account, that works both for normal users and SYSTEM.
* Add a comment to run Certbot under NT AUTHORITY\SYSTEM
Fixes#7548.
This PR udpdates installation instructions to get rid of python2 and certbot-auto in the how-to explaining the Certbot development environment setup.
Instead, Python 3 is used, and appropriate instructions for APT and RPM based distributions are provided.
* Don't call core constants from nginx plugin
* Move constants.py to _internal/
* Move ENHANCEMENTS from now-internal constants to public plugins.enhancements
* Update display.enhancements.ask from its 2015 comment
* Move display/completer.py to _internal/
* Move display/dummy_readline.py to _internal/
* Move display/enhancements.py to _internal/
* Create __init__.py in _internal/display
* Create _internal package for Certbot's non-public modules
* Move account.py to _internal
* Move auth_handler.py to _internal
* Move cert_manager.py to _internal
* Move client.py to _internal
* Move error_handler.py to _internal
* Move lock.py to _internal
* Move main.py to _internal
* Move notify.py to _internal
* Move ocsp.py to _internal
* Move renewal.py to _internal
* Move reporter.py to _internal
* Move storage.py to _internal
* Move updater.py to _internal
* update apache and nginx oldest requirements
* Keep the lock file as certbot.lock
* nginx oldest tests still need to rely on newer certbot
* python doesn't have good dependency resolution, so specify the transitive dependency
* update required minimum versions in nginx setup.py
In Travis, the full test suite doesn't run on PRs for point release branches, just on commits for them. I think this behavior makes sense because what we actually want to test before a point release is the exact commit we want to release after any squashing/merging has been done. This PR modifies Azure to match this behavior.
After this PR lands, I need to update the tests required to pass on GitHub.
This PR uses pipstrap to bootstrap the venv used to build Windows installers. This effectively pin all build dependencies, since pynsist is already installed through pip_install.py script.
* Use pipstrap
* Pin also NSIS version
* acme: re-populate uri in deactivate_authorization
* Use fresh authorizations in dry runs
--dry-run now deactivates 'valid' authorizations if it encounters them
when creating a new order.
Resolves#5116.
* remove unused code
* typo in local-oldest-requirements
* better error handling
* certbot-ci: AUTHREUSE to 100 + unskip dry-run test
* improve test coverage for error cases
* restore newline to local-oldest-requirements.txt
This pull request addresses #7451 by removing the deprecated flags.
* Dropped deprecated flags from commands
* Updated changelog for dropped flags and deleted outdated tests
* removed init-script part of apache test
I tried to finish up #7214 by removing the code in acme but we can't really do that until #7478 is resolved which we cannot do until we release 0.40.0.
Since we have to wait, this PR adds deprecation warnings for code that uses the TLS-SNI-01 code or was only used by the long deprecated TLS-SNI-01 code.
I'd like this PR to land before our next release.
* Deprecate more code related to TLS-SNI-01.
* Assert about warning message.
We don't package rebuild_dependencies.py so I don't think we need to mention changes to it in our changelog which is primarily read by users and packagers.
This pull request ensures that we use distro package in all the distribution version detection. It also replaces the custom systemd /etc/os-release parsing and adds a few version fingerprints to Apache override selection.
Fixes: #7405
* Revert "Try to use platform.linux_distribution() before distro equivalent (#7403)"
This reverts commit ca3077d034.
* Use distro for all os detection code
* Address review comments
* Add changelog entry
* Added tests
* Fix tests to return a consistent os name
* Do not crash on non-linux systems
* Minor fixes to distro compatibility checks
* Make the tests OS independent
* Update certbot/util.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Skip linux specific tests on other platforms
* Test fixes
* Better test state handling
* Lower the coverage target for Windows tests
`certbot-compatibility-test` is using code in `acme` that I proposed making private and not trivially importable in https://github.com/certbot/certbot/issues/5775.
To fix it, I switched to using Certbot's test utilities which I proposed keeping public to help with writing tests for plugins. When doing this I had to change the name of the key because `rsa1024_key.pem` doesn't exist in Certbot.
I also deleted the keys in `certbot-compatibility-test`'s testdata because because they are unused.
This is a big part of #7214. It removes all references to TLS-SNI-01 outside of acme (and pytest.ini). Those changes will come in a subsequent PR. I thought this one was getting big enough.
* Remove references to TLS-SNI-01 in Apache plugin
* Remove references to TLS-SNI-01 from certbot-nginx
* Remove references to TLS-SNI from Certbot.
* Remove TLS-SNI reference from docs
* add certbot changelog
* Clarify test behavior
I wanted to polish the changelog a bit. Changes made are:
* We don't ship our test farm tests so including info about them in our changelog seems unnecessary.
* I combined and expanded the info about the deprecation of Python 3.4.
While working on #7214, I noticed that certbot.plugins.common.TLSSNI01 wasn't printing a deprecation warning and it was still being used in our Apache plugin. This PR fixes that.
Fixes#7007
Python 3.4 is [EOL](https://www.python.org/dev/peps/pep-0429/), and only Python 3.x version available for CentOS 6 through EPEL is this version, and so is used by `certbot-auto`, the only official way to install Certbot on this platform.
This unpleasant situation becomes a little more uncomfortable, considering that the newest `pip` version (19.2) [just dropped Python 3.4 support](https://github.com/pypa/pip/issues/6685) and will refuse to start on this Python version. We can expect a lot of dependencies to follow this path now.
One direct result of this situation is that a fix to support correctly the ARM platforms requires to upgrade `pip` to 19.2 for `certbot-auto`. So this is not possible right now.
Then, let's upgrade Certbot instances on CentOS 6 to a supported version of Python 3.
This PR proposes a new bootstrap approach for CentOS 6 platform, `BootstrapRpmPython3Legacy`, that will install Python 3.6 from [SCL](https://www.softwarecollections.org) (the latest one available for now on CentOS 6). In term of Python 3 specific bootstrap methods, I take the occasion here to completely separate the bootstrap of CentOS 6 as a legacy system, from the RPM-based newest systems (like Fedora 29+) that are simply dropping support for Python 2.x. This is in prevision of future migration for all systems on Python 3.x, that is a different problematic than supporting old systems.
* Add logic
* Rebuilt letsencrypt-auto
* Fix logic
* Focus on specific packages
* Maintain PATH for further invocations of letsencrypt-auto after bootstrap.
* Various corrections
* Fix farm test for RHEL6
* Working centos6 letsencrypt-auto self tests
* Fix test_sdist for CentOS 6
* Corrections
* Work in progress
* Working configuration
* Fix typo
* Remove EPEL. Add a test.
* Update letsencrypt-auto-source/letsencrypt-auto.template
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Improvements after review
* Improvements
* Add a comment
* Add a test
* Update a test
* Corrections
* Update function return
* Work in progress
* Correct behavior on oracle linux 6.
* Corrections
* Rebuild script
* Add letsencrypt-auto tests for oraclelinux6
* Update tox.ini
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update letsencrypt-auto-source/letsencrypt-auto
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update letsencrypt-auto-source/letsencrypt-auto.template
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update letsencrypt-auto-source/letsencrypt-auto
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update letsencrypt-auto-source/letsencrypt-auto
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update letsencrypt-auto-source/letsencrypt-auto.template
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Remove specific code for scientific linux
* Change some variables names
* Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Various corrections
* Fix tests
* Add a comment
* Update message
* Fix test message
* Update letsencrypt-auto-source/letsencrypt-auto.template
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update letsencrypt-auto-source/letsencrypt-auto
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update letsencrypt-auto-source/letsencrypt-auto
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update scripts
* More focused assertion
* Add back a test
* Update script
* Update letsencrypt-auto-source/letsencrypt-auto.template
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update letsencrypt-auto-source/letsencrypt-auto.template
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Check quiet mode
* Add changelog
* Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
This PR creates a pipeline triggered on tag push matching v0.* (eg. v0.40.0).
Once triggered, this pipeline will build the windows installer, and run integration tests on it, like for the pipeline run nightly.
I also add a simple script to extract from CHANGELOG.md file to extract the relevant part to put it in the body of the GitHub release. I believe it makes things nicer.
* Create release pipeline
* Relax condition on tags
* Put beta keyword
* Update job name
* Fix release pipeline
Over the weekend, nightly tests on Windows failed for certbot-dns-google: https://dev.azure.com/certbot/web/build.aspx?pcguid=74ef9c03-9faf-405b-9d03-9acf8c43e8d6&builduri=vstfs%3a%2f%2f%2fBuild%2fBuild%2f72
The error occurred inside `oauth2client`'s locking code and the failure seems spurious as it did not reproduce this morning: https://dev.azure.com/certbot/certbot/_build/results?buildId=73
I could not find a relevant changelog entry in `oauth2client` saying they've fixed the problem, but the problematic code no longer exists in `oauth2client>=4.0`. This PR updates our minimum dependency required in an attempt to avoid spurious failures for us in the future. The only downside I am aware of is it'll make it harder for certbot-dns-google to be packaged in Debian Old Stable or Ubuntu 16.04, but I don't expect either of those things to happen anytime soon.
* bump oauth2client dep
* Update dev_constraints.txt.
* Add changelog entry for packagers.
The value of --server will now be respected, except when it is the
default value, in which case it will be changed to the staging server,
preserving Certbot's existing behavior.
When setting up Azure Pipelines, I didn't like having to define codecov_token for each pipeline. This works around it by using a shared variable group.
You can see this working successfully at https://dev.azure.com/certbot/certbot/_build/results?buildId=3.
* Use certbot-common.
* update instructions
This PR defines pipelines that can be run on Azure Pipelines. Currently there are two:
* `.azure-pipelines/main.yml` is the main one, executed on PRs for master, and pushes to master,
* `.azure-pipelines/advanced.yml` add installer testing on top of the main pipeline, and is executed for `test-*` branches, release branches, and nightly run for master.
These two pipelines covers all existing stuff done by AppVeyor currently, and so AppVeyor can be decommissioned once Azure Pipelines is operational.
You can see working pipeline in my fork:
* a PR for `master` (so using main pipeline): https://github.com/adferrand/certbot/pull/65
* a PR for `test-something` (so using advanced pipeline): https://github.com/adferrand/certbot/pull/66
* uploaded coverage from Azure Pipelines: 499aa2cbf2/build
Once this PR is merged, we need to enable Azure Pipelines for Certbot. Instructions are written in `azure-pipelines/INSTALL.md`. This document also references all access rights required to Azure Pipelines onto GitHub to make the CI process work.
Future work for future PRs:
* create a CD pipeline for the releases that will push the installer to GitHub releases
* implement a solution to generate notification on IRC or Mattermost when a nightly build fails
* Define pipelines
* Update locations
* Update nightly
* Use x86
* Update nightly.yml for Azure Pipelines
* Run script
* Use script
* Update install
* Use local installation
* Register warnings
* Fix pywin32 loading
* Clean context
* Enable coverage publication
* Consume codecov token
* Document installation
* Update tool to upload coverage
* Prepare pipeline artifacts
* Update artifact ignore
* Protect against codecov failures
* Add a comment about codecov
* Add a comment on RW access asked by Azure
* Add instructions
* Rename pipeline file
* Update instructions
* Update .azure-pipelines/templates/tests-suite.yml
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update .azure-pipelines/INSTALL.md
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Modified scheduled pipeline
* Add comment
* Remove dynamic version-based installer name
* Remove unnecessary account ID match check.
Right now the Account object calculates an ID using md5. This is
unnecessary and causes problems on FIPS systems that forbid md5. It's
just as good to pick a random series of bytes for the ID, since the ID
gets read out of renewal/foo.conf.
However, if we switched the algorithm right now, we could wind up
breaking forward compatibility / downgradeability, since older versions
would run into this check.
Removing this check now lays the ground to change the ID-calculation
algorithm in the future.
Related to #1948 and
https://github.com/certbot/certbot/pull/1013#issuecomment-149983479.
* Remove test.
* Remove unused import.
The repo description for the [3rd party Icecast plugin](https://github.com/e00E/lets-encrypt-icecast) says that the plugin isn't currently working and the repository hasn't been updated since 2017. Since it seems broken and unmaintained, let's remove it from the list of third party plugins.
I would happily add it again to the list of third party plugins if people fix and maintain it.
The README for the [3rd party heroku plugin](https://github.com/gboudreau/certbot-heroku) says it has been deprecated. Because of this, let's remove it from the list of third party plugins.
Try to primarily fall back to using `platform.linux_distribution()` if `/etc/os-release` isn't available. Only use `distro.linux_distribution()` on Python >= 3.8.
* Try to use platform.linux_distribution() before distro equivalent
* Fix tests for py38
* Added changelog entry
This PR fixes a regression in #7337 (0.38.0) that certbot cannot run with Apache on RHEL 6.
In RHEL 6, `distro.linux_distribution()` returns `RedHatEnterpriseServer`.
In RHEL 6:
```py
>>> import distro
>>> distro.linux_distribution()
(u'RedHatEnterpriseServer', u'6.10', u'Santiago')
>>> import platform
>>> platform.linux_distribution()
('Red Hat Enterprise Linux Server', '6.10', 'Santiago')
```
In RHEL 7:
```py
>>> import distro
>>> distro.linux_distribution()
('Red Hat Enterprise Linux Server', '7.6', 'Maipo')
>>> import platform
>>> platform.linux_distribution()
('Red Hat Enterprise Linux Server', '7.6', 'Maipo')
```
* fix to run with Apache on RHEL 6
* fix docs
Fixes#7368.
When updating the changelog, I replaced the line about running tests on Python 3.8 because I personally think that support for Python 3.8 is the most relevant information for our users/packagers about our changes in this area.
* List support for Python 3.8.
* Update changelog.
Fixes#7152.
* don't check ocsp if cert is expired when getting cert information
* don't check ocsp if the cert is expired in ocsp_revoked
* update tests
* update changelog
* move pytz import to the top of the test file
This PR implements the item "register a scheduled task for certificate renewal" from the list of requirements described in #7365.
This PR adds required instructions in the NSIS installer for Certbot to create a task, named "Certbot Renew Task" in the Windows Scheduler. This task is run twice a day, to execute the command certbot renew and keep the certificates up-to-date.
Uninstalling Certbot will also remove this scheduled task.
* Implementation
* Corrections
* Update template.nsi
* Improve scripts
* Add a random delay of 12 hours
* Synchronize template against default one in pynsist 2.4
* Clean config of scheduled task
* Install only in AllUsers mode
* Add comments
* Remove the logic of single user install
* Get integration tests working on python 3.8
* Run unit tests on py38
* Update coveragercs to use coverage 4.5+ format
* remove line added to tox.ini
* update changelog
* xenial is the new travis default; no need to specify in .travis.yml
Fixes#7212
This PR forbid os.stat and os.fstat, and fix or provide alternatives to avoid its usage in certbot outside of certbot.compat.filesystem.
* Reimplement private key mode propagation
* Remove other os.stat
* Remove last call of os.stat in certbot package
* Forbid stat and fstat
* Implement mode comparison checks
* Add unit tests
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Handle case where multiple ace concerns a given SID in has_min_permissions
* Add a new test scenario
* Add a simple test for has_same_ownership
* Fix name function
* Add a comment explaining an ACE structure
* Move a test in its dedicated class
* Improve a message error
* Calculate has_min_permission result using effective permission rights to be more generic.
* Change an exception message
* Add comments, avoid to skip a test.
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Find OpenSSL version
* Create and update various config files
* Update logic to use new version constraints
* SSL_OPTIONS_HASHES_NEW and SSL_OPTIONS_HASHES_MEDIUM were just being used for testing, and maintaining them is becoming untenable, so remove them.
* if we don't know the openssl version, we can't turn off session tickets
* add unit test for _get_openssl_version
* add unit tests
* placate lint
* Fix docs and tests and clean up code
* use python correctly
* update changelog
* Lint
* make comment a comment
This PR is the first step to create an official distribution channel of Certbot for Windows. It consists essentially in creating a proper Certbot Windows installer.
Usually distributing an application requires, in a way or another, to stabilize the application logic and its dependencies around a given version. On Windows, this usually takes the form of a freezed application, that vendors its dependencies into a single executable.
There are two well-known solutions to create an executable shipping a Python application on Windows: [py2exe](http://www.py2exe.org/) and [pyinstaller](https://www.pyinstaller.org/). However these solutions create self-executable `.EXE` files: you run the `.EXE` file that launches immediately the software.
This is not a end-user solution. Indeed when a Windows user wants to install a piece of software, he expects to find and download an installer. When run the installer would interface with Windows to setup configuration entries in the Registry, update the environment variable, add shortcuts in the Start Menu, and declare a uninstaller entry into the Uninstaller Manager. Quite similarly, this is what you would get from a `.deb` or `.rpm` package.
A solution that builds proper installers is [pynsis](https://pynsist.readthedocs.io/en/latest/). It is a Python project that constructs installers for Python software using [NSIS](https://sourceforge.net/projects/nsis/), the most known free Windows installer builder solution.
This PR uses pynsist to build a Windows installer. The Python script to launch the installer build is `.\windows-installer\construct.py`. Once finished, the installer is located in `.\windows-installer\build\nsis`.
This installer will do the following operations during the installation:
* copy in the install path a full python distribution used exclusively for Certbot
* copy all Python requirements gathered from the `setup.py` of relevant certbot projects
* copy `certbot` and `acme`
* pre-build python binary assets
* register the existence of the application correctly in Windows Registry
* prepare a procedure to uninstall Certbot
* and of course, expose `certbot` executable to the Windows command line, like on Linux, to be able to launch it as any CLI application from Batch or Powershell
This installer support updates: downloading a new version of it and running it on a Windows with existing installation of Certbot will replace it with the new version.
Future capabilities not included in this PR:
* auto-update of Certbot when a new release is available
* online documentation for Windows
* register a scheduled task for certificate renewal
* installer distribution (continuous deployment + distribution channels)
* method to check the downloaded installer is untampered
* Setup config
* Fix shortcut
* Various improvments
* Update windows-installer/construct.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Split into several method
* Change installer name
* Remove DNS plugins for now
* Add a comment about administrator privileges
* Update welcome
* Control python version
* Control bitness
* Update windows-installer/construct.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update windows-installer/construct.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update windows-installer/construct.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
Smallest possible fix for #7106
* Replace platform.linux_dependencies with distro.linux_dependencies
* run build.py
* Add minimum version of 1.0.1
* Pin back requests package
* Update changelog
This PR supersedes #7353.
It fixes the execution of nginx oldest tests when these tests are executed on top of the modifications made in #7337. This execution failure revealed the fact that in some cases, the wrong version of certbot logic was used during integration tests (namely the logic lying in the codebase of the branch built, instead of the logic from the version of certbot declared by certbot-nginx for instance).
I let you appreciate my inline comment for the explanation and the workaround.
Thanks a lot to @bmw who found this python/pytest madness.
You can see the oldest tests succeeding with the logic of #7337 + this PR here: https://travis-ci.com/certbot/certbot/builds/124816254
* Remove certbot root from PYTHONPATH during integration tests
* Add a biiiiig comment.
On Windows you can have several drives (`C:`, `D:`, ...), that is the roughly (really roughly) equivalent of mount points, since each drive is usually associated to a specific physical partition.
So you can have paths like `C:\one\path`, `D:\another\path`.
In parallel, `os.path.relpath(path, start='.')` calculates the relative path between the given `path` and a `start` path (current directory if not provided). In recent versions of Python, `os.path.relpath` will fail if `path` and `start` are not on the same drive, because a relative path between two paths like `C:\one\path`, `D:\another\path` is not possible.
In saw unit tests failing because of this in two locations. This occurs when the certbot codebase that is tested is on a given drive (like `D:`) while the default temporary directory used by `tempfile` is on another drive (most of the time located in `C:` drive).
This PR fixes that.
* Use travis_retry in travis builds to retry the farm tests
* travis_retry is a bash function, so it can be called only from current bash
* Update .travis.yml
* Update .travis.yml
This PR adds OVERRIDE_CLASS in certbot-apache/entrypoint.py for Scientific Linux. Fixes#7248.
* add OVERRIDE_CLASS for Scientific Linux os name
* add entry for Scientific Linux using "scientific" as key
* Update changelog
See https://community.letsencrypt.org/t/ssl-error-after-cert-renew/99430.
The first commit of this PR is a simple, clean revert of #7191. Subsequent commits add back pieces of that PR we want to keep.
I also reverted #7299 which landed in a separate PR, but needs to be reverted to keep including the TLS config files in the certbot-apache package when it is built.
I tested this on Ubuntu 18.04 by installing a cert to Apache using Certbot master and then running certbot renew with this branch. I watched the Apache plugin update the configuration file to remove SSLSessionTickets off.
* Revert "Disable TLS session tickets for Apache 2.4.11+ (#7191)"
This reverts commit 9174c631d9.
* Keep hashes with TLS session tickets disabled.
* dont delete changelog entries
* add changelog entry
* Revert "Clean the useless entries in MANIFEST.in (#7299)"
This reverts commit f4d17d9a6b.
(cherry picked from commit 120137eb8d)
See https://community.letsencrypt.org/t/ssl-error-after-cert-renew/99430.
The first commit of this PR is a simple, clean revert of #7191. Subsequent commits add back pieces of that PR we want to keep.
I also reverted #7299 which landed in a separate PR, but needs to be reverted to keep including the TLS config files in the certbot-apache package when it is built.
I tested this on Ubuntu 18.04 by installing a cert to Apache using Certbot master and then running certbot renew with this branch. I watched the Apache plugin update the configuration file to remove SSLSessionTickets off.
* Revert "Disable TLS session tickets for Apache 2.4.11+ (#7191)"
This reverts commit 9174c631d9.
* Keep hashes with TLS session tickets disabled.
* dont delete changelog entries
* add changelog entry
* Revert "Clean the useless entries in MANIFEST.in (#7299)"
This reverts commit f4d17d9a6b.
This PR builds off of #7240 to fix#7241.
The code in certbot-auto is unchanged which I +1. Someone else should give it a 2nd review.
For the code in the tests, you can see all tests passing (including test_tests.sh) at https://travis-ci.com/certbot/certbot/builds/122198270.
I created #7301 to track removing the temporary code in test_leauto_upgrades.sh as suggested at #7282 (comment).
One noteworthy thing here is I did not add the RHEL 8 AMI to the Apache tests due to #7273. This problem is not related to support in certbot-auto though, is an edge case, and I do not personally believe it should block this PR.
* Fix account_tests
* Fix hook executable test
* Remove the temporary decorator @broken_on_windows
* Fix util_test
* No broken unit test on Windows anymore
* More elegant mock
* Fix context manager
* Fix lint
* Fix mypy
* Adapt coverage
* Corrections
* Fix lint
* Adapt coverage
* Update certbot/tests/compat/filesystem_test.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update util_test.py
* Fix pylint
* Forbid os.access
* Update os_test.py
* Update os.py
* Fix lint
* Update filesystem.py
* Update filesystem.py
* Update filesystem.py
* Update os.py
* Start fixing tests
* Platform independent hooks
* Fix probe fd close
* Add broken_on_windows for integration tests
* Fix a lot of tests
* Use a python hook script, to prepare cross-platform
* New approach to be compliant with Linux and Windows on hook scripts
* New tests fixed
* Test for permissions on Windows
* Permissions comparison for Windows
* No broken tests in certbot core anymore
* Change mode
* Specific config for appveyor
* Use forked pebble for now
* Various fixes
* Assert file permissions for world on private keys
* Clean code
* Fix several things
* Add integration target
* Optimize integration env
* Re-enable all AppVeyor envs
* Use again official pebble
* Update pebble_artifacts.py
* Set PYTEST_ADDOPTS silently
* Update appveyor.yml
* Pin pywin32 for tests, give a minimal requirement for certbot.
* Remove injection of nginx in PATH
* Clean debug code
* Various cleanup, ensure to remove workspace after tests
* Update tox target
* Improve assertions. Control the keyword echoed in hooks
* Fix for virtualenv on Python 3.7.4 for Windows
* Update certbot-ci/certbot_integration_tests/certbot_tests/assertions.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Add conditionally pywin in certbot-ci like in certbot
/usr/bin/python no longer exists in RHEL 8. This patch updates
the certbot-auto script to use python3 on nodes running RHEL 8.
Also fixed a bug in the RPM_DIST_VERSION logic which would cause
letsencrypt-auto to fail on servers running CentOS/RHEL 6.
Since #7191, TLS configuration files for Apache have been moved to a dedicated folder tls_configs. Then the entries in MANIFEST.in removed by this PR do not correspond to an existing path, and so are not useful anymore.
Following discussions in #7298.
This PR moves the three Nginx TLS configuration files into a specific folder, tls_configs, update the MANIFEST to include this folder and its content into the certbot-nginx package, and update tests accordingly.
* Move tls configuration files in a specific folder
* Move new file
* Follow Mozilla recs for Nginx ssl_protocols, ssl_ciphers, and ssl_prefer_server_ciphers
* Add tests and fix if statement
* Update CHANGELOG.md
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Test that the hashes of all of the current configuration files are in ALL_SSL_OPTIONS_HASHES
* Remove conditioning on OpenSSL version, since Nginx behaves cleanly if its linked OpenSSL doesn't support TLS1.3
* Implement a logic, miss the private key of pebble
* Complete process
* Fix nginx cert path
* Check conditionnally docker
* Update gitignore, fix apacheconftest
* Full object
* Carriage return
* Work in progress
* Move to official v2.1.0 of pebble
* Fix name
* Update acme_server.py
* Link things together with new version of pebble
* Plug the logic to tests
* Update config
* Reinitiate config
* Add OCSP config to pebble
* Working.
* Simplify logic
* Clean code
* Use forked pebble for now
# Conflicts:
# certbot-ci/certbot_integration_tests/utils/pebble_artifacts.py
* Move full logic of mock at the acme server config
* Continue work
* Finish fixing the date parsing
* Update module name
* Use again official pebble
* Activate mock OCSP server
* Clean code
* Update pebble_artifacts.py
* Remove OCSP stale test
* Add executable permissions
* Clean code
* Update setup.py
* Simplify code
* On-demand import of pebble_ocsp_server
* Revert "Remove OCSP stale test"
This reverts commit 2e4c985b427120cc15526bbcfd15806d02a6f3fc.
# Conflicts:
# certbot-ci/certbot_integration_tests/utils/misc.py
* Fix for virtualenv on Python 3.7.4 for Windows
* Update acme_server.py
AppVeyor recently upgrade the Python 3.7.x installed in their VM to 3.7.4. However, virtualenv 16.6.1 is broken on that specific version of Python for Windows.
This PR upgrade virtualenv installed for a dev/test environment from 16.6.1 to 16.6.2 in order to fix this issue, and repair the CI jobs execute by AppVeyor on PRs.
Fixes#6850
This PR makes the last corrections needed to run all unit tests on Windows:
add a function to check if a hook is executable in a cross-platform compatible way
handle correctly the PATH surgery for Windows during hook execution
handle correctly an account compatibility over both ACMEv1 and ACMEv2
remove (finally!) the @broken_on_windows decorator.
* Fix account_tests
* Fix hook executable test
* Remove the temporary decorator @broken_on_windows
* Fix util_test
* No broken unit test on Windows anymore
* More elegant mock
* Fix context manager
* Adapt coverage
* Corrections
* Adapt coverage
* Forbid os.access
* Implement the logic
* Update tests
* Fix lint and changelog
* Update configurator.py
* Move the TLS configs in a dedicated folder. Fix the formalism of their naming and location.
* Improve existing test to check all TLS config have their hash registered in Certbot
* Corrections after review
* Improve a test
* Remove commented useless lines in TLS configs
* Add a nice warning. Because I am nice.
* Fix lint
* Add a test
Resolves#4945. First PR in order to address #5116.
* acme: Implement authz deactivation
Resolves#4945
* update AUTHORS and CHANGELOG
* typos in mypy annotations
* formatting: missing newline
* improve test_deactivate_authorization
* improve deactivate_authorization
* test: s/STATUS_INVALID/STATUS_DEACTIVATED/
* simplify dict to keyword argument
* acme: add UpdateAuthorization
* acme: use UpdateAuthorization in deactivate_authz
and add mypy annotation
This allows deactivate_authorization to succeed for both ACME v1
and v2 servers.
This fixes the test failures which can be seen at
https://travis-ci.com/certbot/certbot/builds/120123338.
The problem here is the path returned by tempfile.mkdtemp() contains a symlink.
For instance, one run of the function produced
'/var/folders/3b/zg8fdh5j71x92yyzc1tyllfw0000gp/T/tmp3k9ytfj1' which is a
symlink to
'/private/var/folders/3b/zg8fdh5j71x92yyzc1tyllfw0000gp/T/tmp3k9ytfj1'.
Removing this symlink before testing filesystem.realpath solves the problem.
You can see the macOS tests passing with this change at https://travis-ci.com/certbot/certbot/builds/120250667.
I think this list maybe had value when distros were first starting to package Certbot, but now I don't think it does. What function does this list serve? The instruction generator at https://certbot.eff.org/instructions does a much better job telling users how to use these packages. On the packaging side, I think anyone capable of packaging Certbot at the various distros would be able to search their repositories to see if a Certbot package is available.
Since this list is hard to maintain as links semi-regularly break and keeping it up to date with all distros and all Certbot components is a fair bit of work, let's just remove it.
This PR was motivated by the Travis failures at https://travis-ci.com/certbot/website/builds/119588518 due to GNU Guix changing the layout of their site.
Fixes#7115
This PR creates a `realpath` method in `filesystem`, whose goal is to replace any call to `os.path.realpath` in Certbot. The reason is that `os.path.realpath` is broken on some versions of Python for Windows. See https://bugs.python.org/issue9949. The function created here works consistently across Linux and Windows.
As for the other forbidden functions in `os` module, our `certbot.compat.os` will raise an exception if its `path.realpath` function is invoked, and using the `os` module from Python is forbidden from the pylint check implemented in our CI.
Every call to `os.path.realpath` is corrected in `certbot` and `certbot-apache` modules.
* Forbid os.path.realpath
* Finish implementation
* Use filesystem.realpath
* Control symlink loops also for Linux
* Add a test for forbidden method
* Import a new object from os.path module
* Use same approach of wrapping than certbot.compat.os
* Correct errors
* Fix dependencies
* Make path module internal
There is a unit test to check that the default directories for Certbot are not diverging, in certbot.tests.cli_test:FlagDefaultTests:test_linux_directories.
But this test is not done on Windows.
This PR fixes that.
We're planning on using the branch apache-parser-v2 allowing us to incrementally work on the new Apache parser and feel comfortable landing temporary test code that we don't really want in master.
The apache-parser-v2 branch is created and locked down, but neither Travis or AppVeyor are configured to run tests on it. See #7230. This PR fixes that problem.
This could probably just land in the apache-parser-v2 branch, but why unnecessarily deviate the branch from master? It doesn't hurt anything there. Once it lands, I'll get this added to the apache-parser-v2 branch too.
* Run tests on apache-parser-v2.
* add comment
* Don't run full test suite on apache-parser-v2.
This PR implements the filesystem.copy_ownership_and_apply_mode method from #6497.
This method is used in two places in Certbot, replacing os.chown, to copy the owner and group owner from a file to another one, and apply to the latter the given POSIX mode.
* Implement copy_ownership_and_apply_mode
* Update certbot/compat/os.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Remove default values
* Rewrite a comment.
* Relaunch CI
* Pass as keyword arguments
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Make the private key permissions transfer platform specific
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Rename variable
* Fix comment0
* Add unit test for copy_ownership_and_apply_mode
* Adapt coverage
* Execute unconditionally chmod with copy_ownership_and_apply_mode. Improve doc.
This PR is a part of the actions necessary to make Certbot-CI work on Windows, in order to execute the integration tests on this platform.
Following #7156, this PR changes how the integration tests are setup against Pebble to not need Docker anymore.
As a reminder, one can check #7156 and letsencrypt/pebble#240 to see the rationale about why using Docker is a problem to run the integration tests on Windows.
Basically, this PR executes directly Pebble using its executable, since it is build using Go, and Go produces self-contained executable that can run without any installation on Linux and on Windows. During the integration tests setup, Certbot-CI will get the Pebble (and Challtestsrv) executables for the defined target version on the GitHub releases. The binaries are persisted on the filesystem, so it is not needed to download them again on the second integration tests execution. Nonetheless, we are talking about 20MB of executables.
Since the setup needs to hold a state, I also took this occasion to refactor the acme_server, in order to use on object oriented approach and improve the readability/maintainability.
Once this PR and #7156 are merged, Docker will not be needed anymore for the main integration tests usecase, that is to use Pebble.
* Complete process
* Fix nginx cert path
* Check conditionnally docker
* Update gitignore, fix apacheconftest
* Full object
* Carriage return
* Move to official v2.1.0 of pebble
* Fix name
* Update acme_server.py
* Relaunch CI
* Update certbot-ci/certbot_integration_tests/utils/acme_server.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update certbot-ci/certbot_integration_tests/utils/acme_server.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update docstring
* Update documentation
* Configure a stdout to ACMEServer
* Map all process through defined stdout
* Remove unused variable
* Handle using signals
* Use failsafe entering context
* Remove failsafe rmtree, that is not needed anymore
* Update virtualenv to the latest version.
* Use venv from pip and pin more packages.
* Pin codecov.
* update appveyor config
* Write the path separator backwards.
* s/pip_install.py install/pip_install.py
* Prefix tools\\pip_install.py with python exe.
* Upgrade py to fix AppVeyor failures.
* add back comment
* Update virtualenv with CERTBOT_NO_PIN.
* Pass -U to upgrade tox and deps.
* Upgrade virtualenv.
2019-07-02 20:02:00 +03:00
857 changed files with 9861 additions and 10513 deletions
Let's begin. All pipelines are defined in `.azure-pipelines`. Currently there are two:
*`.azure-pipelines/main.yml` is the main one, executed on PRs for master, and pushes to master,
*`.azure-pipelines/advanced.yml` add installer testing on top of the main pipeline, and is executed for `test-*` branches, release branches, and nightly run for master.
Several templates are defined in `.azure-pipelines/templates`. These YAML files aggregate common jobs configuration that can be reused in several pipelines.
Unlike Travis, where CodeCov is working without any action required, CodeCov supports Azure Pipelines
using the coverage-bash utility (not python-coverage for now) only if you provide the Codecov repo token
using the `CODECOV_TOKEN` environment variable. So `CODECOV_TOKEN` needs to be set as a secured
environment variable to allow the main pipeline to publish coverage reports to CodeCov.
This INSTALL.md file explains how to configure Azure Pipelines with Certbot in order to execute the CI/CD logic defined in `.azure-pipelines` folder with it.
During this installation step, warnings describing user access and legal comitments will be displayed like this:
```
!!! ACCESS REQUIRED !!!
```
This document suppose that the Azure DevOps organization is named _certbot_, and the Azure DevOps project is also _certbot_.
Use your GitHub user for a normal GitHub account, or a user that has administrative rights to the GitHub organization if relevant.
### Having an Azure DevOps account
- Go to https://dev.azure.com/, click "Start free with GitHub"
- Login to GitHub
```
!!! ACCESS REQUIRED !!!
Personal user data (email + profile info, in read-only)
```
- Microsoft will create a Live account using the email referenced for the GitHub account. This account is also linked to GitHub account (meaning you can log it using GitHub authentication)
- Proceed with account registration (birth date, country), add details about name and email contact
```
!!! ACCESS REQUIRED !!!
Microsoft proposes to send commercial links to this mail
Azure DevOps terms of service need to be accepted
```
_Logged to Azure DevOps, account is ready._
### Installing Azure Pipelines to GitHub
- On GitHub, go to Marketplace
- Select Azure Pipeline, and "Set up a plan"
- Select Free, then "Install it for free"
- Click "Complete order and begin installation"
```
!!! ACCESS !!!
Azure Pipeline needs RW on code, RO on metadata, RW on checks, commit statuses, deployments, issues, pull requests.
RW access here is required to allow update of the pipelines YAML files from Azure DevOps interface, and to
update the status of builds and PRs on GitHub side when Azure Pipelines are triggered.
Note however that no admin access is defined here: this means that Azure Pipelines cannot do anything with
protected branches, like master, and cannot modify the security context around this on GitHub.
Access can be defined for all or only selected repositories, which is nice.
```
- Redirected to Azure DevOps, select the account created in _Having an Azure DevOps account_ section.
- Select the organization, and click "Create a new project" (let's name it the same than the targetted github repo)
- The Visibility is public, to profit from 10 parallel jobs
```
!!! ACCESS !!!
Azure Pipelines needs access to the GitHub account (in term of beeing able to check it is valid), and the Resources shared between the GitHub account and Azure Pipelines.
```
_Done. We can move to pipelines configuration._
## Import an existing pipelines from `.azure-pipelines` folder
- On Azure DevOps, go to your organization (eg. _certbot_) then your project (eg. _certbot_)
- Click "Pipelines" tab
- Click "New pipeline"
- Where is your code?: select "__Use the classic editor__"
__Warning: Do not choose the GitHub option in Where is your code? section. Indeed, this option will trigger an OAuth
grant permissions from Azure Pipelines to GitHub in order to setup a GitHub OAuth Application. The permissions asked
then are way too large (admin level on almost everything), while the classic approach does not add any more
permissions, and works perfectly well.__
- Select GitHub in "Select your repository section", choose certbot/certbot in Repository, master in default branch.
- Click on YAML option for "Select a template"
- Choose a name for the pipeline (eg. test-pipeline), and browse to the actual pipeline YAML definition in the
.. This file contains a series of comments that are used to include sections of this README in other files. Do not modify these comments unless you know what you are doing. tag:intro-begin
Certbot is part of EFF’s effort to encrypt the entire Internet. Secure communication over the Web relies on HTTPS, which requires the use of a digital certificate that lets browsers verify the identity of web servers (e.g., is that really google.com?). Web servers obtain their certificates from trusted third parties called certificate authorities (CAs). Certbot is an easy-to-use client that fetches a certificate from Let’s Encrypt—an open certificate authority launched by the EFF, Mozilla, and others—and deploys it to a web server.
Anyone who has gone through the trouble of setting up a secure website knows what a hassle getting and maintaining a certificate is. Certbot and Let’s Encrypt can automate away the pain and let you turn on and manage HTTPS with simple commands. Using Certbot and Let's Encrypt is free, so there’s no need to arrange payment.
How you use Certbot depends on the configuration of your web server. The best way to get started is to use our `interactive guide <https://certbot.eff.org>`_. It generates instructions based on your configuration settings. In most cases, you’ll need `root or administrator access <https://certbot.eff.org/faq/#does-certbot-require-root-administrator-privileges>`_ to your web server to run Certbot.
Certbot is meant to be run directly on your web server, not on your personal computer. If you’re using a hosted service and don’t have direct access to your web server, you might not be able to use Certbot. Check with your hosting provider for documentation about uploading certificates or using certificates issued by Let’s Encrypt.
Certbot is a fully-featured, extensible client for the Let's
* Can optionally install a http -> https redirect, so your site effectively
runs https only (Apache only)
* Fully automated.
* Configuration changes are logged and can be reverted.
* Supports an interactive text UI, or can be driven entirely from the
command line.
* Free and Open Source Software, made with Python.
.. Do not modify this comment unless you know what you're doing. tag:features-end
For extensive documentation on using and contributing to Certbot, go to https://certbot.eff.org/docs. If you would like to contribute to the project or run the latest code from git, you should read our `developer guide <https://certbot.eff.org/docs/contributing.html>`_.
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.