Re-enable all tests

# Conflicts:
#	CHANGELOG.md

.azure-pipelines/INSTALL.md

@@ -0,0 +1,119 @@
+# Configuring Azure Pipelines with Certbot
+
+Let's begin. All pipelines are defined in `.azure-pipelines`. Currently there are two:
+* `.azure-pipelines/main.yml` is the main one, executed on PRs for master, and pushes to master,
+* `.azure-pipelines/advanced.yml` add installer testing on top of the main pipeline, and is executed for `test-*` branches, release branches, and nightly run for master.
+
+Several templates are defined in `.azure-pipelines/templates`. These YAML files aggregate common jobs configuration that can be reused in several pipelines.
+
+Unlike Travis, where CodeCov is working without any action required, CodeCov supports Azure Pipelines
+using the coverage-bash utility (not python-coverage for now) only if you provide the Codecov repo token
+using the `CODECOV_TOKEN` environment variable. So `CODECOV_TOKEN` needs to be set as a secured
+environment variable to allow the main pipeline to publish coverage reports to CodeCov.
+
+This INSTALL.md file explains how to configure Azure Pipelines with Certbot in order to execute the CI/CD logic defined in `.azure-pipelines` folder with it.
+During this installation step, warnings describing user access and legal comitments will be displayed like this:
+```
+!!! ACCESS REQUIRED !!!
+```
+
+This document suppose that the Azure DevOps organization is named _certbot_, and the Azure DevOps project is also _certbot_.
+
+## Useful links
+
+* https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema
+* https://www.azuredevopslabs.com/labs/azuredevops/github-integration/
+* https://docs.microsoft.com/en-us/azure/devops/pipelines/ecosystems/python?view=azure-devops
+
+## Prerequisites
+
+### Having a GitHub account
+
+Use your GitHub user for a normal GitHub account, or a user that has administrative rights to the GitHub organization if relevant.
+
+### Having an Azure DevOps account
+- Go to https://dev.azure.com/, click "Start free with GitHub"
+- Login to GitHub
+
+```
+!!! ACCESS REQUIRED !!!
+Personal user data (email + profile info, in read-only)
+```
+
+- Microsoft will create a Live account using the email referenced for the GitHub account. This account is also linked to GitHub account (meaning you can log it using GitHub authentication)
+- Proceed with account registration (birth date, country), add details about name and email contact
+
+```
+!!! ACCESS REQUIRED !!!
+Microsoft proposes to send commercial links to this mail
+Azure DevOps terms of service need to be accepted
+```
+
+_Logged to Azure DevOps, account is ready._
+
+### Installing Azure Pipelines to GitHub
+
+- On GitHub, go to Marketplace
+- Select Azure Pipeline, and "Set up a plan"
+- Select Free, then "Install it for free"
+- Click "Complete order and begin installation"
+
+```
+!!! ACCESS !!!
+Azure Pipeline needs RW on code, RO on metadata, RW on checks, commit statuses, deployments, issues, pull requests.
+RW access here is required to allow update of the pipelines YAML files from Azure DevOps interface, and to
+update the status of builds and PRs on GitHub side when Azure Pipelines are triggered.
+Note however that no admin access is defined here: this means that Azure Pipelines cannot do anything with
+protected branches, like master, and cannot modify the security context around this on GitHub.
+Access can be defined for all or only selected repositories, which is nice.
+```
+
+- Redirected to Azure DevOps, select the account created in _Having an Azure DevOps account_ section.
+- Select the organization, and click "Create a new project" (let's name it the same than the targetted github repo)
+- The Visibility is public, to profit from 10 parallel jobs
+
+```
+!!! ACCESS !!!
+Azure Pipelines needs access to the GitHub account (in term of beeing able to check it is valid), and the Resources shared between the GitHub account and Azure Pipelines.
+```
+
+_Done. We can move to pipelines configuration._
+
+## Import an existing pipelines from `.azure-pipelines` folder
+
+- On Azure DevOps, go to your organization (eg. _certbot_) then your project (eg. _certbot_)
+- Click "Pipelines" tab
+- Click "New pipeline"
+- Where is your code?: select "__Use the classic editor__"
+
+__Warning: Do not choose the GitHub option in Where is your code? section. Indeed, this option will trigger an OAuth
+grant permissions from Azure Pipelines to GitHub in order to setup a GitHub OAuth Application. The permissions asked
+then are way too large (admin level on almost everything), while the classic approach does not add any more
+permissions, and works perfectly well.__
+
+- Select GitHub in "Select your repository section", choose certbot/certbot in Repository, master in default branch.
+- Click on YAML option for "Select a template"
+- Choose a name for the pipeline (eg. test-pipeline), and browse to the actual pipeline YAML definition in the
+  "YAML file path" input (eg. `.azure-pipelines/test-pipeline.yml`)
+- Click "Save & queue", choose the master branch to build the first pipeline, and click "Save and run" button.
+
+_Done. Pipeline is operational. Repeat to add more pipelines from existing YAML files in `.azure-pipelines`._
+
+## Add a secret variable to a pipeline (like `CODECOV_TOKEN`)
+
+__NB: Following steps suppose that you already setup the YAML pipeline file to
+consume the secret variable that these steps will create as an environment variable.
+For a variable named `CODECOV_TOKEN` consuming the variable `codecov_token`,
+in the YAML file this setup would take the form of the following:
+```
+steps:
+    - script: ./do_something_that_consumes_CODECOV_TOKEN  # Eg. `codecov -F windows`
+      env:
+        CODECOV_TOKEN: $(codecov_token)
+```
+
+To set up a variable that is shared between pipelines, follow the instructions
+at
+https://docs.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups.
+When adding variables to a group, don't forget to tick "Keep this value secret"
+if it shouldn't be shared publcily.

.azure-pipelines/advanced.yml

@@ -0,0 +1,19 @@
+# Advanced pipeline for isolated checks and release purpose
+trigger:
+  - test-*
+  - '*.x'
+pr:
+  - test-*
+  - '*.x'
+# This pipeline is also nightly run on master
+schedules:
+  - cron: "0 4 * * *"
+    displayName: Nightly build
+    branches:
+      include:
+      - master
+    always: true
+
+jobs:
+  - template: templates/tests-suite.yml
+  - template: templates/installer-tests.yml

.azure-pipelines/main.yml

@@ -0,0 +1,11 @@
+trigger:
+  # apache-parser-v2 is a temporary branch for doing work related to
+  # rewriting the parser in the Apache plugin.
+  - apache-parser-v2
+  - master
+pr:
+  - apache-parser-v2
+  - master
+
+jobs:
+  - template: templates/tests-suite.yml

.azure-pipelines/templates/installer-tests.yml

@@ -0,0 +1,31 @@
+jobs:
+  - job: installer
+    pool:
+      vmImage: vs2017-win2016
+    steps:
+      - task: UsePythonVersion@0
+        inputs:
+          versionSpec: 3.7
+          architecture: x86
+          addToPath: true
+      - script: python windows-installer/construct.py
+        displayName: Build Certbot installer
+      - task: CopyFiles@2
+        inputs:
+          sourceFolder: $(System.DefaultWorkingDirectory)/windows-installer/build/nsis
+          contents: '*.exe'
+          targetFolder: $(Build.ArtifactStagingDirectory)
+      - task: PublishPipelineArtifact@1
+        inputs:
+          path: $(Build.ArtifactStagingDirectory)
+          artifact: WindowsInstaller
+      - script: $(Build.ArtifactStagingDirectory)\certbot-installer-win32.exe /S
+        displayName: Install Certbot
+      - script: |
+          python -m venv venv
+          venv\Scripts\python tools\pip_install.py -e certbot-ci
+        displayName: Prepare Certbot-CI
+      - script: |
+          set PATH=%ProgramFiles(x86)%\Certbot\bin;%PATH%
+          venv\Scripts\python -m pytest certbot-ci\certbot_integration_tests\certbot_tests -n 4
+        displayName: Run integration tests

.azure-pipelines/templates/tests-suite.yml

@@ -0,0 +1,38 @@
+jobs:
+  - job: test
+    pool:
+      vmImage: vs2017-win2016
+    strategy:
+      matrix:
+        py35:
+          PYTHON_VERSION: 3.5
+          TOXENV: py35
+        py37-cover:
+          PYTHON_VERSION: 3.7
+          TOXENV: py37-cover
+        integration-certbot:
+          PYTHON_VERSION: 3.7
+          TOXENV: integration-certbot
+          PYTEST_ADDOPTS: --numprocesses 4
+    variables:
+    - group: certbot-common
+    steps:
+    - task: UsePythonVersion@0
+      inputs:
+        versionSpec: $(PYTHON_VERSION)
+        addToPath: true
+    - script: python tools/pip_install.py -U tox coverage
+      displayName: Install dependencies
+    - script: python -m tox
+      displayName: Run tox
+    # We do not require codecov report upload to succeed. So to avoid to break the pipeline if
+    # something goes wrong, each command is suffixed with a command that hides any non zero exit
+    # codes and echoes an informative message instead.
+    - bash: |
+        curl -s https://codecov.io/bash -o codecov-bash || echo "Failed to download codecov-bash"
+        chmod +x codecov-bash || echo "Failed to apply execute permissions on codecov-bash"
+        ./codecov-bash -F windows  || echo "Codecov did not collect coverage reports"
+      condition: in(variables['TOXENV'], 'py37-cover', 'integration-certbot')
+      env:
+        CODECOV_TOKEN: $(codecov_token)
+      displayName: Publish coverage

.codecov.yml

@@ -6,13 +6,13 @@ coverage:
         flags: linux
         # Fixed target instead of auto set by #7173, can
         # be removed when flags in Codecov are added back.
-        target: 97.5
+        target: 97.4
         threshold: 0.1
         base: auto
       windows:
         flags: windows
         # Fixed target instead of auto set by #7173, can
         # be removed when flags in Codecov are added back.
-        target: 97.6
+        target: 97.7
         threshold: 0.1
         base: auto

.coveragerc

@@ -1,2 +1,5 @@
+[run]
+omit = */setup.py
+
 [report]
 omit = */setup.py

.travis.yml

@@ -1,4 +1,5 @@
 language: python
+dist: xenial
 
 cache:
     directories:
@@ -36,212 +37,239 @@ extended-test-suite: &extended-test-suite
 
 matrix:
   include:
-#    # Main test suite
-#    - python: "2.7"
-#      env: ACME_SERVER=pebble TOXENV=integration
-#      <<: *not-on-master
-#
-#    # This job is always executed, including on master
-#    - python: "2.7"
-#      env: TOXENV=py27-cover FYI="py27 tests + code coverage"
-#
-#    - python: "2.7"
-#      env: TOXENV=lint
-#      <<: *not-on-master
-#    - python: "3.4"
-#      env: TOXENV=mypy
-#      <<: *not-on-master
-#    - python: "3.5"
-#      env: TOXENV=mypy
-#      <<: *not-on-master
-#    - python: "2.7"
-#      # Ubuntu Trusty or older must be used because the oldest version of
-#      # cryptography we support cannot be compiled against the version of
-#      # OpenSSL in Xenial or newer.
-#      dist: trusty
-#      env: TOXENV='py27-{acme,apache,certbot,dns,nginx}-oldest'
-#      <<: *not-on-master
-#    - python: "3.4"
-#      env: TOXENV=py34
-#      <<: *not-on-master
-#    - python: "3.7"
-#      dist: xenial
-#      env: TOXENV=py37
-#      <<: *not-on-master
-#    - sudo: required
-#      env: TOXENV=apache_compat
-#      services: docker
-#      before_install:
-#      addons:
-#      <<: *not-on-master
-#    - sudo: required
-#      env: TOXENV=le_auto_xenial
-#      services: docker
-#      <<: *not-on-master
-#    - python: "2.7"
-#      env: TOXENV=apacheconftest-with-pebble
-#      <<: *not-on-master
-#    - python: "2.7"
-#      env: TOXENV=nginxroundtrip
-#      <<: *not-on-master
-#
-#    # Extended test suite on cron jobs and pushes to tested branches other than master
-#    - sudo: required
-#      env: TOXENV=nginx_compat
-#      services: docker
-#      before_install:
-#      addons:
-#      <<: *extended-test-suite
-#    - python: "2.7"
-#      env:
-#        - TOXENV=travis-test-farm-apache2
-#        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
-#      <<: *extended-test-suite
-#    - python: "2.7"
-#      env:
-#        - TOXENV=travis-test-farm-leauto-upgrades
-#        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
-#      git:
-#        depth: false  # This is needed to have the history to checkout old versions of certbot-auto.
-#      <<: *extended-test-suite
-#    - python: "2.7"
-#      env:
-#        - TOXENV=travis-test-farm-certonly-standalone
-#        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
-#      <<: *extended-test-suite
-#    - python: "2.7"
-#      env:
-#        - TOXENV=travis-test-farm-sdists
-#        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
-#      <<: *extended-test-suite
-#    - python: "3.7"
-#      env: TOXENV=py37 CERTBOT_NO_PIN=1
-#      <<: *extended-test-suite
-#    - python: "2.7"
-#      env: ACME_SERVER=boulder-v1 TOXENV=integration
-#      sudo: required
-#      services: docker
-#      <<: *extended-test-suite
-#    - python: "2.7"
-#      env: ACME_SERVER=boulder-v2 TOXENV=integration
-#      sudo: required
-#      services: docker
-#      <<: *extended-test-suite
+    # Main test suite
+    - python: "2.7"
+      env: ACME_SERVER=pebble TOXENV=integration
+      <<: *not-on-master
+
+    # This job is always executed, including on master
+    - python: "2.7"
+      env: TOXENV=py27-cover FYI="py27 tests + code coverage"
+
+    - python: "2.7"
+      env: TOXENV=lint
+      <<: *not-on-master
+    - python: "3.4"
+      env: TOXENV=mypy
+      <<: *not-on-master
+    - python: "3.5"
+      env: TOXENV=mypy
+      <<: *not-on-master
+    - python: "2.7"
+      # Ubuntu Trusty or older must be used because the oldest version of
+      # cryptography we support cannot be compiled against the version of
+      # OpenSSL in Xenial or newer.
+      dist: trusty
+      env: TOXENV='py27-{acme,apache,certbot,dns,nginx}-oldest'
+      <<: *not-on-master
+    - python: "3.4"
+      env: TOXENV=py34
+      <<: *not-on-master
+    - python: "3.7"
+      env: TOXENV=py37
+      <<: *not-on-master
+    - python: "3.8-dev"
+      env: TOXENV=py38
+      <<: *not-on-master
+    - sudo: required
+      env: TOXENV=apache_compat
+      services: docker
+      before_install:
+      addons:
+      <<: *not-on-master
+    - sudo: required
+      env: TOXENV=le_auto_xenial
+      services: docker
+      <<: *not-on-master
+    - python: "2.7"
+      env: TOXENV=apacheconftest-with-pebble
+      <<: *not-on-master
+    - python: "2.7"
+      env: TOXENV=nginxroundtrip
+      <<: *not-on-master
+
+    # Extended test suite on cron jobs and pushes to tested branches other than master
+    - sudo: required
+      env: TOXENV=nginx_compat
+      services: docker
+      before_install:
+      addons:
+      <<: *extended-test-suite
+    - python: "2.7"
+      env:
+        - TOXENV=travis-test-farm-apache2
+        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
+      <<: *extended-test-suite
+    - python: "2.7"
+      env:
+        - TOXENV=travis-test-farm-leauto-upgrades
+        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
+      git:
+        depth: false  # This is needed to have the history to checkout old versions of certbot-auto.
+      <<: *extended-test-suite
+    - python: "2.7"
+      env:
+        - TOXENV=travis-test-farm-certonly-standalone
+        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
+      <<: *extended-test-suite
+    - python: "2.7"
+      env:
+        - TOXENV=travis-test-farm-sdists
+        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
+      <<: *extended-test-suite
+    - python: "3.7"
+      env: TOXENV=py37 CERTBOT_NO_PIN=1
+      <<: *extended-test-suite
+    - python: "2.7"
+      env: ACME_SERVER=boulder-v1 TOXENV=integration
+      sudo: required
+      services: docker
+      <<: *extended-test-suite
+    - python: "2.7"
+      env: ACME_SERVER=boulder-v2 TOXENV=integration
+      sudo: required
+      services: docker
+      <<: *extended-test-suite
     - python: "2.7"
       env: ACME_SERVER=boulder-v1 TOXENV=integration-certbot-oldest
+      # Ubuntu Trusty or older must be used because the oldest version of
+      # cryptography we support cannot be compiled against the version of
+      # OpenSSL in Xenial or newer.
+      dist: trusty
       sudo: required
       services: docker
       <<: *extended-test-suite
     - python: "2.7"
       env: ACME_SERVER=boulder-v2 TOXENV=integration-certbot-oldest
+      # Ubuntu Trusty or older must be used because the oldest version of
+      # cryptography we support cannot be compiled against the version of
+      # OpenSSL in Xenial or newer.
+      dist: trusty
       sudo: required
       services: docker
       <<: *extended-test-suite
     - python: "2.7"
       env: ACME_SERVER=boulder-v1 TOXENV=integration-nginx-oldest
+      # Ubuntu Trusty or older must be used because the oldest version of
+      # cryptography we support cannot be compiled against the version of
+      # OpenSSL in Xenial or newer.
+      dist: trusty
       sudo: required
       services: docker
       <<: *extended-test-suite
     - python: "2.7"
       env: ACME_SERVER=boulder-v2 TOXENV=integration-nginx-oldest
+      # Ubuntu Trusty or older must be used because the oldest version of
+      # cryptography we support cannot be compiled against the version of
+      # OpenSSL in Xenial or newer.
+      dist: trusty
       sudo: required
       services: docker
       <<: *extended-test-suite
-#    - python: "3.4"
-#      env: TOXENV=py34
-#      <<: *extended-test-suite
-#    - python: "3.5"
-#      env: TOXENV=py35
-#      <<: *extended-test-suite
-#    - python: "3.6"
-#      env: TOXENV=py36
-#      <<: *extended-test-suite
-#    - python: "3.7"
-#      dist: xenial
-#      env: TOXENV=py37
-#      <<: *extended-test-suite
-#    - python: "3.4"
-#      env: ACME_SERVER=boulder-v1 TOXENV=integration
-#      sudo: required
-#      services: docker
-#      <<: *extended-test-suite
-#    - python: "3.4"
-#      env: ACME_SERVER=boulder-v2 TOXENV=integration
-#      sudo: required
-#      services: docker
-#      <<: *extended-test-suite
-#    - python: "3.5"
-#      env: ACME_SERVER=boulder-v1 TOXENV=integration
-#      sudo: required
-#      services: docker
-#      <<: *extended-test-suite
-#    - python: "3.5"
-#      env: ACME_SERVER=boulder-v2 TOXENV=integration
-#      sudo: required
-#      services: docker
-#      <<: *extended-test-suite
-#    - python: "3.6"
-#      env: ACME_SERVER=boulder-v1 TOXENV=integration
-#      sudo: required
-#      services: docker
-#      <<: *extended-test-suite
-#    - python: "3.6"
-#      env: ACME_SERVER=boulder-v2 TOXENV=integration
-#      sudo: required
-#      services: docker
-#      <<: *extended-test-suite
-#    - python: "3.7"
-#      dist: xenial
-#      env: ACME_SERVER=boulder-v1 TOXENV=integration
-#      sudo: required
-#      services: docker
-#      <<: *extended-test-suite
-#    - python: "3.7"
-#      dist: xenial
-#      env: ACME_SERVER=boulder-v2 TOXENV=integration
-#      sudo: required
-#      services: docker
-#      <<: *extended-test-suite
-#    - sudo: required
-#      env: TOXENV=le_auto_jessie
-#      services: docker
-#      <<: *extended-test-suite
-#    - sudo: required
-#      env: TOXENV=le_auto_centos6
-#      services: docker
-#      <<: *extended-test-suite
-#    - sudo: required
-#      env: TOXENV=docker_dev
-#      services: docker
-#      addons:
-#        apt:
-#          packages:  # don't install nginx and apache
-#            - libaugeas0
-#      <<: *extended-test-suite
-#    - language: generic
-#      env: TOXENV=py27
-#      os: osx
-#      # Using this osx_image is a workaround for
-#      # https://travis-ci.community/t/xcode-8-3-homebrew-outdated-error/3798.
-#      osx_image: xcode10.2
-#      addons:
-#        homebrew:
-#          packages:
-#            - augeas
-#            - python2
-#      <<: *extended-test-suite
-#    - language: generic
-#      env: TOXENV=py3
-#      os: osx
-#      # Using this osx_image is a workaround for
-#      # https://travis-ci.community/t/xcode-8-3-homebrew-outdated-error/3798.
-#      osx_image: xcode10.2
-#      addons:
-#        homebrew:
-#          packages:
-#            - augeas
-#            - python3
-#      <<: *extended-test-suite
+    - python: "3.4"
+      env: TOXENV=py34
+      <<: *extended-test-suite
+    - python: "3.5"
+      env: TOXENV=py35
+      <<: *extended-test-suite
+    - python: "3.6"
+      env: TOXENV=py36
+      <<: *extended-test-suite
+    - python: "3.7"
+      env: TOXENV=py37
+      <<: *extended-test-suite
+    - python: "3.8-dev"
+      env: TOXENV=py38
+      <<: *extended-test-suite
+    - python: "3.4"
+      env: ACME_SERVER=boulder-v1 TOXENV=integration
+      sudo: required
+      services: docker
+      <<: *extended-test-suite
+    - python: "3.4"
+      env: ACME_SERVER=boulder-v2 TOXENV=integration
+      sudo: required
+      services: docker
+      <<: *extended-test-suite
+    - python: "3.5"
+      env: ACME_SERVER=boulder-v1 TOXENV=integration
+      sudo: required
+      services: docker
+      <<: *extended-test-suite
+    - python: "3.5"
+      env: ACME_SERVER=boulder-v2 TOXENV=integration
+      sudo: required
+      services: docker
+      <<: *extended-test-suite
+    - python: "3.6"
+      env: ACME_SERVER=boulder-v1 TOXENV=integration
+      sudo: required
+      services: docker
+      <<: *extended-test-suite
+    - python: "3.6"
+      env: ACME_SERVER=boulder-v2 TOXENV=integration
+      sudo: required
+      services: docker
+      <<: *extended-test-suite
+    - python: "3.7"
+      env: ACME_SERVER=boulder-v1 TOXENV=integration
+      sudo: required
+      services: docker
+      <<: *extended-test-suite
+    - python: "3.7"
+      env: ACME_SERVER=boulder-v2 TOXENV=integration
+      sudo: required
+      services: docker
+      <<: *extended-test-suite
+    - python: "3.8-dev"
+      env: ACME_SERVER=boulder-v1 TOXENV=integration
+      <<: *extended-test-suite
+    - python: "3.8-dev"
+      env: ACME_SERVER=boulder-v2 TOXENV=integration
+      <<: *extended-test-suite
+    - sudo: required
+      env: TOXENV=le_auto_jessie
+      services: docker
+      <<: *extended-test-suite
+    - sudo: required
+      env: TOXENV=le_auto_centos6
+      services: docker
+    - sudo: required
+      env: TOXENV=le_auto_oraclelinux6
+      services: docker
+      <<: *extended-test-suite
+    - sudo: required
+      env: TOXENV=docker_dev
+      services: docker
+      addons:
+        apt:
+          packages:  # don't install nginx and apache
+            - libaugeas0
+      <<: *extended-test-suite
+    - language: generic
+      env: TOXENV=py27
+      os: osx
+      # Using this osx_image is a workaround for
+      # https://travis-ci.community/t/xcode-8-3-homebrew-outdated-error/3798.
+      osx_image: xcode10.2
+      addons:
+        homebrew:
+          packages:
+            - augeas
+            - python2
+      <<: *extended-test-suite
+    - language: generic
+      env: TOXENV=py3
+      os: osx
+      # Using this osx_image is a workaround for
+      # https://travis-ci.community/t/xcode-8-3-homebrew-outdated-error/3798.
+      osx_image: xcode10.2
+      addons:
+        homebrew:
+          packages:
+            - augeas
+            - python3
+      <<: *extended-test-suite
 
 # container-based infrastructure
 sudo: false

AUTHORS.md

@@ -18,6 +18,7 @@ Authors
 * [Alex Zorin](https://github.com/alexzorin)
 * [Amjad Mashaal](https://github.com/TheNavigat)
 * [Andrew Murray](https://github.com/radarhere)
+* [Andrzej Górski](https://github.com/andrzej3393)
 * [Anselm Levskaya](https://github.com/levskaya)
 * [Antoine Jacoutot](https://github.com/ajacoutot)
 * [asaph](https://github.com/asaph)
@@ -127,6 +128,7 @@ Authors
 * [Joubin Jabbari](https://github.com/joubin)
 * [Juho Juopperi](https://github.com/jkjuopperi)
 * [Kane York](https://github.com/riking)
+* [Kenichi Maehashi](https://github.com/kmaehashi)
 * [Kenneth Skovhede](https://github.com/kenkendk)
 * [Kevin Burke](https://github.com/kevinburke)
 * [Kevin London](https://github.com/kevinlondon)

CHANGELOG.md

@@ -2,7 +2,7 @@
 
 Certbot adheres to [Semantic Versioning](https://semver.org/).
 
-## 0.38.0 - master
+## 0.40.0 - master
 
 ### Added
 
@@ -10,9 +10,51 @@ Certbot adheres to [Semantic Versioning](https://semver.org/).
 
 ### Changed
 
+* Removed `--fast` flag from the test farm tests
+* `--server` may now be combined with `--dry-run`. Certbot will, as before, use the
+  staging server instead of the live server when `--dry-run` is used.
+* Updated certbot-dns-google to depend on newer versions of
+  google-api-python-client and oauth2client.
+* Migrated CentOS 6 certbot-auto users from Python 3.4 to Python 3.6.
+
+### Fixed
+
+*
+
+More details about these changes can be found on our GitHub repo.
+
+## 0.39.0 - 2019-10-01
+
+### Added
+
+* Support for Python 3.8 was added to Certbot and all of its components.
+* Support for CentOS 8 was added to certbot-auto.
+
+### Changed
+
+* Don't send OCSP requests for expired certificates
+* Return to using platform.linux_distribution instead of distro.linux_distribution in OS fingerprinting for Python < 3.8 
+* Updated the Nginx plugin's TLS configuration to keep support for some versions of IE11.
+
+### Fixed
+
+* Fixed OS detection in the Apache plugin on RHEL 6.
+
+More details about these changes can be found on our GitHub repo.
+
+## 0.38.0 - 2019-09-03
+
+### Added
+
+* Disable session tickets for Nginx users when appropriate.
+
+### Changed
+
 * If Certbot fails to rollback your server configuration, the error message
   links to the Let's Encrypt forum. Change the link to the Help category now
   that the Server category has been closed.
+* Replace platform.linux_distribution with distro.linux_distribution as a step
+  towards Python 3.8 support in Certbot.
 
 ### Fixed
 
@@ -20,6 +62,13 @@ Certbot adheres to [Semantic Versioning](https://semver.org/).
 
 More details about these changes can be found on our GitHub repo.
 
+## 0.37.2 - 2019-08-21
+
+* Stop disabling TLS session tickets in Nginx as it caused TLS failures on
+  some systems.
+
+More details about these changes can be found on our GitHub repo.
+
 ## 0.37.1 - 2019-08-08
 
 ### Fixed

acme/setup.py

@@ -3,7 +3,7 @@ from setuptools import find_packages
 from setuptools.command.test import test as TestCommand
 import sys
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [
@@ -73,6 +73,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
     ],

appveyor.yml

@@ -1,47 +0,0 @@
-image: Visual Studio 2015
-
-environment:
-  matrix:
-    - TOXENV: py35
-    - TOXENV: py37-cover
-    - TOXENV: integration-certbot
-
-branches:
-  only:
-    # apache-parser-v2 is a temporary branch for doing work related to
-    # rewriting the parser in the Apache plugin.
-    - apache-parser-v2
-    - master
-    - /^\d+\.\d+\.x$/ # Version branches like X.X.X
-    - /^test-.*$/
-
-init:
-  # Since master can receive only commits from PR that have already been tested, following
-  # condition avoid to launch all jobs except the coverage one for commits pushed to master.
-  - ps: |
-      if (-Not $Env:APPVEYOR_PULL_REQUEST_NUMBER -And $Env:APPVEYOR_REPO_BRANCH -Eq 'master' `
-          -And -Not ($Env:TOXENV -Like '*-cover'))
-      { $Env:APPVEYOR_SKIP_FINALIZE_ON_EXIT = 'true'; Exit-AppVeyorBuild }
-
-install:
-  # Use Python 3.7 by default
-  - SET PATH=C:\\Python37;C:\\Python37\\Scripts;%PATH%
-  # Using 4 processes is proven to be the most efficient integration tests config for AppVeyor
-  - IF %TOXENV%==integration-certbot SET PYTEST_ADDOPTS=--numprocesses=4
-  # Check env
-  - python --version
-  # Upgrade pip to avoid warnings
-  - python -m pip install --upgrade pip
-  # Ready to install tox and coverage
-  # tools/pip_install.py is used to pin packages to a known working version.
-  - python tools\\pip_install.py tox codecov
-
-build: off
-
-test_script:
-  - set TOX_TESTENV_PASSENV=APPVEYOR
-  # Test env is set by TOXENV env variable
-  - tox
-
-on_success:
-  - if exist .coverage codecov -F windows

certbot-apache/certbot_apache/entrypoint.py

@@ -24,6 +24,7 @@ OVERRIDE_CLASSES = {
     "fedora_old": override_centos.CentOSConfigurator,
     "fedora": override_fedora.FedoraConfigurator,
     "ol": override_centos.CentOSConfigurator,
+    "redhatenterpriseserver": override_centos.CentOSConfigurator,
     "red hat enterprise linux server": override_centos.CentOSConfigurator,
     "rhel": override_centos.CentOSConfigurator,
     "amazon": override_centos.CentOSConfigurator,

certbot-apache/certbot_apache/tests/http_01_test.py

@@ -7,6 +7,7 @@ from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-
 
 from certbot import achallenges
 from certbot import errors
+from certbot.compat import filesystem
 from certbot.compat import os
 from certbot.tests import acme_util
 
@@ -180,7 +181,7 @@ class ApacheHttp01Test(util.ApacheTest):
         self.assertEqual(self.http.perform(), expected_response)
 
         self.assertTrue(os.path.isdir(self.http.challenge_dir))
-        self._has_min_permissions(self.http.challenge_dir, 0o755)
+        self.assertTrue(filesystem.has_min_permissions(self.http.challenge_dir, 0o755))
         self._test_challenge_conf()
 
         for achall in achalls:
@@ -218,15 +219,10 @@ class ApacheHttp01Test(util.ApacheTest):
         name = os.path.join(self.http.challenge_dir, achall.chall.encode("token"))
         validation = achall.validation(self.account_key)
 
-        self._has_min_permissions(name, 0o644)
+        self.assertTrue(filesystem.has_min_permissions(name, 0o644))
         with open(name, 'rb') as f:
             self.assertEqual(f.read(), validation.encode())
 
-    def _has_min_permissions(self, path, min_mode):
-        """Tests the given file has at least the permissions in mode."""
-        st_mode = os.stat(path).st_mode
-        self.assertEqual(st_mode, st_mode | min_mode)
-
 
 if __name__ == "__main__":
     unittest.main()  # pragma: no cover

certbot-apache/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.29.0
-certbot[dev]==0.37.0
+certbot[dev]==0.39.0

certbot-apache/setup.py

@@ -4,13 +4,13 @@ from setuptools.command.test import test as TestCommand
 import sys
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.29.0',
-    'certbot>=0.37.0',
+    'certbot>=0.39.0',
     'mock',
     'python-augeas',
     'setuptools',
@@ -62,6 +62,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-auto

@@ -31,7 +31,7 @@ if [ -z "$VENV_PATH" ]; then
 fi
 VENV_BIN="$VENV_PATH/bin"
 BOOTSTRAP_VERSION_PATH="$VENV_PATH/certbot-auto-bootstrap-version.txt"
-LE_AUTO_VERSION="0.37.1"
+LE_AUTO_VERSION="0.39.0"
 BASENAME=$(basename $0)
 USAGE="Usage: $BASENAME [OPTIONS]
 A self-updating wrapper script for the Certbot ACME client. When run, updates
@@ -775,6 +775,8 @@ elif [ -f /etc/redhat-release ]; then
     RPM_USE_PYTHON_3=1
   elif [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
     RPM_USE_PYTHON_3=1
+  elif [ "$RPM_DIST_NAME" = "centos" -a "$RPM_DIST_VERSION" -ge 8 ]; then
+    RPM_USE_PYTHON_3=1
   else
     RPM_USE_PYTHON_3=0
   fi
@@ -1134,73 +1136,76 @@ if [ "$1" = "--le-auto-phase2" ]; then
 # To generate this, do (with docker and package hashin installed):
 # ```
 # letsencrypt-auto-source/rebuild_dependencies.py \
-#   letsencrypt-auto-sources/pieces/dependency-requirements.txt
+#   letsencrypt-auto-source/pieces/dependency-requirements.txt
+# ```
+# If you want to update a single dependency, run commands similar to these:
+# ```
+# pip install hashin
+# hashin -r dependency-requirements.txt cryptography==1.5.2
 # ```
 ConfigArgParse==0.14.0 \
     --hash=sha256:2e2efe2be3f90577aca9415e32cb629aa2ecd92078adbe27b53a03e53ff12e91
 asn1crypto==0.24.0 \
     --hash=sha256:2f1adbb7546ed199e3c90ef23ec95c5cf3585bac7d11fb7eb562a3fe89c64e87 \
     --hash=sha256:9d5c20441baf0cb60a4ac34cc447c6c189024b6b4c6cd7877034f4965c464e49
-certifi==2019.3.9 \
-    --hash=sha256:59b7658e26ca9c7339e00f8f4636cdfe59d34fa37b9b04f6f9e9926b3cece1a5 \
-    --hash=sha256:b26104d6835d1f5e49452a26eb2ff87fe7090b89dfcaee5ea2212697e1e1d7ae
-cffi==1.12.2 \
-    --hash=sha256:00b97afa72c233495560a0793cdc86c2571721b4271c0667addc83c417f3d90f \
-    --hash=sha256:0ba1b0c90f2124459f6966a10c03794082a2f3985cd699d7d63c4a8dae113e11 \
-    --hash=sha256:0bffb69da295a4fc3349f2ec7cbe16b8ba057b0a593a92cbe8396e535244ee9d \
-    --hash=sha256:21469a2b1082088d11ccd79dd84157ba42d940064abbfa59cf5f024c19cf4891 \
-    --hash=sha256:2e4812f7fa984bf1ab253a40f1f4391b604f7fc424a3e21f7de542a7f8f7aedf \
-    --hash=sha256:2eac2cdd07b9049dd4e68449b90d3ef1adc7c759463af5beb53a84f1db62e36c \
-    --hash=sha256:2f9089979d7456c74d21303c7851f158833d48fb265876923edcb2d0194104ed \
-    --hash=sha256:3dd13feff00bddb0bd2d650cdb7338f815c1789a91a6f68fdc00e5c5ed40329b \
-    --hash=sha256:4065c32b52f4b142f417af6f33a5024edc1336aa845b9d5a8d86071f6fcaac5a \
-    --hash=sha256:51a4ba1256e9003a3acf508e3b4f4661bebd015b8180cc31849da222426ef585 \
-    --hash=sha256:59888faac06403767c0cf8cfb3f4a777b2939b1fbd9f729299b5384f097f05ea \
-    --hash=sha256:59c87886640574d8b14910840327f5cd15954e26ed0bbd4e7cef95fa5aef218f \
-    --hash=sha256:610fc7d6db6c56a244c2701575f6851461753c60f73f2de89c79bbf1cc807f33 \
-    --hash=sha256:70aeadeecb281ea901bf4230c6222af0248c41044d6f57401a614ea59d96d145 \
-    --hash=sha256:71e1296d5e66c59cd2c0f2d72dc476d42afe02aeddc833d8e05630a0551dad7a \
-    --hash=sha256:8fc7a49b440ea752cfdf1d51a586fd08d395ff7a5d555dc69e84b1939f7ddee3 \
-    --hash=sha256:9b5c2afd2d6e3771d516045a6cfa11a8da9a60e3d128746a7fe9ab36dfe7221f \
-    --hash=sha256:9c759051ebcb244d9d55ee791259ddd158188d15adee3c152502d3b69005e6bd \
-    --hash=sha256:b4d1011fec5ec12aa7cc10c05a2f2f12dfa0adfe958e56ae38dc140614035804 \
-    --hash=sha256:b4f1d6332339ecc61275bebd1f7b674098a66fea11a00c84d1c58851e618dc0d \
-    --hash=sha256:c030cda3dc8e62b814831faa4eb93dd9a46498af8cd1d5c178c2de856972fd92 \
-    --hash=sha256:c2e1f2012e56d61390c0e668c20c4fb0ae667c44d6f6a2eeea5d7148dcd3df9f \
-    --hash=sha256:c37c77d6562074452120fc6c02ad86ec928f5710fbc435a181d69334b4de1d84 \
-    --hash=sha256:c8149780c60f8fd02752d0429246088c6c04e234b895c4a42e1ea9b4de8d27fb \
-    --hash=sha256:cbeeef1dc3c4299bd746b774f019de9e4672f7cc666c777cd5b409f0b746dac7 \
-    --hash=sha256:e113878a446c6228669144ae8a56e268c91b7f1fafae927adc4879d9849e0ea7 \
-    --hash=sha256:e21162bf941b85c0cda08224dade5def9360f53b09f9f259adb85fc7dd0e7b35 \
-    --hash=sha256:fb6934ef4744becbda3143d30c6604718871495a5e36c408431bf33d9c146889
+certifi==2019.6.16 \
+    --hash=sha256:046832c04d4e752f37383b628bc601a7ea7211496b4638f6514d0e5b9acc4939 \
+    --hash=sha256:945e3ba63a0b9f577b1395204e13c3a231f9bc0223888be653286534e5873695
+cffi==1.12.3 \
+    --hash=sha256:041c81822e9f84b1d9c401182e174996f0bae9991f33725d059b771744290774 \
+    --hash=sha256:046ef9a22f5d3eed06334d01b1e836977eeef500d9b78e9ef693f9380ad0b83d \
+    --hash=sha256:066bc4c7895c91812eff46f4b1c285220947d4aa46fa0a2651ff85f2afae9c90 \
+    --hash=sha256:066c7ff148ae33040c01058662d6752fd73fbc8e64787229ea8498c7d7f4041b \
+    --hash=sha256:2444d0c61f03dcd26dbf7600cf64354376ee579acad77aef459e34efcb438c63 \
+    --hash=sha256:300832850b8f7967e278870c5d51e3819b9aad8f0a2c8dbe39ab11f119237f45 \
+    --hash=sha256:34c77afe85b6b9e967bd8154e3855e847b70ca42043db6ad17f26899a3df1b25 \
+    --hash=sha256:46de5fa00f7ac09f020729148ff632819649b3e05a007d286242c4882f7b1dc3 \
+    --hash=sha256:4aa8ee7ba27c472d429b980c51e714a24f47ca296d53f4d7868075b175866f4b \
+    --hash=sha256:4d0004eb4351e35ed950c14c11e734182591465a33e960a4ab5e8d4f04d72647 \
+    --hash=sha256:4e3d3f31a1e202b0f5a35ba3bc4eb41e2fc2b11c1eff38b362de710bcffb5016 \
+    --hash=sha256:50bec6d35e6b1aaeb17f7c4e2b9374ebf95a8975d57863546fa83e8d31bdb8c4 \
+    --hash=sha256:55cad9a6df1e2a1d62063f79d0881a414a906a6962bc160ac968cc03ed3efcfb \
+    --hash=sha256:5662ad4e4e84f1eaa8efce5da695c5d2e229c563f9d5ce5b0113f71321bcf753 \
+    --hash=sha256:59b4dc008f98fc6ee2bb4fd7fc786a8d70000d058c2bbe2698275bc53a8d3fa7 \
+    --hash=sha256:73e1ffefe05e4ccd7bcea61af76f36077b914f92b76f95ccf00b0c1b9186f3f9 \
+    --hash=sha256:a1f0fd46eba2d71ce1589f7e50a9e2ffaeb739fb2c11e8192aa2b45d5f6cc41f \
+    --hash=sha256:a2e85dc204556657661051ff4bab75a84e968669765c8a2cd425918699c3d0e8 \
+    --hash=sha256:a5457d47dfff24882a21492e5815f891c0ca35fefae8aa742c6c263dac16ef1f \
+    --hash=sha256:a8dccd61d52a8dae4a825cdbb7735da530179fea472903eb871a5513b5abbfdc \
+    --hash=sha256:ae61af521ed676cf16ae94f30fe202781a38d7178b6b4ab622e4eec8cefaff42 \
+    --hash=sha256:b012a5edb48288f77a63dba0840c92d0504aa215612da4541b7b42d849bc83a3 \
+    --hash=sha256:d2c5cfa536227f57f97c92ac30c8109688ace8fa4ac086d19d0af47d134e2909 \
+    --hash=sha256:d42b5796e20aacc9d15e66befb7a345454eef794fdb0737d1af593447c6c8f45 \
+    --hash=sha256:dee54f5d30d775f525894d67b1495625dd9322945e7fee00731952e0368ff42d \
+    --hash=sha256:e070535507bd6aa07124258171be2ee8dfc19119c28ca94c9dfb7efd23564512 \
+    --hash=sha256:e1ff2748c84d97b065cc95429814cdba39bcbd77c9c85c89344b317dc0d9cbff \
+    --hash=sha256:ed851c75d1e0e043cbf5ca9a8e1b13c4c90f3fbd863dacb01c0808e2b5204201
 chardet==3.0.4 \
     --hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae \
     --hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691
 configobj==5.0.6 \
     --hash=sha256:a2f5650770e1c87fb335af19a9b7eb73fc05ccf22144eb68db7d00cd2bcb0902
-cryptography==2.6.1 \
-    --hash=sha256:066f815f1fe46020877c5983a7e747ae140f517f1b09030ec098503575265ce1 \
-    --hash=sha256:210210d9df0afba9e000636e97810117dc55b7157c903a55716bb73e3ae07705 \
-    --hash=sha256:26c821cbeb683facb966045e2064303029d572a87ee69ca5a1bf54bf55f93ca6 \
-    --hash=sha256:2afb83308dc5c5255149ff7d3fb9964f7c9ee3d59b603ec18ccf5b0a8852e2b1 \
-    --hash=sha256:2db34e5c45988f36f7a08a7ab2b69638994a8923853dec2d4af121f689c66dc8 \
-    --hash=sha256:409c4653e0f719fa78febcb71ac417076ae5e20160aec7270c91d009837b9151 \
-    --hash=sha256:45a4f4cf4f4e6a55c8128f8b76b4c057027b27d4c67e3fe157fa02f27e37830d \
-    --hash=sha256:48eab46ef38faf1031e58dfcc9c3e71756a1108f4c9c966150b605d4a1a7f659 \
-    --hash=sha256:6b9e0ae298ab20d371fc26e2129fd683cfc0cfde4d157c6341722de645146537 \
-    --hash=sha256:6c4778afe50f413707f604828c1ad1ff81fadf6c110cb669579dea7e2e98a75e \
-    --hash=sha256:8c33fb99025d353c9520141f8bc989c2134a1f76bac6369cea060812f5b5c2bb \
-    --hash=sha256:9873a1760a274b620a135054b756f9f218fa61ca030e42df31b409f0fb738b6c \
-    --hash=sha256:9b069768c627f3f5623b1cbd3248c5e7e92aec62f4c98827059eed7053138cc9 \
-    --hash=sha256:9e4ce27a507e4886efbd3c32d120db5089b906979a4debf1d5939ec01b9dd6c5 \
-    --hash=sha256:acb424eaca214cb08735f1a744eceb97d014de6530c1ea23beb86d9c6f13c2ad \
-    --hash=sha256:c8181c7d77388fe26ab8418bb088b1a1ef5fde058c6926790c8a0a3d94075a4a \
-    --hash=sha256:d4afbb0840f489b60f5a580a41a1b9c3622e08ecb5eec8614d4fb4cd914c4460 \
-    --hash=sha256:d9ed28030797c00f4bc43c86bf819266c76a5ea61d006cd4078a93ebf7da6bfd \
-    --hash=sha256:e603aa7bb52e4e8ed4119a58a03b60323918467ef209e6ff9db3ac382e5cf2c6
-# Package enum34 needs to be explicitly limited to Python2.x, in order to avoid
-# certbot-auto failures on Python 3.6+ which enum34 doesn't support. See #5456.
-enum34==1.1.6 ; python_version < '3.4' \
+cryptography==2.7 \
+    --hash=sha256:24b61e5fcb506424d3ec4e18bca995833839bf13c59fc43e530e488f28d46b8c \
+    --hash=sha256:25dd1581a183e9e7a806fe0543f485103232f940fcfc301db65e630512cce643 \
+    --hash=sha256:3452bba7c21c69f2df772762be0066c7ed5dc65df494a1d53a58b683a83e1216 \
+    --hash=sha256:41a0be220dd1ed9e998f5891948306eb8c812b512dc398e5a01846d855050799 \
+    --hash=sha256:5751d8a11b956fbfa314f6553d186b94aa70fdb03d8a4d4f1c82dcacf0cbe28a \
+    --hash=sha256:5f61c7d749048fa6e3322258b4263463bfccefecb0dd731b6561cb617a1d9bb9 \
+    --hash=sha256:72e24c521fa2106f19623a3851e9f89ddfdeb9ac63871c7643790f872a305dfc \
+    --hash=sha256:7b97ae6ef5cba2e3bb14256625423413d5ce8d1abb91d4f29b6d1a081da765f8 \
+    --hash=sha256:961e886d8a3590fd2c723cf07be14e2a91cf53c25f02435c04d39e90780e3b53 \
+    --hash=sha256:96d8473848e984184b6728e2c9d391482008646276c3ff084a1bd89e15ff53a1 \
+    --hash=sha256:ae536da50c7ad1e002c3eee101871d93abdc90d9c5f651818450a0d3af718609 \
+    --hash=sha256:b0db0cecf396033abb4a93c95d1602f268b3a68bb0a9cc06a7cff587bb9a7292 \
+    --hash=sha256:cfee9164954c186b191b91d4193989ca994703b2fff406f71cf454a2d3c7327e \
+    --hash=sha256:e6347742ac8f35ded4a46ff835c60e68c22a536a8ae5c4422966d06946b6d4c6 \
+    --hash=sha256:f27d93f0139a3c056172ebb5d4f9056e770fdf0206c2f422ff2ebbad142e09ed \
+    --hash=sha256:f57b76e46a58b63d1c6375017f4564a28f19a5ca912691fd2e4261b3414b618d
+distro==1.4.0 \
+    --hash=sha256:362dde65d846d23baee4b5c058c8586f219b5a54be1cf5fc6ff55c4578392f57 \
+    --hash=sha256:eedf82a470ebe7d010f1872c17237c79ab04097948800029994fa458e52fb4b4
+enum34==1.1.6 \
     --hash=sha256:2d81cbbe0e73112bdfe6ef8576f2238f2ba27dd0d55752a776c41d38b7da2850 \
     --hash=sha256:644837f692e5f550741432dd3f223bbb9852018674981b1664e5dc339387588a \
     --hash=sha256:6bd0f6ad48ec2aa117d3d141940d484deccda84d4fcd884f5c3d93c23ecd8c79 \
@@ -1216,18 +1221,18 @@ idna==2.8 \
 ipaddress==1.0.22 \
     --hash=sha256:64b28eec5e78e7510698f6d4da08800a5c575caa4a286c93d651c5d3ff7b6794 \
     --hash=sha256:b146c751ea45cad6188dd6cf2d9b757f6f4f8d6ffb96a023e6f2e26eea02a72c
-josepy==1.1.0 \
-    --hash=sha256:1309a25aac3caeff5239729c58ff9b583f7d022ffdb1553406ddfc8e5b52b76e \
-    --hash=sha256:fb5c62c77d26e04df29cb5ecd01b9ce69b6fcc9e521eb1ca193b7faa2afa7086
+josepy==1.2.0 \
+    --hash=sha256:8ea15573203f28653c00f4ac0142520777b1c59d9eddd8da3f256c6ba3cac916 \
+    --hash=sha256:9cec9a839fe9520f0420e4f38e7219525daccce4813296627436fe444cd002d3
 mock==1.3.0 \
     --hash=sha256:1e247dbecc6ce057299eb7ee019ad68314bb93152e81d9a6110d35f4d5eca0f6 \
     --hash=sha256:3f573a18be94de886d1191f27c168427ef693e8dcfcecf95b170577b2eb69cbb
 parsedatetime==2.4 \
     --hash=sha256:3d817c58fb9570d1eec1dd46fa9448cd644eeed4fb612684b02dfda3a79cb84b \
     --hash=sha256:9ee3529454bf35c40a77115f5a596771e59e1aee8c53306f346c461b8e913094
-pbr==5.1.3 \
-    --hash=sha256:8257baf496c8522437e8a6cfe0f15e00aedc6c0e0e7c9d55eeeeab31e0853843 \
-    --hash=sha256:8c361cc353d988e4f5b998555c88098b9d5964c2e11acf7b0d21925a66bb5824
+pbr==5.4.2 \
+    --hash=sha256:56e52299170b9492513c64be44736d27a512fa7e606f21942160b68ce510b4bc \
+    --hash=sha256:9b321c204a88d8ab5082699469f52cc94c5da45c51f114113d01b3d993c24cdf
 pyOpenSSL==19.0.0 \
     --hash=sha256:aeca66338f6de19d1aa46ed634c3b9ae519a64b458f8468aec688e7e3c20f200 \
     --hash=sha256:c727930ad54b10fc157015014b666f2d8b41f70c0d03e83ab67624fd3dd5d1e6
@@ -1236,14 +1241,14 @@ pyRFC3339==1.1 \
     --hash=sha256:81b8cbe1519cdb79bed04910dd6fa4e181faf8c88dff1e1b987b5f7ab23a5b1a
 pycparser==2.19 \
     --hash=sha256:a988718abfad80b6b157acce7bf130a30876d27603738ac39f140993246b25b3
-pyparsing==2.3.1 \
-    --hash=sha256:66c9268862641abcac4a96ba74506e594c884e3f57690a696d21ad8210ed667a \
-    --hash=sha256:f6c5ef0d7480ad048c054c37632c67fca55299990fff127850181659eea33fc3
+pyparsing==2.4.2 \
+    --hash=sha256:6f98a7b9397e206d78cc01df10131398f1c8b8510a2f4d97d9abd82e1aacdd80 \
+    --hash=sha256:d9338df12903bbf5d65a0e4e87c2161968b10d2e489652bb47001d82a9b028b4
 python-augeas==0.5.0 \
     --hash=sha256:67d59d66cdba8d624e0389b87b2a83a176f21f16a87553b50f5703b23f29bac2
-pytz==2018.9 \
-    --hash=sha256:32b0891edff07e28efe91284ed9c31e123d84bea3fd98e1f72be2508f43ef8d9 \
-    --hash=sha256:d5f05e487007e29e03409f9398d074e158d920d36eb82eaf66fb1136b0c5374c
+pytz==2019.2 \
+    --hash=sha256:26c0b32e437e54a18161324a2fca3c4b9846b74a8dccddd843113109e1116b32 \
+    --hash=sha256:c894d57500a4cd2d5c71114aaab77dbab5eabd9022308ce5ac9bb93a60a6f0c7
 requests==2.21.0 \
     --hash=sha256:502a824f31acdacb3a35b6690b5fbf0bc41d63a24a45c4004352b0242707598e \
     --hash=sha256:7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b
@@ -1253,15 +1258,15 @@ requests-toolbelt==0.9.1 \
 six==1.12.0 \
     --hash=sha256:3350809f0555b11f552448330d0b52d5f24c91a322ea4a15ef22629740f3761c \
     --hash=sha256:d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73
-urllib3==1.24.2 \
-    --hash=sha256:4c291ca23bbb55c76518905869ef34bdd5f0e46af7afe6861e8375643ffee1a0 \
-    --hash=sha256:9a247273df709c4fedb38c711e44292304f73f39ab01beda9f6b9fc375669ac3
+urllib3==1.24.3 \
+    --hash=sha256:2393a695cd12afedd0dcb26fe5d50d0cf248e5a66f75dbd89a3d4eb333a61af4 \
+    --hash=sha256:a637e5fae88995b256e3409dc4d52c2e2e0ba32c42a6365fee8bbd2238de3cfb
 zope.component==4.5 \
     --hash=sha256:6edfd626c3b593b72895a8cfcf79bff41f4619194ce996a85bce31ac02b94e55 \
     --hash=sha256:984a06ba3def0b02b1117fa4c45b56e772e8c29c0340820fbf367e440a93a3a4
-zope.deferredimport==4.3 \
-    --hash=sha256:2ddef5a7ecfff132a2dd796253366ecf9748a446e30f1a0b3a636aec9d9c05c5 \
-    --hash=sha256:4aae9cbacb2146cca58e62be0a914f0cec034d3b2d41135ea212ca8a96f4b5ec
+zope.deferredimport==4.3.1 \
+    --hash=sha256:57b2345e7b5eef47efcd4f634ff16c93e4265de3dcf325afc7315ade48d909e1 \
+    --hash=sha256:9a0c211df44aa95f1c4e6d2626f90b400f56989180d3ef96032d708da3d23e0a
 zope.deprecation==4.4.0 \
     --hash=sha256:0d453338f04bacf91bbfba545d8bcdf529aa829e67b705eac8c1a7fdce66e2df \
     --hash=sha256:f1480b74995958b24ce37b0ef04d3663d2683e5d6debc96726eff18acf4ea113
@@ -1309,18 +1314,18 @@ zope.interface==4.6.0 \
     --hash=sha256:d788a3999014ddf416f2dc454efa4a5dbeda657c6aba031cf363741273804c6b \
     --hash=sha256:eed88ae03e1ef3a75a0e96a55a99d7937ed03e53d0cffc2451c208db445a2966 \
     --hash=sha256:f99451f3a579e73b5dd58b1b08d1179791d49084371d9a47baad3b22417f0317
-zope.proxy==4.3.1 \
-    --hash=sha256:0cbcfcafaa3b5fde7ba7a7b9a2b5f09af25c9b90087ad65f9e61359fed0ca63b \
-    --hash=sha256:3de631dd5054a3a20b9ebff0e375f39c0565f1fb9131200d589a6a8f379214cd \
-    --hash=sha256:5429134d04d42262f4dac25f6dea907f6334e9a751ffc62cb1d40226fb52bdeb \
-    --hash=sha256:563c2454b2d0f23bca54d2e0e4d781149b7b06cb5df67e253ca3620f37202dd2 \
-    --hash=sha256:5bcf773345016b1461bb07f70c635b9386e5eaaa08e37d3939dcdf12d3fdbec5 \
-    --hash=sha256:8d84b7aef38c693874e2f2084514522bf73fd720fde0ce2a9352a51315ffa475 \
-    --hash=sha256:90de9473c05819b36816b6cb957097f809691836ed3142648bf62da84b4502fe \
-    --hash=sha256:dd592a69fe872445542a6e1acbefb8e28cbe6b4007b8f5146da917e49b155cc3 \
-    --hash=sha256:e7399ab865399fce322f9cefc6f2f3e4099d087ba581888a9fea1bbe1db42a08 \
-    --hash=sha256:e7d1c280d86d72735a420610df592aac72332194e531a8beff43a592c3a1b8eb \
-    --hash=sha256:e90243fee902adb0c39eceb3c69995c0f2004bc3fdb482fbf629efc656d124ed
+zope.proxy==4.3.2 \
+    --hash=sha256:320a7619992e42142549ebf61e14ce27683b4d14b0cbc45f7c037ba64edb560c \
+    --hash=sha256:824d4dbabbb7deb84f25fdb96ea1eeca436a1802c3c8d323b3eb4ac9d527d41c \
+    --hash=sha256:8a32eb9c94908f3544da2dae3f4a9e6961d78819b88ac6b6f4a51cee2d65f4a0 \
+    --hash=sha256:96265fd3bc3ea646f98482e16307a69de21402eeaaaaf4b841c1161ac2f71bb0 \
+    --hash=sha256:ab6d6975d9c51c13cac828ff03168de21fb562b0664c59bcdc4a4b10f39a5b17 \
+    --hash=sha256:af10cb772391772463f65a58348e2de5ecc06693c16d2078be276dc068bcbb54 \
+    --hash=sha256:b8fd3a3de3f7b6452775e92af22af5977b17b69ac86a38a3ddfe870e40a0d05f \
+    --hash=sha256:bb7088f1bed3b8214284a5e425dc23da56f2f28e8815b7580bfed9e245b6c0b6 \
+    --hash=sha256:bc29b3665eac34f14c4aef5224bef045efcfb1a7d12d78c8685858de5fbf21c0 \
+    --hash=sha256:c39fa6a159affeae5fe31b49d9f5b12bd674fe77271a9a324408b271440c50a7 \
+    --hash=sha256:e946a036ac5b9f897e986ac9dc950a34cffc857d88eae6727b8434fbc4752366
 
 # Contains the requirements for the letsencrypt package.
 #
@@ -1333,18 +1338,18 @@ letsencrypt==0.7.0 \
     --hash=sha256:105a5fb107e45bcd0722eb89696986dcf5f08a86a321d6aef25a0c7c63375ade \
     --hash=sha256:c36e532c486a7e92155ee09da54b436a3c420813ec1c590b98f635d924720de9
 
-certbot==0.37.1 \
-    --hash=sha256:84dbdad204327b8d8ef9ab5b040f2be1e427a9f7e087affcc9a6051ea1b03fe7 \
-    --hash=sha256:aace73e63b0c11cdb4b0bd33e1780c1fbe0ce5669dc72e80c3aa9500145daf16
-acme==0.37.1 \
-    --hash=sha256:83a4f6f3c5eb6a85233d5ba87714b426f2d096df58d711f8a2fc4071eb3fd3fc \
-    --hash=sha256:c069a761990751f7c4bf51d2e87ae10319bf460de6629d2908c9fa6f69e97111
-certbot-apache==0.37.1 \
-    --hash=sha256:3ea832408877b12b3a60d17e8b2ee3387364f8c3023ac267161c25b99087cd42 \
-    --hash=sha256:e46c2644451101c0e216aa1f525a577cc903efaf871e0e4da277224a4439040c
-certbot-nginx==0.37.1 \
-    --hash=sha256:1f9af389d26f06634e2eefaace3354e7679dabb4295e1d55d05a4ee7e23a64bd \
-    --hash=sha256:02a7ec15bd388d0f0e94a34c86a8f8d618ec7d5ffde0c206039bb4c46b294ce4
+certbot==0.39.0 \
+    --hash=sha256:f1a70651a6c5137a448f4a8db17b09af619f80a077326caae6b74278bf1db488 \
+    --hash=sha256:885cee1c4d05888af86b626cbbfc29d3c6c842ef4fe8f4a486994cef9daddfe0
+acme==0.39.0 \
+    --hash=sha256:4f8be913df289b981852042719469cc367a7e436256f232c799d0bd1521db710 \
+    --hash=sha256:a2fcb75d16de6804f4b4d773a457ee2f6434ebaf8fd1aa60862a91d4e8f73608
+certbot-apache==0.39.0 \
+    --hash=sha256:c7a8630a85b753a52ca0b8c19e24b8f85ac4ba028292a95745e250c2e72faab9 \
+    --hash=sha256:4651a0212c9ebc3087281dad92ad3cb355bb2730f432d0180a8d23325d11825a
+certbot-nginx==0.39.0 \
+    --hash=sha256:76e5862ad5cc0fbc099df3502987c101c60dee1c188a579eac990edee7a910df \
+    --hash=sha256:ceac88df52d3b27d14c3052b9e90ada327d7e14ecd6e4af7519918182d6138b4
 
 UNLIKELY_EOF
     # -------------------------------------------------------------------------

certbot-ci/certbot_integration_tests/.coveragerc

@@ -2,6 +2,7 @@
 # Avoid false warnings because certbot packages are not installed in the thread that executes
 # the coverage: indeed, certbot is launched as a CLI from a subprocess.
 disable_warnings = module-not-imported,no-data-collected
+omit = **/*_test.py,**/tests/*,**/dns_common*,**/certbot_nginx/parser_obj.py
 
 [report]
 # Exclude unit tests in coverage during integration tests.

certbot-ci/certbot_integration_tests/utils/certbot_call.py

@@ -37,6 +37,22 @@ def _prepare_environ(workspace):
     new_environ = os.environ.copy()
     new_environ['TMPDIR'] = workspace
 
+    # So, pytest is nice, and a little too nice for our usage.
+    # In order to help user to call seamlessly any piece of python code without requiring to
+    # install it as a full-fledged setuptools distribution for instance, it may inject the path
+    # to the test files into the PYTHONPATH. This allows the python interpreter to import
+    # as modules any python file available at this path.
+    # See https://docs.pytest.org/en/3.2.5/pythonpath.html for the explanation and description.
+    # However this behavior is not good in integration tests, in particular the nginx oldest ones.
+    # Indeed during these kind of tests certbot is installed as a transitive dependency to
+    # certbot-nginx. Here is the trick: this certbot version is not necessarily the same as
+    # the certbot codebase lying in current working directory. For instance in oldest tests
+    # certbot==0.36.0 may be installed while the codebase corresponds to certbot==0.37.0.dev0.
+    # Then during a pytest run, PYTHONPATH contains the path to the Certbot codebase, so invoking
+    # certbot will import the modules from the codebase (0.37.0.dev0), not from the
+    # required/installed version (0.36.0).
+    # This will lead to funny and totally incomprehensible errors. To avoid that, we ensure that
+    # if PYTHONPATH is set, it does not contain the path to the root of the codebase.
     if new_environ.get('PYTHONPATH'):
         # certbot_integration_tests.__file__ is:
         # '/path/to/certbot/certbot-ci/certbot_integration_tests/__init__.pyc'

certbot-ci/setup.py

@@ -51,6 +51,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
     ],

certbot-compatibility-test/setup.py

@@ -4,7 +4,7 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 install_requires = [
     'certbot',
@@ -47,6 +47,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
     ],

certbot-dns-cloudflare/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.29.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-cloudflare/setup.py

@@ -2,13 +2,13 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.29.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'cloudflare>=1.5.1',
     'mock',
     'setuptools',
@@ -43,6 +43,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-cloudxns/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.31.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-cloudxns/setup.py

@@ -2,13 +2,13 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.31.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'dns-lexicon>=2.2.1',  # Support for >1 TXT record per name
     'mock',
     'setuptools',
@@ -43,6 +43,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-digitalocean/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.29.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-digitalocean/setup.py

@@ -2,13 +2,13 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.29.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'mock',
     'python-digitalocean>=1.11',
     'setuptools',
@@ -44,6 +44,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-dnsimple/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.31.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-dnsimple/setup.py

@@ -3,13 +3,13 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.31.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'mock',
     'setuptools',
     'zope.interface',
@@ -55,6 +55,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-dnsmadeeasy/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.31.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-dnsmadeeasy/setup.py

@@ -2,13 +2,13 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.31.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'dns-lexicon>=2.2.1',  # Support for >1 TXT record per name
     'mock',
     'setuptools',
@@ -43,6 +43,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-gehirn/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.31.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-gehirn/setup.py

@@ -2,12 +2,12 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [
     'acme>=0.31.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'dns-lexicon>=2.1.22',
     'mock',
     'setuptools',
@@ -41,6 +41,8 @@ setup(
         'Programming Language :: Python :: 3.4',
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-google/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.29.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-google/setup.py

@@ -2,18 +2,16 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.29.0',
-    'certbot>=0.34.0',
-    # 1.5 is the first version that supports oauth2client>=2.0
-    'google-api-python-client>=1.5',
+    'certbot>=0.39.0',
+    'google-api-python-client>=1.5.5',
     'mock',
-    # for oauth2client.service_account.ServiceAccountCredentials
-    'oauth2client>=2.0',
+    'oauth2client>=4.0',
     'setuptools',
     'zope.interface',
     # already a dependency of google-api-python-client, but added for consistency
@@ -48,6 +46,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-linode/local-oldest-requirements.txt

@@ -1,4 +1,4 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.31.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0
 dns-lexicon==2.2.3

certbot-dns-linode/setup.py

@@ -1,12 +1,12 @@
 from setuptools import setup
 from setuptools import find_packages
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [
     'acme>=0.31.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'dns-lexicon>=2.2.3',
     'mock',
     'setuptools',
@@ -41,6 +41,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-luadns/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.31.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-luadns/setup.py

@@ -2,13 +2,13 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.31.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'dns-lexicon>=2.2.1',  # Support for >1 TXT record per name
     'mock',
     'setuptools',
@@ -43,6 +43,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-nsone/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.31.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-nsone/setup.py

@@ -2,13 +2,13 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.31.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'dns-lexicon>=2.2.1',  # Support for >1 TXT record per name
     'mock',
     'setuptools',
@@ -43,6 +43,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-ovh/local-oldest-requirements.txt

@@ -1,4 +1,4 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.31.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0
 dns-lexicon==2.7.14

certbot-dns-ovh/setup.py

@@ -2,13 +2,13 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.31.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'dns-lexicon>=2.7.14',  # Correct proxy use on OVH provider
     'mock',
     'setuptools',
@@ -42,6 +42,8 @@ setup(
         'Programming Language :: Python :: 3.4',
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-rfc2136/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.29.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-rfc2136/setup.py

@@ -2,13 +2,13 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.29.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'dnspython',
     'mock',
     'setuptools',
@@ -43,6 +43,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-route53/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.29.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-route53/setup.py

@@ -1,13 +1,13 @@
 from setuptools import setup
 from setuptools import find_packages
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
 install_requires = [
     'acme>=0.29.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'boto3',
     'mock',
     'setuptools',
@@ -37,6 +37,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-dns-sakuracloud/local-oldest-requirements.txt

@@ -1,3 +1,3 @@
 # Remember to update setup.py to match the package versions below.
 acme[dev]==0.31.0
-certbot[dev]==0.34.0
+certbot[dev]==0.39.0

certbot-dns-sakuracloud/setup.py

@@ -2,12 +2,12 @@ from setuptools import setup
 from setuptools import find_packages
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [
     'acme>=0.31.0',
-    'certbot>=0.34.0',
+    'certbot>=0.39.0',
     'dns-lexicon>=2.1.23',
     'mock',
     'setuptools',
@@ -41,6 +41,8 @@ setup(
         'Programming Language :: Python :: 3.4',
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot-nginx/certbot_nginx/configurator.py

@@ -1,4 +1,6 @@
 """Nginx Configuration"""
+# https://github.com/PyCQA/pylint/issues/73
+from distutils.version import LooseVersion # pylint: disable=no-name-in-module,import-error
 import logging
 import re
 import socket
@@ -91,8 +93,12 @@ class NginxConfigurator(common.Installer):
         :param tup version: version of Nginx as a tuple (1, 4, 7)
             (used mostly for unittesting)
 
+        :param tup openssl_version: version of OpenSSL linked to Nginx as a tuple (1, 4, 7)
+            (used mostly for unittesting)
+
         """
         version = kwargs.pop("version", None)
+        openssl_version = kwargs.pop("openssl_version", None)
         super(NginxConfigurator, self).__init__(*args, **kwargs)
 
         # Verify that all directories and files exist with proper permissions
@@ -115,6 +121,7 @@ class NginxConfigurator(common.Installer):
         # These will be set in the prepare function
         self.parser = None
         self.version = version
+        self.openssl_version = openssl_version
         self._enhance_func = {"redirect": self._enable_redirect,
                               "ensure-http-header": self._set_http_header,
                               "staple-ocsp": self._enable_ocsp_stapling}
@@ -124,11 +131,33 @@ class NginxConfigurator(common.Installer):
     @property
     def mod_ssl_conf_src(self):
         """Full absolute path to SSL configuration file source."""
-        config_filename = "options-ssl-nginx.conf"
-        if self.version < (1, 5, 9):
-            config_filename = "options-ssl-nginx-old.conf"
-        elif self.version < (1, 13, 0):
-            config_filename = "options-ssl-nginx-tls12-only.conf"
+
+        # Why all this complexity? Well, we want to support Mozilla's intermediate
+        # recommendations. But TLS1.3 is only supported by newer versions of Nginx.
+        # And as for session tickets, our ideal is to turn them off across the board.
+        # But! Turning them off at all is only supported with new enough versions of
+        # Nginx. And older versions of OpenSSL have a bug that leads to browser errors
+        # given certain configurations. While we'd prefer to have forward secrecy, we'd
+        # rather fail open than error out. Unfortunately, Nginx can be compiled against
+        # many versions of OpenSSL. So we have to check both for the two different features,
+        # leading to four different combinations of options.
+        # For a complete history, check out https://github.com/certbot/certbot/issues/7322
+
+        use_tls13 = self.version >= (1, 13, 0)
+        session_tix_off = self.version >= (1, 5, 9) and self.openssl_version and\
+            LooseVersion(self.openssl_version) >= LooseVersion('1.0.2l')
+
+        if use_tls13:
+            if session_tix_off:
+                config_filename = "options-ssl-nginx.conf"
+            else:
+                config_filename = "options-ssl-nginx-tls13-session-tix-on.conf"
+        else:
+            if session_tix_off:
+                config_filename = "options-ssl-nginx-tls12-only.conf"
+            else:
+                config_filename = "options-ssl-nginx-old.conf"
+
         return pkg_resources.resource_filename(
             "certbot_nginx", os.path.join("tls_configs", config_filename))
 
@@ -169,6 +198,9 @@ class NginxConfigurator(common.Installer):
         if self.version is None:
             self.version = self.get_version()
 
+        if self.openssl_version is None:
+            self.openssl_version = self._get_openssl_version()
+
         self.install_ssl_options_conf(self.mod_ssl_conf, self.updated_mod_ssl_conf_digest)
 
         self.install_ssl_dhparams()
@@ -909,17 +941,14 @@ class NginxConfigurator(common.Installer):
         util.make_or_verify_dir(self.config.backup_dir, core_constants.CONFIG_DIRS_MODE)
         util.make_or_verify_dir(self.config.config_dir, core_constants.CONFIG_DIRS_MODE)
 
-    def get_version(self):
-        """Return version of Nginx Server.
+    def _nginx_version(self):
+        """Return results of nginx -V
 
-        Version is returned as tuple. (ie. 2.4.7 = (2, 4, 7))
-
-        :returns: version
-        :rtype: tuple
+        :returns: version text
+        :rtype: str
 
         :raises .PluginError:
-            Unable to find Nginx version or version is unsupported
-
+            Unable to run Nginx version command
         """
         try:
             proc = subprocess.Popen(
@@ -932,6 +961,21 @@ class NginxConfigurator(common.Installer):
             logger.debug(str(error), exc_info=True)
             raise errors.PluginError(
                 "Unable to run %s -V" % self.conf('ctl'))
+        return text
+
+    def get_version(self):
+        """Return version of Nginx Server.
+
+        Version is returned as tuple. (ie. 2.4.7 = (2, 4, 7))
+
+        :returns: version
+        :rtype: tuple
+
+        :raises .PluginError:
+            Unable to find Nginx version or version is unsupported
+
+        """
+        text = self._nginx_version()
 
         version_regex = re.compile(r"nginx version: ([^/]+)/([0-9\.]*)", re.IGNORECASE)
         version_matches = version_regex.findall(text)
@@ -964,6 +1008,28 @@ class NginxConfigurator(common.Installer):
 
         return nginx_version
 
+    def _get_openssl_version(self):
+        """Return version of OpenSSL linked to Nginx.
+
+        Version is returned as string. If no version can be found, empty string is returned.
+
+        :returns: openssl_version
+        :rtype: str
+
+        :raises .PluginError:
+            Unable to run Nginx version command
+        """
+        text = self._nginx_version()
+
+        matches = re.findall(r"running with OpenSSL ([^ ]+) ", text)
+        if not matches:
+            matches = re.findall(r"built with OpenSSL ([^ ]+) ", text)
+            if not matches:
+                logger.warning("NGINX configured with OpenSSL alternatives is not officially"
+                    "supported by Certbot.")
+                return ""
+        return matches[0]
+
     def more_info(self):
         """Human-readable string to help understand the module"""
         return (

certbot-nginx/certbot_nginx/constants.py

@@ -22,19 +22,6 @@ MOD_SSL_CONF_DEST = "options-ssl-nginx.conf"
 UPDATED_MOD_SSL_CONF_DIGEST = ".updated-options-ssl-nginx-conf-digest.txt"
 """Name of the hash of the updated or informed mod_ssl_conf as saved in `IConfig.config_dir`."""
 
-SSL_OPTIONS_HASHES_NEW = [
-    '108c4555058a087496a3893aea5d9e1cee0f20a3085d44a52dc1a66522299ac3',
-]
-"""SHA256 hashes of the contents of versions of MOD_SSL_CONF_SRC for nginx >= 1.13.0"""
-
-SSL_OPTIONS_HASHES_MEDIUM = [
-    '63e2bddebb174a05c9d8a7cf2adf72f7af04349ba59a1a925fe447f73b2f1abf',
-    '2901debc7ecbc10917edd9084c05464c9c5930b463677571eaf8c94bffd11ae2',
-    '30baca73ed9a5b0e9a69ea40e30482241d8b1a7343aa79b49dc5d7db0bf53b6c',
-]
-"""SHA256 hashes of the contents of versions of MOD_SSL_CONF_SRC for nginx >= 1.5.9
-   and nginx < 1.13.0"""
-
 ALL_SSL_OPTIONS_HASHES = [
     '0f81093a1465e3d4eaa8b0c14e77b2a2e93568b0fc1351c2b87893a95f0de87c',
     '9a7b32c49001fed4cff8ad24353329472a50e86ade1ef9b2b9e43566a619612e',
@@ -44,7 +31,17 @@ ALL_SSL_OPTIONS_HASHES = [
     '4b16fec2bcbcd8a2f3296d886f17f9953ffdcc0af54582452ca1e52f5f776f16',
     'c052ffff0ad683f43bffe105f7c606b339536163490930e2632a335c8d191cc4',
     '02329eb19930af73c54b3632b3165d84571383b8c8c73361df940cb3894dd426',
-] + SSL_OPTIONS_HASHES_MEDIUM + SSL_OPTIONS_HASHES_NEW
+    '63e2bddebb174a05c9d8a7cf2adf72f7af04349ba59a1a925fe447f73b2f1abf',
+    '2901debc7ecbc10917edd9084c05464c9c5930b463677571eaf8c94bffd11ae2',
+    '30baca73ed9a5b0e9a69ea40e30482241d8b1a7343aa79b49dc5d7db0bf53b6c',
+    '02329eb19930af73c54b3632b3165d84571383b8c8c73361df940cb3894dd426',
+    '108c4555058a087496a3893aea5d9e1cee0f20a3085d44a52dc1a66522299ac3',
+    'd5e021706ecdccc7090111b0ae9a29ef61523e927f020e410caf0a1fd7063981',
+    'ef11e3fb17213e74d3e1816cde0ec37b8b95b4167cf21e7b8ff1eaa9c6f918ee',
+    'af85f6193808a44789a1d293e6cffa249cad9a21135940800958b8e3c72dbc69',
+    'a2a612fd21b02abaa32d9d11ac63d987d6e3054dbfa356de5800eea0d7ce17f3',
+    '2d9648302e3588a172c318e46bff88ade46fc7a16d6afc85322776a04800d473',
+]
 """SHA256 hashes of the contents of all versions of MOD_SSL_CONF_SRC"""
 
 def os_constant(key):

certbot-nginx/certbot_nginx/tests/configurator_test.py

@@ -394,6 +394,68 @@ class NginxConfiguratorTest(util.NginxTest):
         mock_popen.side_effect = OSError("Can't find program")
         self.assertRaises(errors.PluginError, self.config.get_version)
 
+    @mock.patch("certbot_nginx.configurator.subprocess.Popen")
+    def test_get_openssl_version(self, mock_popen):
+        # pylint: disable=protected-access
+        mock_popen().communicate.return_value = (
+            "", """
+                nginx version: nginx/1.15.5
+                built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)
+                built with OpenSSL 1.0.2g  1 Mar 2016
+                TLS SNI support enabled
+                configure arguments:
+            """)
+        self.assertEqual(self.config._get_openssl_version(), "1.0.2g")
+
+        mock_popen().communicate.return_value = (
+            "", """
+                nginx version: nginx/1.15.5
+                built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)
+                built with OpenSSL 1.0.2-beta1  1 Mar 2016
+                TLS SNI support enabled
+                configure arguments:
+            """)
+        self.assertEqual(self.config._get_openssl_version(), "1.0.2-beta1")
+
+        mock_popen().communicate.return_value = (
+            "", """
+                nginx version: nginx/1.15.5
+                built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)
+                built with OpenSSL 1.0.2  1 Mar 2016
+                TLS SNI support enabled
+                configure arguments:
+            """)
+        self.assertEqual(self.config._get_openssl_version(), "1.0.2")
+
+        mock_popen().communicate.return_value = (
+            "", """
+                nginx version: nginx/1.15.5
+                built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)
+                built with OpenSSL 1.0.2g  1 Mar 2016 (running with OpenSSL 1.0.2a  1 Mar 2016)
+                TLS SNI support enabled
+                configure arguments:
+            """)
+        self.assertEqual(self.config._get_openssl_version(), "1.0.2a")
+
+        mock_popen().communicate.return_value = (
+            "", """
+                nginx version: nginx/1.15.5
+                built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)
+                built with LibreSSL 2.2.2
+                TLS SNI support enabled
+                configure arguments:
+            """)
+        self.assertEqual(self.config._get_openssl_version(), "")
+
+        mock_popen().communicate.return_value = (
+            "", """
+                nginx version: nginx/1.15.5
+                built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)
+                TLS SNI support enabled
+                configure arguments:
+            """)
+        self.assertEqual(self.config._get_openssl_version(), "")
+
     @mock.patch("certbot_nginx.configurator.subprocess.Popen")
     def test_nginx_restart(self, mock_popen):
         mocked = mock_popen()
@@ -920,13 +982,12 @@ class InstallSslOptionsConfTest(util.NginxTest):
         self._assert_current_file()
 
     def test_prev_file_updates_to_current_old_nginx(self):
-        from certbot_nginx.constants import ALL_SSL_OPTIONS_HASHES, SSL_OPTIONS_HASHES_NEW
+        from certbot_nginx.constants import ALL_SSL_OPTIONS_HASHES
         self.config.version = (1, 5, 8)
         with mock.patch('certbot.crypto_util.sha256sum',
                 new=self._mock_hash_except_ssl_conf_src(ALL_SSL_OPTIONS_HASHES[0])):
             self._call()
         self._assert_current_file()
-        self.assertTrue(self._current_ssl_options_hash() not in SSL_OPTIONS_HASHES_NEW)
 
     def test_manually_modified_current_file_does_not_update(self):
         with open(self.config.mod_ssl_conf, "a") as mod_ssl_conf:
@@ -987,11 +1048,13 @@ class InstallSslOptionsConfTest(util.NginxTest):
 
     def test_nginx_version_uses_correct_config(self):
         self.config.version = (1, 5, 8)
+        self.config.openssl_version = "1.0.2g" # shouldn't matter
         self.assertEqual(os.path.basename(self.config.mod_ssl_conf_src),
                          "options-ssl-nginx-old.conf")
         self._call()
         self._assert_current_file()
         self.config.version = (1, 5, 9)
+        self.config.openssl_version = "1.0.2l"
         self.assertEqual(os.path.basename(self.config.mod_ssl_conf_src),
                          "options-ssl-nginx-tls12-only.conf")
         self._call()
@@ -999,6 +1062,12 @@ class InstallSslOptionsConfTest(util.NginxTest):
         self.config.version = (1, 13, 0)
         self.assertEqual(os.path.basename(self.config.mod_ssl_conf_src),
                          "options-ssl-nginx.conf")
+        self._call()
+        self._assert_current_file()
+        self.config.version = (1, 13, 0)
+        self.config.openssl_version = "1.0.2k"
+        self.assertEqual(os.path.basename(self.config.mod_ssl_conf_src),
+                         "options-ssl-nginx-tls13-session-tix-on.conf")
 
 
 class DetermineDefaultServerRootTest(certbot_test_util.ConfigTestCase):

certbot-nginx/certbot_nginx/tests/parser_test.py

@@ -30,8 +30,16 @@ class NginxParserTest(util.NginxTest): #pylint: disable=too-many-public-methods
         self.assertEqual(nparser.root, self.config_path)
 
     def test_root_absolute(self):
-        nparser = parser.NginxParser(os.path.relpath(self.config_path))
-        self.assertEqual(nparser.root, self.config_path)
+        curr_dir = os.getcwd()
+        try:
+            # On Windows current directory may be on a different drive than self.tempdir.
+            # However a relative path between two different drives is invalid. So we move to
+            # self.tempdir to ensure that we stay on the same drive.
+            os.chdir(self.temp_dir)
+            nparser = parser.NginxParser(os.path.relpath(self.config_path))
+            self.assertEqual(nparser.root, self.config_path)
+        finally:
+            os.chdir(curr_dir)
 
     def test_root_no_trailing_slash(self):
         nparser = parser.NginxParser(self.config_path + os.path.sep)

certbot-nginx/certbot_nginx/tests/util.py

@@ -54,7 +54,7 @@ def get_data_filename(filename):
 
 
 def get_nginx_configurator(
-        config_path, config_dir, work_dir, logs_dir, version=(1, 6, 2)):
+        config_path, config_dir, work_dir, logs_dir, version=(1, 6, 2), openssl_version="1.0.2g"):
     """Create an Nginx Configurator with the specified options."""
 
     backups = os.path.join(work_dir, "backups")
@@ -79,7 +79,8 @@ def get_nginx_configurator(
                     https_port=5001,
                 ),
                 name="nginx",
-                version=version)
+                version=version,
+                openssl_version=openssl_version)
             config.prepare()
 
     # Provide general config utility.

certbot-nginx/certbot_nginx/tls_configs/options-ssl-nginx-old.conf

@@ -10,4 +10,4 @@ ssl_session_timeout 1440m;
 ssl_protocols TLSv1.2;
 ssl_prefer_server_ciphers off;
 
-ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
+ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA";

certbot-nginx/certbot_nginx/tls_configs/options-ssl-nginx-tls12-only.conf

@@ -11,4 +11,4 @@ ssl_session_tickets off;
 ssl_protocols TLSv1.2;
 ssl_prefer_server_ciphers off;
 
-ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
+ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA";

certbot-nginx/certbot_nginx/tls_configs/options-ssl-nginx-tls13-session-tix-on.conf

@@ -0,0 +1,13 @@
+# This file contains important security parameters. If you modify this file
+# manually, Certbot will be unable to automatically provide future security
+# updates. Instead, Certbot will print and log an error message with a path to
+# the up-to-date file that you will need to refer to when manually updating
+# this file.
+
+ssl_session_cache shared:le_nginx_SSL:10m;
+ssl_session_timeout 1440m;
+
+ssl_protocols TLSv1.2 TLSv1.3;
+ssl_prefer_server_ciphers off;
+
+ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA";

certbot-nginx/certbot_nginx/tls_configs/options-ssl-nginx.conf

@@ -11,4 +11,4 @@ ssl_session_tickets off;
 ssl_protocols TLSv1.2 TLSv1.3;
 ssl_prefer_server_ciphers off;
 
-ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
+ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA";

certbot-nginx/setup.py

@@ -4,7 +4,7 @@ from setuptools.command.test import test as TestCommand
 import sys
 
 
-version = '0.38.0.dev0'
+version = '0.40.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.
@@ -62,6 +62,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

certbot/__init__.py

@@ -1,4 +1,4 @@
 """Certbot client."""
 
 # version number like 1.2.3a0, must have at least 2 parts, like 1.2
-__version__ = '0.38.0.dev0'
+__version__ = '0.40.0.dev0'

certbot/account.py

@@ -231,12 +231,7 @@ class AccountFileStorage(interfaces.AccountStorage):
         except IOError as error:
             raise errors.AccountStorageError(error)
 
-        acc = Account(regr, key, meta)
-        if acc.id != account_id:
-            raise errors.AccountStorageError(
-                "Account ids mismatch (expected: {0}, found: {1}".format(
-                    account_id, acc.id))
-        return acc
+        return Account(regr, key, meta)
 
     def load(self, account_id):
         return self._load_for_server_path(account_id, self.config.server_path)

certbot/cert_manager.py

@@ -262,7 +262,7 @@ def human_readable_cert_info(config, cert, skip_filter_checks=False):
         reasons.append('TEST_CERT')
     if cert.target_expiry <= now:
         reasons.append('EXPIRED')
-    if checker.ocsp_revoked(cert.cert, cert.chain):
+    elif checker.ocsp_revoked(cert):
         reasons.append('REVOKED')
 
     if reasons:

certbot/cli.py

@@ -163,24 +163,6 @@ def report_config_interaction(modified, modifiers):
         VAR_MODIFIERS.setdefault(var, set()).update(modifiers)
 
 
-def possible_deprecation_warning(config):
-    "A deprecation warning for users with the old, not-self-upgrading letsencrypt-auto."
-    if cli_command != LEAUTO:
-        return
-    if config.no_self_upgrade:
-        # users setting --no-self-upgrade might be hanging on a client version like 0.3.0
-        # or 0.5.0 which is the new script, but doesn't set CERTBOT_AUTO; they don't
-        # need warnings
-        return
-    if "CERTBOT_AUTO" not in os.environ:
-        logger.warning("You are running with an old copy of letsencrypt-auto"
-            " that does not receive updates, and is less reliable than more"
-            " recent versions. The letsencrypt client has also been renamed"
-            " to Certbot. We recommend upgrading to the latest certbot-auto"
-            " script, or using native OS packages.")
-        logger.debug("Deprecation warning circumstances: %s / %s", sys.argv[0], os.environ)
-
-
 class _Default(object):
     """A class to use as a default to detect if a value is set by a user"""
 
@@ -642,20 +624,25 @@ class HelpfulArgumentParser(object):
             raise errors.Error(
                 "Parameters --hsts and --auto-hsts cannot be used simultaneously.")
 
-        possible_deprecation_warning(parsed_args)
-
         return parsed_args
 
     def set_test_server(self, parsed_args):
         """We have --staging/--dry-run; perform sanity check and set config.server"""
 
-        if parsed_args.server not in (flag_default("server"), constants.STAGING_URI):
-            conflicts = ["--staging"] if parsed_args.staging else []
-            conflicts += ["--dry-run"] if parsed_args.dry_run else []
-            raise errors.Error("--server value conflicts with {0}".format(
-                " and ".join(conflicts)))
+        # Flag combinations should produce these results:
+        #                             | --staging      | --dry-run   |
+        # ------------------------------------------------------------
+        # | --server acme-v02         | Use staging    | Use staging |
+        # | --server acme-staging-v02 | Use staging    | Use staging |
+        # | --server <other>          | Conflict error | Use <other> |
 
-        parsed_args.server = constants.STAGING_URI
+        default_servers = (flag_default("server"), constants.STAGING_URI)
+
+        if parsed_args.staging and parsed_args.server not in default_servers:
+            raise errors.Error("--server value conflicts with --staging")
+
+        if parsed_args.server in default_servers:
+            parsed_args.server = constants.STAGING_URI
 
         if parsed_args.dry_run:
             if self.verb not in ["certonly", "renew"]:

certbot/compat/filesystem.py

@@ -20,7 +20,7 @@ except ImportError:
 else:
     POSIX_MODE = False
 
-from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import List, Union, Tuple  # pylint: disable=unused-import, no-name-in-module
 
 
 def chmod(file_path, mode):
@@ -264,6 +264,7 @@ def replace(src, dst):
 
 
 def realpath(file_path):
+    # type: (str) -> str
     """
     Find the real path for the given path. This method resolves symlinks, including
     recursive symlinks, and is protected against symlinks that creates an infinite loop.
@@ -300,10 +301,11 @@ def realpath(file_path):
 # requires to be run under a privileged shell, so the user will always benefit
 # from the highest (privileged one) set of permissions on a given file.
 def is_executable(path):
+    # type: (str) -> bool
     """
     Is path an executable file?
     :param str path: path to test
-    :returns: True if path is an executable file
+    :return: True if path is an executable file
     :rtype: bool
     """
     if POSIX_MODE:
@@ -312,6 +314,118 @@ def is_executable(path):
     return _win_is_executable(path)
 
 
+def has_world_permissions(path):
+    # type: (str) -> bool
+    """
+    Check if everybody/world has any right (read/write/execute) on a file given its path
+    :param str path: path to test
+    :return: True if everybody/world has any right to the file
+    :rtype: bool
+    """
+    if POSIX_MODE:
+        return bool(stat.S_IMODE(os.stat(path).st_mode) & stat.S_IRWXO)
+
+    security = win32security.GetFileSecurity(path, win32security.DACL_SECURITY_INFORMATION)
+    dacl = security.GetSecurityDescriptorDacl()
+
+    return bool(dacl.GetEffectiveRightsFromAcl({
+        'TrusteeForm': win32security.TRUSTEE_IS_SID,
+        'TrusteeType': win32security.TRUSTEE_IS_USER,
+        'Identifier': win32security.ConvertStringSidToSid('S-1-1-0'),
+    }))
+
+
+def compute_private_key_mode(old_key, base_mode):
+    # type: (str, int) -> int
+    """
+    Calculate the POSIX mode to apply to a private key given the previous private key
+    :param str old_key: path to the previous private key
+    :param int base_mode: the minimum modes to apply to a private key
+    :return: the POSIX mode to apply
+    :rtype: int
+    """
+    if POSIX_MODE:
+        # On Linux, we keep read/write/execute permissions
+        # for group and read permissions for everybody.
+        old_mode = (stat.S_IMODE(os.stat(old_key).st_mode) &
+                    (stat.S_IRGRP | stat.S_IWGRP | stat.S_IXGRP | stat.S_IROTH))
+        return base_mode | old_mode
+
+    # On Windows, the mode returned by os.stat is not reliable,
+    # so we do not keep any permission from the previous private key.
+    return base_mode
+
+
+def has_same_ownership(path1, path2):
+    # type: (str, str) -> bool
+    """
+    Return True if the ownership of two files given their respective path is the same.
+    On Windows, ownership is checked against owner only, since files do not have a group owner.
+    :param str path1: path to the first file
+    :param str path2: path to the second file
+    :return: True if both files have the same ownership, False otherwise
+    :rtype: bool
+
+    """
+    if POSIX_MODE:
+        stats1 = os.stat(path1)
+        stats2 = os.stat(path2)
+        return (stats1.st_uid, stats1.st_gid) == (stats2.st_uid, stats2.st_gid)
+
+    security1 = win32security.GetFileSecurity(path1, win32security.OWNER_SECURITY_INFORMATION)
+    user1 = security1.GetSecurityDescriptorOwner()
+
+    security2 = win32security.GetFileSecurity(path2, win32security.OWNER_SECURITY_INFORMATION)
+    user2 = security2.GetSecurityDescriptorOwner()
+
+    return user1 == user2
+
+
+def has_min_permissions(path, min_mode):
+    # type: (str, int) -> bool
+    """
+    Check if a file given its path has at least the permissions defined by the given minimal mode.
+    On Windows, group permissions are ignored since files do not have a group owner.
+    :param str path: path to the file to check
+    :param int min_mode: the minimal permissions expected
+    :return: True if the file matches the minimal permissions expectations, False otherwise
+    :rtype: bool
+    """
+    if POSIX_MODE:
+        st_mode = os.stat(path).st_mode
+        return st_mode == st_mode | min_mode
+
+    # Resolve symlinks, to get a consistent result with os.stat on Linux,
+    # that follows symlinks by default.
+    path = realpath(path)
+
+    # Get owner sid of the file
+    security = win32security.GetFileSecurity(
+        path, win32security.OWNER_SECURITY_INFORMATION | win32security.DACL_SECURITY_INFORMATION)
+    user = security.GetSecurityDescriptorOwner()
+    dacl = security.GetSecurityDescriptorDacl()
+    min_dacl = _generate_dacl(user, min_mode)
+
+    for index in range(min_dacl.GetAceCount()):
+        min_ace = min_dacl.GetAce(index)
+
+        # On a given ACE, index 0 is the ACE type, 1 is the permission mask, and 2 is the SID.
+        # See: http://timgolden.me.uk/pywin32-docs/PyACL__GetAce_meth.html
+        mask = min_ace[1]
+        user = min_ace[2]
+
+        effective_mask = dacl.GetEffectiveRightsFromAcl({
+            'TrusteeForm': win32security.TRUSTEE_IS_SID,
+            'TrusteeType': win32security.TRUSTEE_IS_USER,
+            'Identifier': user,
+        })
+
+        if effective_mask != effective_mask | mask:
+            return False
+
+    return True
+
+
 def _win_is_executable(path):
     if not os.path.isfile(path):
         return False
@@ -432,13 +546,7 @@ def _generate_windows_flags(rights_desc):
     if rights_desc['write']:
         flag = flag | (ntsecuritycon.FILE_ALL_ACCESS
                        ^ ntsecuritycon.FILE_GENERIC_READ
-                       ^ ntsecuritycon.FILE_GENERIC_EXECUTE
-                       # Despite bit `512` being present in ntsecuritycon.FILE_ALL_ACCESS, it is
-                       # not effectively applied to the file or the directory.
-                       # As _generate_windows_flags is also used to compare two dacls, we remove
-                       # it right now to have flags that contain only the bits effectively applied
-                       # by Windows.
-                       ^ 512)
+                       ^ ntsecuritycon.FILE_GENERIC_EXECUTE)
     if rights_desc['execute']:
         flag = flag | ntsecuritycon.FILE_GENERIC_EXECUTE
 
@@ -472,8 +580,8 @@ def _compare_dacls(dacl1, dacl2):
     This method compare the two given DACLs to check if they are identical.
     Identical means here that they contains the same set of ACEs in the same order.
     """
-    return ([dacl1.GetAce(index) for index in range(0, dacl1.GetAceCount())] ==
-            [dacl2.GetAce(index) for index in range(0, dacl2.GetAceCount())])
+    return ([dacl1.GetAce(index) for index in range(dacl1.GetAceCount())] ==
+            [dacl2.GetAce(index) for index in range(dacl2.GetAceCount())])
 
 
 def _get_current_user():

certbot/compat/misc.py

@@ -5,7 +5,6 @@ particular category.
 from __future__ import absolute_import
 
 import select
-import stat
 import sys
 
 try:
@@ -18,18 +17,6 @@ from certbot import errors
 from certbot.compat import os
 
 
-# MASK_FOR_PRIVATE_KEY_PERMISSIONS defines what are the permissions flags to keep
-# when transferring the permissions from an old private key to a new one.
-if POSIX_MODE:
-    # On Linux, we keep read/write/execute permissions
-    # for group and read permissions for everybody.
-    MASK_FOR_PRIVATE_KEY_PERMISSIONS = stat.S_IRGRP | stat.S_IWGRP | stat.S_IXGRP | stat.S_IROTH
-else:
-    # On Windows, the mode returned by os.stat is not reliable,
-    # so we do not keep any permission from the previous private key.
-    MASK_FOR_PRIVATE_KEY_PERMISSIONS = 0
-
-
 # For Linux: define OS specific standard binary directories
 STANDARD_BINARY_DIRS = ["/usr/sbin", "/usr/local/bin", "/usr/local/sbin"] if POSIX_MODE else []
 

certbot/compat/os.py

@@ -116,3 +116,21 @@ def access(*unused_args, **unused_kwargs):
     raise RuntimeError('Usage of os.access() is forbidden. '
                        'Use certbot.compat.filesystem.check_mode() or '
                        'certbot.compat.filesystem.is_executable() instead.')
+
+
+# On Windows os.stat call result is inconsistent, with a lot of flags that are not set or
+# meaningless. We need to use specialized functions from the certbot.compat.filesystem module.
+def stat(*unused_args, **unused_kwargs):
+    """Method os.stat() is forbidden"""
+    raise RuntimeError('Usage of os.stat() is forbidden. '
+                       'Use certbot.compat.filesystem functions instead '
+                       '(eg. has_min_permissions, has_same_ownership).')
+
+
+# Method os.fstat has the same problem than os.stat, since it is the same function,
+# but accepting a file descriptor instead of a path.
+def fstat(*unused_args, **unused_kwargs):
+    """Method os.stat() is forbidden"""
+    raise RuntimeError('Usage of os.fstat() is forbidden. '
+                       'Use certbot.compat.filesystem functions instead '
+                       '(eg. has_min_permissions, has_same_ownership).')

certbot/lock.py

@@ -167,14 +167,18 @@ class _UnixLockMechanism(_BaseLockMechanism):
         :returns: True if the lock was successfully acquired
         :rtype: bool
         """
+        # Normally os module should not be imported in certbot codebase except in certbot.compat
+        # for the sake of compatibility over Windows and Linux.
+        # We make an exception here, since _lock_success is private and called only on Linux.
+        from os import stat, fstat  # pylint: disable=os-module-forbidden
         try:
-            stat1 = os.stat(self._path)
+            stat1 = stat(self._path)
         except OSError as err:
             if err.errno == errno.ENOENT:
                 return False
             raise
 
-        stat2 = os.fstat(fd)
+        stat2 = fstat(fd)
         # If our locked file descriptor and the file on disk refer to
         # the same device and inode, they're the same file.
         return stat1.st_dev == stat2.st_dev and stat1.st_ino == stat2.st_ino

certbot/ocsp.py

@@ -16,11 +16,13 @@ from cryptography.hazmat.backends import default_backend
 from cryptography.hazmat.primitives import serialization
 from cryptography.hazmat.primitives import hashes  # type: ignore
 from cryptography.exceptions import UnsupportedAlgorithm, InvalidSignature
+import pytz
 import requests
 
 from acme.magic_typing import Optional, Tuple  # pylint: disable=unused-import, no-name-in-module
 from certbot import crypto_util
 from certbot import errors
+from certbot.storage import RenewableCert # pylint: disable=unused-import
 from certbot import util
 
 logger = logging.getLogger(__name__)
@@ -48,21 +50,29 @@ class RevocationChecker(object):
             else:
                 self.host_args = lambda host: ["Host", host]
 
-    def ocsp_revoked(self, cert_path, chain_path):
-        # type: (str, str) -> bool
+    def ocsp_revoked(self, cert):
+        # type: (RenewableCert) -> bool
         """Get revoked status for a particular cert version.
 
         .. todo:: Make this a non-blocking call
 
-        :param str cert_path: Path to certificate
-        :param str chain_path: Path to intermediate cert
-        :returns: True if revoked; False if valid or the check failed
+        :param `.storage.RenewableCert` cert: Certificate object
+        :returns: True if revoked; False if valid or the check failed or cert is expired.
         :rtype: bool
 
         """
+        cert_path, chain_path = cert.cert, cert.chain
+
         if self.broken:
             return False
 
+        # Let's Encrypt doesn't update OCSP for expired certificates,
+        # so don't check OCSP if the cert is expired.
+        # https://github.com/certbot/certbot/issues/7152
+        now = pytz.UTC.fromutc(datetime.utcnow())
+        if cert.target_expiry <= now:
+            return False
+
         url, host = _determine_ocsp_server(cert_path)
         if not host or not url:
             return False

certbot/plugins/dns_common.py

@@ -2,7 +2,6 @@
 
 import abc
 import logging
-import stat
 from time import sleep
 
 import configobj
@@ -12,6 +11,7 @@ from acme import challenges
 
 from certbot import errors
 from certbot import interfaces
+from certbot.compat import filesystem
 from certbot.compat import os
 from certbot.display import ops
 from certbot.display import util as display_util
@@ -312,8 +312,7 @@ def validate_file_permissions(filename):
 
     validate_file(filename)
 
-    permissions = stat.S_IMODE(os.stat(filename).st_mode)
-    if permissions & stat.S_IRWXO:
+    if filesystem.has_world_permissions(filename):
         logger.warning('Unsafe permissions on credentials configuration file: %s', filename)
 
 

certbot/plugins/dns_common_test.py

@@ -7,14 +7,15 @@ import unittest
 import mock
 
 from certbot import errors
+from certbot import util
 from certbot.compat import os
 from certbot.display import util as display_util
 from certbot.plugins import dns_common
 from certbot.plugins import dns_test_common
-from certbot.tests import util
+from certbot.tests import util as test_util
 
 
-class DNSAuthenticatorTest(util.TempDirTestCase, dns_test_common.BaseAuthenticatorTest):
+class DNSAuthenticatorTest(test_util.TempDirTestCase, dns_test_common.BaseAuthenticatorTest):
     # pylint: disable=protected-access
 
     class _FakeDNSAuthenticator(dns_common.DNSAuthenticator):
@@ -50,7 +51,7 @@ class DNSAuthenticatorTest(util.TempDirTestCase, dns_test_common.BaseAuthenticat
 
         self.auth._cleanup.assert_called_once_with(dns_test_common.DOMAIN, mock.ANY, mock.ANY)
 
-    @util.patch_get_utility()
+    @test_util.patch_get_utility()
     def test_prompt(self, mock_get_utility):
         mock_display = mock_get_utility()
         mock_display.input.side_effect = ((display_util.OK, "",),
@@ -59,14 +60,14 @@ class DNSAuthenticatorTest(util.TempDirTestCase, dns_test_common.BaseAuthenticat
         self.auth._configure("other_key", "")
         self.assertEqual(self.auth.config.fake_other_key, "value")
 
-    @util.patch_get_utility()
+    @test_util.patch_get_utility()
     def test_prompt_canceled(self, mock_get_utility):
         mock_display = mock_get_utility()
         mock_display.input.side_effect = ((display_util.CANCEL, "c",),)
 
         self.assertRaises(errors.PluginError, self.auth._configure, "other_key", "")
 
-    @util.patch_get_utility()
+    @test_util.patch_get_utility()
     def test_prompt_file(self, mock_get_utility):
         path = os.path.join(self.tempdir, 'file.ini')
         open(path, "wb").close()
@@ -80,7 +81,7 @@ class DNSAuthenticatorTest(util.TempDirTestCase, dns_test_common.BaseAuthenticat
         self.auth._configure_file("file_path", "")
         self.assertEqual(self.auth.config.fake_file_path, path)
 
-    @util.patch_get_utility()
+    @test_util.patch_get_utility()
     def test_prompt_file_canceled(self, mock_get_utility):
         mock_display = mock_get_utility()
         mock_display.directory_select.side_effect = ((display_util.CANCEL, "c",),)
@@ -96,7 +97,7 @@ class DNSAuthenticatorTest(util.TempDirTestCase, dns_test_common.BaseAuthenticat
 
         self.assertEqual(credentials.conf("test"), "value")
 
-    @util.patch_get_utility()
+    @test_util.patch_get_utility()
     def test_prompt_credentials(self, mock_get_utility):
         bad_path = os.path.join(self.tempdir, 'bad-file.ini')
         dns_test_common.write({"fake_other": "other_value"}, bad_path)
@@ -116,7 +117,7 @@ class DNSAuthenticatorTest(util.TempDirTestCase, dns_test_common.BaseAuthenticat
         self.assertEqual(credentials.conf("test"), "value")
 
 
-class CredentialsConfigurationTest(util.TempDirTestCase):
+class CredentialsConfigurationTest(test_util.TempDirTestCase):
     class _MockLoggingHandler(logging.Handler):
         messages = None
 
@@ -150,14 +151,14 @@ class CredentialsConfigurationTest(util.TempDirTestCase):
         dns_common.logger.addHandler(log)
 
         path = os.path.join(self.tempdir, 'too-permissive-file.ini')
-        open(path, "wb").close()
+        util.safe_open(path, "wb", 0o744).close()
 
         dns_common.CredentialsConfiguration(path)
 
         self.assertEqual(1, len([_ for _ in log.messages['warning'] if _.startswith("Unsafe")]))
 
 
-class CredentialsConfigurationRequireTest(util.TempDirTestCase):
+class CredentialsConfigurationRequireTest(test_util.TempDirTestCase):
 
     def setUp(self):
         super(CredentialsConfigurationRequireTest, self).setUp()

certbot/plugins/webroot_test.py

@@ -34,7 +34,13 @@ class AuthenticatorTest(unittest.TestCase):
 
     def setUp(self):
         from certbot.plugins.webroot import Authenticator
-        self.path = tempfile.mkdtemp()
+        # On Linux directories created by tempfile.mkdtemp inherit their permissions from their
+        # parent directory. So the actual permissions are inconsistent over various tests env.
+        # To circumvent this, a dedicated sub-workspace is created under the workspace, using
+        # filesystem.mkdir to get consistent permissions.
+        self.workspace = tempfile.mkdtemp()
+        self.path = os.path.join(self.workspace, 'webroot')
+        filesystem.mkdir(self.path)
         self.partial_root_challenge_path = os.path.join(
             self.path, ".well-known")
         self.root_challenge_path = os.path.join(
@@ -170,17 +176,12 @@ class AuthenticatorTest(unittest.TestCase):
         self.assertTrue(filesystem.check_mode(self.validation_path, 0o644))
 
         # Check permissions of the directories
-
         for dirpath, dirnames, _ in os.walk(self.path):
             for directory in dirnames:
                 full_path = os.path.join(dirpath, directory)
                 self.assertTrue(filesystem.check_mode(full_path, 0o755))
 
-        parent_gid = os.stat(self.path).st_gid
-        parent_uid = os.stat(self.path).st_uid
-
-        self.assertEqual(os.stat(self.validation_path).st_gid, parent_gid)
-        self.assertEqual(os.stat(self.validation_path).st_uid, parent_uid)
+        self.assertTrue(filesystem.has_same_ownership(self.validation_path, self.path))
 
     def test_perform_cleanup(self):
         self.auth.prepare()

certbot/storage.py

@@ -18,7 +18,6 @@ from certbot import crypto_util
 from certbot import error_handler
 from certbot import errors
 from certbot import util
-from certbot.compat import misc
 from certbot.compat import os
 from certbot.compat import filesystem
 from certbot.plugins import common as plugins_common
@@ -1107,9 +1106,7 @@ class RenewableCert(object):
                 f.write(new_privkey)
             # Preserve gid and (mode & MASK_FOR_PRIVATE_KEY_PERMISSIONS)
             # from previous privkey in this lineage.
-            old_mode = (stat.S_IMODE(os.stat(old_privkey).st_mode) &
-                        misc.MASK_FOR_PRIVATE_KEY_PERMISSIONS)
-            mode = BASE_PRIVKEY_MODE | old_mode
+            mode = filesystem.compute_private_key_mode(old_privkey, BASE_PRIVKEY_MODE)
             filesystem.copy_ownership_and_apply_mode(
                 old_privkey, target["privkey"], mode, copy_user=False, copy_group=True)
 

certbot/tests/account_test.py

@@ -1,7 +1,6 @@
 """Tests for certbot.account."""
 import datetime
 import json
-import shutil
 import unittest
 
 import josepy as jose
@@ -170,13 +169,6 @@ class AccountFileStorageTest(test_util.ConfigTestCase):
     def test_load_non_existent_raises_error(self):
         self.assertRaises(errors.AccountNotFound, self.storage.load, "missing")
 
-    def test_load_id_mismatch_raises_error(self):
-        self.storage.save(self.acc, self.mock_client)
-        shutil.move(os.path.join(self.config.accounts_dir, self.acc.id),
-                    os.path.join(self.config.accounts_dir, "x" + self.acc.id))
-        self.assertRaises(errors.AccountStorageError, self.storage.load,
-                          "x" + self.acc.id)
-
     def _set_server(self, server):
         self.config.server = server
         from certbot.account import AccountFileStorage

certbot/tests/cli_test.py

@@ -23,21 +23,27 @@ PLUGINS = disco.PluginsRegistry.find_all()
 
 
 class TestReadFile(TempDirTestCase):
-    '''Test cli.read_file'''
-
-
+    """Test cli.read_file"""
     def test_read_file(self):
-        rel_test_path = os.path.relpath(os.path.join(self.tempdir, 'foo'))
-        self.assertRaises(
-            argparse.ArgumentTypeError, cli.read_file, rel_test_path)
+        curr_dir = os.getcwd()
+        try:
+            # On Windows current directory may be on a different drive than self.tempdir.
+            # However a relative path between two different drives is invalid. So we move to
+            # self.tempdir to ensure that we stay on the same drive.
+            os.chdir(self.tempdir)
+            rel_test_path = os.path.relpath(os.path.join(self.tempdir, 'foo'))
+            self.assertRaises(
+                argparse.ArgumentTypeError, cli.read_file, rel_test_path)
 
-        test_contents = b'bar\n'
-        with open(rel_test_path, 'wb') as f:
-            f.write(test_contents)
+            test_contents = b'bar\n'
+            with open(rel_test_path, 'wb') as f:
+                f.write(test_contents)
 
-        path, contents = cli.read_file(rel_test_path)
-        self.assertEqual(path, os.path.abspath(path))
-        self.assertEqual(contents, test_contents)
+            path, contents = cli.read_file(rel_test_path)
+            self.assertEqual(path, os.path.abspath(path))
+            self.assertEqual(contents, test_contents)
+        finally:
+            os.chdir(curr_dir)
 
 
 class FlagDefaultTest(unittest.TestCase):
@@ -327,16 +333,26 @@ class ParseTest(unittest.TestCase):  # pylint: disable=too-many-public-methods
 
         self._assert_dry_run_flag_worked(self.parse(short_args + ['auth']), True)
         self._assert_dry_run_flag_worked(self.parse(short_args + ['renew']), True)
+        self._assert_dry_run_flag_worked(self.parse(short_args + ['certonly']), True)
+
         short_args += ['certonly']
-        self._assert_dry_run_flag_worked(self.parse(short_args), True)
 
-        short_args += '--server example.com'.split()
-        conflicts = ['--dry-run']
-        self._check_server_conflict_message(short_args, '--dry-run')
+        # `--dry-run --server example.com` should emit example.com
+        self.assertEqual(self.parse(short_args + ['--server', 'example.com']).server,
+                         'example.com')
 
-        short_args += ['--staging']
-        conflicts += ['--staging']
-        self._check_server_conflict_message(short_args, conflicts)
+        # `--dry-run --server STAGING_URI` should emit STAGING_URI
+        self.assertEqual(self.parse(short_args + ['--server', constants.STAGING_URI]).server,
+                         constants.STAGING_URI)
+
+        # `--dry-run --server LIVE` should emit STAGING_URI
+        self.assertEqual(self.parse(short_args + ['--server', cli.flag_default("server")]).server,
+                         constants.STAGING_URI)
+
+        # `--dry-run --server example.com --staging` should emit an error
+        conflicts = ['--staging']
+        self._check_server_conflict_message(short_args + ['--server', 'example.com', '--staging'],
+                                            conflicts)
 
     def test_option_was_set(self):
         key_size_option = 'rsa_key_size'

certbot/tests/compat/filesystem_test.py

@@ -89,8 +89,8 @@ class WindowsChmodTests(TempDirTestCase):
         self.assertEqual(len(system_aces), 1)
         self.assertEqual(len(admin_aces), 1)
 
-        self.assertEqual(system_aces[0][1], ntsecuritycon.FILE_ALL_ACCESS ^ 512)
-        self.assertEqual(admin_aces[0][1], ntsecuritycon.FILE_ALL_ACCESS ^ 512)
+        self.assertEqual(system_aces[0][1], ntsecuritycon.FILE_ALL_ACCESS)
+        self.assertEqual(admin_aces[0][1], ntsecuritycon.FILE_ALL_ACCESS)
 
     def test_read_flag(self):
         self._test_flag(4, ntsecuritycon.FILE_GENERIC_READ)
@@ -101,12 +101,10 @@ class WindowsChmodTests(TempDirTestCase):
     def test_write_flag(self):
         self._test_flag(2, (ntsecuritycon.FILE_ALL_ACCESS
                             ^ ntsecuritycon.FILE_GENERIC_READ
-                            ^ ntsecuritycon.FILE_GENERIC_EXECUTE
-                            ^ 512))
+                            ^ ntsecuritycon.FILE_GENERIC_EXECUTE))
 
     def test_full_flag(self):
-        self._test_flag(7, (ntsecuritycon.FILE_ALL_ACCESS
-                            ^ 512))
+        self._test_flag(7, ntsecuritycon.FILE_ALL_ACCESS)
 
     def _test_flag(self, everyone_mode, windows_flag):
         # Note that flag is tested against `everyone`, not `user`, because practically these unit
@@ -150,6 +148,24 @@ class WindowsChmodTests(TempDirTestCase):
         self.assertEqual(security_dacl.GetSecurityDescriptorDacl().GetAceCount(), 2)
 
 
+class ComputePrivateKeyModeTest(TempDirTestCase):
+    def setUp(self):
+        super(ComputePrivateKeyModeTest, self).setUp()
+        self.probe_path = _create_probe(self.tempdir)
+
+    def test_compute_private_key_mode(self):
+        filesystem.chmod(self.probe_path, 0o777)
+        new_mode = filesystem.compute_private_key_mode(self.probe_path, 0o600)
+
+        if POSIX_MODE:
+            # On Linux RWX permissions for group and R permission for world
+            # are persisted from the existing moe
+            self.assertEqual(new_mode, 0o674)
+        else:
+            # On Windows no permission is persisted
+            self.assertEqual(new_mode, 0o600)
+
+
 @unittest.skipIf(POSIX_MODE, reason='Tests specific to Windows security')
 class WindowsOpenTest(TempDirTestCase):
     def test_new_file_correct_permissions(self):
@@ -262,14 +278,14 @@ class WindowsMkdirTests(test_util.TempDirTestCase):
         self.assertEqual(original_mkdir, std_os.mkdir)
 
 
-class CopyOwnershipTest(test_util.TempDirTestCase):
-    """Tests about replacement of chown: copy_ownership_and_apply_mode"""
+class OwnershipTest(test_util.TempDirTestCase):
+    """Tests about copy_ownership_and_apply_mode and has_same_ownership"""
     def setUp(self):
-        super(CopyOwnershipTest, self).setUp()
+        super(OwnershipTest, self).setUp()
         self.probe_path = _create_probe(self.tempdir)
 
     @unittest.skipIf(POSIX_MODE, reason='Test specific to Windows security')
-    def test_windows(self):
+    def test_copy_ownership_windows(self):
         system = win32security.ConvertStringSidToSid(SYSTEM_SID)
         security = win32security.SECURITY_ATTRIBUTES().SECURITY_DESCRIPTOR
         security.SetSecurityDescriptorOwner(system, False)
@@ -295,7 +311,7 @@ class CopyOwnershipTest(test_util.TempDirTestCase):
                           if dacl.GetAce(index)[2] == everybody])
 
     @unittest.skipUnless(POSIX_MODE, reason='Test specific to Linux security')
-    def test_linux(self):
+    def test_copy_ownership_linux(self):
         with mock.patch('os.chown') as mock_chown:
             with mock.patch('os.chmod') as mock_chmod:
                 with mock.patch('os.stat') as mock_stat:
@@ -307,8 +323,18 @@ class CopyOwnershipTest(test_util.TempDirTestCase):
         mock_chown.assert_called_once_with(self.probe_path, 50, 51)
         mock_chmod.assert_called_once_with(self.probe_path, 0o700)
 
+    def test_has_same_ownership(self):
+        path1 = os.path.join(self.tempdir, 'test1')
+        path2 = os.path.join(self.tempdir, 'test2')
+
+        util.safe_open(path1, 'w').close()
+        util.safe_open(path2, 'w').close()
+
+        self.assertTrue(filesystem.has_same_ownership(path1, path2))
+
 
 class CheckPermissionsTest(test_util.TempDirTestCase):
+    """Tests relative to functions that check modes."""
     def setUp(self):
         super(CheckPermissionsTest, self).setUp()
         self.probe_path = _create_probe(self.tempdir)
@@ -353,6 +379,23 @@ class CheckPermissionsTest(test_util.TempDirTestCase):
             mock_owner.return_value = False
             self.assertFalse(filesystem.check_permissions(self.probe_path, 0o744))
 
+    def test_check_min_permissions(self):
+        filesystem.chmod(self.probe_path, 0o744)
+        self.assertTrue(filesystem.has_min_permissions(self.probe_path, 0o744))
+
+        filesystem.chmod(self.probe_path, 0o700)
+        self.assertFalse(filesystem.has_min_permissions(self.probe_path, 0o744))
+
+        filesystem.chmod(self.probe_path, 0o741)
+        self.assertFalse(filesystem.has_min_permissions(self.probe_path, 0o744))
+
+    def test_is_world_reachable(self):
+        filesystem.chmod(self.probe_path, 0o744)
+        self.assertTrue(filesystem.has_world_permissions(self.probe_path))
+
+        filesystem.chmod(self.probe_path, 0o700)
+        self.assertFalse(filesystem.has_world_permissions(self.probe_path))
+
 
 class OsReplaceTest(test_util.TempDirTestCase):
     """Test to ensure consistent behavior of rename method"""

certbot/tests/compat/os_test.py

@@ -8,8 +8,8 @@ class OsTest(unittest.TestCase):
     """Unit tests for os module."""
     def test_forbidden_methods(self):
         # Checks for os module
-        for method in ['chmod', 'chown', 'open', 'mkdir',
-                       'makedirs', 'rename', 'replace', 'access']:
+        for method in ['chmod', 'chown', 'open', 'mkdir', 'makedirs', 'rename',
+                       'replace', 'access', 'stat', 'fstat']:
             self.assertRaises(RuntimeError, getattr(os, method))
         # Checks for os.path module
         for method in ['realpath']:

certbot/tests/lock_test.py

@@ -82,7 +82,10 @@ class LockFileTest(test_util.TempDirTestCase):
         'Race conditions on lock are specific to the non-blocking file access approach on Linux.')
     def test_race(self):
         should_delete = [True, False]
-        stat = os.stat
+        # Normally os module should not be imported in certbot codebase except in certbot.compat
+        # for the sake of compatibility over Windows and Linux.
+        # We make an exception here, since test_race is a test function called only on Linux.
+        from os import stat  # pylint: disable=os-module-forbidden
 
         def delete_and_stat(path):
             """Wrap os.stat and maybe delete the file first."""
@@ -90,7 +93,7 @@ class LockFileTest(test_util.TempDirTestCase):
                 os.remove(path)
             return stat(path)
 
-        with mock.patch('certbot.lock.os.stat') as mock_stat:
+        with mock.patch('certbot.lock.filesystem.os.stat') as mock_stat:
             mock_stat.side_effect = delete_and_stat
             self._call(self.lock_path)
         self.assertFalse(should_delete)
@@ -117,7 +120,7 @@ class LockFileTest(test_util.TempDirTestCase):
 
     def test_unexpected_os_err(self):
         if POSIX_MODE:
-            mock_function = 'certbot.lock.os.stat'
+            mock_function = 'certbot.lock.filesystem.os.stat'
         else:
             mock_function = 'certbot.lock.msvcrt.locking'
         # The only expected errno are ENOENT and EACCES in lock module.

certbot/tests/ocsp_test.py

@@ -16,6 +16,7 @@ try:
 except (ImportError, AttributeError):  # pragma: no cover
     ocsp_lib = None  # type: ignore
 import mock
+import pytz
 
 from certbot import errors
 from certbot.tests import util as test_util
@@ -72,21 +73,34 @@ class OCSPTestOpenSSL(unittest.TestCase):
     @mock.patch('certbot.ocsp._determine_ocsp_server')
     @mock.patch('certbot.util.run_script')
     def test_ocsp_revoked(self, mock_run, mock_determine):
+        now = pytz.UTC.fromutc(datetime.utcnow())
+        cert_obj = mock.MagicMock()
+        cert_obj.cert = "x"
+        cert_obj.chain = "y"
+        cert_obj.target_expiry = now + timedelta(hours=2)
+
         self.checker.broken = True
         mock_determine.return_value = ("", "")
-        self.assertEqual(self.checker.ocsp_revoked("x", "y"), False)
+        self.assertEqual(self.checker.ocsp_revoked(cert_obj), False)
 
         self.checker.broken = False
         mock_run.return_value = tuple(openssl_happy[1:])
-        self.assertEqual(self.checker.ocsp_revoked("x", "y"), False)
+        self.assertEqual(self.checker.ocsp_revoked(cert_obj), False)
         self.assertEqual(mock_run.call_count, 0)
 
         mock_determine.return_value = ("http://x.co", "x.co")
-        self.assertEqual(self.checker.ocsp_revoked("blah.pem", "chain.pem"), False)
+        self.assertEqual(self.checker.ocsp_revoked(cert_obj), False)
         mock_run.side_effect = errors.SubprocessError("Unable to load certificate launcher")
-        self.assertEqual(self.checker.ocsp_revoked("x", "y"), False)
+        self.assertEqual(self.checker.ocsp_revoked(cert_obj), False)
         self.assertEqual(mock_run.call_count, 2)
 
+        # cert expired
+        cert_obj.target_expiry = now
+        mock_determine.return_value = ("", "")
+        count_before = mock_determine.call_count
+        self.assertEqual(self.checker.ocsp_revoked(cert_obj), False)
+        self.assertEqual(mock_determine.call_count, count_before)
+
     def test_determine_ocsp_server(self):
         cert_path = test_util.vector_path('ocsp_certificate.pem')
 
@@ -131,18 +145,23 @@ class OSCPTestCryptography(unittest.TestCase):
         self.checker = ocsp.RevocationChecker()
         self.cert_path = test_util.vector_path('ocsp_certificate.pem')
         self.chain_path = test_util.vector_path('ocsp_issuer_certificate.pem')
+        self.cert_obj = mock.MagicMock()
+        self.cert_obj.cert = self.cert_path
+        self.cert_obj.chain = self.chain_path
+        now = pytz.UTC.fromutc(datetime.utcnow())
+        self.cert_obj.target_expiry = now + timedelta(hours=2)
 
     @mock.patch('certbot.ocsp._determine_ocsp_server')
     @mock.patch('certbot.ocsp._check_ocsp_cryptography')
     def test_ensure_cryptography_toggled(self, mock_revoke, mock_determine):
         mock_determine.return_value = ('http://example.com', 'example.com')
-        self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+        self.checker.ocsp_revoked(self.cert_obj)
 
         mock_revoke.assert_called_once_with(self.cert_path, self.chain_path, 'http://example.com')
 
     def test_revoke(self):
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.REVOKED, ocsp_lib.OCSPResponseStatus.SUCCESSFUL):
-            revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertTrue(revoked)
 
     def test_responder_is_issuer(self):
@@ -152,7 +171,7 @@ class OSCPTestCryptography(unittest.TestCase):
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.REVOKED,
                         ocsp_lib.OCSPResponseStatus.SUCCESSFUL) as mocks:
             mocks['mock_response'].return_value.responder_name = issuer.subject
-            self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            self.checker.ocsp_revoked(self.cert_obj)
         # Here responder and issuer are the same. So only the signature of the OCSP
         # response is checked (using the issuer/responder public key).
         self.assertEqual(mocks['mock_check'].call_count, 1)
@@ -167,7 +186,7 @@ class OSCPTestCryptography(unittest.TestCase):
 
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.REVOKED,
                         ocsp_lib.OCSPResponseStatus.SUCCESSFUL) as mocks:
-            self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            self.checker.ocsp_revoked(self.cert_obj)
         # Here responder and issuer are not the same. Two signatures will be checked then,
         # first to verify the responder cert (using the issuer public key), second to
         # to verify the OCSP response itself (using the responder public key).
@@ -181,17 +200,17 @@ class OSCPTestCryptography(unittest.TestCase):
         # Server return an invalid HTTP response
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.UNKNOWN, ocsp_lib.OCSPResponseStatus.SUCCESSFUL,
                         http_status_code=400):
-            revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertFalse(revoked)
 
         # OCSP response in invalid
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.UNKNOWN, ocsp_lib.OCSPResponseStatus.UNAUTHORIZED):
-            revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertFalse(revoked)
 
         # OCSP response is valid, but certificate status is unknown
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.UNKNOWN, ocsp_lib.OCSPResponseStatus.SUCCESSFUL):
-            revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertFalse(revoked)
 
         # The OCSP response says that the certificate is revoked, but certificate
@@ -200,32 +219,32 @@ class OSCPTestCryptography(unittest.TestCase):
             with mock.patch('cryptography.x509.Extensions.get_extension_for_class',
                             side_effect=x509.ExtensionNotFound(
                                 'Not found', x509.AuthorityInformationAccessOID.OCSP)):
-                revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+                revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertFalse(revoked)
 
         # OCSP response uses an unsupported signature.
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.REVOKED, ocsp_lib.OCSPResponseStatus.SUCCESSFUL,
                         check_signature_side_effect=UnsupportedAlgorithm('foo')):
-            revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertFalse(revoked)
 
         # OSCP signature response is invalid.
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.REVOKED, ocsp_lib.OCSPResponseStatus.SUCCESSFUL,
                         check_signature_side_effect=InvalidSignature('foo')):
-            revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertFalse(revoked)
 
         # Assertion error on OCSP response validity
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.REVOKED, ocsp_lib.OCSPResponseStatus.SUCCESSFUL,
                         check_signature_side_effect=AssertionError('foo')):
-            revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertFalse(revoked)
 
         # No responder cert in OCSP response
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.REVOKED,
                         ocsp_lib.OCSPResponseStatus.SUCCESSFUL) as mocks:
             mocks['mock_response'].return_value.certificates = []
-            revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertFalse(revoked)
 
         # Responder cert is not signed by certificate issuer
@@ -234,7 +253,7 @@ class OSCPTestCryptography(unittest.TestCase):
             cert = mocks['mock_response'].return_value.certificates[0]
             mocks['mock_response'].return_value.certificates[0] = mock.Mock(
                 issuer='fake', subject=cert.subject)
-            revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+            revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertFalse(revoked)
 
         with _ocsp_mock(ocsp_lib.OCSPCertStatus.REVOKED, ocsp_lib.OCSPResponseStatus.SUCCESSFUL):
@@ -245,7 +264,7 @@ class OSCPTestCryptography(unittest.TestCase):
                 with mock.patch('cryptography.x509.Extensions.get_extension_for_class',
                                 side_effect=x509.ExtensionNotFound(
                                     'Not found', x509.AuthorityInformationAccessOID.OCSP)):
-                    revoked = self.checker.ocsp_revoked(self.cert_path, self.chain_path)
+                    revoked = self.checker.ocsp_revoked(self.cert_obj)
         self.assertFalse(revoked)
 
 

certbot/tests/util_test.py

@@ -520,13 +520,16 @@ class OsInfoTest(unittest.TestCase):
 
             with mock.patch('platform.system_alias',
                             return_value=('linux', '', '')):
-                with mock.patch('distro.linux_distribution',
-                                return_value=('', '', '')):
-                    self.assertEqual(get_python_os_info(), ("linux", ""))
+                with mock.patch('platform.linux_distribution',
+                                side_effect=AttributeError,
+                                create=True):
+                    with mock.patch('distro.linux_distribution',
+                                    return_value=('', '', '')):
+                        self.assertEqual(get_python_os_info(), ("linux", ""))
 
-                with mock.patch('distro.linux_distribution',
-                                return_value=('testdist', '42', '')):
-                    self.assertEqual(get_python_os_info(), ("testdist", "42"))
+                    with mock.patch('distro.linux_distribution',
+                                    return_value=('testdist', '42', '')):
+                        self.assertEqual(get_python_os_info(), ("testdist", "42"))
 
             with mock.patch('platform.system_alias',
                             return_value=('freebsd', '9.3-RC3-p1', '')):

certbot/util.py

@@ -392,7 +392,7 @@ def get_python_os_info():
     os_type, os_ver, _ = info
     os_type = os_type.lower()
     if os_type.startswith('linux'):
-        info = distro.linux_distribution()
+        info = _get_linux_distribution()
         # On arch, distro.linux_distribution() is reportedly ('','',''),
         # so handle it defensively
         if info[0]:
@@ -424,6 +424,14 @@ def get_python_os_info():
         os_ver = ''
     return os_type, os_ver
 
+def _get_linux_distribution():
+    """Gets the linux distribution name from the underlying OS"""
+
+    try:
+        return platform.linux_distribution()
+    except AttributeError:
+        return distro.linux_distribution()
+
 # Just make sure we don't get pwned... Make sure that it also doesn't
 # start with a period or have two consecutive periods <- this needs to
 # be done in addition to the regex

docs/cli-help.txt

@@ -113,7 +113,7 @@ optional arguments:
                         case, and to know when to deprecate support for past
                         Python versions and flags. If you wish to hide this
                         information from the Let's Encrypt server, set this to
-                        "". (default: CertbotACMEClient/0.37.1
+                        "". (default: CertbotACMEClient/0.39.0
                         (certbot(-auto); OS_NAME OS_VERSION) Authenticator/XXX
                         Installer/YYY (SUBCOMMAND; flags: FLAGS)
                         Py/major.minor.patchlevel). The flags encoded in the

docs/install.rst

@@ -200,23 +200,39 @@ Operating System Packages
 
 **Debian**
 
-If you run Debian Stretch or Debian Sid, you can install certbot packages.
+If you run Debian Buster or Debian testing/Sid, you can easily install certbot
+packages through commands like:
 
 .. code-block:: shell
 
    sudo apt-get update
-   sudo apt-get install certbot python-certbot-apache
+   sudo apt-get install certbot
 
-If you don't want to use the Apache plugin, you can omit the
-``python-certbot-apache`` package. Or you can install ``python-certbot-nginx`` instead.
-
-Packages exist for Debian Jessie via backports. First you'll have to follow the
-instructions at http://backports.debian.org/Instructions/ to enable the Jessie backports
-repo, if you have not already done so. Then run:
+If you run Debian Stretch, we recommend you use the packages in Debian
+backports repository. First you'll have to follow the instructions at
+https://backports.debian.org/Instructions/ to enable the Stretch backports repo,
+if you have not already done so. Then run:
 
 .. code-block:: shell
 
-   sudo apt-get install certbot python-certbot-apache -t jessie-backports
+   sudo apt-get install certbot -t stretch-backports
+
+In all of these cases, there also packages available to help Certbot integrate
+with Apache, nginx, or various DNS services. If you are using Apache or nginx,
+we strongly recommend that you install the ``python-certbot-apache`` or
+``python-certbot-nginx`` package so that Certbot can fully automate HTTPS
+configuration for your server. A full list of these packages can be found
+through a command like:
+
+.. code-block:: shell
+
+    apt search 'python-certbot*'
+
+They can be installed by running the same installation command above but
+replacing ``certbot`` with the name of the desired package.
+
+There are no Certbot packages available for Debian Jessie and Jessie users
+should instead use certbot-auto_.
 
 **Ubuntu**
 

docs/using.rst

@@ -276,10 +276,8 @@ s3front_           Y    Y    Integration with Amazon CloudFront distribution of
 gandi_             Y    N    Obtain certificates via the Gandi LiveDNS API
 varnish_           Y    N    Obtain certificates via a Varnish server
 external_          Y    N    A plugin for convenient scripting (See also ticket 2782_)
-icecast_           N    Y    Deploy certificates to Icecast 2 streaming media servers
 pritunl_           N    Y    Install certificates in pritunl distributed OpenVPN servers
 proxmox_           N    Y    Install certificates in Proxmox Virtualization servers
-heroku_            Y    Y    Integration with Heroku SSL
 dns-standalone_    Y    N    Obtain certificates via an integrated DNS server
 dns-ispconfig_     Y    N    DNS Authentication using ISPConfig as DNS server
 ================== ==== ==== ===============================================================
@@ -287,13 +285,11 @@ dns-ispconfig_     Y    N    DNS Authentication using ISPConfig as DNS server
 .. _haproxy: https://github.com/greenhost/certbot-haproxy
 .. _s3front: https://github.com/dlapiduz/letsencrypt-s3front
 .. _gandi: https://github.com/obynio/certbot-plugin-gandi
-.. _icecast: https://github.com/e00E/lets-encrypt-icecast
 .. _varnish: http://git.sesse.net/?p=letsencrypt-varnish-plugin
 .. _2782: https://github.com/certbot/certbot/issues/2782
 .. _pritunl: https://github.com/kharkevich/letsencrypt-pritunl
 .. _proxmox: https://github.com/kharkevich/letsencrypt-proxmox
 .. _external: https://github.com/marcan/letsencrypt-external
-.. _heroku: https://github.com/gboudreau/certbot-heroku
 .. _dns-standalone: https://github.com/siilike/certbot-dns-standalone
 .. _dns-ispconfig: https://github.com/m42e/certbot-dns-ispconfig
 
@@ -921,8 +917,9 @@ Certbot accepts a global configuration file that applies its options to all invo
 of Certbot. Certificate specific configuration choices should be set in the ``.conf``
 files that can be found in ``/etc/letsencrypt/renewal``.
 
-By default no cli.ini file is created, after creating one
-it is possible to specify the location of this configuration file with
+By default no cli.ini file is created (though it may exist already if you installed Certbot
+via a package manager, for instance).
+After creating one it is possible to specify the location of this configuration file with
 ``certbot --config cli.ini`` (or shorter ``-c cli.ini``). An
 example configuration file is shown below:
 

letsencrypt-auto

@@ -31,7 +31,7 @@ if [ -z "$VENV_PATH" ]; then
 fi
 VENV_BIN="$VENV_PATH/bin"
 BOOTSTRAP_VERSION_PATH="$VENV_PATH/certbot-auto-bootstrap-version.txt"
-LE_AUTO_VERSION="0.37.1"
+LE_AUTO_VERSION="0.39.0"
 BASENAME=$(basename $0)
 USAGE="Usage: $BASENAME [OPTIONS]
 A self-updating wrapper script for the Certbot ACME client. When run, updates
@@ -775,6 +775,8 @@ elif [ -f /etc/redhat-release ]; then
     RPM_USE_PYTHON_3=1
   elif [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
     RPM_USE_PYTHON_3=1
+  elif [ "$RPM_DIST_NAME" = "centos" -a "$RPM_DIST_VERSION" -ge 8 ]; then
+    RPM_USE_PYTHON_3=1
   else
     RPM_USE_PYTHON_3=0
   fi
@@ -1134,73 +1136,76 @@ if [ "$1" = "--le-auto-phase2" ]; then
 # To generate this, do (with docker and package hashin installed):
 # ```
 # letsencrypt-auto-source/rebuild_dependencies.py \
-#   letsencrypt-auto-sources/pieces/dependency-requirements.txt
+#   letsencrypt-auto-source/pieces/dependency-requirements.txt
+# ```
+# If you want to update a single dependency, run commands similar to these:
+# ```
+# pip install hashin
+# hashin -r dependency-requirements.txt cryptography==1.5.2
 # ```
 ConfigArgParse==0.14.0 \
     --hash=sha256:2e2efe2be3f90577aca9415e32cb629aa2ecd92078adbe27b53a03e53ff12e91
 asn1crypto==0.24.0 \
     --hash=sha256:2f1adbb7546ed199e3c90ef23ec95c5cf3585bac7d11fb7eb562a3fe89c64e87 \
     --hash=sha256:9d5c20441baf0cb60a4ac34cc447c6c189024b6b4c6cd7877034f4965c464e49
-certifi==2019.3.9 \
-    --hash=sha256:59b7658e26ca9c7339e00f8f4636cdfe59d34fa37b9b04f6f9e9926b3cece1a5 \
-    --hash=sha256:b26104d6835d1f5e49452a26eb2ff87fe7090b89dfcaee5ea2212697e1e1d7ae
-cffi==1.12.2 \
-    --hash=sha256:00b97afa72c233495560a0793cdc86c2571721b4271c0667addc83c417f3d90f \
-    --hash=sha256:0ba1b0c90f2124459f6966a10c03794082a2f3985cd699d7d63c4a8dae113e11 \
-    --hash=sha256:0bffb69da295a4fc3349f2ec7cbe16b8ba057b0a593a92cbe8396e535244ee9d \
-    --hash=sha256:21469a2b1082088d11ccd79dd84157ba42d940064abbfa59cf5f024c19cf4891 \
-    --hash=sha256:2e4812f7fa984bf1ab253a40f1f4391b604f7fc424a3e21f7de542a7f8f7aedf \
-    --hash=sha256:2eac2cdd07b9049dd4e68449b90d3ef1adc7c759463af5beb53a84f1db62e36c \
-    --hash=sha256:2f9089979d7456c74d21303c7851f158833d48fb265876923edcb2d0194104ed \
-    --hash=sha256:3dd13feff00bddb0bd2d650cdb7338f815c1789a91a6f68fdc00e5c5ed40329b \
-    --hash=sha256:4065c32b52f4b142f417af6f33a5024edc1336aa845b9d5a8d86071f6fcaac5a \
-    --hash=sha256:51a4ba1256e9003a3acf508e3b4f4661bebd015b8180cc31849da222426ef585 \
-    --hash=sha256:59888faac06403767c0cf8cfb3f4a777b2939b1fbd9f729299b5384f097f05ea \
-    --hash=sha256:59c87886640574d8b14910840327f5cd15954e26ed0bbd4e7cef95fa5aef218f \
-    --hash=sha256:610fc7d6db6c56a244c2701575f6851461753c60f73f2de89c79bbf1cc807f33 \
-    --hash=sha256:70aeadeecb281ea901bf4230c6222af0248c41044d6f57401a614ea59d96d145 \
-    --hash=sha256:71e1296d5e66c59cd2c0f2d72dc476d42afe02aeddc833d8e05630a0551dad7a \
-    --hash=sha256:8fc7a49b440ea752cfdf1d51a586fd08d395ff7a5d555dc69e84b1939f7ddee3 \
-    --hash=sha256:9b5c2afd2d6e3771d516045a6cfa11a8da9a60e3d128746a7fe9ab36dfe7221f \
-    --hash=sha256:9c759051ebcb244d9d55ee791259ddd158188d15adee3c152502d3b69005e6bd \
-    --hash=sha256:b4d1011fec5ec12aa7cc10c05a2f2f12dfa0adfe958e56ae38dc140614035804 \
-    --hash=sha256:b4f1d6332339ecc61275bebd1f7b674098a66fea11a00c84d1c58851e618dc0d \
-    --hash=sha256:c030cda3dc8e62b814831faa4eb93dd9a46498af8cd1d5c178c2de856972fd92 \
-    --hash=sha256:c2e1f2012e56d61390c0e668c20c4fb0ae667c44d6f6a2eeea5d7148dcd3df9f \
-    --hash=sha256:c37c77d6562074452120fc6c02ad86ec928f5710fbc435a181d69334b4de1d84 \
-    --hash=sha256:c8149780c60f8fd02752d0429246088c6c04e234b895c4a42e1ea9b4de8d27fb \
-    --hash=sha256:cbeeef1dc3c4299bd746b774f019de9e4672f7cc666c777cd5b409f0b746dac7 \
-    --hash=sha256:e113878a446c6228669144ae8a56e268c91b7f1fafae927adc4879d9849e0ea7 \
-    --hash=sha256:e21162bf941b85c0cda08224dade5def9360f53b09f9f259adb85fc7dd0e7b35 \
-    --hash=sha256:fb6934ef4744becbda3143d30c6604718871495a5e36c408431bf33d9c146889
+certifi==2019.6.16 \
+    --hash=sha256:046832c04d4e752f37383b628bc601a7ea7211496b4638f6514d0e5b9acc4939 \
+    --hash=sha256:945e3ba63a0b9f577b1395204e13c3a231f9bc0223888be653286534e5873695
+cffi==1.12.3 \
+    --hash=sha256:041c81822e9f84b1d9c401182e174996f0bae9991f33725d059b771744290774 \
+    --hash=sha256:046ef9a22f5d3eed06334d01b1e836977eeef500d9b78e9ef693f9380ad0b83d \
+    --hash=sha256:066bc4c7895c91812eff46f4b1c285220947d4aa46fa0a2651ff85f2afae9c90 \
+    --hash=sha256:066c7ff148ae33040c01058662d6752fd73fbc8e64787229ea8498c7d7f4041b \
+    --hash=sha256:2444d0c61f03dcd26dbf7600cf64354376ee579acad77aef459e34efcb438c63 \
+    --hash=sha256:300832850b8f7967e278870c5d51e3819b9aad8f0a2c8dbe39ab11f119237f45 \
+    --hash=sha256:34c77afe85b6b9e967bd8154e3855e847b70ca42043db6ad17f26899a3df1b25 \
+    --hash=sha256:46de5fa00f7ac09f020729148ff632819649b3e05a007d286242c4882f7b1dc3 \
+    --hash=sha256:4aa8ee7ba27c472d429b980c51e714a24f47ca296d53f4d7868075b175866f4b \
+    --hash=sha256:4d0004eb4351e35ed950c14c11e734182591465a33e960a4ab5e8d4f04d72647 \
+    --hash=sha256:4e3d3f31a1e202b0f5a35ba3bc4eb41e2fc2b11c1eff38b362de710bcffb5016 \
+    --hash=sha256:50bec6d35e6b1aaeb17f7c4e2b9374ebf95a8975d57863546fa83e8d31bdb8c4 \
+    --hash=sha256:55cad9a6df1e2a1d62063f79d0881a414a906a6962bc160ac968cc03ed3efcfb \
+    --hash=sha256:5662ad4e4e84f1eaa8efce5da695c5d2e229c563f9d5ce5b0113f71321bcf753 \
+    --hash=sha256:59b4dc008f98fc6ee2bb4fd7fc786a8d70000d058c2bbe2698275bc53a8d3fa7 \
+    --hash=sha256:73e1ffefe05e4ccd7bcea61af76f36077b914f92b76f95ccf00b0c1b9186f3f9 \
+    --hash=sha256:a1f0fd46eba2d71ce1589f7e50a9e2ffaeb739fb2c11e8192aa2b45d5f6cc41f \
+    --hash=sha256:a2e85dc204556657661051ff4bab75a84e968669765c8a2cd425918699c3d0e8 \
+    --hash=sha256:a5457d47dfff24882a21492e5815f891c0ca35fefae8aa742c6c263dac16ef1f \
+    --hash=sha256:a8dccd61d52a8dae4a825cdbb7735da530179fea472903eb871a5513b5abbfdc \
+    --hash=sha256:ae61af521ed676cf16ae94f30fe202781a38d7178b6b4ab622e4eec8cefaff42 \
+    --hash=sha256:b012a5edb48288f77a63dba0840c92d0504aa215612da4541b7b42d849bc83a3 \
+    --hash=sha256:d2c5cfa536227f57f97c92ac30c8109688ace8fa4ac086d19d0af47d134e2909 \
+    --hash=sha256:d42b5796e20aacc9d15e66befb7a345454eef794fdb0737d1af593447c6c8f45 \
+    --hash=sha256:dee54f5d30d775f525894d67b1495625dd9322945e7fee00731952e0368ff42d \
+    --hash=sha256:e070535507bd6aa07124258171be2ee8dfc19119c28ca94c9dfb7efd23564512 \
+    --hash=sha256:e1ff2748c84d97b065cc95429814cdba39bcbd77c9c85c89344b317dc0d9cbff \
+    --hash=sha256:ed851c75d1e0e043cbf5ca9a8e1b13c4c90f3fbd863dacb01c0808e2b5204201
 chardet==3.0.4 \
     --hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae \
     --hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691
 configobj==5.0.6 \
     --hash=sha256:a2f5650770e1c87fb335af19a9b7eb73fc05ccf22144eb68db7d00cd2bcb0902
-cryptography==2.6.1 \
-    --hash=sha256:066f815f1fe46020877c5983a7e747ae140f517f1b09030ec098503575265ce1 \
-    --hash=sha256:210210d9df0afba9e000636e97810117dc55b7157c903a55716bb73e3ae07705 \
-    --hash=sha256:26c821cbeb683facb966045e2064303029d572a87ee69ca5a1bf54bf55f93ca6 \
-    --hash=sha256:2afb83308dc5c5255149ff7d3fb9964f7c9ee3d59b603ec18ccf5b0a8852e2b1 \
-    --hash=sha256:2db34e5c45988f36f7a08a7ab2b69638994a8923853dec2d4af121f689c66dc8 \
-    --hash=sha256:409c4653e0f719fa78febcb71ac417076ae5e20160aec7270c91d009837b9151 \
-    --hash=sha256:45a4f4cf4f4e6a55c8128f8b76b4c057027b27d4c67e3fe157fa02f27e37830d \
-    --hash=sha256:48eab46ef38faf1031e58dfcc9c3e71756a1108f4c9c966150b605d4a1a7f659 \
-    --hash=sha256:6b9e0ae298ab20d371fc26e2129fd683cfc0cfde4d157c6341722de645146537 \
-    --hash=sha256:6c4778afe50f413707f604828c1ad1ff81fadf6c110cb669579dea7e2e98a75e \
-    --hash=sha256:8c33fb99025d353c9520141f8bc989c2134a1f76bac6369cea060812f5b5c2bb \
-    --hash=sha256:9873a1760a274b620a135054b756f9f218fa61ca030e42df31b409f0fb738b6c \
-    --hash=sha256:9b069768c627f3f5623b1cbd3248c5e7e92aec62f4c98827059eed7053138cc9 \
-    --hash=sha256:9e4ce27a507e4886efbd3c32d120db5089b906979a4debf1d5939ec01b9dd6c5 \
-    --hash=sha256:acb424eaca214cb08735f1a744eceb97d014de6530c1ea23beb86d9c6f13c2ad \
-    --hash=sha256:c8181c7d77388fe26ab8418bb088b1a1ef5fde058c6926790c8a0a3d94075a4a \
-    --hash=sha256:d4afbb0840f489b60f5a580a41a1b9c3622e08ecb5eec8614d4fb4cd914c4460 \
-    --hash=sha256:d9ed28030797c00f4bc43c86bf819266c76a5ea61d006cd4078a93ebf7da6bfd \
-    --hash=sha256:e603aa7bb52e4e8ed4119a58a03b60323918467ef209e6ff9db3ac382e5cf2c6
-# Package enum34 needs to be explicitly limited to Python2.x, in order to avoid
-# certbot-auto failures on Python 3.6+ which enum34 doesn't support. See #5456.
-enum34==1.1.6 ; python_version < '3.4' \
+cryptography==2.7 \
+    --hash=sha256:24b61e5fcb506424d3ec4e18bca995833839bf13c59fc43e530e488f28d46b8c \
+    --hash=sha256:25dd1581a183e9e7a806fe0543f485103232f940fcfc301db65e630512cce643 \
+    --hash=sha256:3452bba7c21c69f2df772762be0066c7ed5dc65df494a1d53a58b683a83e1216 \
+    --hash=sha256:41a0be220dd1ed9e998f5891948306eb8c812b512dc398e5a01846d855050799 \
+    --hash=sha256:5751d8a11b956fbfa314f6553d186b94aa70fdb03d8a4d4f1c82dcacf0cbe28a \
+    --hash=sha256:5f61c7d749048fa6e3322258b4263463bfccefecb0dd731b6561cb617a1d9bb9 \
+    --hash=sha256:72e24c521fa2106f19623a3851e9f89ddfdeb9ac63871c7643790f872a305dfc \
+    --hash=sha256:7b97ae6ef5cba2e3bb14256625423413d5ce8d1abb91d4f29b6d1a081da765f8 \
+    --hash=sha256:961e886d8a3590fd2c723cf07be14e2a91cf53c25f02435c04d39e90780e3b53 \
+    --hash=sha256:96d8473848e984184b6728e2c9d391482008646276c3ff084a1bd89e15ff53a1 \
+    --hash=sha256:ae536da50c7ad1e002c3eee101871d93abdc90d9c5f651818450a0d3af718609 \
+    --hash=sha256:b0db0cecf396033abb4a93c95d1602f268b3a68bb0a9cc06a7cff587bb9a7292 \
+    --hash=sha256:cfee9164954c186b191b91d4193989ca994703b2fff406f71cf454a2d3c7327e \
+    --hash=sha256:e6347742ac8f35ded4a46ff835c60e68c22a536a8ae5c4422966d06946b6d4c6 \
+    --hash=sha256:f27d93f0139a3c056172ebb5d4f9056e770fdf0206c2f422ff2ebbad142e09ed \
+    --hash=sha256:f57b76e46a58b63d1c6375017f4564a28f19a5ca912691fd2e4261b3414b618d
+distro==1.4.0 \
+    --hash=sha256:362dde65d846d23baee4b5c058c8586f219b5a54be1cf5fc6ff55c4578392f57 \
+    --hash=sha256:eedf82a470ebe7d010f1872c17237c79ab04097948800029994fa458e52fb4b4
+enum34==1.1.6 \
     --hash=sha256:2d81cbbe0e73112bdfe6ef8576f2238f2ba27dd0d55752a776c41d38b7da2850 \
     --hash=sha256:644837f692e5f550741432dd3f223bbb9852018674981b1664e5dc339387588a \
     --hash=sha256:6bd0f6ad48ec2aa117d3d141940d484deccda84d4fcd884f5c3d93c23ecd8c79 \
@@ -1216,18 +1221,18 @@ idna==2.8 \
 ipaddress==1.0.22 \
     --hash=sha256:64b28eec5e78e7510698f6d4da08800a5c575caa4a286c93d651c5d3ff7b6794 \
     --hash=sha256:b146c751ea45cad6188dd6cf2d9b757f6f4f8d6ffb96a023e6f2e26eea02a72c
-josepy==1.1.0 \
-    --hash=sha256:1309a25aac3caeff5239729c58ff9b583f7d022ffdb1553406ddfc8e5b52b76e \
-    --hash=sha256:fb5c62c77d26e04df29cb5ecd01b9ce69b6fcc9e521eb1ca193b7faa2afa7086
+josepy==1.2.0 \
+    --hash=sha256:8ea15573203f28653c00f4ac0142520777b1c59d9eddd8da3f256c6ba3cac916 \
+    --hash=sha256:9cec9a839fe9520f0420e4f38e7219525daccce4813296627436fe444cd002d3
 mock==1.3.0 \
     --hash=sha256:1e247dbecc6ce057299eb7ee019ad68314bb93152e81d9a6110d35f4d5eca0f6 \
     --hash=sha256:3f573a18be94de886d1191f27c168427ef693e8dcfcecf95b170577b2eb69cbb
 parsedatetime==2.4 \
     --hash=sha256:3d817c58fb9570d1eec1dd46fa9448cd644eeed4fb612684b02dfda3a79cb84b \
     --hash=sha256:9ee3529454bf35c40a77115f5a596771e59e1aee8c53306f346c461b8e913094
-pbr==5.1.3 \
-    --hash=sha256:8257baf496c8522437e8a6cfe0f15e00aedc6c0e0e7c9d55eeeeab31e0853843 \
-    --hash=sha256:8c361cc353d988e4f5b998555c88098b9d5964c2e11acf7b0d21925a66bb5824
+pbr==5.4.2 \
+    --hash=sha256:56e52299170b9492513c64be44736d27a512fa7e606f21942160b68ce510b4bc \
+    --hash=sha256:9b321c204a88d8ab5082699469f52cc94c5da45c51f114113d01b3d993c24cdf
 pyOpenSSL==19.0.0 \
     --hash=sha256:aeca66338f6de19d1aa46ed634c3b9ae519a64b458f8468aec688e7e3c20f200 \
     --hash=sha256:c727930ad54b10fc157015014b666f2d8b41f70c0d03e83ab67624fd3dd5d1e6
@@ -1236,14 +1241,14 @@ pyRFC3339==1.1 \
     --hash=sha256:81b8cbe1519cdb79bed04910dd6fa4e181faf8c88dff1e1b987b5f7ab23a5b1a
 pycparser==2.19 \
     --hash=sha256:a988718abfad80b6b157acce7bf130a30876d27603738ac39f140993246b25b3
-pyparsing==2.3.1 \
-    --hash=sha256:66c9268862641abcac4a96ba74506e594c884e3f57690a696d21ad8210ed667a \
-    --hash=sha256:f6c5ef0d7480ad048c054c37632c67fca55299990fff127850181659eea33fc3
+pyparsing==2.4.2 \
+    --hash=sha256:6f98a7b9397e206d78cc01df10131398f1c8b8510a2f4d97d9abd82e1aacdd80 \
+    --hash=sha256:d9338df12903bbf5d65a0e4e87c2161968b10d2e489652bb47001d82a9b028b4
 python-augeas==0.5.0 \
     --hash=sha256:67d59d66cdba8d624e0389b87b2a83a176f21f16a87553b50f5703b23f29bac2
-pytz==2018.9 \
-    --hash=sha256:32b0891edff07e28efe91284ed9c31e123d84bea3fd98e1f72be2508f43ef8d9 \
-    --hash=sha256:d5f05e487007e29e03409f9398d074e158d920d36eb82eaf66fb1136b0c5374c
+pytz==2019.2 \
+    --hash=sha256:26c0b32e437e54a18161324a2fca3c4b9846b74a8dccddd843113109e1116b32 \
+    --hash=sha256:c894d57500a4cd2d5c71114aaab77dbab5eabd9022308ce5ac9bb93a60a6f0c7
 requests==2.21.0 \
     --hash=sha256:502a824f31acdacb3a35b6690b5fbf0bc41d63a24a45c4004352b0242707598e \
     --hash=sha256:7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b
@@ -1253,15 +1258,15 @@ requests-toolbelt==0.9.1 \
 six==1.12.0 \
     --hash=sha256:3350809f0555b11f552448330d0b52d5f24c91a322ea4a15ef22629740f3761c \
     --hash=sha256:d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73
-urllib3==1.24.2 \
-    --hash=sha256:4c291ca23bbb55c76518905869ef34bdd5f0e46af7afe6861e8375643ffee1a0 \
-    --hash=sha256:9a247273df709c4fedb38c711e44292304f73f39ab01beda9f6b9fc375669ac3
+urllib3==1.24.3 \
+    --hash=sha256:2393a695cd12afedd0dcb26fe5d50d0cf248e5a66f75dbd89a3d4eb333a61af4 \
+    --hash=sha256:a637e5fae88995b256e3409dc4d52c2e2e0ba32c42a6365fee8bbd2238de3cfb
 zope.component==4.5 \
     --hash=sha256:6edfd626c3b593b72895a8cfcf79bff41f4619194ce996a85bce31ac02b94e55 \
     --hash=sha256:984a06ba3def0b02b1117fa4c45b56e772e8c29c0340820fbf367e440a93a3a4
-zope.deferredimport==4.3 \
-    --hash=sha256:2ddef5a7ecfff132a2dd796253366ecf9748a446e30f1a0b3a636aec9d9c05c5 \
-    --hash=sha256:4aae9cbacb2146cca58e62be0a914f0cec034d3b2d41135ea212ca8a96f4b5ec
+zope.deferredimport==4.3.1 \
+    --hash=sha256:57b2345e7b5eef47efcd4f634ff16c93e4265de3dcf325afc7315ade48d909e1 \
+    --hash=sha256:9a0c211df44aa95f1c4e6d2626f90b400f56989180d3ef96032d708da3d23e0a
 zope.deprecation==4.4.0 \
     --hash=sha256:0d453338f04bacf91bbfba545d8bcdf529aa829e67b705eac8c1a7fdce66e2df \
     --hash=sha256:f1480b74995958b24ce37b0ef04d3663d2683e5d6debc96726eff18acf4ea113
@@ -1309,18 +1314,18 @@ zope.interface==4.6.0 \
     --hash=sha256:d788a3999014ddf416f2dc454efa4a5dbeda657c6aba031cf363741273804c6b \
     --hash=sha256:eed88ae03e1ef3a75a0e96a55a99d7937ed03e53d0cffc2451c208db445a2966 \
     --hash=sha256:f99451f3a579e73b5dd58b1b08d1179791d49084371d9a47baad3b22417f0317
-zope.proxy==4.3.1 \
-    --hash=sha256:0cbcfcafaa3b5fde7ba7a7b9a2b5f09af25c9b90087ad65f9e61359fed0ca63b \
-    --hash=sha256:3de631dd5054a3a20b9ebff0e375f39c0565f1fb9131200d589a6a8f379214cd \
-    --hash=sha256:5429134d04d42262f4dac25f6dea907f6334e9a751ffc62cb1d40226fb52bdeb \
-    --hash=sha256:563c2454b2d0f23bca54d2e0e4d781149b7b06cb5df67e253ca3620f37202dd2 \
-    --hash=sha256:5bcf773345016b1461bb07f70c635b9386e5eaaa08e37d3939dcdf12d3fdbec5 \
-    --hash=sha256:8d84b7aef38c693874e2f2084514522bf73fd720fde0ce2a9352a51315ffa475 \
-    --hash=sha256:90de9473c05819b36816b6cb957097f809691836ed3142648bf62da84b4502fe \
-    --hash=sha256:dd592a69fe872445542a6e1acbefb8e28cbe6b4007b8f5146da917e49b155cc3 \
-    --hash=sha256:e7399ab865399fce322f9cefc6f2f3e4099d087ba581888a9fea1bbe1db42a08 \
-    --hash=sha256:e7d1c280d86d72735a420610df592aac72332194e531a8beff43a592c3a1b8eb \
-    --hash=sha256:e90243fee902adb0c39eceb3c69995c0f2004bc3fdb482fbf629efc656d124ed
+zope.proxy==4.3.2 \
+    --hash=sha256:320a7619992e42142549ebf61e14ce27683b4d14b0cbc45f7c037ba64edb560c \
+    --hash=sha256:824d4dbabbb7deb84f25fdb96ea1eeca436a1802c3c8d323b3eb4ac9d527d41c \
+    --hash=sha256:8a32eb9c94908f3544da2dae3f4a9e6961d78819b88ac6b6f4a51cee2d65f4a0 \
+    --hash=sha256:96265fd3bc3ea646f98482e16307a69de21402eeaaaaf4b841c1161ac2f71bb0 \
+    --hash=sha256:ab6d6975d9c51c13cac828ff03168de21fb562b0664c59bcdc4a4b10f39a5b17 \
+    --hash=sha256:af10cb772391772463f65a58348e2de5ecc06693c16d2078be276dc068bcbb54 \
+    --hash=sha256:b8fd3a3de3f7b6452775e92af22af5977b17b69ac86a38a3ddfe870e40a0d05f \
+    --hash=sha256:bb7088f1bed3b8214284a5e425dc23da56f2f28e8815b7580bfed9e245b6c0b6 \
+    --hash=sha256:bc29b3665eac34f14c4aef5224bef045efcfb1a7d12d78c8685858de5fbf21c0 \
+    --hash=sha256:c39fa6a159affeae5fe31b49d9f5b12bd674fe77271a9a324408b271440c50a7 \
+    --hash=sha256:e946a036ac5b9f897e986ac9dc950a34cffc857d88eae6727b8434fbc4752366
 
 # Contains the requirements for the letsencrypt package.
 #
@@ -1333,18 +1338,18 @@ letsencrypt==0.7.0 \
     --hash=sha256:105a5fb107e45bcd0722eb89696986dcf5f08a86a321d6aef25a0c7c63375ade \
     --hash=sha256:c36e532c486a7e92155ee09da54b436a3c420813ec1c590b98f635d924720de9
 
-certbot==0.37.1 \
-    --hash=sha256:84dbdad204327b8d8ef9ab5b040f2be1e427a9f7e087affcc9a6051ea1b03fe7 \
-    --hash=sha256:aace73e63b0c11cdb4b0bd33e1780c1fbe0ce5669dc72e80c3aa9500145daf16
-acme==0.37.1 \
-    --hash=sha256:83a4f6f3c5eb6a85233d5ba87714b426f2d096df58d711f8a2fc4071eb3fd3fc \
-    --hash=sha256:c069a761990751f7c4bf51d2e87ae10319bf460de6629d2908c9fa6f69e97111
-certbot-apache==0.37.1 \
-    --hash=sha256:3ea832408877b12b3a60d17e8b2ee3387364f8c3023ac267161c25b99087cd42 \
-    --hash=sha256:e46c2644451101c0e216aa1f525a577cc903efaf871e0e4da277224a4439040c
-certbot-nginx==0.37.1 \
-    --hash=sha256:1f9af389d26f06634e2eefaace3354e7679dabb4295e1d55d05a4ee7e23a64bd \
-    --hash=sha256:02a7ec15bd388d0f0e94a34c86a8f8d618ec7d5ffde0c206039bb4c46b294ce4
+certbot==0.39.0 \
+    --hash=sha256:f1a70651a6c5137a448f4a8db17b09af619f80a077326caae6b74278bf1db488 \
+    --hash=sha256:885cee1c4d05888af86b626cbbfc29d3c6c842ef4fe8f4a486994cef9daddfe0
+acme==0.39.0 \
+    --hash=sha256:4f8be913df289b981852042719469cc367a7e436256f232c799d0bd1521db710 \
+    --hash=sha256:a2fcb75d16de6804f4b4d773a457ee2f6434ebaf8fd1aa60862a91d4e8f73608
+certbot-apache==0.39.0 \
+    --hash=sha256:c7a8630a85b753a52ca0b8c19e24b8f85ac4ba028292a95745e250c2e72faab9 \
+    --hash=sha256:4651a0212c9ebc3087281dad92ad3cb355bb2730f432d0180a8d23325d11825a
+certbot-nginx==0.39.0 \
+    --hash=sha256:76e5862ad5cc0fbc099df3502987c101c60dee1c188a579eac990edee7a910df \
+    --hash=sha256:ceac88df52d3b27d14c3052b9e90ada327d7e14ecd6e4af7519918182d6138b4
 
 UNLIKELY_EOF
     # -------------------------------------------------------------------------

letsencrypt-auto-source/Dockerfile.redhat6

@@ -1,9 +1,13 @@
 # For running tests, build a docker image with a passwordless sudo and a trust
 # store we can manipulate.
 
-FROM centos:6
+ARG REDHAT_DIST_FLAVOR
+FROM ${REDHAT_DIST_FLAVOR}:6
 
-RUN yum install -y epel-release
+ARG REDHAT_DIST_FLAVOR
+
+RUN curl -O https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm \
+ && rpm -ivh epel-release-latest-6.noarch.rpm
 
 # Install pip and sudo:
 RUN yum install -y python-pip sudo
@@ -27,11 +31,18 @@ RUN mkdir -p /home/lea/certbot
 COPY ./tests/certs/ca/my-root-ca.crt.pem /usr/local/share/ca-certificates/
 RUN update-ca-trust
 
-# Copy code:
+# Copy current letsencrypt-auto:
 COPY . /home/lea/certbot/letsencrypt-auto-source
 
+# Fetch previous letsencrypt-auto that was installing python 3.4
+RUN curl https://raw.githubusercontent.com/certbot/certbot/v0.38.0/letsencrypt-auto-source/letsencrypt-auto \
+    -o /home/lea/certbot/letsencrypt-auto-source/letsencrypt-auto_py_34 \
+ && chmod +x /home/lea/certbot/letsencrypt-auto-source/letsencrypt-auto_py_34
+
+RUN cp /home/lea/certbot/letsencrypt-auto-source/tests/${REDHAT_DIST_FLAVOR}6_tests.sh /home/lea/certbot/letsencrypt-auto-source/tests/redhat6_tests.sh \
+ && chmod +x /home/lea/certbot/letsencrypt-auto-source/tests/redhat6_tests.sh
+
 USER lea
 WORKDIR /home/lea
 
-RUN sudo chmod +x certbot/letsencrypt-auto-source/tests/centos6_tests.sh
-CMD sudo certbot/letsencrypt-auto-source/tests/centos6_tests.sh
+CMD ["sudo", "certbot/letsencrypt-auto-source/tests/redhat6_tests.sh"]

letsencrypt-auto-source/certbot-auto.asc

@@ -1,11 +1,11 @@
 -----BEGIN PGP SIGNATURE-----
 
-iQEzBAABCAAdFiEEos+1H6J1pyhiNOeyTRfJlc2XdfIFAl1Mt7UACgkQTRfJlc2X
-dfIALggAhyS29bqwp7L2u31uJalZbZQzK2jb86+YyxYzJ/TNAOVHghZNrF7krXAV
-GCYEV6SXNHlScAtv7eIVbMcbiaSh/+6/1K3HsPBNP/7nR2sTZ/AOSQNPKdgUia5E
-jypTdGYcOiQBCqyP0yDKFXIKxJFOP63tIvidfuT0rBcyusrJ/QPJs6uhKLggOiFv
-9kNgZQsOhE3LpA9Yaqf0lsbKhA154c2Q662JiGCzQ2AST36bdzNEwsUeVoTbJda3
-o3qN5kg+mWZNrc9qgYjDA3gXxepNGxjXmFasJc1k1uVx9gxYhEO+/WC1UKMQJR1O
-Y/7Qrv3sR3KJ/Q/guhEB4jTKOnvXvw==
-=+61j
+iQEzBAABCAAdFiEEos+1H6J1pyhiNOeyTRfJlc2XdfIFAl2TsPMACgkQTRfJlc2X
+dfJHUAf+NcnvHzowhLr1rkR11CSKMCMgwUee7Nm0QHnVPf09+Dd9mvuaRptuua1D
+Qvtcb3F4OQ6/3khy3fzGXIcEe9kuI2+boe+ZA0dfmmzo4ELzpWUadXkuonYybZFE
+JAaICgLLHOkiRL8J8ZTmXZI4tbFSsxTLMNOwoMZ6oGgp2plj2rm85L4Z+vUlfaTf
+wcs/glbBtbYfW3WWapMsMWwgrE62Q/OOhBjbkPCywFRQDwwaXz6QPrvi+k6gLCqs
+Okvg5bY2hP70tU1i9wxp2DAfF/P/5i2hVSWktRdMolUTTTeczLW81allmmDRJcAi
+4xrj6wYhN7olMZrTpakXb7zRR9/MGQ==
+=Ag2y
 -----END PGP SIGNATURE-----

letsencrypt-auto-source/letsencrypt-auto

@@ -31,7 +31,7 @@ if [ -z "$VENV_PATH" ]; then
 fi
 VENV_BIN="$VENV_PATH/bin"
 BOOTSTRAP_VERSION_PATH="$VENV_PATH/certbot-auto-bootstrap-version.txt"
-LE_AUTO_VERSION="0.38.0.dev0"
+LE_AUTO_VERSION="0.40.0.dev0"
 BASENAME=$(basename $0)
 USAGE="Usage: $BASENAME [OPTIONS]
 A self-updating wrapper script for the Certbot ACME client. When run, updates
@@ -256,20 +256,28 @@ DeprecationBootstrap() {
   fi
 }
 
-MIN_PYTHON_VERSION="2.7"
-MIN_PYVER=$(echo "$MIN_PYTHON_VERSION" | sed 's/\.//')
+MIN_PYTHON_2_VERSION="2.7"
+MIN_PYVER2=$(echo "$MIN_PYTHON_2_VERSION" | sed 's/\.//')
+MIN_PYTHON_3_VERSION="3.5"
+MIN_PYVER3=$(echo "$MIN_PYTHON_3_VERSION" | sed 's/\.//')
 # Sets LE_PYTHON to Python version string and PYVER to the first two
-# digits of the python version
+# digits of the python version.
+# MIN_PYVER and MIN_PYTHON_VERSION are also set by this function, and their
+# values depend on if we try to use Python 3 or Python 2.
 DeterminePythonVersion() {
   # Arguments: "NOCRASH" if we shouldn't crash if we don't find a good python
   #
   # If no Python is found, PYVER is set to 0.
   if [ "$USE_PYTHON_3" = 1 ]; then
+    MIN_PYVER=$MIN_PYVER3
+    MIN_PYTHON_VERSION=$MIN_PYTHON_3_VERSION
     for LE_PYTHON in "$LE_PYTHON" python3; do
       # Break (while keeping the LE_PYTHON value) if found.
       $EXISTS "$LE_PYTHON" > /dev/null && break
     done
   else
+    MIN_PYVER=$MIN_PYVER2
+    MIN_PYTHON_VERSION=$MIN_PYTHON_2_VERSION
     for LE_PYTHON in "$LE_PYTHON" python2.7 python27 python2 python; do
       # Break (while keeping the LE_PYTHON value) if found.
       $EXISTS "$LE_PYTHON" > /dev/null && break
@@ -285,7 +293,7 @@ DeterminePythonVersion() {
     fi
   fi
 
-  PYVER=`"$LE_PYTHON" -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//'`
+  PYVER=$("$LE_PYTHON" -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//')
   if [ "$PYVER" -lt "$MIN_PYVER" ]; then
     if [ "$1" != "NOCRASH" ]; then
       error "You have an ancient version of Python entombed in your operating system..."
@@ -368,7 +376,9 @@ BootstrapDebCommon() {
 
 # Sets TOOL to the name of the package manager
 # Sets appropriate values for YES_FLAG and QUIET_FLAG based on $ASSUME_YES and $QUIET_FLAG.
-# Enables EPEL if applicable and possible.
+# Note: this function is called both while selecting the bootstrap scripts and
+# during the actual bootstrap. Some things like prompting to user can be done in the latter
+# case, but not in the former one.
 InitializeRPMCommonBase() {
   if type dnf 2>/dev/null
   then
@@ -388,26 +398,6 @@ InitializeRPMCommonBase() {
   if [ "$QUIET" = 1 ]; then
     QUIET_FLAG='--quiet'
   fi
-
-  if ! $TOOL list *virtualenv >/dev/null 2>&1; then
-    echo "To use Certbot, packages from the EPEL repository need to be installed."
-    if ! $TOOL list epel-release >/dev/null 2>&1; then
-      error "Enable the EPEL repository and try running Certbot again."
-      exit 1
-    fi
-    if [ "$ASSUME_YES" = 1 ]; then
-      /bin/echo -n "Enabling the EPEL repository in 3 seconds..."
-      sleep 1s
-      /bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..."
-      sleep 1s
-      /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 second..."
-      sleep 1s
-    fi
-    if ! $TOOL install $YES_FLAG $QUIET_FLAG epel-release; then
-      error "Could not enable EPEL. Aborting bootstrap!"
-      exit 1
-    fi
-  fi
 }
 
 BootstrapRpmCommonBase() {
@@ -488,13 +478,88 @@ BootstrapRpmCommon() {
   BootstrapRpmCommonBase "$python_pkgs"
 }
 
+# If new packages are installed by BootstrapRpmPython3 below, this version
+# number must be increased.
+BOOTSTRAP_RPM_PYTHON3_LEGACY_VERSION=1
+
+# Checks if rh-python36 can be installed.
+Python36SclIsAvailable() {
+  InitializeRPMCommonBase >/dev/null 2>&1;
+
+  if "${TOOL}" list rh-python36 >/dev/null 2>&1; then
+    return 0
+  fi
+  if "${TOOL}" list centos-release-scl >/dev/null 2>&1; then
+    return 0
+  fi
+  return 1
+}
+
+# Try to enable rh-python36 from SCL if it is necessary and possible.
+EnablePython36SCL() {
+  if "$EXISTS" python3.6 > /dev/null 2> /dev/null; then
+      return 0
+  fi
+  if ! scl --list 2>/dev/null | grep -q rh-python36; then
+      return 0
+  fi
+  set +e
+  . scl_source enable rh-python36
+  set -e
+}
+
+# This bootstrap concerns old RedHat-based distributions that do not ship by default
+# with Python 2.7, but only Python 2.6. We bootstrap them by enabling SCL and installing
+# Python 3.6. Some of these distributions are: CentOS/RHEL/OL/SL 6.
+BootstrapRpmPython3Legacy() {
+  # Tested with:
+  #   - CentOS 6
+
+  InitializeRPMCommonBase
+
+  if ! "${TOOL}" list rh-python36 >/dev/null 2>&1; then
+    echo "To use Certbot on this operating system, packages from the SCL repository need to be installed."
+    if ! "${TOOL}" list centos-release-scl >/dev/null 2>&1; then
+      error "Enable the SCL repository and try running Certbot again."
+      exit 1
+    fi
+    if [ "${ASSUME_YES}" = 1 ]; then
+      /bin/echo -n "Enabling the SCL repository in 3 seconds... (Press Ctrl-C to cancel)"
+      sleep 1s
+      /bin/echo -ne "\e[0K\rEnabling the SCL repository in 2 seconds... (Press Ctrl-C to cancel)"
+      sleep 1s
+      /bin/echo -e "\e[0K\rEnabling the SCL repository in 1 second... (Press Ctrl-C to cancel)"
+      sleep 1s
+    fi
+    if ! "${TOOL}" install "${YES_FLAG}" "${QUIET_FLAG}" centos-release-scl; then
+      error "Could not enable SCL. Aborting bootstrap!"
+      exit 1
+    fi
+  fi
+
+  # CentOS 6 must use rh-python36 from SCL
+  if "${TOOL}" list rh-python36 >/dev/null 2>&1; then
+    python_pkgs="rh-python36-python
+      rh-python36-python-virtualenv
+      rh-python36-python-devel
+    "
+  else
+    error "No supported Python package available to install. Aborting bootstrap!"
+    exit 1
+  fi
+
+  BootstrapRpmCommonBase "${python_pkgs}"
+
+  # Enable SCL rh-python36 after bootstrapping.
+  EnablePython36SCL
+}
+
 # If new packages are installed by BootstrapRpmPython3 below, this version
 # number must be increased.
 BOOTSTRAP_RPM_PYTHON3_VERSION=1
 
 BootstrapRpmPython3() {
   # Tested with:
-  #   - CentOS 6
   #   - Fedora 29
 
   InitializeRPMCommonBase
@@ -505,12 +570,6 @@ BootstrapRpmPython3() {
       python3-virtualenv
       python3-devel
     "
-  # EPEL uses python34
-  elif $TOOL list python34 >/dev/null 2>&1; then
-    python_pkgs="python34
-      python34-devel
-      python34-tools
-    "
   else
     error "No supported Python package available to install. Aborting bootstrap!"
     exit 1
@@ -769,29 +828,50 @@ elif [ -f /etc/redhat-release ]; then
      RPM_DIST_VERSION=0
   fi
 
-  # Starting to Fedora 29, python2 is on a deprecation path. Let's move to python3 then.
-  # RHEL 8 also uses python3 by default.
-  if [ "$RPM_DIST_NAME" = "fedora" -a "$RPM_DIST_VERSION" -ge 29 -o "$PYVER" -eq 26 ]; then
-    RPM_USE_PYTHON_3=1
-  elif [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
-    RPM_USE_PYTHON_3=1
-  else
-    RPM_USE_PYTHON_3=0
-  fi
+  # Handle legacy RPM distributions
+  if [ "$PYVER" -eq 26 ]; then
+    # Check if an automated bootstrap can be achieved on this system.
+    if ! Python36SclIsAvailable; then
+      INTERACTIVE_BOOTSTRAP=1
+    fi
 
-  if [ "$RPM_USE_PYTHON_3" = 1 ]; then
     Bootstrap() {
-      BootstrapMessage "RedHat-based OSes that will use Python3"
-      BootstrapRpmPython3
+      BootstrapMessage "Legacy RedHat-based OSes that will use Python3"
+      BootstrapRpmPython3Legacy
     }
     USE_PYTHON_3=1
-    BOOTSTRAP_VERSION="BootstrapRpmPython3 $BOOTSTRAP_RPM_PYTHON3_VERSION"
+    BOOTSTRAP_VERSION="BootstrapRpmPython3Legacy $BOOTSTRAP_RPM_PYTHON3_LEGACY_VERSION"
+
+    # Try now to enable SCL rh-python36 for systems already bootstrapped
+    # NB: EnablePython36SCL has been defined along with BootstrapRpmPython3Legacy in certbot-auto
+    EnablePython36SCL
   else
-    Bootstrap() {
-      BootstrapMessage "RedHat-based OSes"
-      BootstrapRpmCommon
-    }
-    BOOTSTRAP_VERSION="BootstrapRpmCommon $BOOTSTRAP_RPM_COMMON_VERSION"
+    # Starting to Fedora 29, python2 is on a deprecation path. Let's move to python3 then.
+    # RHEL 8 also uses python3 by default.
+    if [ "$RPM_DIST_NAME" = "fedora" -a "$RPM_DIST_VERSION" -ge 29 ]; then
+      RPM_USE_PYTHON_3=1
+    elif [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
+      RPM_USE_PYTHON_3=1
+    elif [ "$RPM_DIST_NAME" = "centos" -a "$RPM_DIST_VERSION" -ge 8 ]; then
+      RPM_USE_PYTHON_3=1
+    else
+      RPM_USE_PYTHON_3=0
+    fi
+
+    if [ "$RPM_USE_PYTHON_3" = 1 ]; then
+      Bootstrap() {
+        BootstrapMessage "RedHat-based OSes that will use Python3"
+        BootstrapRpmPython3
+      }
+      USE_PYTHON_3=1
+      BOOTSTRAP_VERSION="BootstrapRpmPython3 $BOOTSTRAP_RPM_PYTHON3_VERSION"
+    else
+      Bootstrap() {
+        BootstrapMessage "RedHat-based OSes"
+        BootstrapRpmCommon
+      }
+      BOOTSTRAP_VERSION="BootstrapRpmCommon $BOOTSTRAP_RPM_COMMON_VERSION"
+    fi
   fi
 
   LE_PYTHON="$prev_le_python"
@@ -1076,8 +1156,15 @@ if [ "$1" = "--le-auto-phase2" ]; then
     # If the selected Bootstrap function isn't a noop and it differs from the
     # previously used version
     if [ -n "$BOOTSTRAP_VERSION" -a "$BOOTSTRAP_VERSION" != "$PREV_BOOTSTRAP_VERSION" ]; then
-      # if non-interactive mode or stdin and stdout are connected to a terminal
-      if [ \( "$NONINTERACTIVE" = 1 \) -o \( \( -t 0 \) -a \( -t 1 \) \) ]; then
+      # Check if we can rebootstrap without manual user intervention: this requires that
+      # certbot-auto is in non-interactive mode AND selected bootstrap does not claim to
+      # require a manual user intervention.
+      if [ "$NONINTERACTIVE" = 1 -a "$INTERACTIVE_BOOTSTRAP" != 1 ]; then
+        CAN_REBOOTSTRAP=1
+      fi
+      # Check if rebootstrap can be done non-interactively and current shell is non-interactive
+      # (true if stdin and stdout are not attached to a terminal).
+      if [ \( "$CAN_REBOOTSTRAP" = 1 \) -o \( \( -t 0 \) -a \( -t 1 \) \) ]; then
         if [ -d "$VENV_PATH" ]; then
           rm -rf "$VENV_PATH"
         fi
@@ -1088,12 +1175,21 @@ if [ "$1" = "--le-auto-phase2" ]; then
           ln -s "$VENV_PATH" "$OLD_VENV_PATH"
         fi
         RerunWithArgs "$@"
+      # Otherwise bootstrap needs to be done manually by the user.
       else
-        error "Skipping upgrade because new OS dependencies may need to be installed."
-        error
-        error "To upgrade to a newer version, please run this script again manually so you can"
-        error "approve changes or with --non-interactive on the command line to automatically"
-        error "install any required packages."
+        # If it is because bootstrapping is interactive, --non-interactive will be of no use.
+        if [ "$INTERACTIVE_BOOTSTRAP" = 1 ]; then
+          error "Skipping upgrade because new OS dependencies may need to be installed."
+          error "This requires manual user intervention: please run this script again manually."
+        # If this is because of the environment (eg. non interactive shell without
+        # --non-interactive flag set), help the user in that direction.
+        else
+          error "Skipping upgrade because new OS dependencies may need to be installed."
+          error
+          error "To upgrade to a newer version, please run this script again manually so you can"
+          error "approve changes or with --non-interactive on the command line to automatically"
+          error "install any required packages."
+        fi
         # Set INSTALLED_VERSION to be the same so we don't update the venv
         INSTALLED_VERSION="$LE_AUTO_VERSION"
         # Continue to use OLD_VENV_PATH if the new venv doesn't exist
@@ -1336,18 +1432,18 @@ letsencrypt==0.7.0 \
     --hash=sha256:105a5fb107e45bcd0722eb89696986dcf5f08a86a321d6aef25a0c7c63375ade \
     --hash=sha256:c36e532c486a7e92155ee09da54b436a3c420813ec1c590b98f635d924720de9
 
-certbot==0.37.1 \
-    --hash=sha256:84dbdad204327b8d8ef9ab5b040f2be1e427a9f7e087affcc9a6051ea1b03fe7 \
-    --hash=sha256:aace73e63b0c11cdb4b0bd33e1780c1fbe0ce5669dc72e80c3aa9500145daf16
-acme==0.37.1 \
-    --hash=sha256:83a4f6f3c5eb6a85233d5ba87714b426f2d096df58d711f8a2fc4071eb3fd3fc \
-    --hash=sha256:c069a761990751f7c4bf51d2e87ae10319bf460de6629d2908c9fa6f69e97111
-certbot-apache==0.37.1 \
-    --hash=sha256:3ea832408877b12b3a60d17e8b2ee3387364f8c3023ac267161c25b99087cd42 \
-    --hash=sha256:e46c2644451101c0e216aa1f525a577cc903efaf871e0e4da277224a4439040c
-certbot-nginx==0.37.1 \
-    --hash=sha256:1f9af389d26f06634e2eefaace3354e7679dabb4295e1d55d05a4ee7e23a64bd \
-    --hash=sha256:02a7ec15bd388d0f0e94a34c86a8f8d618ec7d5ffde0c206039bb4c46b294ce4
+certbot==0.39.0 \
+    --hash=sha256:f1a70651a6c5137a448f4a8db17b09af619f80a077326caae6b74278bf1db488 \
+    --hash=sha256:885cee1c4d05888af86b626cbbfc29d3c6c842ef4fe8f4a486994cef9daddfe0
+acme==0.39.0 \
+    --hash=sha256:4f8be913df289b981852042719469cc367a7e436256f232c799d0bd1521db710 \
+    --hash=sha256:a2fcb75d16de6804f4b4d773a457ee2f6434ebaf8fd1aa60862a91d4e8f73608
+certbot-apache==0.39.0 \
+    --hash=sha256:c7a8630a85b753a52ca0b8c19e24b8f85ac4ba028292a95745e250c2e72faab9 \
+    --hash=sha256:4651a0212c9ebc3087281dad92ad3cb355bb2730f432d0180a8d23325d11825a
+certbot-nginx==0.39.0 \
+    --hash=sha256:76e5862ad5cc0fbc099df3502987c101c60dee1c188a579eac990edee7a910df \
+    --hash=sha256:ceac88df52d3b27d14c3052b9e90ada327d7e14ecd6e4af7519918182d6138b4
 
 UNLIKELY_EOF
     # -------------------------------------------------------------------------

letsencrypt-auto-source/letsencrypt-auto.template

@@ -256,20 +256,28 @@ DeprecationBootstrap() {
   fi
 }
 
-MIN_PYTHON_VERSION="2.7"
-MIN_PYVER=$(echo "$MIN_PYTHON_VERSION" | sed 's/\.//')
+MIN_PYTHON_2_VERSION="2.7"
+MIN_PYVER2=$(echo "$MIN_PYTHON_2_VERSION" | sed 's/\.//')
+MIN_PYTHON_3_VERSION="3.5"
+MIN_PYVER3=$(echo "$MIN_PYTHON_3_VERSION" | sed 's/\.//')
 # Sets LE_PYTHON to Python version string and PYVER to the first two
-# digits of the python version
+# digits of the python version.
+# MIN_PYVER and MIN_PYTHON_VERSION are also set by this function, and their
+# values depend on if we try to use Python 3 or Python 2.
 DeterminePythonVersion() {
   # Arguments: "NOCRASH" if we shouldn't crash if we don't find a good python
   #
   # If no Python is found, PYVER is set to 0.
   if [ "$USE_PYTHON_3" = 1 ]; then
+    MIN_PYVER=$MIN_PYVER3
+    MIN_PYTHON_VERSION=$MIN_PYTHON_3_VERSION
     for LE_PYTHON in "$LE_PYTHON" python3; do
       # Break (while keeping the LE_PYTHON value) if found.
       $EXISTS "$LE_PYTHON" > /dev/null && break
     done
   else
+    MIN_PYVER=$MIN_PYVER2
+    MIN_PYTHON_VERSION=$MIN_PYTHON_2_VERSION
     for LE_PYTHON in "$LE_PYTHON" python2.7 python27 python2 python; do
       # Break (while keeping the LE_PYTHON value) if found.
       $EXISTS "$LE_PYTHON" > /dev/null && break
@@ -285,7 +293,7 @@ DeterminePythonVersion() {
     fi
   fi
 
-  PYVER=`"$LE_PYTHON" -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//'`
+  PYVER=$("$LE_PYTHON" -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//')
   if [ "$PYVER" -lt "$MIN_PYVER" ]; then
     if [ "$1" != "NOCRASH" ]; then
       error "You have an ancient version of Python entombed in your operating system..."
@@ -298,6 +306,7 @@ DeterminePythonVersion() {
 {{ bootstrappers/deb_common.sh }}
 {{ bootstrappers/rpm_common_base.sh }}
 {{ bootstrappers/rpm_common.sh }}
+{{ bootstrappers/rpm_python3_legacy.sh }}
 {{ bootstrappers/rpm_python3.sh }}
 {{ bootstrappers/suse_common.sh }}
 {{ bootstrappers/arch_common.sh }}
@@ -344,29 +353,50 @@ elif [ -f /etc/redhat-release ]; then
      RPM_DIST_VERSION=0
   fi
 
-  # Starting to Fedora 29, python2 is on a deprecation path. Let's move to python3 then.
-  # RHEL 8 also uses python3 by default.
-  if [ "$RPM_DIST_NAME" = "fedora" -a "$RPM_DIST_VERSION" -ge 29 -o "$PYVER" -eq 26 ]; then
-    RPM_USE_PYTHON_3=1
-  elif [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
-    RPM_USE_PYTHON_3=1
-  else
-    RPM_USE_PYTHON_3=0
-  fi
+  # Handle legacy RPM distributions
+  if [ "$PYVER" -eq 26 ]; then
+    # Check if an automated bootstrap can be achieved on this system.
+    if ! Python36SclIsAvailable; then
+      INTERACTIVE_BOOTSTRAP=1
+    fi
 
-  if [ "$RPM_USE_PYTHON_3" = 1 ]; then
     Bootstrap() {
-      BootstrapMessage "RedHat-based OSes that will use Python3"
-      BootstrapRpmPython3
+      BootstrapMessage "Legacy RedHat-based OSes that will use Python3"
+      BootstrapRpmPython3Legacy
     }
     USE_PYTHON_3=1
-    BOOTSTRAP_VERSION="BootstrapRpmPython3 $BOOTSTRAP_RPM_PYTHON3_VERSION"
+    BOOTSTRAP_VERSION="BootstrapRpmPython3Legacy $BOOTSTRAP_RPM_PYTHON3_LEGACY_VERSION"
+
+    # Try now to enable SCL rh-python36 for systems already bootstrapped
+    # NB: EnablePython36SCL has been defined along with BootstrapRpmPython3Legacy in certbot-auto
+    EnablePython36SCL
   else
-    Bootstrap() {
-      BootstrapMessage "RedHat-based OSes"
-      BootstrapRpmCommon
-    }
-    BOOTSTRAP_VERSION="BootstrapRpmCommon $BOOTSTRAP_RPM_COMMON_VERSION"
+    # Starting to Fedora 29, python2 is on a deprecation path. Let's move to python3 then.
+    # RHEL 8 also uses python3 by default.
+    if [ "$RPM_DIST_NAME" = "fedora" -a "$RPM_DIST_VERSION" -ge 29 ]; then
+      RPM_USE_PYTHON_3=1
+    elif [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
+      RPM_USE_PYTHON_3=1
+    elif [ "$RPM_DIST_NAME" = "centos" -a "$RPM_DIST_VERSION" -ge 8 ]; then
+      RPM_USE_PYTHON_3=1
+    else
+      RPM_USE_PYTHON_3=0
+    fi
+
+    if [ "$RPM_USE_PYTHON_3" = 1 ]; then
+      Bootstrap() {
+        BootstrapMessage "RedHat-based OSes that will use Python3"
+        BootstrapRpmPython3
+      }
+      USE_PYTHON_3=1
+      BOOTSTRAP_VERSION="BootstrapRpmPython3 $BOOTSTRAP_RPM_PYTHON3_VERSION"
+    else
+      Bootstrap() {
+        BootstrapMessage "RedHat-based OSes"
+        BootstrapRpmCommon
+      }
+      BOOTSTRAP_VERSION="BootstrapRpmCommon $BOOTSTRAP_RPM_COMMON_VERSION"
+    fi
   fi
 
   LE_PYTHON="$prev_le_python"
@@ -543,8 +573,15 @@ if [ "$1" = "--le-auto-phase2" ]; then
     # If the selected Bootstrap function isn't a noop and it differs from the
     # previously used version
     if [ -n "$BOOTSTRAP_VERSION" -a "$BOOTSTRAP_VERSION" != "$PREV_BOOTSTRAP_VERSION" ]; then
-      # if non-interactive mode or stdin and stdout are connected to a terminal
-      if [ \( "$NONINTERACTIVE" = 1 \) -o \( \( -t 0 \) -a \( -t 1 \) \) ]; then
+      # Check if we can rebootstrap without manual user intervention: this requires that
+      # certbot-auto is in non-interactive mode AND selected bootstrap does not claim to
+      # require a manual user intervention.
+      if [ "$NONINTERACTIVE" = 1 -a "$INTERACTIVE_BOOTSTRAP" != 1 ]; then
+        CAN_REBOOTSTRAP=1
+      fi
+      # Check if rebootstrap can be done non-interactively and current shell is non-interactive
+      # (true if stdin and stdout are not attached to a terminal).
+      if [ \( "$CAN_REBOOTSTRAP" = 1 \) -o \( \( -t 0 \) -a \( -t 1 \) \) ]; then
         if [ -d "$VENV_PATH" ]; then
           rm -rf "$VENV_PATH"
         fi
@@ -555,12 +592,21 @@ if [ "$1" = "--le-auto-phase2" ]; then
           ln -s "$VENV_PATH" "$OLD_VENV_PATH"
         fi
         RerunWithArgs "$@"
+      # Otherwise bootstrap needs to be done manually by the user.
       else
-        error "Skipping upgrade because new OS dependencies may need to be installed."
-        error
-        error "To upgrade to a newer version, please run this script again manually so you can"
-        error "approve changes or with --non-interactive on the command line to automatically"
-        error "install any required packages."
+        # If it is because bootstrapping is interactive, --non-interactive will be of no use.
+        if [ "$INTERACTIVE_BOOTSTRAP" = 1 ]; then
+          error "Skipping upgrade because new OS dependencies may need to be installed."
+          error "This requires manual user intervention: please run this script again manually."
+        # If this is because of the environment (eg. non interactive shell without
+        # --non-interactive flag set), help the user in that direction.
+        else
+          error "Skipping upgrade because new OS dependencies may need to be installed."
+          error
+          error "To upgrade to a newer version, please run this script again manually so you can"
+          error "approve changes or with --non-interactive on the command line to automatically"
+          error "install any required packages."
+        fi
         # Set INSTALLED_VERSION to be the same so we don't update the venv
         INSTALLED_VERSION="$LE_AUTO_VERSION"
         # Continue to use OLD_VENV_PATH if the new venv doesn't exist

letsencrypt-auto-source/pieces/bootstrappers/rpm_common_base.sh

@@ -3,7 +3,9 @@
 
 # Sets TOOL to the name of the package manager
 # Sets appropriate values for YES_FLAG and QUIET_FLAG based on $ASSUME_YES and $QUIET_FLAG.
-# Enables EPEL if applicable and possible.
+# Note: this function is called both while selecting the bootstrap scripts and
+# during the actual bootstrap. Some things like prompting to user can be done in the latter
+# case, but not in the former one.
 InitializeRPMCommonBase() {
   if type dnf 2>/dev/null
   then
@@ -23,26 +25,6 @@ InitializeRPMCommonBase() {
   if [ "$QUIET" = 1 ]; then
     QUIET_FLAG='--quiet'
   fi
-
-  if ! $TOOL list *virtualenv >/dev/null 2>&1; then
-    echo "To use Certbot, packages from the EPEL repository need to be installed."
-    if ! $TOOL list epel-release >/dev/null 2>&1; then
-      error "Enable the EPEL repository and try running Certbot again."
-      exit 1
-    fi
-    if [ "$ASSUME_YES" = 1 ]; then
-      /bin/echo -n "Enabling the EPEL repository in 3 seconds..."
-      sleep 1s
-      /bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..."
-      sleep 1s
-      /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 second..."
-      sleep 1s
-    fi
-    if ! $TOOL install $YES_FLAG $QUIET_FLAG epel-release; then
-      error "Could not enable EPEL. Aborting bootstrap!"
-      exit 1
-    fi
-  fi
 }
 
 BootstrapRpmCommonBase() {

letsencrypt-auto-source/pieces/bootstrappers/rpm_python3.sh

@@ -4,7 +4,6 @@ BOOTSTRAP_RPM_PYTHON3_VERSION=1
 
 BootstrapRpmPython3() {
   # Tested with:
-  #   - CentOS 6
   #   - Fedora 29
 
   InitializeRPMCommonBase
@@ -15,12 +14,6 @@ BootstrapRpmPython3() {
       python3-virtualenv
       python3-devel
     "
-  # EPEL uses python34
-  elif $TOOL list python34 >/dev/null 2>&1; then
-    python_pkgs="python34
-      python34-devel
-      python34-tools
-    "
   else
     error "No supported Python package available to install. Aborting bootstrap!"
     exit 1

letsencrypt-auto-source/pieces/bootstrappers/rpm_python3_legacy.sh

@@ -0,0 +1,75 @@
+# If new packages are installed by BootstrapRpmPython3 below, this version
+# number must be increased.
+BOOTSTRAP_RPM_PYTHON3_LEGACY_VERSION=1
+
+# Checks if rh-python36 can be installed.
+Python36SclIsAvailable() {
+  InitializeRPMCommonBase >/dev/null 2>&1;
+
+  if "${TOOL}" list rh-python36 >/dev/null 2>&1; then
+    return 0
+  fi
+  if "${TOOL}" list centos-release-scl >/dev/null 2>&1; then
+    return 0
+  fi
+  return 1
+}
+
+# Try to enable rh-python36 from SCL if it is necessary and possible.
+EnablePython36SCL() {
+  if "$EXISTS" python3.6 > /dev/null 2> /dev/null; then
+      return 0
+  fi
+  if ! scl --list 2>/dev/null | grep -q rh-python36; then
+      return 0
+  fi
+  set +e
+  . scl_source enable rh-python36
+  set -e
+}
+
+# This bootstrap concerns old RedHat-based distributions that do not ship by default
+# with Python 2.7, but only Python 2.6. We bootstrap them by enabling SCL and installing
+# Python 3.6. Some of these distributions are: CentOS/RHEL/OL/SL 6.
+BootstrapRpmPython3Legacy() {
+  # Tested with:
+  #   - CentOS 6
+
+  InitializeRPMCommonBase
+
+  if ! "${TOOL}" list rh-python36 >/dev/null 2>&1; then
+    echo "To use Certbot on this operating system, packages from the SCL repository need to be installed."
+    if ! "${TOOL}" list centos-release-scl >/dev/null 2>&1; then
+      error "Enable the SCL repository and try running Certbot again."
+      exit 1
+    fi
+    if [ "${ASSUME_YES}" = 1 ]; then
+      /bin/echo -n "Enabling the SCL repository in 3 seconds... (Press Ctrl-C to cancel)"
+      sleep 1s
+      /bin/echo -ne "\e[0K\rEnabling the SCL repository in 2 seconds... (Press Ctrl-C to cancel)"
+      sleep 1s
+      /bin/echo -e "\e[0K\rEnabling the SCL repository in 1 second... (Press Ctrl-C to cancel)"
+      sleep 1s
+    fi
+    if ! "${TOOL}" install "${YES_FLAG}" "${QUIET_FLAG}" centos-release-scl; then
+      error "Could not enable SCL. Aborting bootstrap!"
+      exit 1
+    fi
+  fi
+
+  # CentOS 6 must use rh-python36 from SCL
+  if "${TOOL}" list rh-python36 >/dev/null 2>&1; then
+    python_pkgs="rh-python36-python
+      rh-python36-python-virtualenv
+      rh-python36-python-devel
+    "
+  else
+    error "No supported Python package available to install. Aborting bootstrap!"
+    exit 1
+  fi
+
+  BootstrapRpmCommonBase "${python_pkgs}"
+
+  # Enable SCL rh-python36 after bootstrapping.
+  EnablePython36SCL
+}

letsencrypt-auto-source/pieces/certbot-requirements.txt

@@ -1,12 +1,12 @@
-certbot==0.37.1 \
-    --hash=sha256:84dbdad204327b8d8ef9ab5b040f2be1e427a9f7e087affcc9a6051ea1b03fe7 \
-    --hash=sha256:aace73e63b0c11cdb4b0bd33e1780c1fbe0ce5669dc72e80c3aa9500145daf16
-acme==0.37.1 \
-    --hash=sha256:83a4f6f3c5eb6a85233d5ba87714b426f2d096df58d711f8a2fc4071eb3fd3fc \
-    --hash=sha256:c069a761990751f7c4bf51d2e87ae10319bf460de6629d2908c9fa6f69e97111
-certbot-apache==0.37.1 \
-    --hash=sha256:3ea832408877b12b3a60d17e8b2ee3387364f8c3023ac267161c25b99087cd42 \
-    --hash=sha256:e46c2644451101c0e216aa1f525a577cc903efaf871e0e4da277224a4439040c
-certbot-nginx==0.37.1 \
-    --hash=sha256:1f9af389d26f06634e2eefaace3354e7679dabb4295e1d55d05a4ee7e23a64bd \
-    --hash=sha256:02a7ec15bd388d0f0e94a34c86a8f8d618ec7d5ffde0c206039bb4c46b294ce4
+certbot==0.39.0 \
+    --hash=sha256:f1a70651a6c5137a448f4a8db17b09af619f80a077326caae6b74278bf1db488 \
+    --hash=sha256:885cee1c4d05888af86b626cbbfc29d3c6c842ef4fe8f4a486994cef9daddfe0
+acme==0.39.0 \
+    --hash=sha256:4f8be913df289b981852042719469cc367a7e436256f232c799d0bd1521db710 \
+    --hash=sha256:a2fcb75d16de6804f4b4d773a457ee2f6434ebaf8fd1aa60862a91d4e8f73608
+certbot-apache==0.39.0 \
+    --hash=sha256:c7a8630a85b753a52ca0b8c19e24b8f85ac4ba028292a95745e250c2e72faab9 \
+    --hash=sha256:4651a0212c9ebc3087281dad92ad3cb355bb2730f432d0180a8d23325d11825a
+certbot-nginx==0.39.0 \
+    --hash=sha256:76e5862ad5cc0fbc099df3502987c101c60dee1c188a579eac990edee7a910df \
+    --hash=sha256:ceac88df52d3b27d14c3052b9e90ada327d7e14ecd6e4af7519918182d6138b4

letsencrypt-auto-source/tests/centos6_tests.sh

@@ -1,81 +1,140 @@
 #!/bin/bash
 # Start by making sure your system is up-to-date:
-yum update -y > /dev/null
-yum install -y centos-release-scl > /dev/null
-yum install -y python27 > /dev/null 2> /dev/null
+yum update -y >/dev/null
+yum install -y centos-release-scl >/dev/null
+yum install -y python27 >/dev/null 2>/dev/null
 
+LE_AUTO_PY_34="certbot/letsencrypt-auto-source/letsencrypt-auto_py_34"
 LE_AUTO="certbot/letsencrypt-auto-source/letsencrypt-auto"
 
+# Last version of certbot-auto that was bootstraping Python 3.4 for CentOS 6 users
+INITIAL_CERTBOT_VERSION_PY34="certbot 0.38.0"
+
 # we're going to modify env variables, so do this in a subshell
 (
-source /opt/rh/python27/enable
+  . scl_source enable python27
 
-# ensure python 3 isn't installed
-python3 --version 2> /dev/null
-RESULT=$?
-if [ $RESULT -eq 0 ]; then
-  error "Python3 is already installed."
-  exit 1
-fi
+  # ensure python 3 isn't installed
+  python3 --version >/dev/null 2>/dev/null
+  RESULT=$?
+  if [ $RESULT -eq 0 ]; then
+    echo "ERROR: Python3 is already installed."
+    exit 1
+  fi
 
-# ensure python2.7 is available
-python2.7 --version 2> /dev/null
-RESULT=$?
-if [ $RESULT -ne 0 ]; then
-  error "Python3 is not available."
-  exit 1
-fi
+  # ensure python2.7 is available
+  python2.7 --version >/dev/null 2>/dev/null
+  RESULT=$?
+  if [ $RESULT -ne 0 ]; then
+    echo "ERROR: Python2.7 is not available."
+    exit 1
+  fi
 
-# bootstrap, but don't install python 3.
-"$LE_AUTO" --no-self-upgrade -n > /dev/null 2> /dev/null
+  # bootstrap, but don't install python 3.
+  "$LE_AUTO" --no-self-upgrade -n >/dev/null 2>/dev/null
 
-# ensure python 3 isn't installed
-python3 --version 2> /dev/null
-RESULT=$?
-if [ $RESULT -eq 0 ]; then
-  error "letsencrypt-auto installed Python3 even though Python2.7 is present."
-  exit 1
-fi
+  # ensure python 3 isn't installed
+  python3 --version >/dev/null 2>/dev/null
+  RESULT=$?
+  if [ $RESULT -eq 0 ]; then
+    echo "ERROR: letsencrypt-auto installed Python3 even though Python2.7 is present."
+    exit 1
+  fi
 
-echo ""
-echo "PASSED: Did not upgrade to Python3 when Python2.7 is present."
+  echo "PASSED: Did not upgrade to Python3 when Python2.7 is present."
 )
 
 # ensure python2.7 isn't available
-python2.7 --version 2> /dev/null
+python2.7 --version >/dev/null 2>/dev/null
 RESULT=$?
 if [ $RESULT -eq 0 ]; then
-  error "Python2.7 is still available."
+  error "ERROR: Python2.7 is still available."
   exit 1
 fi
 
 # Skip self upgrade due to Python 3 not being available.
 if ! "$LE_AUTO" 2>&1 | grep -q "WARNING: couldn't find Python"; then
-  echo "Python upgrade failure warning not printed!"
+  echo "ERROR: Python upgrade failure warning not printed!"
   exit 1
 fi
 
-# bootstrap, this time installing python3
-"$LE_AUTO" --no-self-upgrade -n > /dev/null 2> /dev/null
+# bootstrap from the old letsencrypt-auto, this time installing python3.4
+"$LE_AUTO_PY_34" --no-self-upgrade -n >/dev/null 2>/dev/null
 
-# ensure python 3 is installed
-python3 --version > /dev/null
+# ensure python 3.4 is installed
+python3.4 --version >/dev/null 2>/dev/null
 RESULT=$?
 if [ $RESULT -ne 0 ]; then
-  error "letsencrypt-auto failed to install Python3 when only Python2.6 is present."
+  echo "ERROR: letsencrypt-auto failed to install Python3.4 using letsencrypt-auto < 0.37.0 when only Python2.6 is present."
   exit 1
 fi
 
-echo "PASSED: Successfully upgraded to Python3 when only Python2.6 is present."
-echo ""
+echo "PASSED: Successfully upgraded to Python3.4 using letsencrypt-auto < 0.37.0 when only Python2.6 is present."
 
-export VENV_PATH=$(mktemp -d)
-"$LE_AUTO" -n --no-bootstrap --no-self-upgrade --version >/dev/null 2>&1
-if [ "$($VENV_PATH/bin/python -V 2>&1 | cut -d" " -f2 | cut -d. -f1)" != 3 ]; then
-  echo "Python 3 wasn't used with --no-bootstrap!"
+# As "certbot-auto" (so without implicit --non-interactive flag set), check that the script
+# refuses to install SCL Python 3.6 when run in a non interactive shell (simulated here
+# using | tee /dev/null) if --non-interactive flag is not provided.
+cp "$LE_AUTO" /tmp/certbot-auto
+# NB: Readline has an issue on all Python versions for CentOS 6, making `certbot --version`
+# output an unprintable ASCII character on a new line at the end.
+# So we take the second last line of the output.
+version=$(/tmp/certbot-auto --version 2>/dev/null | tee /dev/null | tail -2 | head -1)
+
+if [ "$version" != "$INITIAL_CERTBOT_VERSION_PY34" ]; then
+  echo "ERROR: certbot-auto upgraded certbot in a non-interactive shell with --non-interactive flag not set."
   exit 1
 fi
-unset VENV_PATH
+
+echo "PASSED: certbot-auto did not upgrade certbot in a non-interactive shell with --non-interactive flag not set."
+
+if [ -f /opt/rh/rh-python36/enable ]; then
+  echo "ERROR: certbot-auto installed Python3.6 in a non-interactive shell with --non-interactive flag not set."
+  exit 1
+fi
+
+echo "PASSED: certbot-auto did not install Python3.6 in a non-interactive shell with --non-interactive flag not set."
+
+# now bootstrap from current letsencrypt-auto, that will install python3.6 from SCL
+"$LE_AUTO" --no-self-upgrade -n >/dev/null 2>/dev/null
+
+# Following test is exectued in a subshell, to not leak any environment variable
+(
+  # enable SCL rh-python36
+  . scl_source enable rh-python36
+
+  # ensure python 3.6 is installed
+  python3.6 --version >/dev/null 2>/dev/null
+  RESULT=$?
+  if [ $RESULT -ne 0 ]; then
+    echo "ERROR: letsencrypt-auto failed to install Python3.6 using current letsencrypt-auto when only Python2.6/Python3.4 are present."
+    exit 1
+  fi
+
+  echo "PASSED: Successfully upgraded to Python3.6 using current letsencrypt-auto when only Python2.6/Python3.4 are present."
+)
+
+# Following test is exectued in a subshell, to not leak any environment variable
+(
+  export VENV_PATH=$(mktemp -d)
+  "$LE_AUTO" -n --no-bootstrap --no-self-upgrade --version >/dev/null 2>&1
+  if [ "$($VENV_PATH/bin/python -V 2>&1 | cut -d" " -f2 | cut -d. -f1-2)" != "3.6" ]; then
+    echo "ERROR: Python 3.6 wasn't used with --no-bootstrap!"
+    exit 1
+  fi
+)
+
+# Following test is exectued in a subshell, to not leak any environment variable
+(
+  # enable SCL rh-python36
+  . scl_source enable rh-python36
+
+  # ensure everything works fine with certbot-auto bootstrap when python 3.6 is already enabled
+  export VENV_PATH=$(mktemp -d)
+  if ! "$LE_AUTO" --no-self-upgrade -n --version >/dev/null 2>/dev/null; then
+    echo "ERROR: Certbot-auto broke when Python 3.6 SCL is already enabled."
+    exit 1
+  fi
+)
 
 # test using python3
 pytest -v -s certbot/letsencrypt-auto-source/tests

letsencrypt-auto-source/tests/oraclelinux6_tests.sh

@@ -0,0 +1,85 @@
+#!/bin/bash
+set -eo pipefail
+# Start by making sure your system is up-to-date:
+yum update -y >/dev/null
+
+LE_AUTO_PY_34="certbot/letsencrypt-auto-source/letsencrypt-auto_py_34"
+LE_AUTO="certbot/letsencrypt-auto-source/letsencrypt-auto"
+
+# Apply installation instructions from official documentation:
+# https://certbot.eff.org/lets-encrypt/centosrhel6-other
+cp "$LE_AUTO" /usr/local/bin/certbot-auto
+chown root /usr/local/bin/certbot-auto
+chmod 0755 /usr/local/bin/certbot-auto
+LE_AUTO=/usr/local/bin/certbot-auto
+
+# Last version of certbot-auto that was bootstraping Python 3.4 for CentOS 6 users
+INITIAL_CERTBOT_VERSION_PY34="certbot 0.38.0"
+
+# Check bootstrap from current certbot-auto will fail, because SCL is not enabled.
+set +o pipefail
+if ! "$LE_AUTO" -n 2>&1 | grep -q "Enable the SCL repository and try running Certbot again."; then
+  echo "ERROR: Bootstrap was not aborted although SCL was not installed!"
+  exit 1
+fi
+set -o pipefail
+
+echo "PASSED: Bootstrap was aborted since SCL was not installed."
+
+# Bootstrap from the old letsencrypt-auto, Python 3.4 will be installed from EPEL.
+"$LE_AUTO_PY_34" --no-self-upgrade -n --install-only >/dev/null 2>/dev/null
+
+# Ensure Python 3.4 is installed
+if ! command -v python3.4 &>/dev/null; then
+  echo "ERROR: old letsencrypt-auto failed to install Python3.4 using letsencrypt-auto < 0.37.0 when only Python2.6 is present."
+  exit 1
+fi
+
+echo "PASSED: Bootstrap from old letsencrypt-auto succeeded and installed Python 3.4"
+
+# Expect certbot-auto to skip rebootstrapping with a warning since SCL is not installed.
+if ! "$LE_AUTO" --non-interactive --version 2>&1 | grep -q "This requires manual user intervention"; then
+  echo "FAILED: Script certbot-auto managed to start Certbot in interactive shell while SCL is not enabled!"
+  exit 1
+fi
+
+echo "PASSED: Script certbot-auto did not rebootstrap."
+
+# NB: Readline has an issue on all Python versions for OL 6, making `certbot --version`
+# output an unprintable ASCII character on a new line at the end.
+# So we take the second last line of the output.
+version=$($LE_AUTO --version 2>/dev/null | tail -2 | head -1)
+
+if [ "$version" != "$INITIAL_CERTBOT_VERSION_PY34" ]; then
+  echo "ERROR: Script certbot-auto upgraded certbot in a non-interactive shell while SCL was not enabled."
+  exit 1
+fi
+
+echo "PASSED: Script certbot-auto did not upgrade certbot but started it successfully while SCL was not enabled."
+
+# Enable SCL
+yum install -y oracle-softwarecollection-release-el6 >/dev/null
+
+# Expect certbot-auto to bootstrap successfully since SCL is available.
+"$LE_AUTO" -n --version &>/dev/null
+
+if [ "$(/opt/eff.org/certbot/venv/bin/python -V 2>&1 | cut -d" " -f2 | cut -d. -f1-2)" != "3.6" ]; then
+  echo "ERROR: Script certbot-auto failed to bootstrap and install Python 3.6 while SCL is available."
+  exit 1
+fi
+
+if ! /opt/eff.org/certbot/venv/bin/certbot --version > /dev/null 2> /dev/null; then
+  echo "ERROR: Script certbot-auto did not install certbot correctly while SCL is enabled."
+  exit 1
+fi
+
+echo "PASSED: Script certbot-auto correctly bootstraped Certbot using rh-python36 when SCL is available."
+
+# Expect certbot-auto will be totally silent now that everything has been correctly boostraped.
+OUTPUT_LEN=$("$LE_AUTO" --install-only --no-self-upgrade --quiet 2>&1 | wc -c)
+if [ "$OUTPUT_LEN" != 0 ]; then
+    echo certbot-auto produced unexpected output!
+    exit 1
+fi
+
+echo "PASSED: Script certbot-auto did not print anything in quiet mode."

letshelp-certbot/setup.py

@@ -36,6 +36,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

setup.py

@@ -59,11 +59,17 @@ install_requires = [
 # However environment markers are supported only with setuptools >= 36.2.
 # So this dependency is not added for old Linux distributions with old setuptools,
 # in order to allow these systems to build certbot from sources.
+pywin32_req = 'pywin32>=225'
 if StrictVersion(setuptools_version) >= StrictVersion('36.2'):
-    install_requires.append("pywin32>=224 ; sys_platform == 'win32'")
+    install_requires.append(pywin32_req + " ; sys_platform == 'win32'")
 elif 'bdist_wheel' in sys.argv[1:]:
     raise RuntimeError('Error, you are trying to build certbot wheels using an old version '
                        'of setuptools. Version 36.2+ of setuptools is required.')
+elif os.name == 'nt':
+    # This branch exists to improve this package's behavior on Windows. Without
+    # it, if the sdist is installed on Windows with an old version of
+    # setuptools, pywin32 will not be specified as a dependency.
+    install_requires.append(pywin32_req)
 
 dev_extras = [
     'astroid==1.6.5',
@@ -132,6 +138,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
         'Topic :: System :: Installation/Setup',

tests/letstest/apache2_targets.yaml

@@ -18,6 +18,11 @@ targets:
     user: ubuntu
   #-----------------------------------------------------------------------------
   # Debian
+  - ami: ami-01db78123b2b99496
+    name: debian10
+    type: ubuntu
+    virt: hvm
+    user: admin
   - ami: ami-003f19e0e687de1cd
     name: debian9
     type: ubuntu

tests/letstest/multitester.py

@@ -84,9 +84,6 @@ parser.add_argument('--killboulder',
 parser.add_argument('--boulderonly',
                     action='store_true',
                     help="only make a boulder server")
-parser.add_argument('--fast',
-                    action='store_true',
-                    help="use larger instance types to run faster (saves about a minute, probably not worth it)")
 cl_args = parser.parse_args()
 
 # Credential Variables
@@ -310,10 +307,10 @@ def create_client_instance(ec2_client, target, security_group_id, subnet_id):
     if 'machine_type' in target:
         machine_type = target['machine_type']
     elif target['virt'] == 'hvm':
-        machine_type = 't2.medium' if cl_args.fast else 't2.micro'
+        machine_type = 't2.medium'
     else:
         # 32 bit systems
-        machine_type = 'c1.medium' if cl_args.fast else 't1.micro'
+        machine_type = 'c1.medium'
     if 'userdata' in target.keys():
         userdata = target['userdata']
     else:

tests/letstest/scripts/test_leauto_upgrades.sh

@@ -23,18 +23,8 @@ if command -v python && [ $(python -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | se
     INITIAL_VERSION="0.20.0"
     RUN_RHEL6_TESTS=1
 else
-    # 0.33.x is the oldest version of letsencrypt-auto that works on Fedora 29+.
-    INITIAL_VERSION="0.33.1"
-fi
-
-# If we're on RHEL 8, the initial version of certbot-auto will fail until we do
-# a release including https://github.com/certbot/certbot/pull/7240 and update
-# INITIAL_VERSION above to use a version containing this fix. This works around
-# the problem for now so we can successfully run tests on RHEL 8.
-RPM_DIST_NAME=`(. /etc/os-release 2> /dev/null && echo $ID) || echo "unknown"`
-RPM_DIST_VERSION=`(. /etc/os-release 2> /dev/null && echo $VERSION_ID) | cut -d '.' -f1 || echo "0"`
-if [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
-    sudo yum install python3-virtualenv -y
+    # 0.37.x is the oldest version of letsencrypt-auto that works on RHEL 8.
+    INITIAL_VERSION="0.37.1"
 fi
 
 git checkout -f "v$INITIAL_VERSION" letsencrypt-auto
@@ -127,6 +117,8 @@ if ! diff letsencrypt-auto letsencrypt-auto-source/letsencrypt-auto ; then
 fi
 
 if [ "$RUN_RHEL6_TESTS" = 1 ]; then
+    # Add the SCL python release to PATH in order to resolve python3 command
+    PATH="/opt/rh/rh-python36/root/usr/bin:$PATH"
     if ! command -v python3; then
         echo "Python3 wasn't properly installed"
         exit 1