test quietly

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

.azure-pipelines/INSTALL.md

@@ -0,0 +1,119 @@
+# Configuring Azure Pipelines with Certbot
+
+Let's begin. All pipelines are defined in `.azure-pipelines`. Currently there are two:
+* `.azure-pipelines/main.yml` is the main one, executed on PRs for master, and pushes to master,
+* `.azure-pipelines/advanced.yml` add installer testing on top of the main pipeline, and is executed for `test-*` branches, release branches, and nightly run for master.
+
+Several templates are defined in `.azure-pipelines/templates`. These YAML files aggregate common jobs configuration that can be reused in several pipelines.
+
+Unlike Travis, where CodeCov is working without any action required, CodeCov supports Azure Pipelines
+using the coverage-bash utility (not python-coverage for now) only if you provide the Codecov repo token
+using the `CODECOV_TOKEN` environment variable. So `CODECOV_TOKEN` needs to be set as a secured
+environment variable to allow the main pipeline to publish coverage reports to CodeCov.
+
+This INSTALL.md file explains how to configure Azure Pipelines with Certbot in order to execute the CI/CD logic defined in `.azure-pipelines` folder with it.
+During this installation step, warnings describing user access and legal comitments will be displayed like this:
+```
+!!! ACCESS REQUIRED !!!
+```
+
+This document suppose that the Azure DevOps organization is named _certbot_, and the Azure DevOps project is also _certbot_.
+
+## Useful links
+
+* https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema
+* https://www.azuredevopslabs.com/labs/azuredevops/github-integration/
+* https://docs.microsoft.com/en-us/azure/devops/pipelines/ecosystems/python?view=azure-devops
+
+## Prerequisites
+
+### Having a GitHub account
+
+Use your GitHub user for a normal GitHub account, or a user that has administrative rights to the GitHub organization if relevant.
+
+### Having an Azure DevOps account
+- Go to https://dev.azure.com/, click "Start free with GitHub"
+- Login to GitHub
+
+```
+!!! ACCESS REQUIRED !!!
+Personal user data (email + profile info, in read-only)
+```
+
+- Microsoft will create a Live account using the email referenced for the GitHub account. This account is also linked to GitHub account (meaning you can log it using GitHub authentication)
+- Proceed with account registration (birth date, country), add details about name and email contact
+
+```
+!!! ACCESS REQUIRED !!!
+Microsoft proposes to send commercial links to this mail
+Azure DevOps terms of service need to be accepted
+```
+
+_Logged to Azure DevOps, account is ready._
+
+### Installing Azure Pipelines to GitHub
+
+- On GitHub, go to Marketplace
+- Select Azure Pipeline, and "Set up a plan"
+- Select Free, then "Install it for free"
+- Click "Complete order and begin installation"
+
+```
+!!! ACCESS !!!
+Azure Pipeline needs RW on code, RO on metadata, RW on checks, commit statuses, deployments, issues, pull requests.
+RW access here is required to allow update of the pipelines YAML files from Azure DevOps interface, and to
+update the status of builds and PRs on GitHub side when Azure Pipelines are triggered.
+Note however that no admin access is defined here: this means that Azure Pipelines cannot do anything with
+protected branches, like master, and cannot modify the security context around this on GitHub.
+Access can be defined for all or only selected repositories, which is nice.
+```
+
+- Redirected to Azure DevOps, select the account created in _Having an Azure DevOps account_ section.
+- Select the organization, and click "Create a new project" (let's name it the same than the targeted github repo)
+- The Visibility is public, to profit from 10 parallel jobs
+
+```
+!!! ACCESS !!!
+Azure Pipelines needs access to the GitHub account (in term of being able to check it is valid), and the Resources shared between the GitHub account and Azure Pipelines.
+```
+
+_Done. We can move to pipelines configuration._
+
+## Import an existing pipelines from `.azure-pipelines` folder
+
+- On Azure DevOps, go to your organization (eg. _certbot_) then your project (eg. _certbot_)
+- Click "Pipelines" tab
+- Click "New pipeline"
+- Where is your code?: select "__Use the classic editor__"
+
+__Warning: Do not choose the GitHub option in Where is your code? section. Indeed, this option will trigger an OAuth
+grant permissions from Azure Pipelines to GitHub in order to setup a GitHub OAuth Application. The permissions asked
+then are way too large (admin level on almost everything), while the classic approach does not add any more
+permissions, and works perfectly well.__
+
+- Select GitHub in "Select your repository section", choose certbot/certbot in Repository, master in default branch.
+- Click on YAML option for "Select a template"
+- Choose a name for the pipeline (eg. test-pipeline), and browse to the actual pipeline YAML definition in the
+  "YAML file path" input (eg. `.azure-pipelines/test-pipeline.yml`)
+- Click "Save & queue", choose the master branch to build the first pipeline, and click "Save and run" button.
+
+_Done. Pipeline is operational. Repeat to add more pipelines from existing YAML files in `.azure-pipelines`._
+
+## Add a secret variable to a pipeline (like `CODECOV_TOKEN`)
+
+__NB: Following steps suppose that you already setup the YAML pipeline file to
+consume the secret variable that these steps will create as an environment variable.
+For a variable named `CODECOV_TOKEN` consuming the variable `codecov_token`,
+in the YAML file this setup would take the form of the following:
+```
+steps:
+    - script: ./do_something_that_consumes_CODECOV_TOKEN  # Eg. `codecov -F windows`
+      env:
+        CODECOV_TOKEN: $(codecov_token)
+```
+
+To set up a variable that is shared between pipelines, follow the instructions
+at
+https://docs.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups.
+When adding variables to a group, don't forget to tick "Keep this value secret"
+if it shouldn't be shared publcily.

.azure-pipelines/advanced.yml

@@ -0,0 +1,20 @@
+# Advanced pipeline for isolated checks and release purpose
+trigger:
+  - test-*
+  - '*.x'
+pr:
+  - test-*
+# This pipeline is also nightly run on master
+schedules:
+  - cron: "0 4 * * *"
+    displayName: Nightly build
+    branches:
+      include:
+      - master
+    always: true
+
+jobs:
+  # Any addition here should be reflected in the release pipeline.
+  # It is advised to declare all jobs here as templates to improve maintainability.
+  - template: templates/tests-suite.yml
+  - template: templates/installer-tests.yml

.azure-pipelines/main.yml

@@ -0,0 +1,12 @@
+trigger:
+  # apache-parser-v2 is a temporary branch for doing work related to
+  # rewriting the parser in the Apache plugin.
+  - apache-parser-v2
+  - master
+pr:
+  - apache-parser-v2
+  - master
+  - '*.x'
+
+jobs:
+  - template: templates/tests-suite.yml

.azure-pipelines/release.yml

@@ -0,0 +1,13 @@
+# Release pipeline to build and deploy Certbot for Windows for GitHub release tags
+trigger:
+  tags:
+    include:
+      - v*
+pr: none
+
+jobs:
+  # Any addition here should be reflected in the advanced pipeline.
+  # It is advised to declare all jobs here as templates to improve maintainability.
+  - template: templates/tests-suite.yml
+  - template: templates/installer-tests.yml
+  - template: templates/changelog.yml

.azure-pipelines/templates/changelog.yml

@@ -0,0 +1,14 @@
+jobs:
+  - job: changelog
+    pool:
+      vmImage: vs2017-win2016
+    steps:
+      - bash: |
+          CERTBOT_VERSION="$(cd certbot && python -c "import certbot; print(certbot.__version__)" && cd ~-)"
+          "${BUILD_REPOSITORY_LOCALPATH}\tools\extract_changelog.py" "${CERTBOT_VERSION}" >> "${BUILD_ARTIFACTSTAGINGDIRECTORY}/release_notes.md"
+        displayName: Prepare changelog
+      - task: PublishPipelineArtifact@1
+        inputs:
+          path: $(Build.ArtifactStagingDirectory)
+          artifact: changelog
+        displayName: Publish changelog

.azure-pipelines/templates/installer-tests.yml

@@ -0,0 +1,54 @@
+jobs:
+  - job: installer_build
+    pool:
+      vmImage: vs2017-win2016
+    steps:
+      - task: UsePythonVersion@0
+        inputs:
+          versionSpec: 3.7
+          architecture: x86
+          addToPath: true
+      - script: python windows-installer/construct.py
+        displayName: Build Certbot installer
+      - task: CopyFiles@2
+        inputs:
+          sourceFolder: $(System.DefaultWorkingDirectory)/windows-installer/build/nsis
+          contents: '*.exe'
+          targetFolder: $(Build.ArtifactStagingDirectory)
+      - task: PublishPipelineArtifact@1
+        inputs:
+          path: $(Build.ArtifactStagingDirectory)
+          artifact: windows-installer
+        displayName: Publish Windows installer
+  - job: installer_run
+    dependsOn: installer_build
+    strategy:
+      matrix:
+        win2019:
+          imageName: windows-2019
+        win2016:
+          imageName: vs2017-win2016
+        win2012r2:
+          imageName: vs2015-win2012r2
+    pool:
+      vmImage: $(imageName)
+    steps:
+      - task: DownloadPipelineArtifact@2
+        inputs:
+          artifact: windows-installer
+          path: $(Build.SourcesDirectory)/bin
+        displayName: Retrieve Windows installer
+      - script: $(Build.SourcesDirectory)\bin\certbot-beta-installer-win32.exe /S
+        displayName: Install Certbot
+      - powershell: Invoke-WebRequest https://www.python.org/ftp/python/3.8.1/python-3.8.1-amd64-webinstall.exe -OutFile C:\py3-setup.exe
+        displayName: Get Python
+      - script: C:\py3-setup.exe /quiet PrependPath=1 InstallAllUsers=1 Include_launcher=1 InstallLauncherAllUsers=1 Include_test=0 Include_doc=0 Include_dev=1 Include_debug=0 Include_tcltk=0 TargetDir=C:\py3
+        displayName: Install Python
+      - script: |
+          py -3 -m venv venv
+          venv\Scripts\python tools\pip_install.py -e certbot-ci
+        displayName: Prepare Certbot-CI
+      - script: |
+          set PATH=%ProgramFiles(x86)%\Certbot\bin;%PATH%
+          venv\Scripts\python -m pytest certbot-ci\certbot_integration_tests\certbot_tests -n 4
+        displayName: Run integration tests

.azure-pipelines/templates/tests-suite.yml

@@ -0,0 +1,38 @@
+jobs:
+  - job: test
+    pool:
+      vmImage: vs2017-win2016
+    strategy:
+      matrix:
+        py35:
+          PYTHON_VERSION: 3.5
+          TOXENV: py35
+        py37-cover:
+          PYTHON_VERSION: 3.7
+          TOXENV: py37-cover
+        integration-certbot:
+          PYTHON_VERSION: 3.7
+          TOXENV: integration-certbot
+          PYTEST_ADDOPTS: --numprocesses 4
+    variables:
+    - group: certbot-common
+    steps:
+    - task: UsePythonVersion@0
+      inputs:
+        versionSpec: $(PYTHON_VERSION)
+        addToPath: true
+    - script: python tools/pip_install.py -U tox coverage
+      displayName: Install dependencies
+    - script: python -m tox
+      displayName: Run tox
+    # We do not require codecov report upload to succeed. So to avoid to break the pipeline if
+    # something goes wrong, each command is suffixed with a command that hides any non zero exit
+    # codes and echoes an informative message instead.
+    - bash: |
+        curl -s https://codecov.io/bash -o codecov-bash || echo "Failed to download codecov-bash"
+        chmod +x codecov-bash || echo "Failed to apply execute permissions on codecov-bash"
+        ./codecov-bash -F windows  || echo "Codecov did not collect coverage reports"
+      condition: in(variables['TOXENV'], 'py37-cover', 'integration-certbot')
+      env:
+        CODECOV_TOKEN: $(codecov_token)
+      displayName: Publish coverage

.codecov.yml

@@ -6,13 +6,13 @@ coverage:
         flags: linux
         # Fixed target instead of auto set by #7173, can
         # be removed when flags in Codecov are added back.
-        target: 97.6
+        target: 97.4
         threshold: 0.1
         base: auto
       windows:
         flags: windows
         # Fixed target instead of auto set by #7173, can
         # be removed when flags in Codecov are added back.
-        target: 97.0
+        target: 97.4
         threshold: 0.1
         base: auto

.coveragerc

@@ -1,2 +1,5 @@
+[run]
+omit = */setup.py
+
 [report]
 omit = */setup.py

.gitignore

@@ -26,6 +26,7 @@ tags
 \#*#
 .idea
 .ropeproject
+.vscode
 
 # auth --cert-path --chain-path
 /*.pem

.isort.cfg

@@ -0,0 +1,7 @@
+[settings]
+skip_glob=venv*
+skip=letsencrypt-auto-source
+force_sort_within_sections=True
+force_single_line=True
+order_by_type=False
+line_length=400

.pylintrc

@@ -24,6 +24,11 @@ persistent=yes
 # usually to register additional checkers.
 load-plugins=linter_plugin
 
+# A comma-separated list of package or module names from where C extensions may
+# be loaded. Extensions are loading into the active Python interpreter and may
+# run arbitrary code.
+extension-pkg-whitelist=pywintypes,win32api,win32file,win32security
+
 
 [MESSAGES CONTROL]
 
@@ -41,10 +46,14 @@ load-plugins=linter_plugin
 # --enable=similarities". If you want to run only the classes checker, but have
 # no Warning level messages displayed, use"--disable=all --enable=classes
 # --disable=W"
-disable=fixme,locally-disabled,locally-enabled,abstract-class-not-used,abstract-class-little-used,bad-continuation,too-few-public-methods,no-self-use,invalid-name,too-many-instance-attributes,cyclic-import,duplicate-code
-# abstract-class-not-used cannot be disabled locally (at least in
-# pylint 1.4.1), same for abstract-class-little-used
-
+# CERTBOT COMMENT
+# 1) Once certbot codebase is claimed to be compatible exclusively with Python 3,
+#    the useless-object-inheritance check can be enabled again, and code fixed accordingly.
+# 2) Check unsubscriptable-object tends to create a lot of false positives. Let's disable it.
+#    See https://github.com/PyCQA/pylint/issues/1498.
+# 3) Same as point 2 for no-value-for-parameter.
+#    See https://github.com/PyCQA/pylint/issues/2820.
+disable=fixme,locally-disabled,locally-enabled,bad-continuation,no-self-use,invalid-name,cyclic-import,duplicate-code,design,import-outside-toplevel,useless-object-inheritance,unsubscriptable-object,no-value-for-parameter,no-else-return,no-else-raise,no-else-break,no-else-continue
 
 [REPORTS]
 
@@ -297,40 +306,6 @@ valid-classmethod-first-arg=cls
 valid-metaclass-classmethod-first-arg=mcs
 
 
-[DESIGN]
-
-# Maximum number of arguments for function / method
-max-args=6
-
-# Argument names that match this expression will be ignored. Default to name
-# with leading underscore
-ignored-argument-names=(unused)?_.*|dummy
-
-# Maximum number of locals for function / method body
-max-locals=15
-
-# Maximum number of return / yield for function / method body
-max-returns=6
-
-# Maximum number of branch for function / method body
-max-branches=12
-
-# Maximum number of statements in function / method body
-max-statements=50
-
-# Maximum number of parents for a class (see R0901).
-max-parents=12
-
-# Maximum number of attributes for a class (see R0902).
-max-attributes=7
-
-# Minimum number of public methods for a class (see R0903).
-min-public-methods=2
-
-# Maximum number of public methods for a class (see R0904).
-max-public-methods=20
-
-
 [EXCEPTIONS]
 
 # Exceptions that will emit a warning when being caught. Defaults to

.travis.yml

@@ -1,4 +1,5 @@
 language: python
+dist: xenial
 
 cache:
     directories:
@@ -8,6 +9,8 @@ before_script:
   - 'if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then ulimit -n 1024 ; fi'
   # On Travis, the fastest parallelization for integration tests has proved to be 4.
   - 'if [[ "$TOXENV" == *"integration"* ]]; then export PYTEST_ADDOPTS="--numprocesses 4"; fi'
+  # Use Travis retry feature for farm tests since they are flaky
+  - 'if [[ "$TOXENV" == "travis-test-farm"* ]]; then export TRAVIS_RETRY=travis_retry; fi'
   - export TOX_TESTENV_PASSENV=TRAVIS
 
 # Only build pushes to the master branch, PRs, and branches beginning with
@@ -37,38 +40,30 @@ matrix:
     # Main test suite
     - python: "2.7"
       env: ACME_SERVER=pebble TOXENV=integration
-      sudo: required
-      services: docker
       <<: *not-on-master
 
     # This job is always executed, including on master
     - python: "2.7"
       env: TOXENV=py27-cover FYI="py27 tests + code coverage"
 
-    - python: "2.7"
+    - python: "3.7"
       env: TOXENV=lint
       <<: *not-on-master
-    - python: "3.4"
-      env: TOXENV=mypy
-      <<: *not-on-master
     - python: "3.5"
       env: TOXENV=mypy
       <<: *not-on-master
     - python: "2.7"
+      # Ubuntu Trusty or older must be used because the oldest version of
+      # cryptography we support cannot be compiled against the version of
+      # OpenSSL in Xenial or newer.
+      dist: trusty
       env: TOXENV='py27-{acme,apache,certbot,dns,nginx}-oldest'
-      sudo: required
-      services: docker
       <<: *not-on-master
-    - python: "3.4"
-      env: TOXENV=py34
-      sudo: required
-      services: docker
+    - python: "3.5"
+      env: TOXENV=py35
       <<: *not-on-master
-    - python: "3.7"
-      dist: xenial
-      env: TOXENV=py37
-      sudo: required
-      services: docker
+    - python: "3.8"
+      env: TOXENV=py38
       <<: *not-on-master
     - sudo: required
       env: TOXENV=apache_compat
@@ -82,8 +77,6 @@ matrix:
       <<: *not-on-master
     - python: "2.7"
       env: TOXENV=apacheconftest-with-pebble
-      sudo: required
-      services: docker
       <<: *not-on-master
     - python: "2.7"
       env: TOXENV=nginxroundtrip
@@ -119,7 +112,6 @@ matrix:
         - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
       <<: *extended-test-suite
     - python: "3.7"
-      dist: xenial
       env: TOXENV=py37 CERTBOT_NO_PIN=1
       <<: *extended-test-suite
     - python: "2.7"
@@ -132,55 +124,48 @@ matrix:
       sudo: required
       services: docker
       <<: *extended-test-suite
-    - python: "2.7"
-      env: TOXENV=py27-certbot-oldest
-      <<: *extended-test-suite
-    - python: "2.7"
-      env: TOXENV=py27-nginx-oldest
-      <<: *extended-test-suite
     - python: "2.7"
       env: ACME_SERVER=boulder-v1 TOXENV=integration-certbot-oldest
+      # Ubuntu Trusty or older must be used because the oldest version of
+      # cryptography we support cannot be compiled against the version of
+      # OpenSSL in Xenial or newer.
+      dist: trusty
       sudo: required
       services: docker
       <<: *extended-test-suite
     - python: "2.7"
       env: ACME_SERVER=boulder-v2 TOXENV=integration-certbot-oldest
+      # Ubuntu Trusty or older must be used because the oldest version of
+      # cryptography we support cannot be compiled against the version of
+      # OpenSSL in Xenial or newer.
+      dist: trusty
       sudo: required
       services: docker
       <<: *extended-test-suite
     - python: "2.7"
       env: ACME_SERVER=boulder-v1 TOXENV=integration-nginx-oldest
+      # Ubuntu Trusty or older must be used because the oldest version of
+      # cryptography we support cannot be compiled against the version of
+      # OpenSSL in Xenial or newer.
+      dist: trusty
       sudo: required
       services: docker
       <<: *extended-test-suite
     - python: "2.7"
       env: ACME_SERVER=boulder-v2 TOXENV=integration-nginx-oldest
+      # Ubuntu Trusty or older must be used because the oldest version of
+      # cryptography we support cannot be compiled against the version of
+      # OpenSSL in Xenial or newer.
+      dist: trusty
       sudo: required
       services: docker
       <<: *extended-test-suite
-    - python: "3.4"
-      env: TOXENV=py34
-      <<: *extended-test-suite
-    - python: "3.5"
-      env: TOXENV=py35
-      <<: *extended-test-suite
     - python: "3.6"
       env: TOXENV=py36
       <<: *extended-test-suite
     - python: "3.7"
-      dist: xenial
       env: TOXENV=py37
       <<: *extended-test-suite
-    - python: "3.4"
-      env: ACME_SERVER=boulder-v1 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "3.4"
-      env: ACME_SERVER=boulder-v2 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
     - python: "3.5"
       env: ACME_SERVER=boulder-v1 TOXENV=integration
       sudo: required
@@ -202,17 +187,21 @@ matrix:
       services: docker
       <<: *extended-test-suite
     - python: "3.7"
-      dist: xenial
       env: ACME_SERVER=boulder-v1 TOXENV=integration
       sudo: required
       services: docker
       <<: *extended-test-suite
     - python: "3.7"
-      dist: xenial
       env: ACME_SERVER=boulder-v2 TOXENV=integration
       sudo: required
       services: docker
       <<: *extended-test-suite
+    - python: "3.8"
+      env: ACME_SERVER=boulder-v1 TOXENV=integration
+      <<: *extended-test-suite
+    - python: "3.8"
+      env: ACME_SERVER=boulder-v2 TOXENV=integration
+      <<: *extended-test-suite
     - sudo: required
       env: TOXENV=le_auto_jessie
       services: docker
@@ -221,6 +210,10 @@ matrix:
       env: TOXENV=le_auto_centos6
       services: docker
       <<: *extended-test-suite
+    - sudo: required
+      env: TOXENV=le_auto_oraclelinux6
+      services: docker
+      <<: *extended-test-suite
     - sudo: required
       env: TOXENV=docker_dev
       services: docker
@@ -232,9 +225,6 @@ matrix:
     - language: generic
       env: TOXENV=py27
       os: osx
-      # Using this osx_image is a workaround for
-      # https://travis-ci.community/t/xcode-8-3-homebrew-outdated-error/3798.
-      osx_image: xcode10.2
       addons:
         homebrew:
           packages:
@@ -244,9 +234,6 @@ matrix:
     - language: generic
       env: TOXENV=py3
       os: osx
-      # Using this osx_image is a workaround for
-      # https://travis-ci.community/t/xcode-8-3-homebrew-outdated-error/3798.
-      osx_image: xcode10.2
       addons:
         homebrew:
           packages:
@@ -275,20 +262,14 @@ addons:
 # virtualenv is listed here explicitly to make sure it is upgraded when
 # CERTBOT_NO_PIN is set to work around failures we've seen when using an older
 # version of virtualenv.
-install: "tools/pip_install.py -U codecov tox virtualenv"
-script: tox
+install: 'tools/pip_install.py -U codecov tox virtualenv'
+# Most of the time TRAVIS_RETRY is an empty string, and has no effect on the
+# script command. It is set only to `travis_retry` during farm tests, in
+# order to trigger the Travis retry feature, and compensate the inherent
+# flakiness of these specific tests.
+script: '$TRAVIS_RETRY tox'
 
 after_success: '[ "$TOXENV" == "py27-cover" ] && codecov -F linux'
 
 notifications:
   email: false
-  irc:
-    channels:
-      # This is set to a secure variable to prevent forks from sending
-      # notifications. This value was created by installing
-      # https://github.com/travis-ci/travis.rb and running
-      # `travis encrypt "chat.freenode.net#certbot-devel"`.
-      - secure: "EWW66E2+KVPZyIPR8ViENZwfcup4Gx3/dlimmAZE0WuLwxDCshBBOd3O8Rf6pBokEoZlXM5eDT6XdyJj8n0DLslgjO62pExdunXpbcMwdY7l1ELxX2/UbnDTE6UnPYa09qVBHNG7156Z6yE0x2lH4M9Ykvp0G0cubjPQHylAwo0="
-    on_cancel: never
-    on_success: never
-    on_failure: always

AUTHORS.md

@@ -15,8 +15,10 @@ Authors
 * [Alex Gaynor](https://github.com/alex)
 * [Alex Halderman](https://github.com/jhalderm)
 * [Alex Jordan](https://github.com/strugee)
+* [Alex Zorin](https://github.com/alexzorin)
 * [Amjad Mashaal](https://github.com/TheNavigat)
 * [Andrew Murray](https://github.com/radarhere)
+* [Andrzej Górski](https://github.com/andrzej3393)
 * [Anselm Levskaya](https://github.com/levskaya)
 * [Antoine Jacoutot](https://github.com/ajacoutot)
 * [asaph](https://github.com/asaph)
@@ -126,6 +128,7 @@ Authors
 * [Joubin Jabbari](https://github.com/joubin)
 * [Juho Juopperi](https://github.com/jkjuopperi)
 * [Kane York](https://github.com/riking)
+* [Kenichi Maehashi](https://github.com/kmaehashi)
 * [Kenneth Skovhede](https://github.com/kenkendk)
 * [Kevin Burke](https://github.com/kevinburke)
 * [Kevin London](https://github.com/kevinlondon)
@@ -161,8 +164,10 @@ Authors
 * [Michael Schumacher](https://github.com/schumaml)
 * [Michael Strache](https://github.com/Jarodiv)
 * [Michael Sverdlin](https://github.com/sveder)
+* [Michael Watters](https://github.com/blackknight36)
 * [Michal Moravec](https://github.com/https://github.com/Majkl578)
 * [Michal Papis](https://github.com/mpapis)
+* [Mickaël Schoentgen](https://github.com/BoboTiG)
 * [Minn Soe](https://github.com/MinnSoe)
 * [Min RK](https://github.com/minrk)
 * [Miquel Ruiz](https://github.com/miquelruiz)
@@ -226,6 +231,7 @@ Authors
 * [Stavros Korokithakis](https://github.com/skorokithakis)
 * [Stefan Weil](https://github.com/stweil)
 * [Steve Desmond](https://github.com/stevedesmond-ca)
+* [sydneyli](https://github.com/sydneyli)
 * [Tan Jay Jun](https://github.com/jayjun)
 * [Tapple Gao](https://github.com/tapple)
 * [Telepenin Nikolay](https://github.com/telepenin)

CHANGELOG.md

@@ -0,0 +1 @@
+certbot/CHANGELOG.md

Dockerfile-dev

@@ -6,16 +6,15 @@ EXPOSE 80 443
 
 WORKDIR /opt/certbot/src
 
-# TODO: Install Apache/Nginx for plugin development.
 COPY . .
 RUN apt-get update && \
-    apt-get install apache2 git nginx-light -y && \
-    letsencrypt-auto-source/letsencrypt-auto --os-packages-only && \
+    apt-get install apache2 git python3-dev python3-venv gcc libaugeas0 \
+                    libssl-dev libffi-dev ca-certificates openssl nginx-light -y && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/* \
            /tmp/* \
            /var/tmp/*
 
-RUN VENV_NAME="../venv" python tools/venv.py
+RUN VENV_NAME="../venv3" python3 tools/venv3.py
 
-ENV PATH /opt/certbot/venv/bin:$PATH
+ENV PATH /opt/certbot/venv3/bin:$PATH

README.rst

@@ -1,131 +0,0 @@
-.. This file contains a series of comments that are used to include sections of this README in other files. Do not modify these comments unless you know what you are doing. tag:intro-begin
-
-Certbot is part of EFF’s effort to encrypt the entire Internet. Secure communication over the Web relies on HTTPS, which requires the use of a digital certificate that lets browsers verify the identity of web servers (e.g., is that really google.com?). Web servers obtain their certificates from trusted third parties called certificate authorities (CAs). Certbot is an easy-to-use client that fetches a certificate from Let’s Encrypt—an open certificate authority launched by the EFF, Mozilla, and others—and deploys it to a web server.
-
-Anyone who has gone through the trouble of setting up a secure website knows what a hassle getting and maintaining a certificate is. Certbot and Let’s Encrypt can automate away the pain and let you turn on and manage HTTPS with simple commands. Using Certbot and Let's Encrypt is free, so there’s no need to arrange payment.
-
-How you use Certbot depends on the configuration of your web server. The best way to get started is to use our `interactive guide <https://certbot.eff.org>`_. It generates instructions based on your configuration settings. In most cases, you’ll need `root or administrator access <https://certbot.eff.org/faq/#does-certbot-require-root-administrator-privileges>`_ to your web server to run Certbot.
-
-Certbot is meant to be run directly on your web server, not on your personal computer. If you’re using a hosted service and don’t have direct access to your web server, you might not be able to use Certbot. Check with your hosting provider for documentation about uploading certificates or using certificates issued by Let’s Encrypt.
-
-Certbot is a fully-featured, extensible client for the Let's
-Encrypt CA (or any other CA that speaks the `ACME
-<https://github.com/ietf-wg-acme/acme/blob/master/draft-ietf-acme-acme.md>`_
-protocol) that can automate the tasks of obtaining certificates and
-configuring webservers to use them. This client runs on Unix-based operating
-systems.
-
-To see the changes made to Certbot between versions please refer to our
-`changelog <https://github.com/certbot/certbot/blob/master/CHANGELOG.md>`_.
-
-Until May 2016, Certbot was named simply ``letsencrypt`` or ``letsencrypt-auto``,
-depending on install method. Instructions on the Internet, and some pieces of the
-software, may still refer to this older name.
-
-Contributing
-------------
-
-If you'd like to contribute to this project please read `Developer Guide
-<https://certbot.eff.org/docs/contributing.html>`_.
-
-This project is governed by `EFF's Public Projects Code of Conduct <https://www.eff.org/pages/eppcode>`_.
-
-.. _installation:
-
-How to run the client
----------------------
-
-The easiest way to install and run Certbot is by visiting `certbot.eff.org`_,
-where you can find the correct instructions for many web server and OS
-combinations.  For more information, see `Get Certbot
-<https://certbot.eff.org/docs/install.html>`_.
-
-.. _certbot.eff.org: https://certbot.eff.org/
-
-Understanding the client in more depth
---------------------------------------
-
-To understand what the client is doing in detail, it's important to
-understand the way it uses plugins.  Please see the `explanation of
-plugins <https://certbot.eff.org/docs/using.html#plugins>`_ in
-the User Guide.
-
-Links
-=====
-
-.. Do not modify this comment unless you know what you're doing. tag:links-begin
-
-Documentation: https://certbot.eff.org/docs
-
-Software project: https://github.com/certbot/certbot
-
-Notes for developers: https://certbot.eff.org/docs/contributing.html
-
-Main Website: https://certbot.eff.org
-
-Let's Encrypt Website: https://letsencrypt.org
-
-Community: https://community.letsencrypt.org
-
-ACME spec: http://ietf-wg-acme.github.io/acme/
-
-ACME working area in github: https://github.com/ietf-wg-acme/acme
-
-|build-status| |coverage| |docs| |container|
-
-.. |build-status| image:: https://travis-ci.com/certbot/certbot.svg?branch=master
-   :target: https://travis-ci.com/certbot/certbot
-   :alt: Travis CI status
-
-.. |coverage| image:: https://codecov.io/gh/certbot/certbot/branch/master/graph/badge.svg
-   :target: https://codecov.io/gh/certbot/certbot
-   :alt: Coverage status
-
-.. |docs| image:: https://readthedocs.org/projects/letsencrypt/badge/
-   :target: https://readthedocs.org/projects/letsencrypt/
-   :alt: Documentation status
-
-.. |container| image:: https://quay.io/repository/letsencrypt/letsencrypt/status
-   :target: https://quay.io/repository/letsencrypt/letsencrypt
-   :alt: Docker Repository on Quay.io
-
-.. Do not modify this comment unless you know what you're doing. tag:links-end
-
-System Requirements
-===================
-
-See https://certbot.eff.org/docs/install.html#system-requirements.
-
-.. Do not modify this comment unless you know what you're doing. tag:intro-end
-
-.. Do not modify this comment unless you know what you're doing. tag:features-begin
-
-Current Features
-=====================
-
-* Supports multiple web servers:
-
-  - apache/2.x
-  - nginx/0.8.48+
-  - webroot (adds files to webroot directories in order to prove control of
-    domains and obtain certs)
-  - standalone (runs its own simple webserver to prove you control a domain)
-  - other server software via `third party plugins <https://certbot.eff.org/docs/using.html#third-party-plugins>`_
-
-* The private key is generated locally on your system.
-* Can talk to the Let's Encrypt CA or optionally to other ACME
-  compliant services.
-* Can get domain-validated (DV) certificates.
-* Can revoke certificates.
-* Adjustable RSA key bit-length (2048 (default), 4096, ...).
-* Can optionally install a http -> https redirect, so your site effectively
-  runs https only (Apache only)
-* Fully automated.
-* Configuration changes are logged and can be reverted.
-* Supports an interactive text UI, or can be driven entirely from the
-  command line.
-* Free and Open Source Software, made with Python.
-
-.. Do not modify this comment unless you know what you're doing. tag:features-end
-
-For extensive documentation on using and contributing to Certbot, go to https://certbot.eff.org/docs. If you would like to contribute to the project or run the latest code from git, you should read our `developer guide <https://certbot.eff.org/docs/contributing.html>`_.

README.rst

@@ -0,0 +1 @@
+certbot/README.rst

acme/MANIFEST.in

@@ -3,4 +3,6 @@ include README.rst
 include pytest.ini
 recursive-include docs *
 recursive-include examples *
-recursive-include acme/testdata *
+recursive-include tests *
+global-exclude __pycache__
+global-exclude *.py[cod]

acme/acme/__init__.py

@@ -13,7 +13,6 @@ import warnings
 #
 # It is based on
 # https://github.com/requests/requests/blob/1278ecdf71a312dc2268f3bfc0aabfab3c006dcf/requests/packages.py
-
 import josepy as jose
 
 for mod in list(sys.modules):
@@ -21,30 +20,3 @@ for mod in list(sys.modules):
     # preserved (acme.jose.* is josepy.*)
     if mod == 'josepy' or mod.startswith('josepy.'):
         sys.modules['acme.' + mod.replace('josepy', 'jose', 1)] = sys.modules[mod]
-
-
-# This class takes a similar approach to the cryptography project to deprecate attributes
-# in public modules. See the _ModuleWithDeprecation class here:
-# https://github.com/pyca/cryptography/blob/91105952739442a74582d3e62b3d2111365b0dc7/src/cryptography/utils.py#L129
-class _TLSSNI01DeprecationModule(object):
-    """
-    Internal class delegating to a module, and displaying warnings when
-    attributes related to TLS-SNI-01 are accessed.
-    """
-    def __init__(self, module):
-        self.__dict__['_module'] = module
-
-    def __getattr__(self, attr):
-        if 'TLSSNI01' in attr:
-            warnings.warn('{0} attribute is deprecated, and will be removed soon.'.format(attr),
-                          DeprecationWarning, stacklevel=2)
-        return getattr(self._module, attr)
-
-    def __setattr__(self, attr, value):  # pragma: no cover
-        setattr(self._module, attr, value)
-
-    def __delattr__(self, attr):  # pragma: no cover
-        delattr(self._module, attr)
-
-    def __dir__(self):  # pragma: no cover
-        return ['_module'] + dir(self._module)

acme/acme/challenges.py

@@ -3,19 +3,13 @@ import abc
 import functools
 import hashlib
 import logging
-import socket
-import sys
 
 from cryptography.hazmat.primitives import hashes  # type: ignore
 import josepy as jose
-import OpenSSL
 import requests
 import six
 
-from acme import errors
-from acme import crypto_util
 from acme import fields
-from acme import _TLSSNI01DeprecationModule
 
 logger = logging.getLogger(__name__)
 
@@ -60,8 +54,7 @@ class UnrecognizedChallenge(Challenge):
         object.__setattr__(self, "jobj", jobj)
 
     def to_partial_json(self):
-        # pylint: disable=no-member
-        return self.jobj
+        return self.jobj  # pylint: disable=no-member
 
     @classmethod
     def from_json(cls, jobj):
@@ -119,7 +112,7 @@ class KeyAuthorizationChallengeResponse(ChallengeResponse):
         :rtype: bool
 
         """
-        parts = self.key_authorization.split('.')  # pylint: disable=no-member
+        parts = self.key_authorization.split('.')
         if len(parts) != 2:
             logger.debug("Key authorization (%r) is not well formed",
                          self.key_authorization)
@@ -237,7 +230,7 @@ class DNS01Response(KeyAuthorizationChallengeResponse):
         return verified
 
 
-@Challenge.register  # pylint: disable=too-many-ancestors
+@Challenge.register
 class DNS01(KeyAuthorizationChallenge):
     """ACME dns-01 challenge."""
     response_cls = DNS01Response
@@ -327,7 +320,7 @@ class HTTP01Response(KeyAuthorizationChallengeResponse):
         return True
 
 
-@Challenge.register  # pylint: disable=too-many-ancestors
+@Challenge.register
 class HTTP01(KeyAuthorizationChallenge):
     """ACME http-01 challenge."""
     response_cls = HTTP01Response
@@ -367,148 +360,6 @@ class HTTP01(KeyAuthorizationChallenge):
         return self.key_authorization(account_key)
 
 
-@ChallengeResponse.register
-class TLSSNI01Response(KeyAuthorizationChallengeResponse):
-    """ACME tls-sni-01 challenge response."""
-    typ = "tls-sni-01"
-
-    DOMAIN_SUFFIX = b".acme.invalid"
-    """Domain name suffix."""
-
-    PORT = 443
-    """Verification port as defined by the protocol.
-
-    You can override it (e.g. for testing) by passing ``port`` to
-    `simple_verify`.
-
-    """
-
-    @property
-    def z(self):  # pylint: disable=invalid-name
-        """``z`` value used for verification.
-
-        :rtype bytes:
-
-        """
-        return hashlib.sha256(
-            self.key_authorization.encode("utf-8")).hexdigest().lower().encode()
-
-    @property
-    def z_domain(self):
-        """Domain name used for verification, generated from `z`.
-
-        :rtype bytes:
-
-        """
-        return self.z[:32] + b'.' + self.z[32:] + self.DOMAIN_SUFFIX
-
-    def gen_cert(self, key=None, bits=2048):
-        """Generate tls-sni-01 certificate.
-
-        :param OpenSSL.crypto.PKey key: Optional private key used in
-            certificate generation. If not provided (``None``), then
-            fresh key will be generated.
-        :param int bits: Number of bits for newly generated key.
-
-        :rtype: `tuple` of `OpenSSL.crypto.X509` and `OpenSSL.crypto.PKey`
-
-        """
-        if key is None:
-            key = OpenSSL.crypto.PKey()
-            key.generate_key(OpenSSL.crypto.TYPE_RSA, bits)
-        return crypto_util.gen_ss_cert(key, [
-            # z_domain is too big to fit into CN, hence first dummy domain
-            'dummy', self.z_domain.decode()], force_san=True), key
-
-    def probe_cert(self, domain, **kwargs):
-        """Probe tls-sni-01 challenge certificate.
-
-        :param unicode domain:
-
-        """
-        # TODO: domain is not necessary if host is provided
-        if "host" not in kwargs:
-            host = socket.gethostbyname(domain)
-            logger.debug('%s resolved to %s', domain, host)
-            kwargs["host"] = host
-
-        kwargs.setdefault("port", self.PORT)
-        kwargs["name"] = self.z_domain
-        # TODO: try different methods?
-        return crypto_util.probe_sni(**kwargs)
-
-    def verify_cert(self, cert):
-        """Verify tls-sni-01 challenge certificate.
-
-        :param OpensSSL.crypto.X509 cert: Challenge certificate.
-
-        :returns: Whether the certificate was successfully verified.
-        :rtype: bool
-
-        """
-        # pylint: disable=protected-access
-        sans = crypto_util._pyopenssl_cert_or_req_san(cert)
-        logger.debug('Certificate %s. SANs: %s', cert.digest('sha256'), sans)
-        return self.z_domain.decode() in sans
-
-    def simple_verify(self, chall, domain, account_public_key,
-                      cert=None, **kwargs):
-        """Simple verify.
-
-        Verify ``validation`` using ``account_public_key``, optionally
-        probe tls-sni-01 certificate and check using `verify_cert`.
-
-        :param .challenges.TLSSNI01 chall: Corresponding challenge.
-        :param str domain: Domain name being validated.
-        :param JWK account_public_key:
-        :param OpenSSL.crypto.X509 cert: Optional certificate. If not
-            provided (``None``) certificate will be retrieved using
-            `probe_cert`.
-        :param int port: Port used to probe the certificate.
-
-
-        :returns: ``True`` iff client's control of the domain has been
-            verified.
-        :rtype: bool
-
-        """
-        if not self.verify(chall, account_public_key):
-            logger.debug("Verification of key authorization in response failed")
-            return False
-
-        if cert is None:
-            try:
-                cert = self.probe_cert(domain=domain, **kwargs)
-            except errors.Error as error:
-                logger.debug(str(error), exc_info=True)
-                return False
-
-        return self.verify_cert(cert)
-
-
-@Challenge.register  # pylint: disable=too-many-ancestors
-class TLSSNI01(KeyAuthorizationChallenge):
-    """ACME tls-sni-01 challenge."""
-    response_cls = TLSSNI01Response
-    typ = response_cls.typ
-
-    # boulder#962, ietf-wg-acme#22
-    #n = jose.Field("n", encoder=int, decoder=int)
-
-    def validation(self, account_key, **kwargs):
-        """Generate validation.
-
-        :param JWK account_key:
-        :param OpenSSL.crypto.PKey cert_key: Optional private key used
-            in certificate generation. If not provided (``None``), then
-            fresh key will be generated.
-
-        :rtype: `tuple` of `OpenSSL.crypto.X509` and `OpenSSL.crypto.PKey`
-
-        """
-        return self.response(account_key).gen_cert(key=kwargs.get('cert_key'))
-
-
 @ChallengeResponse.register
 class TLSALPN01Response(KeyAuthorizationChallengeResponse):
     """ACME TLS-ALPN-01 challenge response.
@@ -520,7 +371,7 @@ class TLSALPN01Response(KeyAuthorizationChallengeResponse):
     typ = "tls-alpn-01"
 
 
-@Challenge.register  # pylint: disable=too-many-ancestors
+@Challenge.register
 class TLSALPN01(KeyAuthorizationChallenge):
     """ACME tls-alpn-01 challenge.
 
@@ -617,7 +468,3 @@ class DNSResponse(ChallengeResponse):
 
         """
         return chall.check_validation(self.validation, account_public_key)
-
-
-# Patching ourselves to warn about TLS-SNI challenge deprecation and removal.
-sys.modules[__name__] = _TLSSNI01DeprecationModule(sys.modules[__name__])

acme/acme/client.py

@@ -5,25 +5,26 @@ import datetime
 from email.utils import parsedate_tz
 import heapq
 import logging
-import time
 import re
 import sys
+import time
 
-import six
-from six.moves import http_client  # pylint: disable=import-error
 import josepy as jose
 import OpenSSL
 import requests
 from requests.adapters import HTTPAdapter
 from requests_toolbelt.adapters.source import SourceAddressAdapter
+import six
+from six.moves import http_client  # pylint: disable=import-error
 
 from acme import crypto_util
 from acme import errors
 from acme import jws
 from acme import messages
-# pylint: disable=unused-import, no-name-in-module
-from acme.magic_typing import Dict, List, Set, Text
-
+from acme.magic_typing import Dict  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Set  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Text  # pylint: disable=unused-import, no-name-in-module
 
 logger = logging.getLogger(__name__)
 
@@ -33,7 +34,6 @@ logger = logging.getLogger(__name__)
 # https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning
 if sys.version_info < (2, 7, 9):  # pragma: no cover
     try:
-        # pylint: disable=no-member
         requests.packages.urllib3.contrib.pyopenssl.inject_into_urllib3()  # type: ignore
     except AttributeError:
         import urllib3.contrib.pyopenssl  # pylint: disable=import-error
@@ -44,7 +44,7 @@ DEFAULT_NETWORK_TIMEOUT = 45
 DER_CONTENT_TYPE = 'application/pkix-cert'
 
 
-class ClientBase(object):  # pylint: disable=too-many-instance-attributes
+class ClientBase(object):
     """ACME client base object.
 
     :ivar messages.Directory directory:
@@ -123,6 +123,22 @@ class ClientBase(object):  # pylint: disable=too-many-instance-attributes
         """
         return self.update_registration(regr, update={'status': 'deactivated'})
 
+    def deactivate_authorization(self, authzr):
+        # type: (messages.AuthorizationResource) -> messages.AuthorizationResource
+        """Deactivate authorization.
+
+        :param messages.AuthorizationResource authzr: The Authorization resource
+            to be deactivated.
+
+        :returns: The Authorization resource that was deactivated.
+        :rtype: `.AuthorizationResource`
+
+        """
+        body = messages.UpdateAuthorization(status='deactivated')
+        response = self._post(authzr.uri, body)
+        return self._authzr_from_response(response,
+            authzr.body.identifier, authzr.uri)
+
     def _authzr_from_response(self, response, identifier=None, uri=None):
         authzr = messages.AuthorizationResource(
             body=messages.Authorization.from_json(response.json()),
@@ -238,7 +254,6 @@ class Client(ClientBase):
             URI from which the resource will be downloaded.
 
         """
-        # pylint: disable=too-many-arguments
         self.key = key
         if net is None:
             net = ClientNetwork(key, alg=alg, verify_ssl=verify_ssl)
@@ -264,7 +279,6 @@ class Client(ClientBase):
         assert response.status_code == http_client.CREATED
 
         # "Instance of 'Field' has no key/contact member" bug:
-        # pylint: disable=no-member
         return self._regr_from_response(response)
 
     def query_registration(self, regr):
@@ -419,7 +433,6 @@ class Client(ClientBase):
             was marked by the CA as invalid
 
         """
-        # pylint: disable=too-many-locals
         assert max_attempts > 0
         attempts = collections.defaultdict(int) # type: Dict[messages.AuthorizationResource, int]
         exhausted = set()
@@ -450,7 +463,6 @@ class Client(ClientBase):
             updated[authzr] = updated_authzr
 
             attempts[authzr] += 1
-            # pylint: disable=no-member
             if updated_authzr.body.status not in (
                     messages.STATUS_VALID, messages.STATUS_INVALID):
                 if attempts[authzr] < max_attempts:
@@ -591,7 +603,6 @@ class ClientV2(ClientBase):
         if response.status_code == 200 and 'Location' in response.headers:
             raise errors.ConflictError(response.headers.get('Location'))
         # "Instance of 'Field' has no key/contact member" bug:
-        # pylint: disable=no-member
         regr = self._regr_from_response(response)
         self.net.account = regr
         return regr
@@ -715,7 +726,7 @@ class ClientV2(ClientBase):
         for authzr in responses:
             if authzr.body.status != messages.STATUS_VALID:
                 for chall in authzr.body.challenges:
-                    if chall.error != None:
+                    if chall.error is not None:
                         failed.append(authzr)
         if failed:
             raise errors.ValidationError(failed)
@@ -765,29 +776,13 @@ class ClientV2(ClientBase):
 
     def _post_as_get(self, *args, **kwargs):
         """
-        Send GET request using the POST-as-GET protocol if needed.
-        The request will be first issued using POST-as-GET for ACME v2. If the ACME CA servers do
-        not support this yet and return an error, request will be retried using GET.
-        For ACME v1, only GET request will be tried, as POST-as-GET is not supported.
+        Send GET request using the POST-as-GET protocol.
         :param args:
         :param kwargs:
         :return:
         """
-        if self.acme_version >= 2:
-            # We add an empty payload for POST-as-GET requests
-            new_args = args[:1] + (None,) + args[1:]
-            try:
-                return self._post(*new_args, **kwargs)
-            except messages.Error as error:
-                if error.code == 'malformed':
-                    logger.debug('Error during a POST-as-GET request, '
-                                 'your ACME CA server may not support it:\n%s', error)
-                    logger.debug('Retrying request with GET.')
-                else:  # pragma: no cover
-                    raise
-
-        # If POST-as-GET is not supported yet, we use a GET instead.
-        return self.net.get(*args, **kwargs)
+        new_args = args[:1] + (None,) + args[1:]
+        return self._post(*new_args, **kwargs)
 
 
 class BackwardsCompatibleClientV2(object):
@@ -931,7 +926,7 @@ class BackwardsCompatibleClientV2(object):
         return self.client.external_account_required()
 
 
-class ClientNetwork(object):  # pylint: disable=too-many-instance-attributes
+class ClientNetwork(object):
     """Wrapper around requests that signs POSTs for authentication.
 
     Also adds user agent, and handles Content-Type.
@@ -947,7 +942,7 @@ class ClientNetwork(object):  # pylint: disable=too-many-instance-attributes
     :param messages.RegistrationResource account: Account object. Required if you are
             planning to use .post() with acme_version=2 for anything other than
             creating a new account; may be set later after registering.
-    :param josepy.JWASignature alg: Algoritm to use in signing JWS.
+    :param josepy.JWASignature alg: Algorithm to use in signing JWS.
     :param bool verify_ssl: Whether to verify certificates on SSL connections.
     :param str user_agent: String to send as User-Agent header.
     :param float timeout: Timeout for requests.
@@ -957,7 +952,6 @@ class ClientNetwork(object):  # pylint: disable=too-many-instance-attributes
     def __init__(self, key, account=None, alg=jose.RS256, verify_ssl=True,
                  user_agent='acme-python', timeout=DEFAULT_NETWORK_TIMEOUT,
                  source_address=None):
-        # pylint: disable=too-many-arguments
         self.key = key
         self.account = account
         self.alg = alg
@@ -1065,7 +1059,6 @@ class ClientNetwork(object):  # pylint: disable=too-many-instance-attributes
         return response
 
     def _send_request(self, method, url, *args, **kwargs):
-        # pylint: disable=too-many-locals
         """Send HTTP request.
 
         Makes sure that `verify_ssl` is respected. Logs request and
@@ -1112,10 +1105,9 @@ class ClientNetwork(object):  # pylint: disable=too-many-instance-attributes
             err_regex = r".*host='(\S*)'.*Max retries exceeded with url\: (\/\w*).*(\[Errno \d+\])([A-Za-z ]*)"
             m = re.match(err_regex, str(e))
             if m is None:
-                raise # pragma: no cover
-            else:
-                host, path, _err_no, err_msg = m.groups()
-                raise ValueError("Requesting {0}{1}:{2}".format(host, path, err_msg))
+                raise  # pragma: no cover
+            host, path, _err_no, err_msg = m.groups()
+            raise ValueError("Requesting {0}{1}:{2}".format(host, path, err_msg))
 
         # If content is DER, log the base64 of it instead of raw bytes, to keep
         # binary data out of the logs.
@@ -1181,8 +1173,7 @@ class ClientNetwork(object):  # pylint: disable=too-many-instance-attributes
             if error.code == 'badNonce':
                 logger.debug('Retrying request after error:\n%s', error)
                 return self._post_once(*args, **kwargs)
-            else:
-                raise
+            raise
 
     def _post_once(self, url, obj, content_type=JOSE_CONTENT_TYPE,
             acme_version=1, **kwargs):

acme/acme/crypto_util.py

@@ -6,15 +6,15 @@ import os
 import re
 import socket
 
-from OpenSSL import crypto
-from OpenSSL import SSL # type: ignore # https://github.com/python/typeshed/issues/2052
 import josepy as jose
+from OpenSSL import crypto
+from OpenSSL import SSL  # type: ignore # https://github.com/python/typeshed/issues/2052
 
 from acme import errors
-# pylint: disable=unused-import, no-name-in-module
-from acme.magic_typing import Callable, Union, Tuple, Optional
-# pylint: enable=unused-import, no-name-in-module
-
+from acme.magic_typing import Callable  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Optional  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Tuple  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Union  # pylint: disable=unused-import, no-name-in-module
 
 logger = logging.getLogger(__name__)
 
@@ -28,7 +28,7 @@ logger = logging.getLogger(__name__)
 _DEFAULT_SSL_METHOD = SSL.SSLv23_METHOD  # type: ignore
 
 
-class SSLSocket(object):  # pylint: disable=too-few-public-methods
+class SSLSocket(object):
     """SSL wrapper for sockets.
 
     :ivar socket sock: Original wrapped socket.
@@ -74,7 +74,7 @@ class SSLSocket(object):  # pylint: disable=too-few-public-methods
     class FakeConnection(object):
         """Fake OpenSSL.SSL.Connection."""
 
-        # pylint: disable=too-few-public-methods,missing-docstring
+        # pylint: disable=missing-docstring
 
         def __init__(self, connection):
             self._wrapped = connection

acme/acme/errors.py

@@ -29,7 +29,12 @@ class NonceError(ClientError):
 class BadNonce(NonceError):
     """Bad nonce error."""
     def __init__(self, nonce, error, *args, **kwargs):
-        super(BadNonce, self).__init__(*args, **kwargs)
+        # MyPy complains here that there is too many arguments for BaseException constructor.
+        # This is an error fixed in typeshed, see https://github.com/python/mypy/issues/4183
+        # The fix is included in MyPy>=0.740, but upgrading it would bring dozen of errors due to
+        #   new types definitions. So we ignore the error until the code base is fixed to match
+        #   with MyPy>=0.740 referential.
+        super(BadNonce, self).__init__(*args, **kwargs)  # type: ignore
         self.nonce = nonce
         self.error = error
 
@@ -48,7 +53,8 @@ class MissingNonce(NonceError):
 
     """
     def __init__(self, response, *args, **kwargs):
-        super(MissingNonce, self).__init__(*args, **kwargs)
+        # See comment in BadNonce constructor above for an explanation of type: ignore here.
+        super(MissingNonce, self).__init__(*args, **kwargs)  # type: ignore
         self.response = response
 
     def __str__(self):
@@ -83,6 +89,7 @@ class PollError(ClientError):
         return '{0}(exhausted={1!r}, updated={2!r})'.format(
             self.__class__.__name__, self.exhausted, self.updated)
 
+
 class ValidationError(Error):
     """Error for authorization failures. Contains a list of authorization
     resources, each of which is invalid and should have an error field.
@@ -91,9 +98,11 @@ class ValidationError(Error):
         self.failed_authzrs = failed_authzrs
         super(ValidationError, self).__init__()
 
-class TimeoutError(Error):
+
+class TimeoutError(Error):  # pylint: disable=redefined-builtin
     """Error for when polling an authorization or an order times out."""
 
+
 class IssuanceError(Error):
     """Error sent by the server after requesting issuance of a certificate."""
 
@@ -105,6 +114,7 @@ class IssuanceError(Error):
         self.error = error
         super(IssuanceError, self).__init__()
 
+
 class ConflictError(ClientError):
     """Error for when the server returns a 409 (Conflict) HTTP status.
 

acme/acme/fields.py

@@ -4,7 +4,6 @@ import logging
 import josepy as jose
 import pyrfc3339
 
-
 logger = logging.getLogger(__name__)
 
 

acme/acme/jws.py

@@ -40,10 +40,10 @@ class Signature(jose.Signature):
 class JWS(jose.JWS):
     """ACME-specific JWS. Includes none, url, and kid in protected header."""
     signature_cls = Signature
-    __slots__ = jose.JWS._orig_slots  # pylint: disable=no-member
+    __slots__ = jose.JWS._orig_slots
 
     @classmethod
-    # pylint: disable=arguments-differ,too-many-arguments
+    # pylint: disable=arguments-differ
     def sign(cls, payload, key, alg, nonce, url=None, kid=None):
         # Per ACME spec, jwk and kid are mutually exclusive, so only include a
         # jwk field if kid is not provided.

acme/acme/magic_typing.py

@@ -1,6 +1,7 @@
 """Shim class to not have to depend on typing module in prod."""
 import sys
 
+
 class TypingClass(object):
     """Ignore import errors by getting anything"""
     def __getattr__(self, name):

acme/acme/messages.py

@@ -1,18 +1,21 @@
 """ACME protocol messages."""
 import json
+
+import josepy as jose
 import six
+
+from acme import challenges
+from acme import errors
+from acme import fields
+from acme import jws
+from acme import util
+
 try:
     from collections.abc import Hashable  # pylint: disable=no-name-in-module
 except ImportError:  # pragma: no cover
     from collections import Hashable
 
-import josepy as jose
 
-from acme import challenges
-from acme import errors
-from acme import fields
-from acme import util
-from acme import jws
 
 OLD_ERROR_PREFIX = "urn:acme:error:"
 ERROR_PREFIX = "urn:ietf:params:acme:error:"
@@ -33,7 +36,7 @@ ERROR_CODES = {
                    ' domain'),
     'dns': 'There was a problem with a DNS query during identifier validation',
     'dnssec': 'The server could not validate a DNSSEC signed domain',
-    'incorrectResponse': 'Response recieved didn\'t match the challenge\'s requirements',
+    'incorrectResponse': 'Response received didn\'t match the challenge\'s requirements',
     # deprecate invalidEmail
     'invalidEmail': 'The provided email for a registration was invalid',
     'invalidContact': 'The provided contact URI was invalid',
@@ -143,7 +146,7 @@ class _Constant(jose.JSONDeSerializable, Hashable):  # type: ignore
         if jobj not in cls.POSSIBLE_NAMES:  # pylint: disable=unsupported-membership-test
             raise jose.DeserializationError(
                 '{0} not recognized'.format(cls.__name__))
-        return cls.POSSIBLE_NAMES[jobj]  # pylint: disable=unsubscriptable-object
+        return cls.POSSIBLE_NAMES[jobj]
 
     def __repr__(self):
         return '{0}({1})'.format(self.__class__.__name__, self.name)
@@ -168,6 +171,7 @@ STATUS_VALID = Status('valid')
 STATUS_INVALID = Status('invalid')
 STATUS_REVOKED = Status('revoked')
 STATUS_READY = Status('ready')
+STATUS_DEACTIVATED = Status('deactivated')
 
 
 class IdentifierType(_Constant):
@@ -241,13 +245,13 @@ class Directory(jose.JSONDeSerializable):
         try:
             return self[name.replace('_', '-')]
         except KeyError as error:
-            raise AttributeError(str(error) + ': ' + name)
+            raise AttributeError(str(error))
 
     def __getitem__(self, name):
         try:
             return self._jobj[self._canon_key(name)]
         except KeyError:
-            raise KeyError('Directory field not found')
+            raise KeyError('Directory field "' + self._canon_key(name) + '" not found')
 
     def to_partial_json(self):
         return self._jobj
@@ -471,7 +475,7 @@ class Authorization(ResourceBody):
     :ivar datetime.datetime expires:
 
     """
-    identifier = jose.Field('identifier', decoder=Identifier.from_json)
+    identifier = jose.Field('identifier', decoder=Identifier.from_json, omitempty=True)
     challenges = jose.Field('challenges', omitempty=True)
     combinations = jose.Field('combinations', omitempty=True)
 
@@ -501,6 +505,12 @@ class NewAuthorization(Authorization):
     resource = fields.Resource(resource_type)
 
 
+class UpdateAuthorization(Authorization):
+    """Update authorization."""
+    resource_type = 'authz'
+    resource = fields.Resource(resource_type)
+
+
 class AuthorizationResource(ResourceWithURI):
     """Authorization Resource.
 

acme/acme/standalone.py

@@ -1,29 +1,22 @@
 """Support for standalone client challenge solvers. """
-import argparse
 import collections
 import functools
 import logging
-import os
 import socket
-import sys
 import threading
 
 from six.moves import BaseHTTPServer  # type: ignore  # pylint: disable=import-error
 from six.moves import http_client  # pylint: disable=import-error
 from six.moves import socketserver  # type: ignore  # pylint: disable=import-error
 
-import OpenSSL
-
 from acme import challenges
 from acme import crypto_util
-from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
-from acme import _TLSSNI01DeprecationModule
-
+from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-module
 
 logger = logging.getLogger(__name__)
 
 # six.moves.* | pylint: disable=no-member,attribute-defined-outside-init
-# pylint: disable=too-few-public-methods,no-init
+# pylint: disable=no-init
 
 
 class TLSServer(socketserver.TCPServer):
@@ -51,7 +44,7 @@ class TLSServer(socketserver.TCPServer):
         return socketserver.TCPServer.server_bind(self)
 
 
-class ACMEServerMixin:  # pylint: disable=old-style-class
+class ACMEServerMixin:
     """ACME server common settings mixin."""
     # TODO: c.f. #858
     server_version = "ACME client standalone challenge solver"
@@ -112,7 +105,6 @@ class BaseDualNetworkedServers(object):
         """Wraps socketserver.TCPServer.serve_forever"""
         for server in self.servers:
             thread = threading.Thread(
-                # pylint: disable=no-member
                 target=server.serve_forever)
             thread.start()
             self.threads.append(thread)
@@ -132,35 +124,6 @@ class BaseDualNetworkedServers(object):
         self.threads = []
 
 
-class TLSSNI01Server(TLSServer, ACMEServerMixin):
-    """TLSSNI01 Server."""
-
-    def __init__(self, server_address, certs, ipv6=False):
-        TLSServer.__init__(
-            self, server_address, BaseRequestHandlerWithLogging, certs=certs, ipv6=ipv6)
-
-
-class TLSSNI01DualNetworkedServers(BaseDualNetworkedServers):
-    """TLSSNI01Server Wrapper. Tries everything for both. Failures for one don't
-       affect the other."""
-
-    def __init__(self, *args, **kwargs):
-        BaseDualNetworkedServers.__init__(self, TLSSNI01Server, *args, **kwargs)
-
-
-class BaseRequestHandlerWithLogging(socketserver.BaseRequestHandler):
-    """BaseRequestHandler with logging."""
-
-    def log_message(self, format, *args):  # pylint: disable=redefined-builtin
-        """Log arbitrary message."""
-        logger.debug("%s - - %s", self.client_address[0], format % args)
-
-    def handle(self):
-        """Handle request."""
-        self.log_message("Incoming request")
-        socketserver.BaseRequestHandler.handle(self)
-
-
 class HTTPServer(BaseHTTPServer.HTTPServer):
     """Generic HTTP Server."""
 
@@ -263,43 +226,3 @@ class HTTP01RequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
         """
         return functools.partial(
             cls, simple_http_resources=simple_http_resources)
-
-
-def simple_tls_sni_01_server(cli_args, forever=True):
-    """Run simple standalone TLSSNI01 server."""
-    logging.basicConfig(level=logging.DEBUG)
-
-    parser = argparse.ArgumentParser()
-    parser.add_argument(
-        "-p", "--port", default=0, help="Port to serve at. By default "
-        "picks random free port.")
-    args = parser.parse_args(cli_args[1:])
-
-    certs = {}
-
-    _, hosts, _ = next(os.walk('.')) # type: ignore # https://github.com/python/mypy/issues/465
-    for host in hosts:
-        with open(os.path.join(host, "cert.pem")) as cert_file:
-            cert_contents = cert_file.read()
-        with open(os.path.join(host, "key.pem")) as key_file:
-            key_contents = key_file.read()
-        certs[host.encode()] = (
-            OpenSSL.crypto.load_privatekey(
-                OpenSSL.crypto.FILETYPE_PEM, key_contents),
-            OpenSSL.crypto.load_certificate(
-                OpenSSL.crypto.FILETYPE_PEM, cert_contents))
-
-    server = TLSSNI01Server(('', int(args.port)), certs=certs)
-    logger.info("Serving at https://%s:%s...", *server.socket.getsockname()[:2])
-    if forever:  # pragma: no cover
-        server.serve_forever()
-    else:
-        server.handle_request()
-
-
-# Patching ourselves to warn about TLS-SNI challenge deprecation and removal.
-sys.modules[__name__] = _TLSSNI01DeprecationModule(sys.modules[__name__])
-
-
-if __name__ == "__main__":
-    sys.exit(simple_tls_sni_01_server(sys.argv))  # pragma: no cover

acme/docs/conf.py

@@ -12,10 +12,9 @@
 # All configuration values have a default; values that are commented out
 # serve to show the default.
 
-import sys
 import os
 import shlex
-
+import sys
 
 here = os.path.abspath(os.path.dirname(__file__))
 
@@ -42,7 +41,7 @@ extensions = [
 ]
 
 autodoc_member_order = 'bysource'
-autodoc_default_flags = ['show-inheritance', 'private-members']
+autodoc_default_flags = ['show-inheritance']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']

acme/examples/http01_example.py

@@ -26,8 +26,10 @@ Workflow:
     - Deactivate Account
 """
 from contextlib import contextmanager
+
 from cryptography.hazmat.backends import default_backend
 from cryptography.hazmat.primitives.asymmetric import rsa
+import josepy as jose
 import OpenSSL
 
 from acme import challenges
@@ -36,7 +38,6 @@ from acme import crypto_util
 from acme import errors
 from acme import messages
 from acme import standalone
-import josepy as jose
 
 # Constants:
 

acme/readthedocs.org.requirements.txt

@@ -1,10 +1,10 @@
 # readthedocs.org gives no way to change the install command to "pip
-# install -e .[docs]" (that would in turn install documentation
+# install -e acme[docs]" (that would in turn install documentation
 # dependencies), but it allows to specify a requirements.txt file at
 # https://readthedocs.org/dashboard/letsencrypt/advanced/ (c.f. #259)
 
 # Although ReadTheDocs certainly doesn't need to install the project
-# in --editable mode (-e), just "pip install .[docs]" does not work as
-# expected and "pip install -e .[docs]" must be used instead
+# in --editable mode (-e), just "pip install acme[docs]" does not work as
+# expected and "pip install -e acme[docs]" must be used instead
 
 -e acme[docs]

acme/setup.py

@@ -1,9 +1,10 @@
-from setuptools import setup
-from setuptools import find_packages
-from setuptools.command.test import test as TestCommand
 import sys
 
-version = '0.37.0.dev0'
+from setuptools import find_packages
+from setuptools import setup
+from setuptools.command.test import test as TestCommand
+
+version = '1.2.0.dev0'
 
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [
@@ -14,8 +15,8 @@ install_requires = [
     # 1.1.0+ is required to avoid the warnings described at
     # https://github.com/certbot/josepy/issues/13.
     'josepy>=1.1.0',
-    # Connection.set_tlsext_host_name (>=0.13)
     'mock',
+    # Connection.set_tlsext_host_name (>=0.13)
     'PyOpenSSL>=0.13.1',
     'pyrfc3339',
     'pytz',
@@ -73,6 +74,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
     ],

acme/tests/challenges_test.py

@@ -3,13 +3,10 @@ import unittest
 
 import josepy as jose
 import mock
-import OpenSSL
 import requests
+from six.moves.urllib import parse as urllib_parse
 
-from six.moves.urllib import parse as urllib_parse  # pylint: disable=relative-import
-
-from acme import errors
-from acme import test_util
+import test_util
 
 CERT = test_util.load_comparable_cert('cert.pem')
 KEY = jose.JWKRSA(key=test_util.load_rsa_private_key('rsa512_key.pem'))
@@ -21,7 +18,6 @@ class ChallengeTest(unittest.TestCase):
         from acme.challenges import Challenge
         from acme.challenges import UnrecognizedChallenge
         chall = UnrecognizedChallenge({"type": "foo"})
-        # pylint: disable=no-member
         self.assertEqual(chall, Challenge.from_json(chall.jobj))
 
 
@@ -77,7 +73,6 @@ class KeyAuthorizationChallengeResponseTest(unittest.TestCase):
 
 
 class DNS01ResponseTest(unittest.TestCase):
-    # pylint: disable=too-many-instance-attributes
 
     def setUp(self):
         from acme.challenges import DNS01Response
@@ -149,7 +144,6 @@ class DNS01Test(unittest.TestCase):
 
 
 class HTTP01ResponseTest(unittest.TestCase):
-    # pylint: disable=too-many-instance-attributes
 
     def setUp(self):
         from acme.challenges import HTTP01Response
@@ -259,152 +253,7 @@ class HTTP01Test(unittest.TestCase):
             self.msg.update(token=b'..').good_token)
 
 
-class TLSSNI01ResponseTest(unittest.TestCase):
-    # pylint: disable=too-many-instance-attributes
-
-    def setUp(self):
-        from acme.challenges import TLSSNI01
-        self.chall = TLSSNI01(
-            token=jose.b64decode(b'a82d5ff8ef740d12881f6d3c2277ab2e'))
-
-        self.response = self.chall.response(KEY)
-        self.jmsg = {
-            'resource': 'challenge',
-            'type': 'tls-sni-01',
-            'keyAuthorization': self.response.key_authorization,
-        }
-
-        # pylint: disable=invalid-name
-        label1 = b'dc38d9c3fa1a4fdcc3a5501f2d38583f'
-        label2 = b'b7793728f084394f2a1afd459556bb5c'
-        self.z = label1 + label2
-        self.z_domain = label1 + b'.' + label2 + b'.acme.invalid'
-        self.domain = 'foo.com'
-
-    def test_z_and_domain(self):
-        self.assertEqual(self.z, self.response.z)
-        self.assertEqual(self.z_domain, self.response.z_domain)
-
-    def test_to_partial_json(self):
-        self.assertEqual({k: v for k, v in self.jmsg.items() if k != 'keyAuthorization'},
-                         self.response.to_partial_json())
-
-    def test_from_json(self):
-        from acme.challenges import TLSSNI01Response
-        self.assertEqual(self.response, TLSSNI01Response.from_json(self.jmsg))
-
-    def test_from_json_hashable(self):
-        from acme.challenges import TLSSNI01Response
-        hash(TLSSNI01Response.from_json(self.jmsg))
-
-    @mock.patch('acme.challenges.socket.gethostbyname')
-    @mock.patch('acme.challenges.crypto_util.probe_sni')
-    def test_probe_cert(self, mock_probe_sni, mock_gethostbyname):
-        mock_gethostbyname.return_value = '127.0.0.1'
-        self.response.probe_cert('foo.com')
-        mock_gethostbyname.assert_called_once_with('foo.com')
-        mock_probe_sni.assert_called_once_with(
-            host='127.0.0.1', port=self.response.PORT,
-            name=self.z_domain)
-
-        self.response.probe_cert('foo.com', host='8.8.8.8')
-        mock_probe_sni.assert_called_with(
-            host='8.8.8.8', port=mock.ANY, name=mock.ANY)
-
-        self.response.probe_cert('foo.com', port=1234)
-        mock_probe_sni.assert_called_with(
-            host=mock.ANY, port=1234, name=mock.ANY)
-
-        self.response.probe_cert('foo.com', bar='baz')
-        mock_probe_sni.assert_called_with(
-            host=mock.ANY, port=mock.ANY, name=mock.ANY, bar='baz')
-
-        self.response.probe_cert('foo.com', name=b'xxx')
-        mock_probe_sni.assert_called_with(
-            host=mock.ANY, port=mock.ANY,
-            name=self.z_domain)
-
-    def test_gen_verify_cert(self):
-        key1 = test_util.load_pyopenssl_private_key('rsa512_key.pem')
-        cert, key2 = self.response.gen_cert(key1)
-        self.assertEqual(key1, key2)
-        self.assertTrue(self.response.verify_cert(cert))
-
-    def test_gen_verify_cert_gen_key(self):
-        cert, key = self.response.gen_cert()
-        self.assertTrue(isinstance(key, OpenSSL.crypto.PKey))
-        self.assertTrue(self.response.verify_cert(cert))
-
-    def test_verify_bad_cert(self):
-        self.assertFalse(self.response.verify_cert(
-            test_util.load_cert('cert.pem')))
-
-    def test_simple_verify_bad_key_authorization(self):
-        key2 = jose.JWKRSA.load(test_util.load_vector('rsa256_key.pem'))
-        self.response.simple_verify(self.chall, "local", key2.public_key())
-
-    @mock.patch('acme.challenges.TLSSNI01Response.verify_cert', autospec=True)
-    def test_simple_verify(self, mock_verify_cert):
-        mock_verify_cert.return_value = mock.sentinel.verification
-        self.assertEqual(
-            mock.sentinel.verification, self.response.simple_verify(
-                self.chall, self.domain, KEY.public_key(),
-                cert=mock.sentinel.cert))
-        mock_verify_cert.assert_called_once_with(
-            self.response, mock.sentinel.cert)
-
-    @mock.patch('acme.challenges.TLSSNI01Response.probe_cert')
-    def test_simple_verify_false_on_probe_error(self, mock_probe_cert):
-        mock_probe_cert.side_effect = errors.Error
-        self.assertFalse(self.response.simple_verify(
-            self.chall, self.domain, KEY.public_key()))
-
-
-class TLSSNI01Test(unittest.TestCase):
-
-    def setUp(self):
-        self.jmsg = {
-            'type': 'tls-sni-01',
-            'token': 'a82d5ff8ef740d12881f6d3c2277ab2e',
-        }
-        from acme.challenges import TLSSNI01
-        self.msg = TLSSNI01(
-            token=jose.b64decode('a82d5ff8ef740d12881f6d3c2277ab2e'))
-
-    def test_to_partial_json(self):
-        self.assertEqual(self.jmsg, self.msg.to_partial_json())
-
-    def test_from_json(self):
-        from acme.challenges import TLSSNI01
-        self.assertEqual(self.msg, TLSSNI01.from_json(self.jmsg))
-
-    def test_from_json_hashable(self):
-        from acme.challenges import TLSSNI01
-        hash(TLSSNI01.from_json(self.jmsg))
-
-    def test_from_json_invalid_token_length(self):
-        from acme.challenges import TLSSNI01
-        self.jmsg['token'] = jose.encode_b64jose(b'abcd')
-        self.assertRaises(
-            jose.DeserializationError, TLSSNI01.from_json, self.jmsg)
-
-    @mock.patch('acme.challenges.TLSSNI01Response.gen_cert')
-    def test_validation(self, mock_gen_cert):
-        mock_gen_cert.return_value = ('cert', 'key')
-        self.assertEqual(('cert', 'key'), self.msg.validation(
-            KEY, cert_key=mock.sentinel.cert_key))
-        mock_gen_cert.assert_called_once_with(key=mock.sentinel.cert_key)
-
-    def test_deprecation_message(self):
-        with mock.patch('acme.warnings.warn') as mock_warn:
-            from acme.challenges import TLSSNI01
-            assert TLSSNI01
-        self.assertEqual(mock_warn.call_count, 1)
-        self.assertTrue('deprecated' in mock_warn.call_args[0][0])
-
-
 class TLSALPN01ResponseTest(unittest.TestCase):
-    # pylint: disable=too-many-instance-attributes
 
     def setUp(self):
         from acme.challenges import TLSALPN01Response

acme/tests/client_test.py

@@ -5,21 +5,19 @@ import datetime
 import json
 import unittest
 
-from six.moves import http_client  # pylint: disable=import-error
-
 import josepy as jose
 import mock
 import OpenSSL
 import requests
+from six.moves import http_client  # pylint: disable=import-error
 
 from acme import challenges
 from acme import errors
 from acme import jws as acme_jws
 from acme import messages
-from acme import messages_test
-from acme import test_util
-from acme.magic_typing import Dict # pylint: disable=unused-import, no-name-in-module
-
+from acme.magic_typing import Dict  # pylint: disable=unused-import, no-name-in-module
+import messages_test
+import test_util
 
 CERT_DER = test_util.load_vector('cert.der')
 CERT_SAN_PEM = test_util.load_vector('cert-san.pem')
@@ -63,7 +61,7 @@ class ClientTestBase(unittest.TestCase):
         self.contact = ('mailto:cert-admin@example.com', 'tel:+12025551212')
         reg = messages.Registration(
             contact=self.contact, key=KEY.public_key())
-        the_arg = dict(reg) # type: Dict
+        the_arg = dict(reg)  # type: Dict
         self.new_reg = messages.NewRegistration(**the_arg)
         self.regr = messages.RegistrationResource(
             body=reg, uri='https://www.letsencrypt-demo.org/acme/reg/1')
@@ -318,7 +316,6 @@ class BackwardsCompatibleClientV2Test(ClientTestBase):
 
 class ClientTest(ClientTestBase):
     """Tests for acme.client.Client."""
-    # pylint: disable=too-many-instance-attributes,too-many-public-methods
 
     def setUp(self):
         super(ClientTest, self).setUp()
@@ -637,6 +634,14 @@ class ClientTest(ClientTestBase):
             errors.PollError, self.client.poll_and_request_issuance,
             csr, authzrs, mintime=mintime, max_attempts=2)
 
+    def test_deactivate_authorization(self):
+        authzb = self.authzr.body.update(status=messages.STATUS_DEACTIVATED)
+        self.response.json.return_value = authzb.to_json()
+        authzr = self.client.deactivate_authorization(self.authzr)
+        self.assertEqual(authzb, authzr.body)
+        self.assertEqual(self.client.net.post.call_count, 1)
+        self.assertTrue(self.authzr.uri in self.net.post.call_args_list[0][0])
+
     def test_check_cert(self):
         self.response.headers['Location'] = self.certr.uri
         self.response.content = CERT_DER
@@ -880,19 +885,6 @@ class ClientV2Test(ClientTestBase):
                 new_nonce_url='https://www.letsencrypt-demo.org/acme/new-nonce')
             self.client.net.get.assert_not_called()
 
-            class FakeError(messages.Error):  # pylint: disable=too-many-ancestors
-                """Fake error to reproduce a malformed request ACME error"""
-                def __init__(self):  # pylint: disable=super-init-not-called
-                    pass
-                @property
-                def code(self):
-                    return 'malformed'
-            self.client.net.post.side_effect = FakeError()
-
-            self.client.poll(self.authzr2)  # pylint: disable=protected-access
-
-            self.client.net.get.assert_called_once_with(self.authzr2.uri)
-
 
 class MockJSONDeSerializable(jose.JSONDeSerializable):
     # pylint: disable=missing-docstring
@@ -909,7 +901,6 @@ class MockJSONDeSerializable(jose.JSONDeSerializable):
 
 class ClientNetworkTest(unittest.TestCase):
     """Tests for acme.client.ClientNetwork."""
-    # pylint: disable=too-many-public-methods
 
     def setUp(self):
         self.verify_ssl = mock.MagicMock()
@@ -959,8 +950,8 @@ class ClientNetworkTest(unittest.TestCase):
 
     def test_check_response_not_ok_jobj_error(self):
         self.response.ok = False
-        self.response.json.return_value = messages.Error(
-            detail='foo', typ='serverInternal', title='some title').to_json()
+        self.response.json.return_value = messages.Error.with_code(
+            'serverInternal', detail='foo', title='some title').to_json()
         # pylint: disable=protected-access
         self.assertRaises(
             messages.Error, self.net._check_response, self.response)
@@ -985,7 +976,7 @@ class ClientNetworkTest(unittest.TestCase):
         self.response.json.side_effect = ValueError
         for response_ct in [self.net.JSON_CONTENT_TYPE, 'foo']:
             self.response.headers['Content-Type'] = response_ct
-            # pylint: disable=protected-access,no-value-for-parameter
+            # pylint: disable=protected-access
             self.assertEqual(
                 self.response, self.net._check_response(self.response))
 
@@ -999,7 +990,7 @@ class ClientNetworkTest(unittest.TestCase):
         self.response.json.return_value = {}
         for response_ct in [self.net.JSON_CONTENT_TYPE, 'foo']:
             self.response.headers['Content-Type'] = response_ct
-            # pylint: disable=protected-access,no-value-for-parameter
+            # pylint: disable=protected-access
             self.assertEqual(
                 self.response, self.net._check_response(self.response))
 
@@ -1115,7 +1106,6 @@ class ClientNetworkTest(unittest.TestCase):
 
 class ClientNetworkWithMockedResponseTest(unittest.TestCase):
     """Tests for acme.client.ClientNetwork which mock out response."""
-    # pylint: disable=too-many-instance-attributes
 
     def setUp(self):
         from acme.client import ClientNetwork
@@ -1125,8 +1115,8 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase):
         self.response.headers = {}
         self.response.links = {}
         self.response.checked = False
-        self.acmev1_nonce_response = mock.MagicMock(ok=False,
-            status_code=http_client.METHOD_NOT_ALLOWED)
+        self.acmev1_nonce_response = mock.MagicMock(
+            ok=False, status_code=http_client.METHOD_NOT_ALLOWED)
         self.acmev1_nonce_response.headers = {}
         self.obj = mock.MagicMock()
         self.wrapped_obj = mock.MagicMock()

acme/tests/crypto_util_test.py

@@ -5,15 +5,14 @@ import threading
 import time
 import unittest
 
-import six
-from six.moves import socketserver  #type: ignore  # pylint: disable=import-error
-
 import josepy as jose
 import OpenSSL
+import six
+from six.moves import socketserver  # type: ignore  # pylint: disable=import-error
 
 from acme import errors
-from acme import test_util
-from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-module
+import test_util
 
 
 class SSLSocketAndProbeSNITest(unittest.TestCase):
@@ -30,7 +29,6 @@ class SSLSocketAndProbeSNITest(unittest.TestCase):
 
         class _TestServer(socketserver.TCPServer):
 
-            # pylint: disable=too-few-public-methods
             # six.moves.* | pylint: disable=attribute-defined-outside-init,no-init
 
             def server_bind(self):  # pylint: disable=missing-docstring
@@ -40,7 +38,6 @@ class SSLSocketAndProbeSNITest(unittest.TestCase):
         self.server = _TestServer(('', 0), socketserver.BaseRequestHandler)
         self.port = self.server.socket.getsockname()[1]
         self.server_thread = threading.Thread(
-            # pylint: disable=no-member
             target=self.server.handle_request)
 
     def tearDown(self):
@@ -67,7 +64,7 @@ class SSLSocketAndProbeSNITest(unittest.TestCase):
 
     def test_probe_connection_error(self):
         # pylint has a hard time with six
-        self.server.server_close()  # pylint: disable=no-member
+        self.server.server_close()
         original_timeout = socket.getdefaulttimeout()
         try:
             socket.setdefaulttimeout(1)

acme/tests/jose_test.py

@@ -2,6 +2,7 @@
 import importlib
 import unittest
 
+
 class JoseTest(unittest.TestCase):
     """Tests for acme.jose shim."""
 
@@ -20,11 +21,10 @@ class JoseTest(unittest.TestCase):
 
         # We use the imports below with eval, but pylint doesn't
         # understand that.
-        # pylint: disable=eval-used,unused-variable
-        import acme
-        import josepy
-        acme_jose_mod = eval(acme_jose_path)
-        josepy_mod = eval(josepy_path)
+        import acme  # pylint: disable=unused-import
+        import josepy  # pylint: disable=unused-import
+        acme_jose_mod = eval(acme_jose_path)  # pylint: disable=eval-used
+        josepy_mod = eval(josepy_path)  # pylint: disable=eval-used
         self.assertIs(acme_jose_mod, josepy_mod)
         self.assertIs(getattr(acme_jose_mod, attribute), getattr(josepy_mod, attribute))
 

acme/tests/jws_test.py

@@ -3,8 +3,7 @@ import unittest
 
 import josepy as jose
 
-from acme import test_util
-
+import test_util
 
 KEY = jose.JWKRSA.load(test_util.load_vector('rsa512_key.pem'))
 

acme/tests/messages_test.py

@@ -5,9 +5,8 @@ import josepy as jose
 import mock
 
 from acme import challenges
-from acme import test_util
-from acme.magic_typing import Dict # pylint: disable=unused-import, no-name-in-module
-
+from acme.magic_typing import Dict  # pylint: disable=unused-import, no-name-in-module
+import test_util
 
 CERT = test_util.load_comparable_cert('cert.der')
 CSR = test_util.load_comparable_csr('csr.der')
@@ -19,8 +18,7 @@ class ErrorTest(unittest.TestCase):
 
     def setUp(self):
         from acme.messages import Error, ERROR_PREFIX
-        self.error = Error(
-            detail='foo', typ=ERROR_PREFIX + 'malformed', title='title')
+        self.error = Error.with_code('malformed', detail='foo', title='title')
         self.jobj = {
             'detail': 'foo',
             'title': 'some title',
@@ -28,7 +26,6 @@ class ErrorTest(unittest.TestCase):
         }
         self.error_custom = Error(typ='custom', detail='bar')
         self.empty_error = Error()
-        self.jobj_custom = {'type': 'custom', 'detail': 'bar'}
 
     def test_default_typ(self):
         from acme.messages import Error
@@ -43,8 +40,7 @@ class ErrorTest(unittest.TestCase):
         hash(Error.from_json(self.error.to_json()))
 
     def test_description(self):
-        self.assertEqual(
-            'The request message was malformed', self.error.description)
+        self.assertEqual('The request message was malformed', self.error.description)
         self.assertTrue(self.error_custom.description is None)
 
     def test_code(self):
@@ -54,17 +50,17 @@ class ErrorTest(unittest.TestCase):
         self.assertEqual(None, Error().code)
 
     def test_is_acme_error(self):
-        from acme.messages import is_acme_error
+        from acme.messages import is_acme_error, Error
         self.assertTrue(is_acme_error(self.error))
         self.assertFalse(is_acme_error(self.error_custom))
+        self.assertFalse(is_acme_error(Error()))
         self.assertFalse(is_acme_error(self.empty_error))
         self.assertFalse(is_acme_error("must pet all the {dogs|rabbits}"))
 
     def test_unicode_error(self):
-        from acme.messages import Error, ERROR_PREFIX, is_acme_error
-        arabic_error = Error(
-                detail=u'\u0639\u062f\u0627\u0644\u0629', typ=ERROR_PREFIX + 'malformed',
-            title='title')
+        from acme.messages import Error, is_acme_error
+        arabic_error = Error.with_code(
+            'malformed', detail=u'\u0639\u062f\u0627\u0644\u0629', title='title')
         self.assertTrue(is_acme_error(arabic_error))
 
     def test_with_code(self):
@@ -305,8 +301,7 @@ class ChallengeBodyTest(unittest.TestCase):
         from acme.messages import Error
         from acme.messages import STATUS_INVALID
         self.status = STATUS_INVALID
-        error = Error(typ='urn:ietf:params:acme:error:serverInternal',
-                      detail='Unable to communicate with DNS server')
+        error = Error.with_code('serverInternal', detail='Unable to communicate with DNS server')
         self.challb = ChallengeBody(
             uri='http://challb', chall=self.chall, status=self.status,
             error=error)

acme/tests/standalone_test.py

@@ -1,26 +1,17 @@
 """Tests for acme.standalone."""
-import multiprocessing
-import os
-import shutil
 import socket
 import threading
-import tempfile
 import unittest
-import time
-from contextlib import closing
-
-from six.moves import http_client  # pylint: disable=import-error
-from six.moves import socketserver  # type: ignore  # pylint: disable=import-error
 
 import josepy as jose
 import mock
 import requests
+from six.moves import http_client  # pylint: disable=import-error
+from six.moves import socketserver  # type: ignore  # pylint: disable=import-error
 
 from acme import challenges
-from acme import crypto_util
-from acme import errors
-from acme import test_util
-from acme.magic_typing import Set # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Set  # pylint: disable=unused-import, no-name-in-module
+import test_util
 
 
 class TLSServerTest(unittest.TestCase):
@@ -41,32 +32,6 @@ class TLSServerTest(unittest.TestCase):
             server.server_close()
 
 
-class TLSSNI01ServerTest(unittest.TestCase):
-    """Test for acme.standalone.TLSSNI01Server."""
-
-
-    def setUp(self):
-        self.certs = {b'localhost': (
-            test_util.load_pyopenssl_private_key('rsa2048_key.pem'),
-            test_util.load_cert('rsa2048_cert.pem'),
-        )}
-        from acme.standalone import TLSSNI01Server
-        self.server = TLSSNI01Server(('localhost', 0), certs=self.certs)
-        self.thread = threading.Thread(target=self.server.serve_forever)
-        self.thread.start()
-
-    def tearDown(self):
-        self.server.shutdown()
-        self.thread.join()
-
-    def test_it(self):
-        host, port = self.server.socket.getsockname()[:2]
-        cert = crypto_util.probe_sni(
-            b'localhost', host=host, port=port, timeout=1)
-        self.assertEqual(jose.ComparableX509(cert),
-                         jose.ComparableX509(self.certs[b'localhost'][1]))
-
-
 class HTTP01ServerTest(unittest.TestCase):
     """Tests for acme.standalone.HTTP01Server."""
 
@@ -170,33 +135,6 @@ class BaseDualNetworkedServersTest(unittest.TestCase):
             prev_port = port
 
 
-class TLSSNI01DualNetworkedServersTest(unittest.TestCase):
-    """Test for acme.standalone.TLSSNI01DualNetworkedServers."""
-
-
-    def setUp(self):
-        self.certs = {b'localhost': (
-            test_util.load_pyopenssl_private_key('rsa2048_key.pem'),
-            test_util.load_cert('rsa2048_cert.pem'),
-        )}
-        from acme.standalone import TLSSNI01DualNetworkedServers
-        self.servers = TLSSNI01DualNetworkedServers(('localhost', 0), certs=self.certs)
-        self.servers.serve_forever()
-
-    def tearDown(self):
-        self.servers.shutdown_and_server_close()
-
-    def test_connect(self):
-        socknames = self.servers.getsocknames()
-        # connect to all addresses
-        for sockname in socknames:
-            host, port = sockname[:2]
-            cert = crypto_util.probe_sni(
-                b'localhost', host=host, port=port, timeout=1)
-            self.assertEqual(jose.ComparableX509(cert),
-                             jose.ComparableX509(self.certs[b'localhost'][1]))
-
-
 class HTTP01DualNetworkedServersTest(unittest.TestCase):
     """Tests for acme.standalone.HTTP01DualNetworkedServers."""
 
@@ -247,60 +185,5 @@ class HTTP01DualNetworkedServersTest(unittest.TestCase):
         self.assertFalse(self._test_http01(add=False))
 
 
-class TestSimpleTLSSNI01Server(unittest.TestCase):
-    """Tests for acme.standalone.simple_tls_sni_01_server."""
-
-
-    def setUp(self):
-        # mirror ../examples/standalone
-        self.test_cwd = tempfile.mkdtemp()
-        localhost_dir = os.path.join(self.test_cwd, 'localhost')
-        os.makedirs(localhost_dir)
-        shutil.copy(test_util.vector_path('rsa2048_cert.pem'),
-                    os.path.join(localhost_dir, 'cert.pem'))
-        shutil.copy(test_util.vector_path('rsa2048_key.pem'),
-                    os.path.join(localhost_dir, 'key.pem'))
-
-        with closing(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) as sock:
-            sock.bind(('', 0))
-            sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-            self.port = sock.getsockname()[1]
-
-        from acme.standalone import simple_tls_sni_01_server
-        self.process = multiprocessing.Process(target=simple_tls_sni_01_server,
-                                               args=(['path', '-p', str(self.port)],))
-        self.old_cwd = os.getcwd()
-        os.chdir(self.test_cwd)
-
-    def tearDown(self):
-        os.chdir(self.old_cwd)
-        if self.process.is_alive():
-            self.process.terminate()
-            self.process.join(timeout=5)
-            # Check that we didn't timeout waiting for the process to
-            # terminate.
-            self.assertNotEqual(self.process.exitcode, None)
-        shutil.rmtree(self.test_cwd)
-
-    @mock.patch('acme.standalone.TLSSNI01Server.handle_request')
-    def test_mock(self, handle):
-        from acme.standalone import simple_tls_sni_01_server
-        simple_tls_sni_01_server(cli_args=['path', '-p', str(self.port)], forever=False)
-        self.assertEqual(handle.call_count, 1)
-
-    def test_live(self):
-        self.process.start()
-        cert = None
-        for _ in range(50):
-            time.sleep(0.1)
-            try:
-                cert = crypto_util.probe_sni(b'localhost', b'127.0.0.1', self.port)
-                break
-            except errors.Error:  # pragma: no cover
-                pass
-        self.assertEqual(jose.ComparableX509(cert),
-                         test_util.load_comparable_cert('rsa2048_cert.pem'))
-
-
 if __name__ == "__main__":
     unittest.main()  # pragma: no cover

acme/tests/test_util.py

@@ -4,19 +4,12 @@
 
 """
 import os
-import unittest
-import pkg_resources
 
 from cryptography.hazmat.backends import default_backend
 from cryptography.hazmat.primitives import serialization
 import josepy as jose
 from OpenSSL import crypto
-
-
-def vector_path(*names):
-    """Path to a test vector."""
-    return pkg_resources.resource_filename(
-        __name__, os.path.join('testdata', *names))
+import pkg_resources
 
 
 def load_vector(*names):
@@ -32,8 +25,7 @@ def _guess_loader(filename, loader_pem, loader_der):
         return loader_pem
     elif ext.lower() == '.der':
         return loader_der
-    else:  # pragma: no cover
-        raise ValueError("Loader could not be recognized based on extension")
+    raise ValueError("Loader could not be recognized based on extension")  # pragma: no cover
 
 
 def load_cert(*names):
@@ -73,23 +65,3 @@ def load_pyopenssl_private_key(*names):
     loader = _guess_loader(
         names[-1], crypto.FILETYPE_PEM, crypto.FILETYPE_ASN1)
     return crypto.load_privatekey(loader, load_vector(*names))
-
-
-def skip_unless(condition, reason):  # pragma: no cover
-    """Skip tests unless a condition holds.
-
-    This implements the basic functionality of unittest.skipUnless
-    which is only available on Python 2.7+.
-
-    :param bool condition: If ``False``, the test will be skipped
-    :param str reason: the reason for skipping the test
-
-    :rtype: callable
-    :returns: decorator that hides tests unless condition is ``True``
-
-    """
-    if hasattr(unittest, "skipUnless"):
-        return unittest.skipUnless(condition, reason)
-    elif condition:
-        return lambda cls: cls
-    return lambda cls: None

appveyor.yml

@@ -1,44 +0,0 @@
-image: Visual Studio 2015
-
-environment:
-  matrix:
-    - TOXENV: py35
-    - TOXENV: py37-cover
-
-branches:
-  only:
-    # apache-parser-v2 is a temporary branch for doing work related to
-    # rewriting the parser in the Apache plugin.
-    - apache-parser-v2
-    - master
-    - /^\d+\.\d+\.x$/ # Version branches like X.X.X
-    - /^test-.*$/
-
-init:
-  # Since master can receive only commits from PR that have already been tested, following
-  # condition avoid to launch all jobs except the coverage one for commits pushed to master.
-  - ps: |
-      if (-Not $Env:APPVEYOR_PULL_REQUEST_NUMBER -And $Env:APPVEYOR_REPO_BRANCH -Eq 'master' `
-          -And -Not ($Env:TOXENV -Like '*-cover'))
-      { $Env:APPVEYOR_SKIP_FINALIZE_ON_EXIT = 'true'; Exit-AppVeyorBuild }
-
-install:
-  # Use Python 3.7 by default
-  - "SET PATH=C:\\Python37;C:\\Python37\\Scripts;%PATH%"
-  # Check env
-  - "python --version"
-  # Upgrade pip to avoid warnings
-  - "python -m pip install --upgrade pip"
-  # Ready to install tox and coverage
-  # tools/pip_install.py is used to pin packages to a known working version.
-  - "python tools\\pip_install.py tox codecov"
-
-build: off
-
-test_script:
-  - set TOX_TESTENV_PASSENV=APPVEYOR
-  # Test env is set by TOXENV env variable
-  - tox
-
-on_success:
-  - if exist .coverage codecov -F windows

certbot-apache/MANIFEST.in

@@ -1,7 +1,8 @@
 include LICENSE.txt
 include README.rst
-recursive-include docs *
-recursive-include certbot_apache/tests/testdata *
-include certbot_apache/centos-options-ssl-apache.conf
-include certbot_apache/options-ssl-apache.conf
-recursive-include certbot_apache/augeas_lens *.aug
+recursive-include tests *
+include certbot_apache/_internal/centos-options-ssl-apache.conf
+include certbot_apache/_internal/options-ssl-apache.conf
+recursive-include certbot_apache/_internal/augeas_lens *.aug
+global-exclude __pycache__
+global-exclude *.py[cod]

certbot-apache/certbot_apache/_internal/__init__.py

@@ -0,0 +1 @@
+"""Certbot Apache plugin."""

certbot-apache/certbot_apache/_internal/apache_util.py

@@ -1,10 +1,104 @@
 """ Utility functions for certbot-apache plugin """
 import binascii
+import hashlib
+import struct
+import shutil
+import time
 
+from certbot import crypto_util
+from certbot import errors
 from certbot import util
 from certbot.compat import os
 
 
+def get_apache_ocsp_struct(ttl, ocsp_response):
+    """Create Apache OCSP response structure to be used in response cache
+
+    :param int ttl: Time-To-Live in seconds
+    :param str ocsp_response: OCSP response data
+
+    :returns: Apache OCSP structure
+    :rtype: `str`
+
+    """
+    ttl = time.time() + ttl
+    # As microseconds
+    ttl_struct = struct.pack('l', int(ttl*1000000))
+    return b'\x01'.join([ttl_struct, ocsp_response])
+
+
+def certid_sha1_hex(cert_path):
+    """Hex representation of certificate SHA1 fingerprint
+
+    :param str cert_path: File path to certificate
+
+    :returns: Hex representation SHA1 fingerprint of certificate
+    :rtype: `str`
+
+    """
+    sha1_hex = binascii.hexlify(certid_sha1(cert_path))
+    return sha1_hex.decode('utf-8')
+
+
+def certid_sha1(cert_path):
+    """SHA1 fingerprint of certificate
+
+    :param str cert_path: File path to certificate
+
+    :returns: SHA1 fingerprint bytestring
+    :rtype: `str`
+
+    """
+    return crypto_util.cert_sha1_fingerprint(cert_path)
+
+
+def safe_copy(source, target):
+    """Copies a file, while verifying the target integrity
+    with the source. Retries twice if the initial
+    copy fails.
+
+    :param str source: File path of the source file
+    :param str target: File path of the target file
+
+    :raises: .errors.PluginError: If file cannot be
+        copied or the target file hash does not match
+        with the source file.
+    """
+    for _ in range(3):
+        try:
+            shutil.copy2(source, target)
+        except IOError as e:
+            emsg = "Could not copy {} to {}: {}".format(
+                source, target, e
+            )
+            raise errors.PluginError(emsg)
+        try:
+            source_hash = _file_hash(source)
+            target_hash = _file_hash(target)
+        except IOError:
+            continue
+        if source_hash == target_hash:
+            return
+    raise errors.PluginError(
+        "Safe copy failed. The file integrity does not match"
+    )
+
+
+def _file_hash(filepath):
+    """Helper function for safe_copy that calculates a
+    sha-256 hash of file.
+
+    :param str filepath: Path of file to calculate hash for
+
+    :returns: File sha-256 hash
+    :rtype: str
+    """
+    fhash = hashlib.sha256()
+    with open(filepath, 'rb') as fh:
+        fhash.update(fh.read())
+    return fhash.hexdigest()
+
+
 def get_mod_deps(mod_name):
     """Get known module dependencies.
 

certbot-apache/certbot_apache/_internal/configurator.py

@@ -1,5 +1,6 @@
 """Apache Configurator."""
 # pylint: disable=too-many-lines
+from collections import defaultdict
 import copy
 import fnmatch
 import logging
@@ -7,17 +8,17 @@ import re
 import socket
 import time
 
-from collections import defaultdict
-
 import pkg_resources
 import six
-
 import zope.component
 import zope.interface
 
 from acme import challenges
-from acme.magic_typing import DefaultDict, Dict, List, Set, Union  # pylint: disable=unused-import, no-name-in-module
-
+from acme.magic_typing import DefaultDict  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Dict  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Set  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Union  # pylint: disable=unused-import, no-name-in-module
 from certbot import errors
 from certbot import interfaces
 from certbot import util
@@ -26,15 +27,14 @@ from certbot.achallenges import KeyAuthorizationAnnotatedChallenge  # pylint: di
 from certbot.compat import filesystem
 from certbot.compat import os
 from certbot.plugins import common
-from certbot.plugins.util import path_surgery
 from certbot.plugins.enhancements import AutoHSTSEnhancement
-
-from certbot_apache import apache_util
-from certbot_apache import constants
-from certbot_apache import display_ops
-from certbot_apache import http_01
-from certbot_apache import obj
-from certbot_apache import parser
+from certbot.plugins.util import path_surgery
+from certbot_apache._internal import apache_util
+from certbot_apache._internal import constants
+from certbot_apache._internal import display_ops
+from certbot_apache._internal import http_01
+from certbot_apache._internal import obj
+from certbot_apache._internal import parser
 
 logger = logging.getLogger(__name__)
 
@@ -71,18 +71,17 @@ logger = logging.getLogger(__name__)
 @zope.interface.implementer(interfaces.IAuthenticator, interfaces.IInstaller)
 @zope.interface.provider(interfaces.IPluginFactory)
 class ApacheConfigurator(common.Installer):
-    # pylint: disable=too-many-instance-attributes,too-many-public-methods
     """Apache configurator.
 
     :ivar config: Configuration.
     :type config: :class:`~certbot.interfaces.IConfig`
 
     :ivar parser: Handles low level parsing
-    :type parser: :class:`~certbot_apache.parser`
+    :type parser: :class:`~certbot_apache._internal.parser`
 
     :ivar tup version: version of Apache
     :ivar list vhosts: All vhosts found in the configuration
-        (:class:`list` of :class:`~certbot_apache.obj.VirtualHost`)
+        (:class:`list` of :class:`~certbot_apache._internal.obj.VirtualHost`)
 
     :ivar dict assoc: Mapping between domains and vhosts
 
@@ -111,7 +110,7 @@ class ApacheConfigurator(common.Installer):
         handle_sites=False,
         challenge_location="/etc/apache2",
         MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
+            "certbot_apache", os.path.join("_internal", "options-ssl-apache.conf"))
     )
 
     def option(self, key):
@@ -174,8 +173,6 @@ class ApacheConfigurator(common.Installer):
                  "(Only Ubuntu/Debian currently)")
         add("ctl", default=DEFAULTS["ctl"],
             help="Full path to Apache control script")
-        util.add_deprecated_argument(
-            add, argument_name="init-script", nargs=1)
 
     def __init__(self, *args, **kwargs):
         """Initialize an Apache Configurator.
@@ -394,7 +391,7 @@ class ApacheConfigurator(common.Installer):
             counterpart, should one get created
 
         :returns: List of VirtualHosts or None
-        :rtype: `list` of :class:`~certbot_apache.obj.VirtualHost`
+        :rtype: `list` of :class:`~certbot_apache._internal.obj.VirtualHost`
         """
 
         if self._wildcard_domain(domain):
@@ -453,7 +450,7 @@ class ApacheConfigurator(common.Installer):
                     filtered_vhosts[name] = vhost
 
         # Only unique VHost objects
-        dialog_input = set([vhost for vhost in filtered_vhosts.values()])
+        dialog_input = set(filtered_vhosts.values())
 
         # Ask the user which of names to enable, expect list of names back
         dialog_output = display_ops.select_vhost_multiple(list(dialog_input))
@@ -569,7 +566,7 @@ class ApacheConfigurator(common.Installer):
             counterpart, should one get created
 
         :returns: vhost associated with name
-        :rtype: :class:`~certbot_apache.obj.VirtualHost`
+        :rtype: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :raises .errors.PluginError: If no vhost is available or chosen
 
@@ -604,9 +601,9 @@ class ApacheConfigurator(common.Installer):
                 "in the Apache config.",
                 target_name)
             raise errors.PluginError("No vhost selected")
-        elif temp:
+        if temp:
             return vhost
-        elif not vhost.ssl:
+        if not vhost.ssl:
             addrs = self._get_proposed_addrs(vhost, "443")
             # TODO: Conflicts is too conservative
             if not any(vhost.enabled and vhost.conflicts(addrs) for
@@ -672,7 +669,7 @@ class ApacheConfigurator(common.Installer):
 
         :param str target_name: domain handled by the desired vhost
         :param vhosts: vhosts to consider
-        :type vhosts: `collections.Iterable` of :class:`~certbot_apache.obj.VirtualHost`
+        :type vhosts: `collections.Iterable` of :class:`~certbot_apache._internal.obj.VirtualHost`
         :param bool filter_defaults: whether a vhost with a _default_
             addr is acceptable
 
@@ -814,7 +811,7 @@ class ApacheConfigurator(common.Installer):
         """Helper function for get_virtual_hosts().
 
         :param host: In progress vhost whose names will be added
-        :type host: :class:`~certbot_apache.obj.VirtualHost`
+        :type host: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         """
 
@@ -833,7 +830,7 @@ class ApacheConfigurator(common.Installer):
         :param str path: Augeas path to virtual host
 
         :returns: newly created vhost
-        :rtype: :class:`~certbot_apache.obj.VirtualHost`
+        :rtype: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         """
         addrs = set()
@@ -874,7 +871,7 @@ class ApacheConfigurator(common.Installer):
     def get_virtual_hosts(self):
         """Returns list of virtual hosts found in the Apache configuration.
 
-        :returns: List of :class:`~certbot_apache.obj.VirtualHost`
+        :returns: List of :class:`~certbot_apache._internal.obj.VirtualHost`
             objects found in configuration
         :rtype: list
 
@@ -931,7 +928,7 @@ class ApacheConfigurator(common.Installer):
         now NameVirtualHosts. If version is earlier than 2.4, check if addr
         has a NameVirtualHost directive in the Apache config
 
-        :param certbot_apache.obj.Addr target_addr: vhost address
+        :param certbot_apache._internal.obj.Addr target_addr: vhost address
 
         :returns: Success
         :rtype: bool
@@ -949,19 +946,18 @@ class ApacheConfigurator(common.Installer):
         """Adds NameVirtualHost directive for given address.
 
         :param addr: Address that will be added as NameVirtualHost directive
-        :type addr: :class:`~certbot_apache.obj.Addr`
+        :type addr: :class:`~certbot_apache._internal.obj.Addr`
 
         """
 
         loc = parser.get_aug_path(self.parser.loc["name"])
         if addr.get_port() == "443":
-            path = self.parser.add_dir_to_ifmodssl(
+            self.parser.add_dir_to_ifmodssl(
                 loc, "NameVirtualHost", [str(addr)])
         else:
-            path = self.parser.add_dir(loc, "NameVirtualHost", [str(addr)])
+            self.parser.add_dir(loc, "NameVirtualHost", [str(addr)])
 
-        msg = ("Setting %s to be NameBasedVirtualHost\n"
-               "\tDirective added to %s\n" % (addr, path))
+        msg = "Setting {0} to be NameBasedVirtualHost\n".format(addr)
         logger.debug(msg)
         self.save_notes += msg
 
@@ -1118,7 +1114,7 @@ class ApacheConfigurator(common.Installer):
             if "ssl_module" not in self.parser.modules:
                 self.enable_mod("ssl", temp=temp)
 
-    def make_vhost_ssl(self, nonssl_vhost):  # pylint: disable=too-many-locals
+    def make_vhost_ssl(self, nonssl_vhost):
         """Makes an ssl_vhost version of a nonssl_vhost.
 
         Duplicates vhost and adds default ssl options
@@ -1128,10 +1124,10 @@ class ApacheConfigurator(common.Installer):
         .. note:: This function saves the configuration
 
         :param nonssl_vhost: Valid VH that doesn't have SSLEngine on
-        :type nonssl_vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type nonssl_vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :returns: SSL vhost
-        :rtype: :class:`~certbot_apache.obj.VirtualHost`
+        :rtype: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :raises .errors.PluginError: If more than one virtual host is in
             the file or if plugin is unable to write/read vhost files.
@@ -1369,12 +1365,9 @@ class ApacheConfigurator(common.Installer):
                         result.append(comment)
                         sift = True
 
-                    result.append('\n'.join(
-                        ['# ' + l for l in chunk]))
-                    continue
+                    result.append('\n'.join(['# ' + l for l in chunk]))
                 else:
                     result.append('\n'.join(chunk))
-                    continue
         return result, sift
 
     def _get_vhost_block(self, vhost):
@@ -1502,7 +1495,7 @@ class ApacheConfigurator(common.Installer):
         https://httpd.apache.org/docs/2.2/mod/core.html#namevirtualhost
 
         :param vhost: New virtual host that was recently created.
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         """
         need_to_save = False
@@ -1537,7 +1530,7 @@ class ApacheConfigurator(common.Installer):
         :param str id_str: Id string for matching
 
         :returns: The matched VirtualHost or None
-        :rtype: :class:`~certbot_apache.obj.VirtualHost` or None
+        :rtype: :class:`~certbot_apache._internal.obj.VirtualHost` or None
 
         :raises .errors.PluginError: If no VirtualHost is found
         """
@@ -1554,7 +1547,7 @@ class ApacheConfigurator(common.Installer):
         used for keeping track of VirtualHost directive over time.
 
         :param vhost: Virtual host to add the id
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :returns: The unique ID or None
         :rtype: str or None
@@ -1576,7 +1569,7 @@ class ApacheConfigurator(common.Installer):
         If ID already exists, returns that instead.
 
         :param vhost: Virtual host to add or find the id
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :returns: The unique ID for vhost
         :rtype: str or None
@@ -1614,9 +1607,9 @@ class ApacheConfigurator(common.Installer):
 
         :param str domain: domain to enhance
         :param str enhancement: enhancement type defined in
-            :const:`~certbot.constants.ENHANCEMENTS`
+            :const:`~certbot.plugins.enhancements.ENHANCEMENTS`
         :param options: options for the enhancement
-            See :const:`~certbot.constants.ENHANCEMENTS`
+            See :const:`~certbot.plugins.enhancements.ENHANCEMENTS`
             documentation for appropriate parameter.
 
         :raises .errors.PluginError: If Enhancement is not supported, or if
@@ -1654,7 +1647,7 @@ class ApacheConfigurator(common.Installer):
         """Increase the AutoHSTS max-age value
 
         :param vhost: Virtual host object to modify
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :param str id_str: The unique ID string of VirtualHost
 
@@ -1722,7 +1715,7 @@ class ApacheConfigurator(common.Installer):
             self.parser.find_dir("SSLCertificateKeyFile",
                                  lineage.key_path, vhost.path))
 
-    def _enable_ocsp_stapling(self, ssl_vhost, unused_options):
+    def _enable_ocsp_stapling(self, ssl_vhost, unused_options, prefetch=False):
         """Enables OCSP Stapling
 
         In OCSP, each client (e.g. browser) would have to query the
@@ -1738,13 +1731,16 @@ class ApacheConfigurator(common.Installer):
         .. note:: This function saves the configuration
 
         :param ssl_vhost: Destination of traffic, an ssl enabled vhost
-        :type ssl_vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type ssl_vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :param unused_options: Not currently used
         :type unused_options: Not Available
 
+        :param prefetch: Use OCSP prefetching
+        :type prefetch: bool
+
         :returns: Success, general_vhost (HTTP vhost)
-        :rtype: (bool, :class:`~certbot_apache.obj.VirtualHost`)
+        :rtype: (bool, :class:`~certbot_apache._internal.obj.VirtualHost`)
 
         """
         min_apache_ver = (2, 3, 3)
@@ -1753,8 +1749,15 @@ class ApacheConfigurator(common.Installer):
                 "Unable to set OCSP directives.\n"
                 "Apache version is below 2.3.3.")
 
-        if "socache_shmcb_module" not in self.parser.modules:
-            self.enable_mod("socache_shmcb")
+        if prefetch:
+            if "socache_dbm_module" not in self.parser.modules:
+                self.enable_mod("socache_dbm")
+            cache_path = os.path.join(self.config.work_dir, "ocsp", "ocsp_cache.db")
+            cache_dir = ["dbm:"+cache_path]
+        else:
+            if "socache_shmcb_module" not in self.parser.modules:
+                self.enable_mod("socache_shmcb")
+            cache_dir = ["shmcb:/var/run/apache2/stapling_cache(128000)"]
 
         # Check if there's an existing SSLUseStapling directive on.
         use_stapling_aug_path = self.parser.find_dir("SSLUseStapling",
@@ -1774,9 +1777,7 @@ class ApacheConfigurator(common.Installer):
             self.parser.aug.remove(
                     re.sub(r"/\w*$", "", stapling_cache_aug_path[0]))
 
-        self.parser.add_dir_to_ifmodssl(ssl_vhost_aug_path,
-                "SSLStaplingCache",
-                ["shmcb:/var/run/apache2/stapling_cache(128000)"])
+        self.parser.add_dir_to_ifmodssl(ssl_vhost_aug_path, "SSLStaplingCache", cache_dir)
 
         msg = "OCSP Stapling was enabled on SSL Vhost: %s.\n"%(
                 ssl_vhost.filep)
@@ -1794,14 +1795,14 @@ class ApacheConfigurator(common.Installer):
         .. note:: This function saves the configuration
 
         :param ssl_vhost: Destination of traffic, an ssl enabled vhost
-        :type ssl_vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type ssl_vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :param header_substring: string that uniquely identifies a header.
                 e.g: Strict-Transport-Security, Upgrade-Insecure-Requests.
         :type str
 
         :returns: Success, general_vhost (HTTP vhost)
-        :rtype: (bool, :class:`~certbot_apache.obj.VirtualHost`)
+        :rtype: (bool, :class:`~certbot_apache._internal.obj.VirtualHost`)
 
         :raises .errors.PluginError: If no viable HTTP host can be created or
             set with header header_substring.
@@ -1825,11 +1826,11 @@ class ApacheConfigurator(common.Installer):
                     ssl_vhost.filep)
 
     def _verify_no_matching_http_header(self, ssl_vhost, header_substring):
-        """Checks to see if an there is an existing Header directive that
+        """Checks to see if there is an existing Header directive that
         contains the string header_substring.
 
         :param ssl_vhost: vhost to check
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :param header_substring: string that uniquely identifies a header.
                 e.g: Strict-Transport-Security, Upgrade-Insecure-Requests.
@@ -1866,7 +1867,7 @@ class ApacheConfigurator(common.Installer):
         .. note:: This function saves the configuration
 
         :param ssl_vhost: Destination of traffic, an ssl enabled vhost
-        :type ssl_vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type ssl_vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :param unused_options: Not currently used
         :type unused_options: Not Available
@@ -1951,7 +1952,7 @@ class ApacheConfigurator(common.Installer):
         delete certbot's old rewrite rules and set the new one instead.
 
         :param vhost: vhost to check
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :raises errors.PluginEnhancementAlreadyPresent: When the exact
                 certbot redirection WriteRule exists in virtual host.
@@ -1993,7 +1994,7 @@ class ApacheConfigurator(common.Installer):
         """Checks if there exists a RewriteRule directive in vhost
 
         :param vhost: vhost to check
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :returns: True if a RewriteRule directive exists.
         :rtype: bool
@@ -2007,7 +2008,7 @@ class ApacheConfigurator(common.Installer):
         """Checks if a RewriteEngine directive is on
 
         :param vhost: vhost to check
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         """
         rewrite_engine_path_list = self.parser.find_dir("RewriteEngine", "on",
@@ -2024,10 +2025,10 @@ class ApacheConfigurator(common.Installer):
         """Creates an http_vhost specifically to redirect for the ssl_vhost.
 
         :param ssl_vhost: ssl vhost
-        :type ssl_vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type ssl_vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :returns: tuple of the form
-            (`success`, :class:`~certbot_apache.obj.VirtualHost`)
+            (`success`, :class:`~certbot_apache._internal.obj.VirtualHost`)
         :rtype: tuple
 
         """
@@ -2153,7 +2154,7 @@ class ApacheConfigurator(common.Installer):
                   of this method where available.
 
         :param vhost: vhost to enable
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :raises .errors.NotSupportedError: If filesystem layout is not
             supported.
@@ -2347,7 +2348,7 @@ class ApacheConfigurator(common.Installer):
         Enable the AutoHSTS enhancement for defined domains
 
         :param _unused_lineage: Certificate lineage object, unused
-        :type _unused_lineage: certbot.storage.RenewableCert
+        :type _unused_lineage: certbot._internal.storage.RenewableCert
 
         :param domains: List of domains in certificate to enhance
         :type domains: str
@@ -2390,7 +2391,7 @@ class ApacheConfigurator(common.Installer):
         """Do the initial AutoHSTS deployment to a vhost
 
         :param ssl_vhost: The VirtualHost object to deploy the AutoHSTS
-        :type ssl_vhost: :class:`~certbot_apache.obj.VirtualHost` or None
+        :type ssl_vhost: :class:`~certbot_apache._internal.obj.VirtualHost` or None
 
         :raises errors.PluginEnhancementAlreadyPresent: When already enhanced
 
@@ -2472,7 +2473,7 @@ class ApacheConfigurator(common.Installer):
         and changes the HSTS max-age to a high value.
 
         :param lineage: Certificate lineage object
-        :type lineage: certbot.storage.RenewableCert
+        :type lineage: certbot._internal.storage.RenewableCert
         """
         self._autohsts_fetch_state()
         if not self._autohsts:
@@ -2517,4 +2518,4 @@ class ApacheConfigurator(common.Installer):
         self._autohsts_save_state()
 
 
-AutoHSTSEnhancement.register(ApacheConfigurator)  # pylint: disable=no-member
+AutoHSTSEnhancement.register(ApacheConfigurator)

certbot-apache/certbot_apache/_internal/constants.py

@@ -1,6 +1,7 @@
 """Apache plugin constants."""
 import pkg_resources
 
+from certbot.compat import os
 
 MOD_SSL_CONF_DEST = "options-ssl-apache.conf"
 """Name of the mod_ssl config file as saved in `IConfig.config_dir`."""
@@ -9,6 +10,7 @@ MOD_SSL_CONF_DEST = "options-ssl-apache.conf"
 UPDATED_MOD_SSL_CONF_DIGEST = ".updated-options-ssl-apache-conf-digest.txt"
 """Name of the hash of the updated or informed mod_ssl_conf as saved in `IConfig.config_dir`."""
 
+# NEVER REMOVE A SINGLE HASH FROM THIS LIST UNLESS YOU KNOW EXACTLY WHAT YOU ARE DOING!
 ALL_SSL_OPTIONS_HASHES = [
     '2086bca02db48daf93468332543c60ac6acdb6f0b58c7bfdf578a5d47092f82a',
     '4844d36c9a0f587172d9fa10f4f1c9518e3bcfa1947379f155e16a70a728c21a',
@@ -18,11 +20,15 @@ ALL_SSL_OPTIONS_HASHES = [
     'cfdd7c18d2025836ea3307399f509cfb1ebf2612c87dd600a65da2a8e2f2797b',
     '80720bd171ccdc2e6b917ded340defae66919e4624962396b992b7218a561791',
     'c0c022ea6b8a51ecc8f1003d0a04af6c3f2bc1c3ce506b3c2dfc1f11ef931082',
+    '717b0a89f5e4c39b09a42813ac6e747cfbdeb93439499e73f4f70a1fe1473f20',
+    '0fcdc81280cd179a07ec4d29d3595068b9326b455c488de4b09f585d5dafc137',
+    '86cc09ad5415cd6d5f09a947fe2501a9344328b1e8a8b458107ea903e80baa6c',
+    '06675349e457eae856120cdebb564efe546f0b87399f2264baeb41e442c724c7',
 ]
 """SHA256 hashes of the contents of previous versions of all versions of MOD_SSL_CONF_SRC"""
 
 AUGEAS_LENS_DIR = pkg_resources.resource_filename(
-    "certbot_apache", "augeas_lens")
+    "certbot_apache", os.path.join("_internal", "augeas_lens"))
 """Path to the Augeas lens directory"""
 
 REWRITE_HTTPS_ARGS = [
@@ -61,3 +67,9 @@ AUTOHSTS_FREQ = 172800
 MANAGED_COMMENT = "DO NOT REMOVE - Managed by Certbot"
 MANAGED_COMMENT_ID = MANAGED_COMMENT+", VirtualHost id: {0}"
 """Managed by Certbot comments and the VirtualHost identification template"""
+
+OCSP_APACHE_TTL = 432000
+"""Apache TTL for OCSP response in seconds: 5 days"""
+
+OCSP_INTERNAL_TTL = 86400
+"""Internal TTL for OCSP response in seconds: 1 day"""

certbot-apache/certbot_apache/_internal/display_ops.py

@@ -3,10 +3,10 @@ import logging
 
 import zope.component
 
-import certbot.display.util as display_util
 from certbot import errors
 from certbot import interfaces
 from certbot.compat import os
+import certbot.display.util as display_util
 
 logger = logging.getLogger(__name__)
 
@@ -77,7 +77,7 @@ def _vhost_menu(domain, vhosts):
 
     if free_chars < 2:
         logger.debug("Display size is too small for "
-                     "certbot_apache.display_ops._vhost_menu()")
+                     "certbot_apache._internal.display_ops._vhost_menu()")
         # This runs the edge off the screen, but it doesn't cause an "error"
         filename_size = 1
         disp_name_size = 1

certbot-apache/certbot_apache/_internal/entrypoint.py

@@ -4,18 +4,18 @@
 from distutils.version import LooseVersion  # pylint: disable=no-name-in-module,import-error
 
 from certbot import util
-
-from certbot_apache import configurator
-from certbot_apache import override_arch
-from certbot_apache import override_fedora
-from certbot_apache import override_darwin
-from certbot_apache import override_debian
-from certbot_apache import override_centos
-from certbot_apache import override_gentoo
-from certbot_apache import override_suse
+from certbot_apache._internal import configurator
+from certbot_apache._internal import override_arch
+from certbot_apache._internal import override_centos
+from certbot_apache._internal import override_darwin
+from certbot_apache._internal import override_debian
+from certbot_apache._internal import override_fedora
+from certbot_apache._internal import override_gentoo
+from certbot_apache._internal import override_suse
 
 OVERRIDE_CLASSES = {
     "arch": override_arch.ArchConfigurator,
+    "cloudlinux": override_centos.CentOSConfigurator,
     "darwin": override_darwin.DarwinConfigurator,
     "debian": override_debian.DebianConfigurator,
     "ubuntu": override_debian.DebianConfigurator,
@@ -23,7 +23,10 @@ OVERRIDE_CLASSES = {
     "centos linux": override_centos.CentOSConfigurator,
     "fedora_old": override_centos.CentOSConfigurator,
     "fedora": override_fedora.FedoraConfigurator,
+    "linuxmint": override_debian.DebianConfigurator,
     "ol": override_centos.CentOSConfigurator,
+    "oracle": override_centos.CentOSConfigurator,
+    "redhatenterpriseserver": override_centos.CentOSConfigurator,
     "red hat enterprise linux server": override_centos.CentOSConfigurator,
     "rhel": override_centos.CentOSConfigurator,
     "amazon": override_centos.CentOSConfigurator,
@@ -31,6 +34,9 @@ OVERRIDE_CLASSES = {
     "gentoo base system": override_gentoo.GentooConfigurator,
     "opensuse": override_suse.OpenSUSEConfigurator,
     "suse": override_suse.OpenSUSEConfigurator,
+    "sles": override_suse.OpenSUSEConfigurator,
+    "scientific": override_centos.CentOSConfigurator,
+    "scientific linux": override_centos.CentOSConfigurator,
 }
 
 

certbot-apache/certbot_apache/_internal/http_01.py

@@ -1,20 +1,19 @@
 """A class that performs HTTP-01 challenges for Apache"""
 import logging
 
-from acme.magic_typing import List, Set  # pylint: disable=unused-import, no-name-in-module
-
+from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Set  # pylint: disable=unused-import, no-name-in-module
 from certbot import errors
-from certbot.compat import os
 from certbot.compat import filesystem
+from certbot.compat import os
 from certbot.plugins import common
-
-from certbot_apache.obj import VirtualHost  # pylint: disable=unused-import
-from certbot_apache.parser import get_aug_path
+from certbot_apache._internal.obj import VirtualHost  # pylint: disable=unused-import
+from certbot_apache._internal.parser import get_aug_path
 
 logger = logging.getLogger(__name__)
 
 
-class ApacheHttp01(common.TLSSNI01):
+class ApacheHttp01(common.ChallengePerformer):
     """Class that performs HTTP-01 challenges within the Apache configurator."""
 
     CONFIG_TEMPLATE22_PRE = """\
@@ -195,8 +194,8 @@ class ApacheHttp01(common.TLSSNI01):
 
         if vhost not in self.moded_vhosts:
             logger.debug(
-                "Adding a temporary challenge validation Include for name: %s " +
-                "in: %s", vhost.name, vhost.filep)
+                "Adding a temporary challenge validation Include for name: %s in: %s",
+                vhost.name, vhost.filep)
             self.configurator.parser.add_dir_beginning(
                 vhost.path, "Include", self.challenge_conf_pre)
             self.configurator.parser.add_dir(

certbot-apache/certbot_apache/_internal/obj.py

@@ -1,7 +1,7 @@
 """Module contains classes used by the Apache Configurator."""
 import re
 
-from acme.magic_typing import Set # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Set  # pylint: disable=unused-import, no-name-in-module
 from certbot.plugins import common
 
 
@@ -24,7 +24,7 @@ class Addr(common.Addr):
         return not self.__eq__(other)
 
     def __repr__(self):
-        return "certbot_apache.obj.Addr(" + repr(self.tup) + ")"
+        return "certbot_apache._internal.obj.Addr(" + repr(self.tup) + ")"
 
     def __hash__(self):  # pylint: disable=useless-super-delegation
         # Python 3 requires explicit overridden for __hash__ if __eq__ or
@@ -98,7 +98,7 @@ class Addr(common.Addr):
         return self.get_addr_obj(port)
 
 
-class VirtualHost(object):  # pylint: disable=too-few-public-methods
+class VirtualHost(object):
     """Represents an Apache Virtualhost.
 
     :ivar str filep: file path of VH
@@ -126,7 +126,6 @@ class VirtualHost(object):  # pylint: disable=too-few-public-methods
     def __init__(self, filep, path, addrs, ssl, enabled, name=None,
                  aliases=None, modmacro=False, ancestor=None):
 
-        # pylint: disable=too-many-arguments
         """Initialize a VH."""
         self.filep = filep
         self.path = path

certbot-apache/certbot_apache/_internal/override_arch.py

@@ -1,11 +1,11 @@
 """ Distribution specific override class for Arch Linux """
 import pkg_resources
-
 import zope.interface
 
 from certbot import interfaces
+from certbot.compat import os
+from certbot_apache._internal import configurator
 
-from certbot_apache import configurator
 
 @zope.interface.provider(interfaces.IPluginFactory)
 class ArchConfigurator(configurator.ApacheConfigurator):
@@ -27,5 +27,5 @@ class ArchConfigurator(configurator.ApacheConfigurator):
         handle_sites=False,
         challenge_location="/etc/httpd/conf",
         MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
+            "certbot_apache", os.path.join("_internal", "options-ssl-apache.conf"))
     )

certbot-apache/certbot_apache/_internal/override_centos.py

@@ -4,17 +4,15 @@ import logging
 import pkg_resources
 import zope.interface
 
+from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-module
 from certbot import errors
 from certbot import interfaces
 from certbot import util
+from certbot.compat import os
 from certbot.errors import MisconfigurationError
-
-from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-module
-
-from certbot_apache import apache_util
-from certbot_apache import configurator
-from certbot_apache import parser
-
+from certbot_apache._internal import apache_util
+from certbot_apache._internal import configurator
+from certbot_apache._internal import parser
 
 logger = logging.getLogger(__name__)
 
@@ -40,7 +38,7 @@ class CentOSConfigurator(configurator.ApacheConfigurator):
         handle_sites=False,
         challenge_location="/etc/httpd/conf.d",
         MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "centos-options-ssl-apache.conf")
+            "certbot_apache", os.path.join("_internal", "centos-options-ssl-apache.conf"))
     )
 
     def config_test(self):

certbot-apache/certbot_apache/_internal/override_darwin.py

@@ -1,11 +1,11 @@
 """ Distribution specific override class for macOS """
 import pkg_resources
-
 import zope.interface
 
 from certbot import interfaces
+from certbot.compat import os
+from certbot_apache._internal import configurator
 
-from certbot_apache import configurator
 
 @zope.interface.provider(interfaces.IPluginFactory)
 class DarwinConfigurator(configurator.ApacheConfigurator):
@@ -27,5 +27,5 @@ class DarwinConfigurator(configurator.ApacheConfigurator):
         handle_sites=False,
         challenge_location="/etc/apache2/other",
         MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
+            "certbot_apache", os.path.join("_internal", "options-ssl-apache.conf"))
     )

certbot-apache/certbot_apache/_internal/override_debian.py

@@ -10,14 +10,15 @@ from certbot import util
 from certbot.compat import filesystem
 from certbot.compat import os
 
-from certbot_apache import apache_util
-from certbot_apache import configurator
+from certbot_apache._internal import apache_util
+from certbot_apache._internal import configurator
+from certbot_apache._internal import prefetch_ocsp
 
 logger = logging.getLogger(__name__)
 
 
 @zope.interface.provider(interfaces.IPluginFactory)
-class DebianConfigurator(configurator.ApacheConfigurator):
+class DebianConfigurator(prefetch_ocsp.OCSPPrefetchMixin, configurator.ApacheConfigurator):
     """Debian specific ApacheConfigurator override class"""
 
     OS_DEFAULTS = dict(
@@ -36,7 +37,7 @@ class DebianConfigurator(configurator.ApacheConfigurator):
         handle_sites=True,
         challenge_location="/etc/apache2",
         MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
+            "certbot_apache", os.path.join("_internal", "options-ssl-apache.conf"))
     )
 
     def enable_site(self, vhost):
@@ -46,7 +47,7 @@ class DebianConfigurator(configurator.ApacheConfigurator):
                   modules are enabled appropriately.
 
         :param vhost: vhost to enable
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
+        :type vhost: :class:`~certbot_apache._internal.obj.VirtualHost`
 
         :raises .errors.NotSupportedError: If filesystem layout is not
             supported.
@@ -71,15 +72,14 @@ class DebianConfigurator(configurator.ApacheConfigurator):
                 # Already in shape
                 vhost.enabled = True
                 return None
-            else:
-                logger.warning(
-                    "Could not symlink %s to %s, got error: %s", enabled_path,
-                    vhost.filep, err.strerror)
-                errstring = ("Encountered error while trying to enable a " +
-                             "newly created VirtualHost located at {0} by " +
-                             "linking to it from {1}")
-                raise errors.NotSupportedError(errstring.format(vhost.filep,
-                                                                enabled_path))
+            logger.warning(
+                "Could not symlink %s to %s, got error: %s", enabled_path,
+                vhost.filep, err.strerror)
+            errstring = ("Encountered error while trying to enable a " +
+                         "newly created VirtualHost located at {0} by " +
+                         "linking to it from {1}")
+            raise errors.NotSupportedError(errstring.format(vhost.filep,
+                                                            enabled_path))
         vhost.enabled = True
         logger.info("Enabling available site: %s", vhost.filep)
         self.save_notes += "Enabled site %s\n" % vhost.filep

certbot-apache/certbot_apache/_internal/override_fedora.py

@@ -5,10 +5,10 @@ import zope.interface
 from certbot import errors
 from certbot import interfaces
 from certbot import util
-
-from certbot_apache import apache_util
-from certbot_apache import configurator
-from certbot_apache import parser
+from certbot.compat import os
+from certbot_apache._internal import apache_util
+from certbot_apache._internal import configurator
+from certbot_apache._internal import parser
 
 
 @zope.interface.provider(interfaces.IPluginFactory)
@@ -33,7 +33,7 @@ class FedoraConfigurator(configurator.ApacheConfigurator):
         challenge_location="/etc/httpd/conf.d",
         MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
             # TODO: eventually newest version of Fedora will need their own config
-            "certbot_apache", "centos-options-ssl-apache.conf")
+            "certbot_apache", os.path.join("_internal", "centos-options-ssl-apache.conf"))
     )
 
     def config_test(self):

certbot-apache/certbot_apache/_internal/override_gentoo.py

@@ -1,13 +1,13 @@
 """ Distribution specific override class for Gentoo Linux """
 import pkg_resources
-
 import zope.interface
 
 from certbot import interfaces
+from certbot.compat import os
+from certbot_apache._internal import apache_util
+from certbot_apache._internal import configurator
+from certbot_apache._internal import parser
 
-from certbot_apache import apache_util
-from certbot_apache import configurator
-from certbot_apache import parser
 
 @zope.interface.provider(interfaces.IPluginFactory)
 class GentooConfigurator(configurator.ApacheConfigurator):
@@ -30,7 +30,7 @@ class GentooConfigurator(configurator.ApacheConfigurator):
         handle_sites=False,
         challenge_location="/etc/apache2/vhosts.d",
         MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
+            "certbot_apache", os.path.join("_internal", "options-ssl-apache.conf"))
     )
 
     def _prepare_options(self):

certbot-apache/certbot_apache/_internal/override_suse.py

@@ -1,11 +1,11 @@
 """ Distribution specific override class for OpenSUSE """
 import pkg_resources
-
 import zope.interface
 
 from certbot import interfaces
+from certbot.compat import os
+from certbot_apache._internal import configurator
 
-from certbot_apache import configurator
 
 @zope.interface.provider(interfaces.IPluginFactory)
 class OpenSUSEConfigurator(configurator.ApacheConfigurator):
@@ -27,5 +27,5 @@ class OpenSUSEConfigurator(configurator.ApacheConfigurator):
         handle_sites=False,
         challenge_location="/etc/apache2/vhosts.d",
         MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
+            "certbot_apache", os.path.join("_internal", "options-ssl-apache.conf"))
     )

certbot-apache/certbot_apache/_internal/parser.py

@@ -8,18 +8,17 @@ import sys
 
 import six
 
-from acme.magic_typing import Dict, List, Set  # pylint: disable=unused-import, no-name-in-module
-
+from acme.magic_typing import Dict  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-module
+from acme.magic_typing import Set  # pylint: disable=unused-import, no-name-in-module
 from certbot import errors
 from certbot.compat import os
-
-from certbot_apache import constants
+from certbot_apache._internal import constants
 
 logger = logging.getLogger(__name__)
 
 
 class ApacheParser(object):
-    # pylint: disable=too-many-public-methods
     """Class handles the fine details of parsing the Apache Configuration.
 
     .. todo:: Make parsing general... remove sites-available etc...
@@ -285,8 +284,8 @@ class ApacheParser(object):
                     mods.add(mod_name)
                     mods.add(os.path.basename(mod_filename)[:-2] + "c")
                 else:
-                    logger.debug("Could not read LoadModule directive from " +
-                                 "Augeas path: %s", match_name[6:])
+                    logger.debug("Could not read LoadModule directive from Augeas path: %s",
+                                 match_name[6:])
         self.modules.update(mods)
 
     def update_runtime_variables(self):
@@ -626,7 +625,7 @@ class ApacheParser(object):
         # https://httpd.apache.org/docs/2.4/mod/core.html#include
         for match in matches:
             dir_ = self.aug.get(match).lower()
-            if dir_ == "include" or dir_ == "includeoptional":
+            if dir_ in ("include", "includeoptional"):
                 ordered_matches.extend(self.find_dir(
                     directive, arg,
                     self._get_include_path(self.get_arg(match + "/arg")),
@@ -666,8 +665,7 @@ class ApacheParser(object):
         # e.g. strip now, not later
         if not value:
             return None
-        else:
-            value = value.strip("'\"")
+        value = value.strip("'\"")
 
         variables = ApacheParser.arg_var_interpreter.findall(value)
 
@@ -766,7 +764,7 @@ class ApacheParser(object):
         split_arg = arg.split("/")
         for idx, split in enumerate(split_arg):
             if any(char in ApacheParser.fnmatch_chars for char in split):
-                # Turn it into a augeas regex
+                # Turn it into an augeas regex
                 # TODO: Can this instead be an augeas glob instead of regex
                 split_arg[idx] = ("* [label()=~regexp('%s')]" %
                                   self.fnmatch_to_re(split))

certbot-apache/certbot_apache/_internal/prefetch_ocsp.py

@@ -0,0 +1,394 @@
+"""A mixin class for OCSP response prefetching for Apache plugin.
+
+The OCSP prefetching functionality solves multiple issues in Apache httpd
+that make using OCSP must-staple error prone.
+
+The prefetching functionality works by storing a value to PluginStorage,
+noting certificates that Certbot should keep OCSP staples (OCSP responses)
+updated for alongside of the information when the last response was
+updated by Certbot.
+
+When Certbot is invoked, typically by scheduled "certbot renew" and the
+TTL from "lastupdate" value in PluginStorage entry has expired,
+Certbot then proceeds to fetch a new OCSP response from the OCSP servers
+pointed by the certificate.
+
+The OCSP response is validated and if valid, stored to Apache DBM
+cache. A high internal cache expiry value is set for Apache in order
+to make it to not to discard the stored response and try to renew
+the staple itself letting Certbot to renew it on its subsequent run
+instead.
+
+The DBM cache file used by Apache is a lightweight key-value storage.
+For OCSP response caching, the sha1 hash of certificate fingerprint
+is used as a key. The value consists of expiry time as timestamp
+in microseconds, \x01 delimiter and the raw OCSP response.
+
+When restarting Apache, Certbot backups the current OCSP response
+cache, and restores it after the restart has happened. This is
+done because Apache deletes and then recreates the file upon
+restart.
+"""
+
+from datetime import datetime
+import logging
+import shutil
+import time
+
+from acme.magic_typing import Dict  # pylint: disable=unused-import, no-name-in-module
+
+from certbot import errors
+from certbot._internal import ocsp
+from certbot.plugins.enhancements import OCSPPrefetchEnhancement
+
+from certbot.compat import filesystem
+from certbot.compat import os
+from certbot_apache._internal import apache_util
+from certbot_apache._internal import constants
+
+logger = logging.getLogger(__name__)
+
+
+class DBMHandler(object):
+    """Context manager to handle DBM file reads and writes"""
+
+    def __init__(self, filename, mode):
+        self.filename = filename
+        self.filemode = mode
+        self.bsddb = False
+        self.database = None
+
+    def __enter__(self):
+        """Open the DBM file and return the filehandle"""
+
+        try:
+            import bsddb
+            self.bsddb = True
+            try:
+                self.database = bsddb.hashopen(self.filename, self.filemode)
+            except Exception:
+                raise errors.PluginError("Unable to open dbm database file.")
+        except ImportError:
+            # Python3 doesn't have bsddb module, so we use dbm.ndbm instead
+            import dbm
+            if self.filename.endswith(".db"):
+                self.filename = self.filename[:-3]
+            try:
+                self.database = dbm.ndbm.open(self.filename, self.filemode)  # pylint: disable=no-member
+            except Exception:
+                # This is raised if a file cannot be found
+                raise errors.PluginError("Unable to open dbm database file.")
+        return self.database
+
+    def __exit__(self, *args):
+        """Close the DBM file"""
+        if self.bsddb:
+            self.database.sync()
+        self.database.close()
+
+
+class OCSPPrefetchMixin(object):
+    """OCSPPrefetchMixin implements OCSP response prefetching"""
+
+    def __init__(self, *args, **kwargs):
+        self._ocsp_prefetch = {}  # type: Dict[str, str]
+        # This is required because of python super() call chain.
+        # Additionally, mypy isn't able to figure the chain out and needs to be
+        # disabled for this line. See https://github.com/python/mypy/issues/5887
+        super(OCSPPrefetchMixin, self).__init__(*args, **kwargs)  # type: ignore
+
+    def _ensure_ocsp_dirs(self):
+        """Makes sure that the OCSP directory paths exist."""
+        ocsp_work = os.path.join(self.config.work_dir, "ocsp_work")
+        ocsp_save = os.path.join(self.config.work_dir, "ocsp")
+        for path in [ocsp_work, ocsp_save]:
+            if not os.path.isdir(path):
+                filesystem.makedirs(path, 0o755)
+
+    def _ensure_ocsp_prefetch_compatibility(self):
+        """Make sure that the operating system supports the required libraries
+        to manage Apache DBM files.
+
+        :raises: errors.NotSupportedError
+        """
+        try:
+            import bsddb  # pylint: disable=unused-variable
+        except ImportError:
+            import dbm
+            if not hasattr(dbm, 'ndbm') or dbm.ndbm.library != 'Berkeley DB':  # pylint: disable=no-member
+                msg = ("Unfortunately your operating system does not have a "
+                       "compatible database module available for managing "
+                       "Apache OCSP stapling cache database.")
+                raise errors.NotSupportedError(msg)
+
+    def _ocsp_refresh_needed(self, pf_obj):
+        """Refreshes OCSP response for a certiifcate if it's due
+
+        :param dict pf_obj: OCSP prefetch object from pluginstorage
+
+        :returns: If OCSP response was updated
+        :rtype: bool
+
+        """
+        ttl = pf_obj["lastupdate"] + constants.OCSP_INTERNAL_TTL
+        if ttl < time.time():
+            self._ocsp_refresh(pf_obj["cert_path"], pf_obj["chain_path"])
+            return True
+        return False
+
+
+    def _ocsp_refresh(self, cert_path, chain_path):
+        """Refresh the OCSP response for a certificate
+
+        :param str cert_path: Filesystem path to certificate file
+        :param str chain_path: Filesystem path to certificate chain file
+
+        """
+
+        self._ensure_ocsp_dirs()
+        ocsp_workfile = os.path.join(
+            self.config.work_dir, "ocsp_work",
+            apache_util.certid_sha1_hex(cert_path))
+        handler = ocsp.RevocationChecker()
+        if not handler.ocsp_revoked_by_paths(cert_path, chain_path, ocsp_workfile):
+            # Guaranteed good response
+            cache_path = os.path.join(self.config.work_dir, "ocsp", "ocsp_cache.db")
+            cert_sha = apache_util.certid_sha1(cert_path)
+            # dbm.open automatically adds the file extension
+            self._write_to_dbm(cache_path, cert_sha, self._ocsp_response_dbm(ocsp_workfile))
+        else:
+            logger.warning("Encountered an issue while trying to prefetch OCSP "
+                           "response for certificate: %s", cert_path)
+        # Clean up
+        try:
+            os.remove(ocsp_workfile)
+        except OSError:
+            # The OCSP workfile did not exist because of an OCSP response fetching error
+            return
+
+    def _write_to_dbm(self, filename, key, value):
+        """Helper method to write an OCSP response cache value to DBM.
+
+        :param filename: DBM database filename
+        :param bytes key: Database key name
+        :param bytes value: Database entry value
+        """
+        tmp_file = os.path.join(
+            self.config.work_dir,
+            "ocsp_work",
+            "tmp_" + os.path.basename(filename)
+        )
+
+        apache_util.safe_copy(filename, tmp_file)
+
+        with DBMHandler(tmp_file, 'w') as db:
+            db[key] = value
+
+        shutil.copy2(tmp_file, filename)
+        os.remove(tmp_file)
+
+    def _read_dbm(self, filename):
+        """Helper method for reading the dbm using context manager.
+        Used for tests.
+
+        :param str filename: DBM database filename
+
+        :returns: Dictionary of database keys and values
+        :rtype: dict
+        """
+
+        ret = dict()
+        with DBMHandler(filename, 'r') as db:
+            for k in db.keys():
+                ret[k] = db[k]
+        return ret
+
+    def _ocsp_ttl(self, next_update):
+        """Calculates Apache internal TTL for the next OCSP staple
+        update.
+
+        The resulting TTL is half of the time between now
+        and the time noted by nextUpdate field in OCSP response.
+
+        If nextUpdate value is None, a default value will be
+        used instead.
+
+        :param next_update: datetime value for nextUpdate or None
+
+        :returns: TTL in seconds.
+        :rtype: int
+        """
+
+        if next_update is not None:
+            now = datetime.fromtimestamp(time.time())
+            res_ttl = int((next_update - now).total_seconds())
+            if res_ttl > 0:
+                return res_ttl/2
+        return constants.OCSP_APACHE_TTL
+
+    def _ocsp_response_dbm(self, workfile):
+        """Creates a dbm entry for OCSP response data
+
+        :param str workfile: File path for raw OCSP response
+
+        :returns: OCSP response cache data that Apache can use
+        :rtype: string
+
+        """
+
+        handler = ocsp.RevocationChecker()
+        _, _, next_update = handler.ocsp_times(workfile)
+        ttl = self._ocsp_ttl(next_update)
+
+        with open(workfile, 'rb') as fh:
+            response = fh.read()
+        return apache_util.get_apache_ocsp_struct(ttl, response)
+
+    def _ocsp_prefetch_save(self, cert_path, chain_path):
+        """Saves status of current OCSP prefetch, including the last update
+        time to determine if an update is needed on later run.
+
+        :param str cert_path: Filesystem path to certificate
+        :param str chain_path: Filesystem path to certificate chain file
+
+        """
+        status = {
+            "lastupdate": time.time(),
+            "cert_path": cert_path,
+            "chain_path": chain_path
+        }
+        cert_id = apache_util.certid_sha1_hex(cert_path)
+        self._ocsp_prefetch[cert_id] = status
+        self.storage.put("ocsp_prefetch", self._ocsp_prefetch)
+        self.storage.save()
+
+    def _ocsp_prefetch_fetch_state(self):
+        """
+        Populates the OCSP prefetch state from the pluginstorage.
+        """
+        try:
+            self._ocsp_prefetch = self.storage.fetch("ocsp_prefetch")
+        except KeyError:
+            self._ocsp_prefetch = dict()
+
+    def _ocsp_prefetch_backup_db(self):
+        """
+        Copies the active dbm file to work directory. Logs a debug error
+        message if unable to copy, but does not error out as it would
+        prevent other critical functions that need to be carried out for
+        Apache httpd.
+        """
+        self._ensure_ocsp_dirs()
+        cache_path = os.path.join(self.config.work_dir, "ocsp", "ocsp_cache.db")
+        try:
+            shutil.copy2(cache_path, os.path.join(self.config.work_dir, "ocsp_work"))
+        except IOError:
+            logger.debug("Encountered an issue while trying to backup OCSP dbm file")
+
+    def _ocsp_prefetch_restore_db(self):
+        """
+        Restores the active dbm file from work directory. Logs a debug error
+        message if unable to restore, but does not error out as it would
+        prevent other critical functions that need to be carried out for
+        Apache httpd.
+
+        """
+        self._ensure_ocsp_dirs()
+        cache_path = os.path.join(self.config.work_dir, "ocsp", "ocsp_cache.db")
+        work_file_path = os.path.join(self.config.work_dir, "ocsp_work", "ocsp_cache.db")
+        try:
+            shutil.copy2(work_file_path, cache_path)
+        except IOError:
+            logger.debug("Encountered an issue when trying to restore OCSP dbm file")
+
+    def enable_ocsp_prefetch(self, lineage, domains):
+        """Enable OCSP Stapling and prefetching of the responses.
+
+        In OCSP, each client (e.g. browser) would have to query the
+        OCSP Responder to validate that the site certificate was not revoked.
+
+        Enabling OCSP Stapling, would allow the web-server to query the OCSP
+        Responder, and staple its response to the offered certificate during
+        TLS. i.e. clients would not have to query the OCSP responder.
+
+        """
+
+        # Fail early if we are not able to support this
+        self._ensure_ocsp_prefetch_compatibility()
+        prefetch_vhosts = set()
+        for domain in domains:
+            matched_vhosts = self.choose_vhosts(domain, create_if_no_ssl=False)
+            # We should be handling only SSL vhosts
+            for vh in matched_vhosts:
+                if vh.ssl:
+                    prefetch_vhosts.add(vh)
+
+        if not prefetch_vhosts:
+            raise errors.MisconfigurationError(
+                "Could not find VirtualHost to enable OCSP prefetching on."
+            )
+
+        try:
+            # The try - block is huge, but required for handling rollback properly.
+            for vh in prefetch_vhosts:
+                self._enable_ocsp_stapling(vh, None, prefetch=True)
+
+            self._ensure_ocsp_dirs()
+            self.restart()
+            # Ensure Apache has enough time to properly restart and create the file
+            time.sleep(2)
+            self._ocsp_refresh(lineage.cert_path, lineage.chain_path)
+            self._ocsp_prefetch_save(lineage.cert_path, lineage.chain_path)
+            self.save("Enabled OCSP prefetching")
+        except errors.PluginError as err:
+            # Revert the OCSP prefetch configuration
+            self.recovery_routine()
+            self.restart()
+            msg = ("Encountered an error while trying to enable OCSP prefetch "
+                "enhancement: %s\nOCSP prefetch was not enabled.")
+            raise errors.PluginError(msg % str(err))
+
+    def update_ocsp_prefetch(self, _unused_lineage):
+        """Checks all certificates that are managed by OCSP prefetch, and
+        refreshes OCSP responses for them if required."""
+
+        self._ocsp_prefetch_fetch_state()
+        if not self._ocsp_prefetch:
+            # No OCSP prefetching enabled for any certificate
+            return
+
+        for _, pf in self._ocsp_prefetch.items():
+            if not self._ocsp_refresh_needed(pf):
+                continue
+            # Save the status to pluginstorage
+            self._ocsp_prefetch_save(pf["cert_path"], pf["chain_path"])
+
+    def restart(self):
+        """Reloads the Apache server. When restarting, Apache deletes
+        the DBM cache file used to store OCSP staples. In this override
+        function, Certbot checks the pluginstorage if we're supposed to
+        manage OCSP prefetching. If needed, Certbot will backup the DBM
+        file, restoring it after calling restart.
+
+        :raises .errors.MisconfigurationError: If either the config test
+            or reload fails.
+
+        """
+        if not self._ocsp_prefetch:
+            # Try to populate OCSP prefetch structure from pluginstorage
+            self._ocsp_prefetch_fetch_state()
+        if self._ocsp_prefetch:
+            # OCSP prefetching is enabled, so back up the db
+            self._ocsp_prefetch_backup_db()
+
+        try:
+            # Ignored because mypy doesn't know that this class is used as
+            # a mixin and fails because object has no restart method.
+            super(OCSPPrefetchMixin, self).restart()  # type: ignore
+        finally:
+            if self._ocsp_prefetch:
+                # Restore the backed up dbm database
+                self._ocsp_prefetch_restore_db()
+
+
+OCSPPrefetchEnhancement.register(OCSPPrefetchMixin)  # pylint: disable=no-member

certbot-apache/certbot_apache/tests/__init__.py

@@ -1 +0,0 @@
-"""Certbot Apache Tests"""

certbot-apache/docs/Makefile

@@ -1,192 +0,0 @@
-# Makefile for Sphinx documentation
-#
-
-# You can set these variables from the command line.
-SPHINXOPTS    =
-SPHINXBUILD   = sphinx-build
-PAPER         =
-BUILDDIR      = _build
-
-# User-friendly check for sphinx-build
-ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
-$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
-endif
-
-# Internal variables.
-PAPEROPT_a4     = -D latex_paper_size=a4
-PAPEROPT_letter = -D latex_paper_size=letter
-ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
-# the i18n builder cannot share the environment and doctrees with the others
-I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
-
-.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest coverage gettext
-
-help:
-	@echo "Please use \`make <target>' where <target> is one of"
-	@echo "  html       to make standalone HTML files"
-	@echo "  dirhtml    to make HTML files named index.html in directories"
-	@echo "  singlehtml to make a single large HTML file"
-	@echo "  pickle     to make pickle files"
-	@echo "  json       to make JSON files"
-	@echo "  htmlhelp   to make HTML files and a HTML help project"
-	@echo "  qthelp     to make HTML files and a qthelp project"
-	@echo "  applehelp  to make an Apple Help Book"
-	@echo "  devhelp    to make HTML files and a Devhelp project"
-	@echo "  epub       to make an epub"
-	@echo "  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
-	@echo "  latexpdf   to make LaTeX files and run them through pdflatex"
-	@echo "  latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
-	@echo "  text       to make text files"
-	@echo "  man        to make manual pages"
-	@echo "  texinfo    to make Texinfo files"
-	@echo "  info       to make Texinfo files and run them through makeinfo"
-	@echo "  gettext    to make PO message catalogs"
-	@echo "  changes    to make an overview of all changed/added/deprecated items"
-	@echo "  xml        to make Docutils-native XML files"
-	@echo "  pseudoxml  to make pseudoxml-XML files for display purposes"
-	@echo "  linkcheck  to check all external links for integrity"
-	@echo "  doctest    to run all doctests embedded in the documentation (if enabled)"
-	@echo "  coverage   to run coverage check of the documentation (if enabled)"
-
-clean:
-	rm -rf $(BUILDDIR)/*
-
-html:
-	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
-	@echo
-	@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
-
-dirhtml:
-	$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
-	@echo
-	@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
-
-singlehtml:
-	$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
-	@echo
-	@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
-
-pickle:
-	$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
-	@echo
-	@echo "Build finished; now you can process the pickle files."
-
-json:
-	$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
-	@echo
-	@echo "Build finished; now you can process the JSON files."
-
-htmlhelp:
-	$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
-	@echo
-	@echo "Build finished; now you can run HTML Help Workshop with the" \
-	      ".hhp project file in $(BUILDDIR)/htmlhelp."
-
-qthelp:
-	$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
-	@echo
-	@echo "Build finished; now you can run "qcollectiongenerator" with the" \
-	      ".qhcp project file in $(BUILDDIR)/qthelp, like this:"
-	@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/certbot-apache.qhcp"
-	@echo "To view the help file:"
-	@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/certbot-apache.qhc"
-
-applehelp:
-	$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
-	@echo
-	@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
-	@echo "N.B. You won't be able to view it unless you put it in" \
-	      "~/Library/Documentation/Help or install it in your application" \
-	      "bundle."
-
-devhelp:
-	$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
-	@echo
-	@echo "Build finished."
-	@echo "To view the help file:"
-	@echo "# mkdir -p $$HOME/.local/share/devhelp/certbot-apache"
-	@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/certbot-apache"
-	@echo "# devhelp"
-
-epub:
-	$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
-	@echo
-	@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
-
-latex:
-	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
-	@echo
-	@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
-	@echo "Run \`make' in that directory to run these through (pdf)latex" \
-	      "(use \`make latexpdf' here to do that automatically)."
-
-latexpdf:
-	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
-	@echo "Running LaTeX files through pdflatex..."
-	$(MAKE) -C $(BUILDDIR)/latex all-pdf
-	@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
-
-latexpdfja:
-	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
-	@echo "Running LaTeX files through platex and dvipdfmx..."
-	$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
-	@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
-
-text:
-	$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
-	@echo
-	@echo "Build finished. The text files are in $(BUILDDIR)/text."
-
-man:
-	$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
-	@echo
-	@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
-
-texinfo:
-	$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
-	@echo
-	@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
-	@echo "Run \`make' in that directory to run these through makeinfo" \
-	      "(use \`make info' here to do that automatically)."
-
-info:
-	$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
-	@echo "Running Texinfo files through makeinfo..."
-	make -C $(BUILDDIR)/texinfo info
-	@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
-
-gettext:
-	$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
-	@echo
-	@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
-
-changes:
-	$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
-	@echo
-	@echo "The overview file is in $(BUILDDIR)/changes."
-
-linkcheck:
-	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
-	@echo
-	@echo "Link check complete; look for any errors in the above output " \
-	      "or in $(BUILDDIR)/linkcheck/output.txt."
-
-doctest:
-	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
-	@echo "Testing of doctests in the sources finished, look at the " \
-	      "results in $(BUILDDIR)/doctest/output.txt."
-
-coverage:
-	$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
-	@echo "Testing of coverage in the sources finished, look at the " \
-	      "results in $(BUILDDIR)/coverage/python.txt."
-
-xml:
-	$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
-	@echo
-	@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
-
-pseudoxml:
-	$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
-	@echo
-	@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."

certbot-apache/docs/api/augeas_configurator.rst

@@ -1,5 +0,0 @@
-:mod:`certbot_apache.augeas_configurator`
----------------------------------------------
-
-.. automodule:: certbot_apache.augeas_configurator
-   :members:

certbot-apache/docs/api/configurator.rst

@@ -1,5 +0,0 @@
-:mod:`certbot_apache.configurator`
---------------------------------------
-
-.. automodule:: certbot_apache.configurator
-   :members:

certbot-apache/docs/api/display_ops.rst

@@ -1,5 +0,0 @@
-:mod:`certbot_apache.display_ops`
--------------------------------------
-
-.. automodule:: certbot_apache.display_ops
-   :members:

certbot-apache/docs/api/obj.rst

@@ -1,5 +0,0 @@
-:mod:`certbot_apache.obj`
------------------------------
-
-.. automodule:: certbot_apache.obj
-   :members:

certbot-apache/docs/api/parser.rst

@@ -1,5 +0,0 @@
-:mod:`certbot_apache.parser`
---------------------------------
-
-.. automodule:: certbot_apache.parser
-   :members:

certbot-apache/docs/api/tls_sni_01.rst

@@ -1,5 +0,0 @@
-:mod:`certbot_apache.tls_sni_01`
-------------------------------------
-
-.. automodule:: certbot_apache.tls_sni_01
-   :members:

certbot-apache/docs/conf.py

@@ -1,318 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-# certbot-apache documentation build configuration file, created by
-# sphinx-quickstart on Sun Oct 18 13:39:26 2015.
-#
-# This file is execfile()d with the current directory set to its
-# containing dir.
-#
-# Note that not all possible configuration values are present in this
-# autogenerated file.
-#
-# All configuration values have a default; values that are commented out
-# serve to show the default.
-
-import sys
-from certbot.compat import os
-import shlex
-
-import mock
-
-
-# http://docs.readthedocs.org/en/latest/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
-# c.f. #262
-sys.modules.update(
-    (mod_name, mock.MagicMock()) for mod_name in ['augeas'])
-
-here = os.path.abspath(os.path.dirname(__file__))
-
-# If extensions (or modules to document with autodoc) are in another directory,
-# add these directories to sys.path here. If the directory is relative to the
-# documentation root, use os.path.abspath to make it absolute, like shown here.
-sys.path.insert(0, os.path.abspath(os.path.join(here, '..')))
-
-# -- General configuration ------------------------------------------------
-
-# If your documentation needs a minimal Sphinx version, state it here.
-needs_sphinx = '1.0'
-
-# Add any Sphinx extension module names here, as strings. They can be
-# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
-# ones.
-extensions = [
-    'sphinx.ext.autodoc',
-    'sphinx.ext.intersphinx',
-    'sphinx.ext.todo',
-    'sphinx.ext.coverage',
-    'sphinx.ext.viewcode',
-]
-
-autodoc_member_order = 'bysource'
-autodoc_default_flags = ['show-inheritance', 'private-members']
-
-# Add any paths that contain templates here, relative to this directory.
-templates_path = ['_templates']
-
-# The suffix(es) of source filenames.
-# You can specify multiple suffix as a list of string:
-# source_suffix = ['.rst', '.md']
-source_suffix = '.rst'
-
-# The encoding of source files.
-#source_encoding = 'utf-8-sig'
-
-# The master toctree document.
-master_doc = 'index'
-
-# General information about the project.
-project = u'certbot-apache'
-copyright = u'2014-2015, Let\'s Encrypt Project'
-author = u'Certbot Project'
-
-# The version info for the project you're documenting, acts as replacement for
-# |version| and |release|, also used in various other places throughout the
-# built documents.
-#
-# The short X.Y version.
-version = '0'
-# The full version, including alpha/beta/rc tags.
-release = '0'
-
-# The language for content autogenerated by Sphinx. Refer to documentation
-# for a list of supported languages.
-#
-# This is also used if you do content translation via gettext catalogs.
-# Usually you set "language" from the command line for these cases.
-language = 'en'
-
-# There are two options for replacing |today|: either, you set today to some
-# non-false value, then it is used:
-#today = ''
-# Else, today_fmt is used as the format for a strftime call.
-#today_fmt = '%B %d, %Y'
-
-# List of patterns, relative to source directory, that match files and
-# directories to ignore when looking for source files.
-exclude_patterns = ['_build']
-
-# The reST default role (used for this markup: `text`) to use for all
-# documents.
-default_role = 'py:obj'
-
-# If true, '()' will be appended to :func: etc. cross-reference text.
-#add_function_parentheses = True
-
-# If true, the current module name will be prepended to all description
-# unit titles (such as .. function::).
-#add_module_names = True
-
-# If true, sectionauthor and moduleauthor directives will be shown in the
-# output. They are ignored by default.
-#show_authors = False
-
-# The name of the Pygments (syntax highlighting) style to use.
-pygments_style = 'sphinx'
-
-# A list of ignored prefixes for module index sorting.
-#modindex_common_prefix = []
-
-# If true, keep warnings as "system message" paragraphs in the built documents.
-#keep_warnings = False
-
-# If true, `todo` and `todoList` produce output, else they produce nothing.
-todo_include_todos = True
-
-
-# -- Options for HTML output ----------------------------------------------
-
-# The theme to use for HTML and HTML Help pages.  See the documentation for
-# a list of builtin themes.
-
-# http://docs.readthedocs.org/en/latest/theme.html#how-do-i-use-this-locally-and-on-read-the-docs
-# on_rtd is whether we are on readthedocs.org
-on_rtd = os.environ.get('READTHEDOCS', None) == 'True'
-if not on_rtd:  # only import and set the theme if we're building docs locally
-    import sphinx_rtd_theme
-    html_theme = 'sphinx_rtd_theme'
-    html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
-# otherwise, readthedocs.org uses their theme by default, so no need to specify it
-
-# Theme options are theme-specific and customize the look and feel of a theme
-# further.  For a list of options available for each theme, see the
-# documentation.
-#html_theme_options = {}
-
-# Add any paths that contain custom themes here, relative to this directory.
-#html_theme_path = []
-
-# The name for this set of Sphinx documents.  If None, it defaults to
-# "<project> v<release> documentation".
-#html_title = None
-
-# A shorter title for the navigation bar.  Default is the same as html_title.
-#html_short_title = None
-
-# The name of an image file (relative to this directory) to place at the top
-# of the sidebar.
-#html_logo = None
-
-# The name of an image file (within the static path) to use as favicon of the
-# docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
-# pixels large.
-#html_favicon = None
-
-# Add any paths that contain custom static files (such as style sheets) here,
-# relative to this directory. They are copied after the builtin static files,
-# so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ['_static']
-
-# Add any extra paths that contain custom files (such as robots.txt or
-# .htaccess) here, relative to this directory. These files are copied
-# directly to the root of the documentation.
-#html_extra_path = []
-
-# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
-# using the given strftime format.
-#html_last_updated_fmt = '%b %d, %Y'
-
-# If true, SmartyPants will be used to convert quotes and dashes to
-# typographically correct entities.
-#html_use_smartypants = True
-
-# Custom sidebar templates, maps document names to template names.
-#html_sidebars = {}
-
-# Additional templates that should be rendered to pages, maps page names to
-# template names.
-#html_additional_pages = {}
-
-# If false, no module index is generated.
-#html_domain_indices = True
-
-# If false, no index is generated.
-#html_use_index = True
-
-# If true, the index is split into individual pages for each letter.
-#html_split_index = False
-
-# If true, links to the reST sources are added to the pages.
-#html_show_sourcelink = True
-
-# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
-#html_show_sphinx = True
-
-# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
-#html_show_copyright = True
-
-# If true, an OpenSearch description file will be output, and all pages will
-# contain a <link> tag referring to it.  The value of this option must be the
-# base URL from which the finished HTML is served.
-#html_use_opensearch = ''
-
-# This is the file name suffix for HTML files (e.g. ".xhtml").
-#html_file_suffix = None
-
-# Language to be used for generating the HTML full-text search index.
-# Sphinx supports the following languages:
-#   'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja'
-#   'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr'
-#html_search_language = 'en'
-
-# A dictionary with options for the search language support, empty by default.
-# Now only 'ja' uses this config value
-#html_search_options = {'type': 'default'}
-
-# The name of a javascript file (relative to the configuration directory) that
-# implements a search results scorer. If empty, the default will be used.
-#html_search_scorer = 'scorer.js'
-
-# Output file base name for HTML help builder.
-htmlhelp_basename = 'certbot-apachedoc'
-
-# -- Options for LaTeX output ---------------------------------------------
-
-latex_elements = {
-    # The paper size ('letterpaper' or 'a4paper').
-    #'papersize': 'letterpaper',
-
-    # The font size ('10pt', '11pt' or '12pt').
-    #'pointsize': '10pt',
-
-    # Additional stuff for the LaTeX preamble.
-    #'preamble': '',
-
-    # Latex figure (float) alignment
-    #'figure_align': 'htbp',
-}
-
-# Grouping the document tree into LaTeX files. List of tuples
-# (source start file, target name, title,
-#  author, documentclass [howto, manual, or own class]).
-latex_documents = [
-    (master_doc, 'certbot-apache.tex', u'certbot-apache Documentation',
-     u'Certbot Project', 'manual'),
-]
-
-# The name of an image file (relative to this directory) to place at the top of
-# the title page.
-#latex_logo = None
-
-# For "manual" documents, if this is true, then toplevel headings are parts,
-# not chapters.
-#latex_use_parts = False
-
-# If true, show page references after internal links.
-#latex_show_pagerefs = False
-
-# If true, show URL addresses after external links.
-#latex_show_urls = False
-
-# Documents to append as an appendix to all manuals.
-#latex_appendices = []
-
-# If false, no module index is generated.
-#latex_domain_indices = True
-
-
-# -- Options for manual page output ---------------------------------------
-
-# One entry per manual page. List of tuples
-# (source start file, name, description, authors, manual section).
-man_pages = [
-    (master_doc, 'certbot-apache', u'certbot-apache Documentation',
-     [author], 1)
-]
-
-# If true, show URL addresses after external links.
-#man_show_urls = False
-
-
-# -- Options for Texinfo output -------------------------------------------
-
-# Grouping the document tree into Texinfo files. List of tuples
-# (source start file, target name, title, author,
-#  dir menu entry, description, category)
-texinfo_documents = [
-    (master_doc, 'certbot-apache', u'certbot-apache Documentation',
-     author, 'certbot-apache', 'One line description of project.',
-     'Miscellaneous'),
-]
-
-# Documents to append as an appendix to all manuals.
-#texinfo_appendices = []
-
-# If false, no module index is generated.
-#texinfo_domain_indices = True
-
-# How to display URL addresses: 'footnote', 'no', or 'inline'.
-#texinfo_show_urls = 'footnote'
-
-# If true, do not generate a @detailmenu in the "Top" node's menu.
-#texinfo_no_detailmenu = False
-
-
-intersphinx_mapping = {
-    'python': ('https://docs.python.org/', None),
-    'acme': ('https://acme-python.readthedocs.org/en/latest/', None),
-    'certbot': ('https://certbot.eff.org/docs/', None),
-}