remove the backup prompt

.azure-pipelines/INSTALL.md

@@ -1,119 +0,0 @@
-# Configuring Azure Pipelines with Certbot
-
-Let's begin. All pipelines are defined in `.azure-pipelines`. Currently there are two:
-* `.azure-pipelines/main.yml` is the main one, executed on PRs for master, and pushes to master,
-* `.azure-pipelines/advanced.yml` add installer testing on top of the main pipeline, and is executed for `test-*` branches, release branches, and nightly run for master.
-
-Several templates are defined in `.azure-pipelines/templates`. These YAML files aggregate common jobs configuration that can be reused in several pipelines.
-
-Unlike Travis, where CodeCov is working without any action required, CodeCov supports Azure Pipelines
-using the coverage-bash utility (not python-coverage for now) only if you provide the Codecov repo token
-using the `CODECOV_TOKEN` environment variable. So `CODECOV_TOKEN` needs to be set as a secured
-environment variable to allow the main pipeline to publish coverage reports to CodeCov.
-
-This INSTALL.md file explains how to configure Azure Pipelines with Certbot in order to execute the CI/CD logic defined in `.azure-pipelines` folder with it.
-During this installation step, warnings describing user access and legal comitments will be displayed like this:
-```
-!!! ACCESS REQUIRED !!!
-```
-
-This document suppose that the Azure DevOps organization is named _certbot_, and the Azure DevOps project is also _certbot_.
-
-## Useful links
-
-* https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema
-* https://www.azuredevopslabs.com/labs/azuredevops/github-integration/
-* https://docs.microsoft.com/en-us/azure/devops/pipelines/ecosystems/python?view=azure-devops
-
-## Prerequisites
-
-### Having a GitHub account
-
-Use your GitHub user for a normal GitHub account, or a user that has administrative rights to the GitHub organization if relevant.
-
-### Having an Azure DevOps account
-- Go to https://dev.azure.com/, click "Start free with GitHub"
-- Login to GitHub
-
-```
-!!! ACCESS REQUIRED !!!
-Personal user data (email + profile info, in read-only)
-```
-
-- Microsoft will create a Live account using the email referenced for the GitHub account. This account is also linked to GitHub account (meaning you can log it using GitHub authentication)
-- Proceed with account registration (birth date, country), add details about name and email contact
-
-```
-!!! ACCESS REQUIRED !!!
-Microsoft proposes to send commercial links to this mail
-Azure DevOps terms of service need to be accepted
-```
-
-_Logged to Azure DevOps, account is ready._
-
-### Installing Azure Pipelines to GitHub
-
-- On GitHub, go to Marketplace
-- Select Azure Pipeline, and "Set up a plan"
-- Select Free, then "Install it for free"
-- Click "Complete order and begin installation"
-
-```
-!!! ACCESS !!!
-Azure Pipeline needs RW on code, RO on metadata, RW on checks, commit statuses, deployments, issues, pull requests.
-RW access here is required to allow update of the pipelines YAML files from Azure DevOps interface, and to
-update the status of builds and PRs on GitHub side when Azure Pipelines are triggered.
-Note however that no admin access is defined here: this means that Azure Pipelines cannot do anything with
-protected branches, like master, and cannot modify the security context around this on GitHub.
-Access can be defined for all or only selected repositories, which is nice.
-```
-
-- Redirected to Azure DevOps, select the account created in _Having an Azure DevOps account_ section.
-- Select the organization, and click "Create a new project" (let's name it the same than the targetted github repo)
-- The Visibility is public, to profit from 10 parallel jobs
-
-```
-!!! ACCESS !!!
-Azure Pipelines needs access to the GitHub account (in term of beeing able to check it is valid), and the Resources shared between the GitHub account and Azure Pipelines.
-```
-
-_Done. We can move to pipelines configuration._
-
-## Import an existing pipelines from `.azure-pipelines` folder
-
-- On Azure DevOps, go to your organization (eg. _certbot_) then your project (eg. _certbot_)
-- Click "Pipelines" tab
-- Click "New pipeline"
-- Where is your code?: select "__Use the classic editor__"
-
-__Warning: Do not choose the GitHub option in Where is your code? section. Indeed, this option will trigger an OAuth
-grant permissions from Azure Pipelines to GitHub in order to setup a GitHub OAuth Application. The permissions asked
-then are way too large (admin level on almost everything), while the classic approach does not add any more
-permissions, and works perfectly well.__
-
-- Select GitHub in "Select your repository section", choose certbot/certbot in Repository, master in default branch.
-- Click on YAML option for "Select a template"
-- Choose a name for the pipeline (eg. test-pipeline), and browse to the actual pipeline YAML definition in the
-  "YAML file path" input (eg. `.azure-pipelines/test-pipeline.yml`)
-- Click "Save & queue", choose the master branch to build the first pipeline, and click "Save and run" button.
-
-_Done. Pipeline is operational. Repeat to add more pipelines from existing YAML files in `.azure-pipelines`._
-
-## Add a secret variable to a pipeline (like `CODECOV_TOKEN`)
-
-__NB: Following steps suppose that you already setup the YAML pipeline file to
-consume the secret variable that these steps will create as an environment variable.
-For a variable named `CODECOV_TOKEN` consuming the variable `codecov_token`,
-in the YAML file this setup would take the form of the following:
-```
-steps:
-    - script: ./do_something_that_consumes_CODECOV_TOKEN  # Eg. `codecov -F windows`
-      env:
-        CODECOV_TOKEN: $(codecov_token)
-```
-
-To set up a variable that is shared between pipelines, follow the instructions
-at
-https://docs.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups.
-When adding variables to a group, don't forget to tick "Keep this value secret"
-if it shouldn't be shared publcily.

.azure-pipelines/advanced.yml

@@ -1,20 +0,0 @@
-# Advanced pipeline for isolated checks and release purpose
-trigger:
-  - test-*
-  - '*.x'
-pr:
-  - test-*
-# This pipeline is also nightly run on master
-schedules:
-  - cron: "0 4 * * *"
-    displayName: Nightly build
-    branches:
-      include:
-      - master
-    always: true
-
-jobs:
-  # Any addition here should be reflected in the release pipeline.
-  # It is advised to declare all jobs here as templates to improve maintainability.
-  - template: templates/tests-suite.yml
-  - template: templates/installer-tests.yml

.azure-pipelines/main.yml

@@ -1,12 +0,0 @@
-trigger:
-  # apache-parser-v2 is a temporary branch for doing work related to
-  # rewriting the parser in the Apache plugin.
-  - apache-parser-v2
-  - master
-pr:
-  - apache-parser-v2
-  - master
-  - '*.x'
-
-jobs:
-  - template: templates/tests-suite.yml

.azure-pipelines/release.yml

@@ -1,13 +0,0 @@
-# Release pipeline to build and deploy Certbot for Windows for GitHub release tags
-trigger:
-  tags:
-    include:
-      - v*
-pr: none
-
-jobs:
-  # Any addition here should be reflected in the advanced pipeline.
-  # It is advised to declare all jobs here as templates to improve maintainability.
-  - template: templates/tests-suite.yml
-  - template: templates/installer-tests.yml
-  - template: templates/changelog.yml

.azure-pipelines/templates/changelog.yml

@@ -1,14 +0,0 @@
-jobs:
-  - job: changelog
-    pool:
-      vmImage: vs2017-win2016
-    steps:
-      - bash: |
-          CERTBOT_VERSION="$(python -c "import certbot; print(certbot.__version__)")"
-          "${BUILD_REPOSITORY_LOCALPATH}\tools\extract_changelog.py" "${CERTBOT_VERSION}" >> "${BUILD_ARTIFACTSTAGINGDIRECTORY}/release_notes.md"
-        displayName: Prepare changelog
-      - task: PublishPipelineArtifact@1
-        inputs:
-          path: $(Build.ArtifactStagingDirectory)
-          artifact: changelog
-        displayName: Publish changelog

.azure-pipelines/templates/installer-tests.yml

@@ -1,32 +0,0 @@
-jobs:
-  - job: installer
-    pool:
-      vmImage: vs2017-win2016
-    steps:
-      - task: UsePythonVersion@0
-        inputs:
-          versionSpec: 3.7
-          architecture: x86
-          addToPath: true
-      - script: python windows-installer/construct.py
-        displayName: Build Certbot installer
-      - task: CopyFiles@2
-        inputs:
-          sourceFolder: $(System.DefaultWorkingDirectory)/windows-installer/build/nsis
-          contents: '*.exe'
-          targetFolder: $(Build.ArtifactStagingDirectory)
-      - task: PublishPipelineArtifact@1
-        inputs:
-          path: $(Build.ArtifactStagingDirectory)
-          artifact: windows-installer
-        displayName: Publish Windows installer
-      - script: $(Build.ArtifactStagingDirectory)\certbot-beta-installer-win32.exe /S
-        displayName: Install Certbot
-      - script: |
-          python -m venv venv
-          venv\Scripts\python tools\pip_install.py -e certbot-ci
-        displayName: Prepare Certbot-CI
-      - script: |
-          set PATH=%ProgramFiles(x86)%\Certbot\bin;%PATH%
-          venv\Scripts\python -m pytest certbot-ci\certbot_integration_tests\certbot_tests -n 4
-        displayName: Run integration tests

.azure-pipelines/templates/tests-suite.yml

@@ -1,38 +0,0 @@
-jobs:
-  - job: test
-    pool:
-      vmImage: vs2017-win2016
-    strategy:
-      matrix:
-        py35:
-          PYTHON_VERSION: 3.5
-          TOXENV: py35
-        py37-cover:
-          PYTHON_VERSION: 3.7
-          TOXENV: py37-cover
-        integration-certbot:
-          PYTHON_VERSION: 3.7
-          TOXENV: integration-certbot
-          PYTEST_ADDOPTS: --numprocesses 4
-    variables:
-    - group: certbot-common
-    steps:
-    - task: UsePythonVersion@0
-      inputs:
-        versionSpec: $(PYTHON_VERSION)
-        addToPath: true
-    - script: python tools/pip_install.py -U tox coverage
-      displayName: Install dependencies
-    - script: python -m tox
-      displayName: Run tox
-    # We do not require codecov report upload to succeed. So to avoid to break the pipeline if
-    # something goes wrong, each command is suffixed with a command that hides any non zero exit
-    # codes and echoes an informative message instead.
-    - bash: |
-        curl -s https://codecov.io/bash -o codecov-bash || echo "Failed to download codecov-bash"
-        chmod +x codecov-bash || echo "Failed to apply execute permissions on codecov-bash"
-        ./codecov-bash -F windows  || echo "Codecov did not collect coverage reports"
-      condition: in(variables['TOXENV'], 'py37-cover', 'integration-certbot')
-      env:
-        CODECOV_TOKEN: $(codecov_token)
-      displayName: Publish coverage

.codecov.yml

@@ -1,18 +0,0 @@
-coverage:
-  status:
-    project:
-      default: off
-      linux:
-        flags: linux
-        # Fixed target instead of auto set by #7173, can
-        # be removed when flags in Codecov are added back.
-        target: 97.4
-        threshold: 0.1
-        base: auto
-      windows:
-        flags: windows
-        # Fixed target instead of auto set by #7173, can
-        # be removed when flags in Codecov are added back.
-        target: 97.4
-        threshold: 0.1
-        base: auto

.coveragerc

@@ -1,5 +1,3 @@
-[run]
-omit = */setup.py
-
 [report]
-omit = */setup.py
+# show lines missing coverage in output
+show_missing = True

.github/stale.yml

@@ -1,35 +0,0 @@
-# Configuration for https://github.com/marketplace/stale
-
-# Number of days of inactivity before an Issue or Pull Request becomes stale
-daysUntilStale: 365
-
-# Number of days of inactivity before an Issue or Pull Request with the stale label is closed.
-# Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale.
-# When changing this value, be sure to also update markComment below.
-daysUntilClose: 30
-
-# Ignore issues with an assignee (defaults to false)
-exemptAssignees: true
-
-# Label to use when marking as stale
-staleLabel: needs-update
-
-# Comment to post when marking as stale. Set to `false` to disable
-markComment: >
-  We've made a lot of changes to Certbot since this issue was opened. If you
-  still have this issue with an up-to-date version of Certbot, can you please
-  add a comment letting us know? This helps us to better see what issues are
-  still affecting our users. If there is no activity in the next 30 days, this
-  issue will be automatically closed.
-
-# Comment to post when closing a stale Issue or Pull Request.
-closeComment: >
-  This issue has been closed due to lack of activity, but if you think it
-  should be reopened, please open a new issue with a link to this one and we'll
-  take a look.
-
-# Limit the number of actions per hour, from 1-30. Default is 30
-limitPerRun: 1
-
-# Don't mark pull requests as stale.
-only: issues

.gitignore

@@ -6,8 +6,7 @@ dist*/
 /venv*/
 /kgs/
 /.tox/
-/releases*/
-/log*
+/releases/
 letsencrypt.log
 certbot.log
 letsencrypt-auto-source/letsencrypt-auto.sig.lzma.base64
@@ -36,16 +35,3 @@ tests/letstest/*.pem
 tests/letstest/venv/
 
 .venv
-
-# pytest cache
-.cache
-.mypy_cache/
-.pytest_cache/
-
-# docker files
-.docker
-
-# certbot tests
-.certbot_test_workspace
-**/assets/pebble*
-**/assets/challtestsrv*

.pylintrc

@@ -41,7 +41,7 @@ load-plugins=linter_plugin
 # --enable=similarities". If you want to run only the classes checker, but have
 # no Warning level messages displayed, use"--disable=all --enable=classes
 # --disable=W"
-disable=fixme,locally-disabled,locally-enabled,abstract-class-not-used,abstract-class-little-used,bad-continuation,no-self-use,invalid-name,cyclic-import,duplicate-code,design
+disable=fixme,locally-disabled,abstract-class-not-used,abstract-class-little-used,bad-continuation,too-few-public-methods,no-self-use,invalid-name,too-many-instance-attributes,cyclic-import
 # abstract-class-not-used cannot be disabled locally (at least in
 # pylint 1.4.1), same for abstract-class-little-used
 
@@ -251,7 +251,7 @@ ignored-modules=pkg_resources,confargparse,argparse,six.moves,six.moves.urllib
 
 # List of classes names for which member attributes should not be checked
 # (useful for classes with attributes dynamically set).
-ignored-classes=Field,Header,JWS,closing
+ignored-classes=SQLObject
 
 # When zope mode is activated, add a predefined set of Zope acquired attributes
 # to generated-members.
@@ -297,6 +297,40 @@ valid-classmethod-first-arg=cls
 valid-metaclass-classmethod-first-arg=mcs
 
 
+[DESIGN]
+
+# Maximum number of arguments for function / method
+max-args=6
+
+# Argument names that match this expression will be ignored. Default to name
+# with leading underscore
+ignored-argument-names=_.*
+
+# Maximum number of locals for function / method body
+max-locals=15
+
+# Maximum number of return / yield for function / method body
+max-returns=6
+
+# Maximum number of branch for function / method body
+max-branches=12
+
+# Maximum number of statements in function / method body
+max-statements=50
+
+# Maximum number of parents for a class (see R0901).
+max-parents=12
+
+# Maximum number of attributes for a class (see R0902).
+max-attributes=7
+
+# Minimum number of public methods for a class (see R0903).
+min-public-methods=2
+
+# Maximum number of public methods for a class (see R0904).
+max-public-methods=20
+
+
 [EXCEPTIONS]
 
 # Exceptions that will emit a warning when being caught. Defaults to

.travis.yml

@@ -1,17 +1,120 @@
 language: python
-dist: xenial
 
 cache:
     directories:
         - $HOME/.cache/pip
 
+before_install:
+  - '[ $TRAVIS_OS_NAME == linux ] && dpkg -s libaugeas0 || brew install augeas python3'
+
 before_script:
-  - 'if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then ulimit -n 1024 ; fi'
-  # On Travis, the fastest parallelization for integration tests has proved to be 4.
-  - 'if [[ "$TOXENV" == *"integration"* ]]; then export PYTEST_ADDOPTS="--numprocesses 4"; fi'
-  # Use Travis retry feature for farm tests since they are flaky
-  - 'if [[ "$TOXENV" == "travis-test-farm"* ]]; then export TRAVIS_RETRY=travis_retry; fi'
-  - export TOX_TESTENV_PASSENV=TRAVIS
+  - 'if [ $TRAVIS_OS_NAME = osx ] ; then ulimit -n 1024 ; fi'
+
+# using separate envs with different TOXENVs creates 4x1 Travis build
+# matrix, which allows us to clearly distinguish which component under
+# test has failed
+matrix:
+  include:
+    - python: "2.7"
+      env: TOXENV=cover
+    - python: "2.7"
+      env: TOXENV=lint
+    - python: "2.7"
+      env: TOXENV=py27-oldest BOULDER_INTEGRATION=1
+      sudo: required
+      after_failure:
+        - sudo cat /var/log/mysql/error.log
+        - ps aux | grep mysql
+      services: docker
+    - python: "2.6"
+      env: TOXENV=py26 BOULDER_INTEGRATION=1
+      sudo: required
+      after_failure:
+        - sudo cat /var/log/mysql/error.log
+        - ps aux | grep mysql
+      services: docker
+    - python: "2.7"
+      env: TOXENV=py27_install BOULDER_INTEGRATION=1
+      sudo: required
+      after_failure:
+        - sudo cat /var/log/mysql/error.log
+        - ps aux | grep mysql
+      services: docker
+    - sudo: required
+      env: TOXENV=apache_compat
+      services: docker
+      before_install:
+      addons:
+    - sudo: required
+      env: TOXENV=nginx_compat
+      services: docker
+      before_install:
+      addons:
+    - sudo: required
+      env: TOXENV=le_auto_precise
+      services: docker
+      before_install:
+      addons:
+    - sudo: required
+      env: TOXENV=le_auto_trusty
+      services: docker
+      before_install:
+      addons:
+    - sudo: required
+      env: TOXENV=le_auto_wheezy
+      services: docker
+      before_install:
+      addons:
+    - sudo: required
+      env: TOXENV=le_auto_centos6
+      services: docker
+      before_install:
+      addons:
+    - sudo: required
+      env: TOXENV=docker_dev
+      services: docker
+      before_install:
+      addons:
+    - python: "2.7"
+      env: TOXENV=apacheconftest
+      sudo: required
+    - python: "3.3"
+      env: TOXENV=py33 BOULDER_INTEGRATION=1
+      sudo: required
+      after_failure:
+        - sudo cat /var/log/mysql/error.log
+        - ps aux | grep mysql
+      services: docker
+    - python: "3.4"
+      env: TOXENV=py34 BOULDER_INTEGRATION=1
+      sudo: required
+      after_failure:
+        - sudo cat /var/log/mysql/error.log
+        - ps aux | grep mysql
+      services: docker
+    - python: "3.5"
+      env: TOXENV=py35 BOULDER_INTEGRATION=1
+      sudo: required
+      after_failure:
+        - sudo cat /var/log/mysql/error.log
+        - ps aux | grep mysql
+      services: docker
+    - python: "3.6"
+      env: TOXENV=py36 BOULDER_INTEGRATION=1
+      sudo: required
+      after_failure:
+        - sudo cat /var/log/mysql/error.log
+        - ps aux | grep mysql
+      services: docker
+    - python: "2.7"
+      env: TOXENV=nginxroundtrip
+    - language: generic
+      env: TOXENV=py27
+      os: osx
+    - language: generic
+      env: TOXENV=py36
+      os: osx
+
 
 # Only build pushes to the master branch, PRs, and branches beginning with
 # `test-` or of the form `digit(s).digit(s).x`. This reduces the number of
@@ -19,262 +122,31 @@ before_script:
 # is a cap of on the number of simultaneous runs.
 branches:
   only:
-    # apache-parser-v2 is a temporary branch for doing work related to
-    # rewriting the parser in the Apache plugin.
-    - apache-parser-v2
     - master
     - /^\d+\.\d+\.x$/
     - /^test-.*$/
 
-# Jobs for the main test suite are always executed (including on PRs) except for pushes on master.
-not-on-master: &not-on-master
-  if: NOT (type = push AND branch = master)
-
-# Jobs for the extended test suite are executed for cron jobs and pushes to
-# non-development branches. See the explanation for apache-parser-v2 above.
-extended-test-suite: &extended-test-suite
-  if: type = cron OR (type = push AND branch NOT IN (apache-parser-v2, master))
-
-matrix:
-  include:
-    # Main test suite
-    - python: "2.7"
-      env: ACME_SERVER=pebble TOXENV=integration
-      <<: *not-on-master
-
-    # This job is always executed, including on master
-    - python: "2.7"
-      env: TOXENV=py27-cover FYI="py27 tests + code coverage"
-
-    - python: "2.7"
-      env: TOXENV=lint
-      <<: *not-on-master
-    - python: "3.4"
-      env: TOXENV=mypy
-      <<: *not-on-master
-    - python: "3.5"
-      env: TOXENV=mypy
-      <<: *not-on-master
-    - python: "2.7"
-      # Ubuntu Trusty or older must be used because the oldest version of
-      # cryptography we support cannot be compiled against the version of
-      # OpenSSL in Xenial or newer.
-      dist: trusty
-      env: TOXENV='py27-{acme,apache,certbot,dns,nginx}-oldest'
-      <<: *not-on-master
-    - python: "3.4"
-      env: TOXENV=py34
-      <<: *not-on-master
-    - python: "3.7"
-      env: TOXENV=py37
-      <<: *not-on-master
-    - python: "3.8"
-      env: TOXENV=py38
-      <<: *not-on-master
-    - sudo: required
-      env: TOXENV=apache_compat
-      services: docker
-      before_install:
-      addons:
-      <<: *not-on-master
-    - sudo: required
-      env: TOXENV=le_auto_xenial
-      services: docker
-      <<: *not-on-master
-    - python: "2.7"
-      env: TOXENV=apacheconftest-with-pebble
-      <<: *not-on-master
-    - python: "2.7"
-      env: TOXENV=nginxroundtrip
-      <<: *not-on-master
-
-    # Extended test suite on cron jobs and pushes to tested branches other than master
-    - sudo: required
-      env: TOXENV=nginx_compat
-      services: docker
-      before_install:
-      addons:
-      <<: *extended-test-suite
-    - python: "2.7"
-      env:
-        - TOXENV=travis-test-farm-apache2
-        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
-      <<: *extended-test-suite
-    - python: "2.7"
-      env:
-        - TOXENV=travis-test-farm-leauto-upgrades
-        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
-      git:
-        depth: false  # This is needed to have the history to checkout old versions of certbot-auto.
-      <<: *extended-test-suite
-    - python: "2.7"
-      env:
-        - TOXENV=travis-test-farm-certonly-standalone
-        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
-      <<: *extended-test-suite
-    - python: "2.7"
-      env:
-        - TOXENV=travis-test-farm-sdists
-        - secure: "f+j/Lj9s1lcuKo5sEFrlRd1kIAMnIJI4z0MTI7QF8jl9Fkmbx7KECGzw31TNgzrOSzxSapHbcueFYvNCLKST+kE/8ogMZBbwqXfEDuKpyF6BY3uYoJn+wPVE5pIb8Hhe08xPte8TTDSMIyHI3EyTfcAKrIreauoArePvh/cRvSw="
-      <<: *extended-test-suite
-    - python: "3.7"
-      env: TOXENV=py37 CERTBOT_NO_PIN=1
-      <<: *extended-test-suite
-    - python: "2.7"
-      env: ACME_SERVER=boulder-v1 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "2.7"
-      env: ACME_SERVER=boulder-v2 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "2.7"
-      env: ACME_SERVER=boulder-v1 TOXENV=integration-certbot-oldest
-      # Ubuntu Trusty or older must be used because the oldest version of
-      # cryptography we support cannot be compiled against the version of
-      # OpenSSL in Xenial or newer.
-      dist: trusty
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "2.7"
-      env: ACME_SERVER=boulder-v2 TOXENV=integration-certbot-oldest
-      # Ubuntu Trusty or older must be used because the oldest version of
-      # cryptography we support cannot be compiled against the version of
-      # OpenSSL in Xenial or newer.
-      dist: trusty
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "2.7"
-      env: ACME_SERVER=boulder-v1 TOXENV=integration-nginx-oldest
-      # Ubuntu Trusty or older must be used because the oldest version of
-      # cryptography we support cannot be compiled against the version of
-      # OpenSSL in Xenial or newer.
-      dist: trusty
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "2.7"
-      env: ACME_SERVER=boulder-v2 TOXENV=integration-nginx-oldest
-      # Ubuntu Trusty or older must be used because the oldest version of
-      # cryptography we support cannot be compiled against the version of
-      # OpenSSL in Xenial or newer.
-      dist: trusty
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "3.4"
-      env: TOXENV=py34
-      <<: *extended-test-suite
-    - python: "3.5"
-      env: TOXENV=py35
-      <<: *extended-test-suite
-    - python: "3.6"
-      env: TOXENV=py36
-      <<: *extended-test-suite
-    - python: "3.7"
-      env: TOXENV=py37
-      <<: *extended-test-suite
-    - python: "3.8-dev"
-      env: TOXENV=py38
-      <<: *extended-test-suite
-    - python: "3.4"
-      env: ACME_SERVER=boulder-v1 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "3.4"
-      env: ACME_SERVER=boulder-v2 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "3.5"
-      env: ACME_SERVER=boulder-v1 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "3.5"
-      env: ACME_SERVER=boulder-v2 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "3.6"
-      env: ACME_SERVER=boulder-v1 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "3.6"
-      env: ACME_SERVER=boulder-v2 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "3.7"
-      env: ACME_SERVER=boulder-v1 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "3.7"
-      env: ACME_SERVER=boulder-v2 TOXENV=integration
-      sudo: required
-      services: docker
-      <<: *extended-test-suite
-    - python: "3.8-dev"
-      env: ACME_SERVER=boulder-v1 TOXENV=integration
-      <<: *extended-test-suite
-    - python: "3.8-dev"
-      env: ACME_SERVER=boulder-v2 TOXENV=integration
-      <<: *extended-test-suite
-    - sudo: required
-      env: TOXENV=le_auto_jessie
-      services: docker
-      <<: *extended-test-suite
-    - sudo: required
-      env: TOXENV=le_auto_centos6
-      services: docker
-      <<: *extended-test-suite
-    - sudo: required
-      env: TOXENV=docker_dev
-      services: docker
-      addons:
-        apt:
-          packages:  # don't install nginx and apache
-            - libaugeas0
-      <<: *extended-test-suite
-    - language: generic
-      env: TOXENV=py27
-      os: osx
-      # Using this osx_image is a workaround for
-      # https://travis-ci.community/t/xcode-8-3-homebrew-outdated-error/3798.
-      osx_image: xcode10.2
-      addons:
-        homebrew:
-          packages:
-            - augeas
-            - python2
-      <<: *extended-test-suite
-    - language: generic
-      env: TOXENV=py3
-      os: osx
-      # Using this osx_image is a workaround for
-      # https://travis-ci.community/t/xcode-8-3-homebrew-outdated-error/3798.
-      osx_image: xcode10.2
-      addons:
-        homebrew:
-          packages:
-            - augeas
-            - python3
-      <<: *extended-test-suite
-
 # container-based infrastructure
 sudo: false
 
 addons:
+  # Custom /etc/hosts required for simple verification of http-01
+  # and tls-sni-01, and for certbot_test_nginx
+  hosts:
+    - le.wtf
+    - le1.wtf
+    - le2.wtf
+    - le3.wtf
+    - nginx.wtf
+    - boulder
+    - boulder-mysql
+    - boulder-rabbitmq
   apt:
+    sources:
+    - augeas
     packages:  # Keep in sync with letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh and Boulder.
     - python-dev
+    - python-virtualenv
     - gcc
     - libaugeas0
     - libssl-dev
@@ -283,30 +155,22 @@ addons:
     # For certbot-nginx integration testing
     - nginx-light
     - openssl
+    # for apacheconftest
+    - apache2
+    - libapache2-mod-wsgi
+    - libapache2-mod-macro
 
-# tools/pip_install.py is used to pin packages to a known working version
-# except in tests where the environment variable CERTBOT_NO_PIN is set.
-# virtualenv is listed here explicitly to make sure it is upgraded when
-# CERTBOT_NO_PIN is set to work around failures we've seen when using an older
-# version of virtualenv.
-install: 'tools/pip_install.py -U codecov tox virtualenv'
-# Most of the time TRAVIS_RETRY is an empty string, and has no effect on the
-# script command. It is set only to `travis_retry` during farm tests, in
-# order to trigger the Travis retry feature, and compensate the inherent
-# flakiness of these specific tests.
-script: '$TRAVIS_RETRY tox'
+install: "travis_retry pip install tox coveralls"
+script: 'travis_retry tox && ([ "xxx$BOULDER_INTEGRATION" = "xxx" ] || ./tests/travis-integration.sh)'
 
-after_success: '[ "$TOXENV" == "py27-cover" ] && codecov -F linux'
+after_success: '[ "$TOXENV" == "cover" ] && coveralls'
 
 notifications:
   email: false
   irc:
     channels:
-      # This is set to a secure variable to prevent forks from sending
-      # notifications. This value was created by installing
-      # https://github.com/travis-ci/travis.rb and running
-      # `travis encrypt "chat.freenode.net#certbot-devel"`.
-      - secure: "EWW66E2+KVPZyIPR8ViENZwfcup4Gx3/dlimmAZE0WuLwxDCshBBOd3O8Rf6pBokEoZlXM5eDT6XdyJj8n0DLslgjO62pExdunXpbcMwdY7l1ELxX2/UbnDTE6UnPYa09qVBHNG7156Z6yE0x2lH4M9Ykvp0G0cubjPQHylAwo0="
-    on_cancel: never
+      - "chat.freenode.net#letsencrypt"
     on_success: never
     on_failure: always
+    use_notice: true
+    skip_join: true

AUTHORS.md

@@ -5,7 +5,6 @@ Authors
 * Aaron Zuehlke
 * Ada Lovelace
 * [Adam Woodbeck](https://github.com/awoodbeck)
-* [Adrien Ferrand](https://github.com/adferrand)
 * [Aidin Gharibnavaz](https://github.com/aidin36)
 * [AJ ONeal](https://github.com/coolaj86)
 * [Alcaro](https://github.com/Alcaro)
@@ -15,10 +14,8 @@ Authors
 * [Alex Gaynor](https://github.com/alex)
 * [Alex Halderman](https://github.com/jhalderm)
 * [Alex Jordan](https://github.com/strugee)
-* [Alex Zorin](https://github.com/alexzorin)
 * [Amjad Mashaal](https://github.com/TheNavigat)
 * [Andrew Murray](https://github.com/radarhere)
-* [Andrzej Górski](https://github.com/andrzej3393)
 * [Anselm Levskaya](https://github.com/levskaya)
 * [Antoine Jacoutot](https://github.com/ajacoutot)
 * [asaph](https://github.com/asaph)
@@ -78,7 +75,6 @@ Authors
 * [Fabian](https://github.com/faerbit)
 * [Faidon Liambotis](https://github.com/paravoid)
 * [Fan Jiang](https://github.com/tcz001)
-* [Felix Lechner](https://github.com/lechner)
 * [Felix Schwarz](https://github.com/FelixSchwarz)
 * [Felix Yan](https://github.com/felixonmars)
 * [Filip Ochnik](https://github.com/filipochnik)
@@ -128,7 +124,6 @@ Authors
 * [Joubin Jabbari](https://github.com/joubin)
 * [Juho Juopperi](https://github.com/jkjuopperi)
 * [Kane York](https://github.com/riking)
-* [Kenichi Maehashi](https://github.com/kmaehashi)
 * [Kenneth Skovhede](https://github.com/kenkendk)
 * [Kevin Burke](https://github.com/kevinburke)
 * [Kevin London](https://github.com/kevinlondon)
@@ -164,10 +159,8 @@ Authors
 * [Michael Schumacher](https://github.com/schumaml)
 * [Michael Strache](https://github.com/Jarodiv)
 * [Michael Sverdlin](https://github.com/sveder)
-* [Michael Watters](https://github.com/blackknight36)
 * [Michal Moravec](https://github.com/https://github.com/Majkl578)
 * [Michal Papis](https://github.com/mpapis)
-* [Mickaël Schoentgen](https://github.com/BoboTiG)
 * [Minn Soe](https://github.com/MinnSoe)
 * [Min RK](https://github.com/minrk)
 * [Miquel Ruiz](https://github.com/miquelruiz)
@@ -231,7 +224,6 @@ Authors
 * [Stavros Korokithakis](https://github.com/skorokithakis)
 * [Stefan Weil](https://github.com/stweil)
 * [Steve Desmond](https://github.com/stevedesmond-ca)
-* [sydneyli](https://github.com/sydneyli)
 * [Tan Jay Jun](https://github.com/jayjun)
 * [Tapple Gao](https://github.com/tapple)
 * [Telepenin Nikolay](https://github.com/telepenin)

CHANGES.rst

@@ -0,0 +1,8 @@
+ChangeLog
+=========
+
+To see the changes in a given release, view the issues closed in a given
+release's GitHub milestone:
+
+ - `Past releases <https://github.com/certbot/certbot/milestones?state=closed>`_
+ - `Upcoming releases <https://github.com/certbot/certbot/milestones>`_

CODE_OF_CONDUCT.md

@@ -1 +0,0 @@
-This project is governed by [EFF's Public Projects Code of Conduct](https://www.eff.org/pages/eppcode).

CONTRIBUTING.md

@@ -33,5 +33,3 @@ started. In particular, we recommend you read these sections
  - [Finding issues to work on](https://certbot.eff.org/docs/contributing.html#find-issues-to-work-on)
  - [Coding style](https://certbot.eff.org/docs/contributing.html#coding-style)
  - [Submitting a pull request](https://certbot.eff.org/docs/contributing.html#submitting-a-pull-request)
- - [EFF's Public Projects Code of Conduct](https://www.eff.org/pages/eppcode)
-

Dockerfile

@@ -0,0 +1,26 @@
+FROM python:2-alpine
+
+ENTRYPOINT [ "certbot" ]
+EXPOSE 80 443
+VOLUME /etc/letsencrypt /var/lib/letsencrypt
+WORKDIR /opt/certbot
+
+COPY CHANGES.rst README.rst setup.py src/
+COPY acme src/acme
+COPY certbot src/certbot
+
+RUN apk add --no-cache --virtual .certbot-deps \
+        libffi \
+        libssl1.0 \
+        ca-certificates \
+        binutils
+RUN apk add --no-cache --virtual .build-deps \
+        gcc \
+        linux-headers \
+        openssl-dev \
+        musl-dev \
+        libffi-dev \
+    && pip install --no-cache-dir \
+        --editable /opt/certbot/src/acme \
+        --editable /opt/certbot/src \
+    && apk del .build-deps

Dockerfile-dev

@@ -1,21 +1,70 @@
 # This Dockerfile builds an image for development.
-FROM debian:buster
+FROM ubuntu:trusty
+MAINTAINER Jakub Warmuz <jakub@warmuz.org>
+MAINTAINER William Budington <bill@eff.org>
+MAINTAINER Yan <yan@eff.org>
 
-# Note: this only exposes the port to other docker containers.
-EXPOSE 80 443
+# Note: this only exposes the port to other docker containers. You
+# still have to bind to 443@host at runtime, as per the ACME spec.
+EXPOSE 443
+
+# TODO: make sure --config-dir and --work-dir cannot be changed
+# through the CLI (certbot-docker wrapper that uses standalone
+# authenticator and text mode only?)
+VOLUME /etc/letsencrypt /var/lib/letsencrypt
 
 WORKDIR /opt/certbot/src
 
+# no need to mkdir anything:
+# https://docs.docker.com/reference/builder/#copy
+# If <dest> doesn't exist, it is created along with all missing
+# directories in its path.
+
 # TODO: Install Apache/Nginx for plugin development.
-COPY . .
-RUN apt-get update && \
-    apt-get install apache2 git nginx-light -y && \
-    letsencrypt-auto-source/letsencrypt-auto --os-packages-only && \
+COPY letsencrypt-auto-source/letsencrypt-auto /opt/certbot/src/letsencrypt-auto-source/letsencrypt-auto
+RUN /opt/certbot/src/letsencrypt-auto-source/letsencrypt-auto --os-packages-only && \
+    apt-get install python3-dev git -y && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/* \
            /tmp/* \
            /var/tmp/*
 
-RUN VENV_NAME="../venv" python tools/venv.py
+# the above is not likely to change, so by putting it further up the
+# Dockerfile we make sure we cache as much as possible
+
+COPY setup.py README.rst CHANGES.rst MANIFEST.in linter_plugin.py tox.cover.sh tox.ini .pylintrc /opt/certbot/src/
+
+# all above files are necessary for setup.py, however, package source
+# code directory has to be copied separately to a subdirectory...
+# https://docs.docker.com/reference/builder/#copy: "If <src> is a
+# directory, the entire contents of the directory are copied,
+# including filesystem metadata. Note: The directory itself is not
+# copied, just its contents." Order again matters, three files are far
+# more likely to be cached than the whole project directory
+
+COPY certbot /opt/certbot/src/certbot/
+COPY acme /opt/certbot/src/acme/
+COPY certbot-apache /opt/certbot/src/certbot-apache/
+COPY certbot-nginx /opt/certbot/src/certbot-nginx/
+COPY letshelp-certbot /opt/certbot/src/letshelp-certbot/
+COPY certbot-compatibility-test /opt/certbot/src/certbot-compatibility-test/
+COPY tests /opt/certbot/src/tests/
+
+RUN virtualenv --no-site-packages -p python2 /opt/certbot/venv && \
+    /opt/certbot/venv/bin/pip install -U pip && \
+    /opt/certbot/venv/bin/pip install -U setuptools && \
+    /opt/certbot/venv/bin/pip install \
+    -e /opt/certbot/src/acme \
+    -e /opt/certbot/src \
+    -e /opt/certbot/src/certbot-apache \
+    -e /opt/certbot/src/certbot-nginx \
+    -e /opt/certbot/src/letshelp-certbot \
+    -e /opt/certbot/src/certbot-compatibility-test \
+    -e /opt/certbot/src[dev,docs]
+
+# install in editable mode (-e) to save space: it's not possible to
+# "rm -rf /opt/certbot/src" (it's stays in the underlaying image);
+# this might also help in debugging: you can "docker run --entrypoint
+# bash" and investigate, apply patches, etc.
 
 ENV PATH /opt/certbot/venv/bin:$PATH

Dockerfile-old

@@ -0,0 +1,75 @@
+# https://github.com/letsencrypt/letsencrypt/pull/431#issuecomment-103659297
+# it is more likely developers will already have ubuntu:trusty rather
+# than e.g. debian:jessie and image size differences are negligible
+FROM ubuntu:trusty
+MAINTAINER Jakub Warmuz <jakub@warmuz.org>
+MAINTAINER William Budington <bill@eff.org>
+
+# Note: this only exposes the port to other docker containers. You
+# still have to bind to 443@host at runtime, as per the ACME spec.
+EXPOSE 443
+
+# TODO: make sure --config-dir and --work-dir cannot be changed
+# through the CLI (certbot-docker wrapper that uses standalone
+# authenticator and text mode only?)
+VOLUME /etc/letsencrypt /var/lib/letsencrypt
+
+WORKDIR /opt/certbot
+
+# no need to mkdir anything:
+# https://docs.docker.com/reference/builder/#copy
+# If <dest> doesn't exist, it is created along with all missing
+# directories in its path.
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+COPY letsencrypt-auto-source/letsencrypt-auto /opt/certbot/src/letsencrypt-auto-source/letsencrypt-auto
+RUN /opt/certbot/src/letsencrypt-auto-source/letsencrypt-auto --os-packages-only && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/* \
+           /tmp/* \
+           /var/tmp/*
+
+# the above is not likely to change, so by putting it further up the
+# Dockerfile we make sure we cache as much as possible
+
+
+COPY setup.py README.rst CHANGES.rst MANIFEST.in letsencrypt-auto-source/pieces/pipstrap.py /opt/certbot/src/
+
+# all above files are necessary for setup.py and venv setup, however,
+# package source code directory has to be copied separately to a
+# subdirectory...
+# https://docs.docker.com/reference/builder/#copy: "If <src> is a
+# directory, the entire contents of the directory are copied,
+# including filesystem metadata. Note: The directory itself is not
+# copied, just its contents." Order again matters, three files are far
+# more likely to be cached than the whole project directory
+
+COPY certbot /opt/certbot/src/certbot/
+COPY acme /opt/certbot/src/acme/
+COPY certbot-apache /opt/certbot/src/certbot-apache/
+COPY certbot-nginx /opt/certbot/src/certbot-nginx/
+
+
+RUN virtualenv --no-site-packages -p python2 /opt/certbot/venv
+
+# PATH is set now so pipstrap upgrades the correct (v)env
+ENV PATH /opt/certbot/venv/bin:$PATH
+RUN /opt/certbot/venv/bin/python /opt/certbot/src/pipstrap.py && \
+    /opt/certbot/venv/bin/pip install \
+    -e /opt/certbot/src/acme \
+    -e /opt/certbot/src \
+    -e /opt/certbot/src/certbot-apache \
+    -e /opt/certbot/src/certbot-nginx
+
+# install in editable mode (-e) to save space: it's not possible to
+# "rm -rf /opt/certbot/src" (it's stays in the underlaying image);
+# this might also help in debugging: you can "docker run --entrypoint
+# bash" and investigate, apply patches, etc.
+
+# set up certbot/letsencrypt wrapper to warn people about Dockerfile changes
+COPY tools/docker-warning.sh /opt/certbot/bin/certbot
+RUN ln -s /opt/certbot/bin/certbot /opt/certbot/bin/letsencrypt
+ENV PATH /opt/certbot/bin:$PATH
+
+ENTRYPOINT [ "certbot" ]

ISSUE_TEMPLATE.md

@@ -1,9 +1,3 @@
-If you're having trouble using Certbot and aren't sure you've found a bug or
-request for a new feature, please first try asking for help at
-https://community.letsencrypt.org/. There is a much larger community there of
-people familiar with the project who will be able to more quickly answer your
-questions.
-
 ## My operating system is (include version):
 
 

MANIFEST.in

@@ -1,9 +1,8 @@
 include README.rst
-include CHANGELOG.md
+include CHANGES.rst
 include CONTRIBUTING.md
 include LICENSE.txt
 include linter_plugin.py
 recursive-include docs *
 recursive-include examples *
 recursive-include certbot/tests/testdata *
-include certbot/ssl-dhparams.pem

README.rst

@@ -6,7 +6,7 @@ Anyone who has gone through the trouble of setting up a secure website knows wha
 
 How you use Certbot depends on the configuration of your web server. The best way to get started is to use our `interactive guide <https://certbot.eff.org>`_. It generates instructions based on your configuration settings. In most cases, you’ll need `root or administrator access <https://certbot.eff.org/faq/#does-certbot-require-root-administrator-privileges>`_ to your web server to run Certbot.
 
-Certbot is meant to be run directly on your web server, not on your personal computer. If you’re using a hosted service and don’t have direct access to your web server, you might not be able to use Certbot. Check with your hosting provider for documentation about uploading certificates or using certificates issued by Let’s Encrypt.
+If you’re using a hosted service and don’t have direct access to your web server, you might not be able to use Certbot. Check with your hosting provider for documentation about uploading certificates or using certificates issued by Let’s Encrypt.
 
 Certbot is a fully-featured, extensible client for the Let's
 Encrypt CA (or any other CA that speaks the `ACME
@@ -15,9 +15,6 @@ protocol) that can automate the tasks of obtaining certificates and
 configuring webservers to use them. This client runs on Unix-based operating
 systems.
 
-To see the changes made to Certbot between versions please refer to our
-`changelog <https://github.com/certbot/certbot/blob/master/CHANGELOG.md>`_.
-
 Until May 2016, Certbot was named simply ``letsencrypt`` or ``letsencrypt-auto``,
 depending on install method. Instructions on the Internet, and some pieces of the
 software, may still refer to this older name.
@@ -28,19 +25,45 @@ Contributing
 If you'd like to contribute to this project please read `Developer Guide
 <https://certbot.eff.org/docs/contributing.html>`_.
 
-This project is governed by `EFF's Public Projects Code of Conduct <https://www.eff.org/pages/eppcode>`_.
-
 .. _installation:
 
+Installation
+------------
+
+The easiest way to install Certbot is by visiting `certbot.eff.org`_, where you can
+find the correct installation instructions for many web server and OS combinations.
+For more information, see `Get Certbot <https://certbot.eff.org/docs/install.html>`_.
+
+.. _certbot.eff.org: https://certbot.eff.org/
+
 How to run the client
 ---------------------
 
-The easiest way to install and run Certbot is by visiting `certbot.eff.org`_,
-where you can find the correct instructions for many web server and OS
-combinations.  For more information, see `Get Certbot
-<https://certbot.eff.org/docs/install.html>`_.
+In many cases, you can just run ``certbot-auto`` or ``certbot``, and the
+client will guide you through the process of obtaining and installing certs
+interactively.
+
+For full command line help, you can type::
+
+  ./certbot-auto --help all
+
+
+You can also tell it exactly what you want it to do from the command line.
+For instance, if you want to obtain a cert for ``example.com``,
+``www.example.com``, and ``other.example.net``, using the Apache plugin to both
+obtain and install the certs, you could do this::
+
+  ./certbot-auto --apache -d example.com -d www.example.com -d other.example.net
+
+(The first time you run the command, it will make an account, and ask for an
+email and agreement to the Let's Encrypt Subscriber Agreement; you can
+automate those with ``--email`` and ``--agree-tos``)
+
+If you want to use a webserver that doesn't have full plugin support yet, you
+can still use "standalone" or "webroot" plugins to obtain a certificate::
+
+  ./certbot-auto certonly --standalone --email admin@example.com -d example.com -d www.example.com -d other.example.net
 
-.. _certbot.eff.org: https://certbot.eff.org/
 
 Understanding the client in more depth
 --------------------------------------
@@ -65,6 +88,8 @@ Main Website: https://certbot.eff.org
 
 Let's Encrypt Website: https://letsencrypt.org
 
+IRC Channel: #letsencrypt on `Freenode`_
+
 Community: https://community.letsencrypt.org
 
 ACME spec: http://ietf-wg-acme.github.io/acme/
@@ -73,12 +98,14 @@ ACME working area in github: https://github.com/ietf-wg-acme/acme
 
 |build-status| |coverage| |docs| |container|
 
-.. |build-status| image:: https://travis-ci.com/certbot/certbot.svg?branch=master
-   :target: https://travis-ci.com/certbot/certbot
+.. _Freenode: https://webchat.freenode.net?channels=%23letsencrypt
+
+.. |build-status| image:: https://travis-ci.org/certbot/certbot.svg?branch=master
+   :target: https://travis-ci.org/certbot/certbot
    :alt: Travis CI status
 
-.. |coverage| image:: https://codecov.io/gh/certbot/certbot/branch/master/graph/badge.svg
-   :target: https://codecov.io/gh/certbot/certbot
+.. |coverage| image:: https://coveralls.io/repos/certbot/certbot/badge.svg?branch=master
+   :target: https://coveralls.io/r/certbot/certbot
    :alt: Coverage status
 
 .. |docs| image:: https://readthedocs.org/projects/letsencrypt/badge/

acme/MANIFEST.in

@@ -1,6 +1,5 @@
 include LICENSE.txt
 include README.rst
-include pytest.ini
 recursive-include docs *
 recursive-include examples *
 recursive-include acme/testdata *

acme/acme/__init__.py

@@ -1,23 +1,12 @@
 """ACME protocol implementation.
 
-This module is an implementation of the `ACME protocol`_.
+This module is an implementation of the `ACME protocol`_. Latest
+supported version: `draft-ietf-acme-01`_.
+
 
 .. _`ACME protocol`: https://ietf-wg-acme.github.io/acme
 
+.. _`draft-ietf-acme-01`:
+  https://github.com/ietf-wg-acme/acme/tree/draft-ietf-acme-acme-01
+
 """
-import sys
-import warnings
-
-# This code exists to keep backwards compatibility with people using acme.jose
-# before it became the standalone josepy package.
-#
-# It is based on
-# https://github.com/requests/requests/blob/1278ecdf71a312dc2268f3bfc0aabfab3c006dcf/requests/packages.py
-
-import josepy as jose
-
-for mod in list(sys.modules):
-    # This traversal is apparently necessary such that the identities are
-    # preserved (acme.jose.* is josepy.*)
-    if mod == 'josepy' or mod.startswith('josepy.'):
-        sys.modules['acme.' + mod.replace('josepy', 'jose', 1)] = sys.modules[mod]

acme/acme/challenges.py

@@ -3,19 +3,25 @@ import abc
 import functools
 import hashlib
 import logging
+import socket
 
 from cryptography.hazmat.primitives import hashes  # type: ignore
-import josepy as jose
+import OpenSSL
 import requests
-import six
 
+from acme import errors
+from acme import crypto_util
 from acme import fields
+from acme import jose
 
 logger = logging.getLogger(__name__)
 
 
+# pylint: disable=too-few-public-methods
+
+
 class Challenge(jose.TypedJSONObjectWithFields):
-    # _fields_to_partial_json
+    # _fields_to_partial_json | pylint: disable=abstract-method
     """ACME challenge."""
     TYPES = {}  # type: dict
 
@@ -29,7 +35,7 @@ class Challenge(jose.TypedJSONObjectWithFields):
 
 
 class ChallengeResponse(jose.TypedJSONObjectWithFields):
-    # _fields_to_partial_json
+    # _fields_to_partial_json | pylint: disable=abstract-method
     """ACME challenge response."""
     TYPES = {}  # type: dict
     resource_type = 'challenge'
@@ -88,7 +94,6 @@ class _TokenChallenge(Challenge):
         """
         # TODO: check that path combined with uri does not go above
         # URI_ROOT_PATH!
-        # pylint: disable=unsupported-membership-test
         return b'..' not in self.token and b'/' not in self.token
 
 
@@ -133,21 +138,17 @@ class KeyAuthorizationChallengeResponse(ChallengeResponse):
 
         return True
 
-    def to_partial_json(self):
-        jobj = super(KeyAuthorizationChallengeResponse, self).to_partial_json()
-        jobj.pop('keyAuthorization', None)
-        return jobj
 
-
-@six.add_metaclass(abc.ABCMeta)
 class KeyAuthorizationChallenge(_TokenChallenge):
+    # pylint: disable=abstract-class-little-used,too-many-ancestors
     """Challenge based on Key Authorization.
 
     :param response_cls: Subclass of `KeyAuthorizationChallengeResponse`
         that will be used to generate `response`.
-    :param str typ: type of the challenge
+
     """
-    typ = NotImplemented
+    __metaclass__ = abc.ABCMeta
+
     response_cls = NotImplemented
     thumbprint_hash_function = (
         KeyAuthorizationChallengeResponse.thumbprint_hash_function)
@@ -172,7 +173,7 @@ class KeyAuthorizationChallenge(_TokenChallenge):
         :rtype: KeyAuthorizationChallengeResponse
 
         """
-        return self.response_cls(  # pylint: disable=not-callable
+        return self.response_cls(
             key_authorization=self.key_authorization(account_key))
 
     @abc.abstractmethod
@@ -209,7 +210,7 @@ class DNS01Response(KeyAuthorizationChallengeResponse):
     """ACME dns-01 challenge response."""
     typ = "dns-01"
 
-    def simple_verify(self, chall, domain, account_public_key):  # pylint: disable=unused-argument
+    def simple_verify(self, chall, domain, account_public_key):
         """Simple verify.
 
         This method no longer checks DNS records and is a simple wrapper
@@ -225,13 +226,14 @@ class DNS01Response(KeyAuthorizationChallengeResponse):
         :rtype: bool
 
         """
+        # pylint: disable=unused-argument
         verified = self.verify(chall, account_public_key)
         if not verified:
             logger.debug("Verification of key authorization in response failed")
         return verified
 
 
-@Challenge.register
+@Challenge.register  # pylint: disable=too-many-ancestors
 class DNS01(KeyAuthorizationChallenge):
     """ACME dns-01 challenge."""
     response_cls = DNS01Response
@@ -321,7 +323,7 @@ class HTTP01Response(KeyAuthorizationChallengeResponse):
         return True
 
 
-@Challenge.register
+@Challenge.register  # pylint: disable=too-many-ancestors
 class HTTP01(KeyAuthorizationChallenge):
     """ACME http-01 challenge."""
     response_cls = HTTP01Response
@@ -362,33 +364,149 @@ class HTTP01(KeyAuthorizationChallenge):
 
 
 @ChallengeResponse.register
-class TLSALPN01Response(KeyAuthorizationChallengeResponse):
-    """ACME TLS-ALPN-01 challenge response.
+class TLSSNI01Response(KeyAuthorizationChallengeResponse):
+    """ACME tls-sni-01 challenge response."""
+    typ = "tls-sni-01"
 
-    This class only allows initiating a TLS-ALPN-01 challenge returned from the
-    CA. Full support for responding to TLS-ALPN-01 challenges by generating and
-    serving the expected response certificate is not currently provided.
-    """
-    typ = "tls-alpn-01"
+    DOMAIN_SUFFIX = b".acme.invalid"
+    """Domain name suffix."""
 
+    PORT = 443
+    """Verification port as defined by the protocol.
 
-@Challenge.register
-class TLSALPN01(KeyAuthorizationChallenge):
-    """ACME tls-alpn-01 challenge.
-
-    This class simply allows parsing the TLS-ALPN-01 challenge returned from
-    the CA. Full TLS-ALPN-01 support is not currently provided.
+    You can override it (e.g. for testing) by passing ``port`` to
+    `simple_verify`.
 
     """
-    typ = "tls-alpn-01"
-    response_cls = TLSALPN01Response
+
+    @property
+    def z(self):  # pylint: disable=invalid-name
+        """``z`` value used for verification.
+
+        :rtype bytes:
+
+        """
+        return hashlib.sha256(
+            self.key_authorization.encode("utf-8")).hexdigest().lower().encode()
+
+    @property
+    def z_domain(self):
+        """Domain name used for verification, generated from `z`.
+
+        :rtype bytes:
+
+        """
+        return self.z[:32] + b'.' + self.z[32:] + self.DOMAIN_SUFFIX
+
+    def gen_cert(self, key=None, bits=2048):
+        """Generate tls-sni-01 certificate.
+
+        :param OpenSSL.crypto.PKey key: Optional private key used in
+            certificate generation. If not provided (``None``), then
+            fresh key will be generated.
+        :param int bits: Number of bits for newly generated key.
+
+        :rtype: `tuple` of `OpenSSL.crypto.X509` and `OpenSSL.crypto.PKey`
+
+        """
+        if key is None:
+            key = OpenSSL.crypto.PKey()
+            key.generate_key(OpenSSL.crypto.TYPE_RSA, bits)
+        return crypto_util.gen_ss_cert(key, [
+            # z_domain is too big to fit into CN, hence first dummy domain
+            'dummy', self.z_domain.decode()], force_san=True), key
+
+    def probe_cert(self, domain, **kwargs):
+        """Probe tls-sni-01 challenge certificate.
+
+        :param unicode domain:
+
+        """
+        # TODO: domain is not necessary if host is provided
+        if "host" not in kwargs:
+            host = socket.gethostbyname(domain)
+            logger.debug('%s resolved to %s', domain, host)
+            kwargs["host"] = host
+
+        kwargs.setdefault("port", self.PORT)
+        kwargs["name"] = self.z_domain
+        # TODO: try different methods?
+        # pylint: disable=protected-access
+        return crypto_util.probe_sni(**kwargs)
+
+    def verify_cert(self, cert):
+        """Verify tls-sni-01 challenge certificate.
+
+        :param OpensSSL.crypto.X509 cert: Challenge certificate.
+
+        :returns: Whether the certificate was successfully verified.
+        :rtype: bool
+
+        """
+        # pylint: disable=protected-access
+        sans = crypto_util._pyopenssl_cert_or_req_san(cert)
+        logger.debug('Certificate %s. SANs: %s', cert.digest('sha256'), sans)
+        return self.z_domain.decode() in sans
+
+    def simple_verify(self, chall, domain, account_public_key,
+                      cert=None, **kwargs):
+        """Simple verify.
+
+        Verify ``validation`` using ``account_public_key``, optionally
+        probe tls-sni-01 certificate and check using `verify_cert`.
+
+        :param .challenges.TLSSNI01 chall: Corresponding challenge.
+        :param str domain: Domain name being validated.
+        :param JWK account_public_key:
+        :param OpenSSL.crypto.X509 cert: Optional certificate. If not
+            provided (``None``) certificate will be retrieved using
+            `probe_cert`.
+        :param int port: Port used to probe the certificate.
+
+
+        :returns: ``True`` iff client's control of the domain has been
+            verified.
+        :rtype: bool
+
+        """
+        if not self.verify(chall, account_public_key):
+            logger.debug("Verification of key authorization in response failed")
+            return False
+
+        if cert is None:
+            try:
+                cert = self.probe_cert(domain=domain, **kwargs)
+            except errors.Error as error:
+                logger.debug(error, exc_info=True)
+                return False
+
+        return self.verify_cert(cert)
+
+
+@Challenge.register  # pylint: disable=too-many-ancestors
+class TLSSNI01(KeyAuthorizationChallenge):
+    """ACME tls-sni-01 challenge."""
+    response_cls = TLSSNI01Response
+    typ = response_cls.typ
+
+    # boulder#962, ietf-wg-acme#22
+    #n = jose.Field("n", encoder=int, decoder=int)
 
     def validation(self, account_key, **kwargs):
-        """Generate validation for the challenge."""
-        raise NotImplementedError()
+        """Generate validation.
+
+        :param JWK account_key:
+        :param OpenSSL.crypto.PKey cert_key: Optional private key used
+            in certificate generation. If not provided (``None``), then
+            fresh key will be generated.
+
+        :rtype: `tuple` of `OpenSSL.crypto.X509` and `OpenSSL.crypto.PKey`
+
+        """
+        return self.response(account_key).gen_cert(key=kwargs.get('cert_key'))
 
 
-@Challenge.register
+@Challenge.register  # pylint: disable=too-many-ancestors
 class DNS(_TokenChallenge):
     """ACME "dns" challenge."""
     typ = "dns"

acme/acme/challenges_test.py

@@ -1,12 +1,14 @@
 """Tests for acme.challenges."""
 import unittest
 
-import josepy as jose
 import mock
+import OpenSSL
 import requests
 
-from six.moves.urllib import parse as urllib_parse  # pylint: disable=relative-import
+from six.moves.urllib import parse as urllib_parse  # pylint: disable=import-error
 
+from acme import errors
+from acme import jose
 from acme import test_util
 
 CERT = test_util.load_comparable_cert('cert.pem')
@@ -75,6 +77,7 @@ class KeyAuthorizationChallengeResponseTest(unittest.TestCase):
 
 
 class DNS01ResponseTest(unittest.TestCase):
+    # pylint: disable=too-many-instance-attributes
 
     def setUp(self):
         from acme.challenges import DNS01Response
@@ -90,8 +93,7 @@ class DNS01ResponseTest(unittest.TestCase):
         self.response = self.chall.response(KEY)
 
     def test_to_partial_json(self):
-        self.assertEqual({k: v for k, v in self.jmsg.items() if k != 'keyAuthorization'},
-                         self.msg.to_partial_json())
+        self.assertEqual(self.jmsg, self.msg.to_partial_json())
 
     def test_from_json(self):
         from acme.challenges import DNS01Response
@@ -146,6 +148,7 @@ class DNS01Test(unittest.TestCase):
 
 
 class HTTP01ResponseTest(unittest.TestCase):
+    # pylint: disable=too-many-instance-attributes
 
     def setUp(self):
         from acme.challenges import HTTP01Response
@@ -161,8 +164,7 @@ class HTTP01ResponseTest(unittest.TestCase):
         self.response = self.chall.response(KEY)
 
     def test_to_partial_json(self):
-        self.assertEqual({k: v for k, v in self.jmsg.items() if k != 'keyAuthorization'},
-                         self.msg.to_partial_json())
+        self.assertEqual(self.jmsg, self.msg.to_partial_json())
 
     def test_from_json(self):
         from acme.challenges import HTTP01Response
@@ -255,42 +257,114 @@ class HTTP01Test(unittest.TestCase):
             self.msg.update(token=b'..').good_token)
 
 
-class TLSALPN01ResponseTest(unittest.TestCase):
+class TLSSNI01ResponseTest(unittest.TestCase):
+    # pylint: disable=too-many-instance-attributes
 
     def setUp(self):
-        from acme.challenges import TLSALPN01Response
-        self.msg = TLSALPN01Response(key_authorization=u'foo')
+        from acme.challenges import TLSSNI01
+        self.chall = TLSSNI01(
+            token=jose.b64decode(b'a82d5ff8ef740d12881f6d3c2277ab2e'))
+
+        self.response = self.chall.response(KEY)
         self.jmsg = {
             'resource': 'challenge',
-            'type': 'tls-alpn-01',
-            'keyAuthorization': u'foo',
+            'type': 'tls-sni-01',
+            'keyAuthorization': self.response.key_authorization,
         }
 
-        from acme.challenges import TLSALPN01
-        self.chall = TLSALPN01(token=(b'x' * 16))
-        self.response = self.chall.response(KEY)
+        # pylint: disable=invalid-name
+        label1 = b'dc38d9c3fa1a4fdcc3a5501f2d38583f'
+        label2 = b'b7793728f084394f2a1afd459556bb5c'
+        self.z = label1 + label2
+        self.z_domain = label1 + b'.' + label2 + b'.acme.invalid'
+        self.domain = 'foo.com'
+
+    def test_z_and_domain(self):
+        self.assertEqual(self.z, self.response.z)
+        self.assertEqual(self.z_domain, self.response.z_domain)
 
     def test_to_partial_json(self):
-        self.assertEqual({k: v for k, v in self.jmsg.items() if k != 'keyAuthorization'},
-                         self.msg.to_partial_json())
+        self.assertEqual(self.jmsg, self.response.to_partial_json())
 
     def test_from_json(self):
-        from acme.challenges import TLSALPN01Response
-        self.assertEqual(self.msg, TLSALPN01Response.from_json(self.jmsg))
+        from acme.challenges import TLSSNI01Response
+        self.assertEqual(self.response, TLSSNI01Response.from_json(self.jmsg))
 
     def test_from_json_hashable(self):
-        from acme.challenges import TLSALPN01Response
-        hash(TLSALPN01Response.from_json(self.jmsg))
+        from acme.challenges import TLSSNI01Response
+        hash(TLSSNI01Response.from_json(self.jmsg))
+
+    @mock.patch('acme.challenges.socket.gethostbyname')
+    @mock.patch('acme.challenges.crypto_util.probe_sni')
+    def test_probe_cert(self, mock_probe_sni, mock_gethostbyname):
+        mock_gethostbyname.return_value = '127.0.0.1'
+        self.response.probe_cert('foo.com')
+        mock_gethostbyname.assert_called_once_with('foo.com')
+        mock_probe_sni.assert_called_once_with(
+            host='127.0.0.1', port=self.response.PORT,
+            name=self.z_domain)
+
+        self.response.probe_cert('foo.com', host='8.8.8.8')
+        mock_probe_sni.assert_called_with(
+            host='8.8.8.8', port=mock.ANY, name=mock.ANY)
+
+        self.response.probe_cert('foo.com', port=1234)
+        mock_probe_sni.assert_called_with(
+            host=mock.ANY, port=1234, name=mock.ANY)
+
+        self.response.probe_cert('foo.com', bar='baz')
+        mock_probe_sni.assert_called_with(
+            host=mock.ANY, port=mock.ANY, name=mock.ANY, bar='baz')
+
+        self.response.probe_cert('foo.com', name=b'xxx')
+        mock_probe_sni.assert_called_with(
+            host=mock.ANY, port=mock.ANY,
+            name=self.z_domain)
+
+    def test_gen_verify_cert(self):
+        key1 = test_util.load_pyopenssl_private_key('rsa512_key.pem')
+        cert, key2 = self.response.gen_cert(key1)
+        self.assertEqual(key1, key2)
+        self.assertTrue(self.response.verify_cert(cert))
+
+    def test_gen_verify_cert_gen_key(self):
+        cert, key = self.response.gen_cert()
+        self.assertTrue(isinstance(key, OpenSSL.crypto.PKey))
+        self.assertTrue(self.response.verify_cert(cert))
+
+    def test_verify_bad_cert(self):
+        self.assertFalse(self.response.verify_cert(
+            test_util.load_cert('cert.pem')))
+
+    def test_simple_verify_bad_key_authorization(self):
+        key2 = jose.JWKRSA.load(test_util.load_vector('rsa256_key.pem'))
+        self.response.simple_verify(self.chall, "local", key2.public_key())
+
+    @mock.patch('acme.challenges.TLSSNI01Response.verify_cert', autospec=True)
+    def test_simple_verify(self, mock_verify_cert):
+        mock_verify_cert.return_value = mock.sentinel.verification
+        self.assertEqual(
+            mock.sentinel.verification, self.response.simple_verify(
+                self.chall, self.domain, KEY.public_key(),
+                cert=mock.sentinel.cert))
+        mock_verify_cert.assert_called_once_with(
+            self.response, mock.sentinel.cert)
+
+    @mock.patch('acme.challenges.TLSSNI01Response.probe_cert')
+    def test_simple_verify_false_on_probe_error(self, mock_probe_cert):
+        mock_probe_cert.side_effect = errors.Error
+        self.assertFalse(self.response.simple_verify(
+            self.chall, self.domain, KEY.public_key()))
 
 
-class TLSALPN01Test(unittest.TestCase):
+class TLSSNI01Test(unittest.TestCase):
 
     def setUp(self):
-        from acme.challenges import TLSALPN01
-        self.msg = TLSALPN01(
+        from acme.challenges import TLSSNI01
+        self.msg = TLSSNI01(
             token=jose.b64decode('a82d5ff8ef740d12881f6d3c2277ab2e'))
         self.jmsg = {
-            'type': 'tls-alpn-01',
+            'type': 'tls-sni-01',
             'token': 'a82d5ff8ef740d12881f6d3c2277ab2e',
         }
 
@@ -298,21 +372,25 @@ class TLSALPN01Test(unittest.TestCase):
         self.assertEqual(self.jmsg, self.msg.to_partial_json())
 
     def test_from_json(self):
-        from acme.challenges import TLSALPN01
-        self.assertEqual(self.msg, TLSALPN01.from_json(self.jmsg))
+        from acme.challenges import TLSSNI01
+        self.assertEqual(self.msg, TLSSNI01.from_json(self.jmsg))
 
     def test_from_json_hashable(self):
-        from acme.challenges import TLSALPN01
-        hash(TLSALPN01.from_json(self.jmsg))
+        from acme.challenges import TLSSNI01
+        hash(TLSSNI01.from_json(self.jmsg))
 
     def test_from_json_invalid_token_length(self):
-        from acme.challenges import TLSALPN01
+        from acme.challenges import TLSSNI01
         self.jmsg['token'] = jose.encode_b64jose(b'abcd')
         self.assertRaises(
-            jose.DeserializationError, TLSALPN01.from_json, self.jmsg)
+            jose.DeserializationError, TLSSNI01.from_json, self.jmsg)
 
-    def test_validation(self):
-        self.assertRaises(NotImplementedError, self.msg.validation, KEY)
+    @mock.patch('acme.challenges.TLSSNI01Response.gen_cert')
+    def test_validation(self, mock_gen_cert):
+        mock_gen_cert.return_value = ('cert', 'key')
+        self.assertEqual(('cert', 'key'), self.msg.validation(
+            KEY, cert_key=mock.sentinel.cert_key))
+        mock_gen_cert.assert_called_once_with(key=mock.sentinel.cert_key)
 
 
 class DNSTest(unittest.TestCase):

acme/acme/client.py

@@ -6,23 +6,18 @@ from email.utils import parsedate_tz
 import heapq
 import logging
 import time
-import re
-import sys
 
 import six
 from six.moves import http_client  # pylint: disable=import-error
-import josepy as jose
+
 import OpenSSL
 import requests
-from requests.adapters import HTTPAdapter
-from requests_toolbelt.adapters.source import SourceAddressAdapter
+import sys
 
-from acme import crypto_util
 from acme import errors
+from acme import jose
 from acme import jws
 from acme import messages
-# pylint: disable=unused-import, no-name-in-module
-from acme.magic_typing import Dict, List, Set, Text
 
 
 logger = logging.getLogger(__name__)
@@ -33,7 +28,6 @@ logger = logging.getLogger(__name__)
 # https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning
 if sys.version_info < (2, 7, 9):  # pragma: no cover
     try:
-        # pylint: disable=no-member
         requests.packages.urllib3.contrib.pyopenssl.inject_into_urllib3()  # type: ignore
     except AttributeError:
         import urllib3.contrib.pyopenssl  # pylint: disable=import-error
@@ -44,23 +38,39 @@ DEFAULT_NETWORK_TIMEOUT = 45
 DER_CONTENT_TYPE = 'application/pkix-cert'
 
 
-class ClientBase(object):
-    """ACME client base object.
+class Client(object):  # pylint: disable=too-many-instance-attributes
+    """ACME client.
+
+    .. todo::
+       Clean up raised error types hierarchy, document, and handle (wrap)
+       instances of `.DeserializationError` raised in `from_json()`.
 
     :ivar messages.Directory directory:
-    :ivar .ClientNetwork net: Client network.
-    :ivar int acme_version: ACME protocol version. 1 or 2.
+    :ivar key: `.JWK` (private)
+    :ivar alg: `.JWASignature`
+    :ivar bool verify_ssl: Verify SSL certificates?
+    :ivar .ClientNetwork net: Client network. Useful for testing. If not
+        supplied, it will be initialized using `key`, `alg` and
+        `verify_ssl`.
+
     """
-    def __init__(self, directory, net, acme_version):
+
+    def __init__(self, directory, key, alg=jose.RS256, verify_ssl=True,
+                 net=None):
         """Initialize.
 
-        :param .messages.Directory directory: Directory Resource
-        :param .ClientNetwork net: Client network.
-        :param int acme_version: ACME protocol version. 1 or 2.
+        :param directory: Directory Resource (`.messages.Directory`) or
+            URI from which the resource will be downloaded.
+
         """
-        self.directory = directory
-        self.net = net
-        self.acme_version = acme_version
+        self.key = key
+        self.net = ClientNetwork(key, alg, verify_ssl) if net is None else net
+
+        if isinstance(directory, six.string_types):
+            self.directory = messages.Directory.from_json(
+                self.net.get(directory).json())
+        else:
+            self.directory = directory
 
     @classmethod
     def _regr_from_response(cls, response, uri=None, terms_of_service=None):
@@ -72,8 +82,28 @@ class ClientBase(object):
             uri=response.headers.get('Location', uri),
             terms_of_service=terms_of_service)
 
+    def register(self, new_reg=None):
+        """Register.
+
+        :param .NewRegistration new_reg:
+
+        :returns: Registration Resource.
+        :rtype: `.RegistrationResource`
+
+        """
+        new_reg = messages.NewRegistration() if new_reg is None else new_reg
+        assert isinstance(new_reg, messages.NewRegistration)
+
+        response = self.net.post(self.directory[new_reg], new_reg)
+        # TODO: handle errors
+        assert response.status_code == http_client.CREATED
+
+        # "Instance of 'Field' has no key/contact member" bug:
+        # pylint: disable=no-member
+        return self._regr_from_response(response)
+
     def _send_recv_regr(self, regr, body):
-        response = self._post(regr.uri, body)
+        response = self.net.post(regr.uri, body)
 
         # TODO: Boulder returns httplib.ACCEPTED
         #assert response.status_code == httplib.OK
@@ -85,15 +115,6 @@ class ClientBase(object):
             response, uri=regr.uri,
             terms_of_service=regr.terms_of_service)
 
-    def _post(self, *args, **kwargs):
-        """Wrapper around self.net.post that adds the acme_version.
-
-        """
-        kwargs.setdefault('acme_version', self.acme_version)
-        if hasattr(self.directory, 'newNonce'):
-            kwargs.setdefault('new_nonce_url', getattr(self.directory, 'newNonce'))
-        return self.net.post(*args, **kwargs)
-
     def update_registration(self, regr, update=None):
         """Update registration.
 
@@ -108,7 +129,6 @@ class ClientBase(object):
         update = regr.body if update is None else update
         body = messages.UpdateRegistration(**dict(update))
         updated_regr = self._send_recv_regr(regr, body=body)
-        self.net.account = updated_regr
         return updated_regr
 
     def deactivate_registration(self, regr):
@@ -123,30 +143,74 @@ class ClientBase(object):
         """
         return self.update_registration(regr, update={'status': 'deactivated'})
 
-    def deactivate_authorization(self, authzr):
-        # type: (messages.AuthorizationResource) -> messages.AuthorizationResource
-        """Deactivate authorization.
+    def query_registration(self, regr):
+        """Query server about registration.
 
-        :param messages.AuthorizationResource authzr: The Authorization resource
-            to be deactivated.
-
-        :returns: The Authorization resource that was deactivated.
-        :rtype: `.AuthorizationResource`
+        :param messages.RegistrationResource: Existing Registration
+            Resource.
 
         """
-        body = messages.UpdateAuthorization(status='deactivated')
-        response = self._post(authzr.uri, body)
-        return self._authzr_from_response(response,
-            authzr.body.identifier, authzr.uri)
+        return self._send_recv_regr(regr, messages.UpdateRegistration())
 
-    def _authzr_from_response(self, response, identifier=None, uri=None):
+    def agree_to_tos(self, regr):
+        """Agree to the terms-of-service.
+
+        Agree to the terms-of-service in a Registration Resource.
+
+        :param regr: Registration Resource.
+        :type regr: `.RegistrationResource`
+
+        :returns: Updated Registration Resource.
+        :rtype: `.RegistrationResource`
+
+        """
+        return self.update_registration(
+            regr.update(body=regr.body.update(agreement=regr.terms_of_service)))
+
+    def _authzr_from_response(self, response, identifier, uri=None):
         authzr = messages.AuthorizationResource(
             body=messages.Authorization.from_json(response.json()),
             uri=response.headers.get('Location', uri))
-        if identifier is not None and authzr.body.identifier != identifier:
+        if authzr.body.identifier != identifier:
             raise errors.UnexpectedUpdate(authzr)
         return authzr
 
+    def request_challenges(self, identifier, new_authzr_uri=None):
+        """Request challenges.
+
+        :param .messages.Identifier identifier: Identifier to be challenged.
+        :param str new_authzr_uri: Deprecated. Do not use.
+
+        :returns: Authorization Resource.
+        :rtype: `.AuthorizationResource`
+
+        """
+        if new_authzr_uri is not None:
+            logger.debug("request_challenges with new_authzr_uri deprecated.")
+        new_authz = messages.NewAuthorization(identifier=identifier)
+        response = self.net.post(self.directory.new_authz, new_authz)
+        # TODO: handle errors
+        assert response.status_code == http_client.CREATED
+        return self._authzr_from_response(response, identifier)
+
+    def request_domain_challenges(self, domain, new_authzr_uri=None):
+        """Request challenges for domain names.
+
+        This is simply a convenience function that wraps around
+        `request_challenges`, but works with domain names instead of
+        generic identifiers. See ``request_challenges`` for more
+        documentation.
+
+        :param str domain: Domain name to be challenged.
+        :param str new_authzr_uri: Deprecated. Do not use.
+
+        :returns: Authorization Resource.
+        :rtype: `.AuthorizationResource`
+
+        """
+        return self.request_challenges(messages.Identifier(
+            typ=messages.IDENTIFIER_FQDN, value=domain), new_authzr_uri)
+
     def answer_challenge(self, challb, response):
         """Answer challenge.
 
@@ -162,7 +226,7 @@ class ClientBase(object):
         :raises .UnexpectedUpdate:
 
         """
-        response = self._post(challb.uri, response)
+        response = self.net.post(challb.uri, response)
         try:
             authzr_uri = response.links['up']['url']
         except KeyError:
@@ -207,150 +271,21 @@ class ClientBase(object):
 
         return datetime.datetime.now() + datetime.timedelta(seconds=seconds)
 
-    def _revoke(self, cert, rsn, url):
-        """Revoke certificate.
+    def poll(self, authzr):
+        """Poll Authorization Resource for status.
 
-        :param .ComparableX509 cert: `OpenSSL.crypto.X509` wrapped in
-            `.ComparableX509`
+        :param authzr: Authorization Resource
+        :type authzr: `.AuthorizationResource`
 
-        :param int rsn: Reason code for certificate revocation.
+        :returns: Updated Authorization Resource and HTTP response.
 
-        :param str url: ACME URL to post to
-
-        :raises .ClientError: If revocation is unsuccessful.
+        :rtype: (`.AuthorizationResource`, `requests.Response`)
 
         """
-        response = self._post(url,
-                              messages.Revocation(
-                                certificate=cert,
-                                reason=rsn))
-        if response.status_code != http_client.OK:
-            raise errors.ClientError(
-                'Successful revocation must return HTTP OK status')
-
-
-class Client(ClientBase):
-    """ACME client for a v1 API.
-
-    .. todo::
-       Clean up raised error types hierarchy, document, and handle (wrap)
-       instances of `.DeserializationError` raised in `from_json()`.
-
-    :ivar messages.Directory directory:
-    :ivar key: `josepy.JWK` (private)
-    :ivar alg: `josepy.JWASignature`
-    :ivar bool verify_ssl: Verify SSL certificates?
-    :ivar .ClientNetwork net: Client network. Useful for testing. If not
-        supplied, it will be initialized using `key`, `alg` and
-        `verify_ssl`.
-
-    """
-
-    def __init__(self, directory, key, alg=jose.RS256, verify_ssl=True,
-                 net=None):
-        """Initialize.
-
-        :param directory: Directory Resource (`.messages.Directory`) or
-            URI from which the resource will be downloaded.
-
-        """
-        self.key = key
-        if net is None:
-            net = ClientNetwork(key, alg=alg, verify_ssl=verify_ssl)
-
-        if isinstance(directory, six.string_types):
-            directory = messages.Directory.from_json(
-                net.get(directory).json())
-        super(Client, self).__init__(directory=directory,
-            net=net, acme_version=1)
-
-    def register(self, new_reg=None):
-        """Register.
-
-        :param .NewRegistration new_reg:
-
-        :returns: Registration Resource.
-        :rtype: `.RegistrationResource`
-
-        """
-        new_reg = messages.NewRegistration() if new_reg is None else new_reg
-        response = self._post(self.directory[new_reg], new_reg)
-        # TODO: handle errors
-        assert response.status_code == http_client.CREATED
-
-        # "Instance of 'Field' has no key/contact member" bug:
-        # pylint: disable=no-member
-        return self._regr_from_response(response)
-
-    def query_registration(self, regr):
-        """Query server about registration.
-
-        :param messages.RegistrationResource: Existing Registration
-            Resource.
-
-        """
-        return self._send_recv_regr(regr, messages.UpdateRegistration())
-
-    def agree_to_tos(self, regr):
-        """Agree to the terms-of-service.
-
-        Agree to the terms-of-service in a Registration Resource.
-
-        :param regr: Registration Resource.
-        :type regr: `.RegistrationResource`
-
-        :returns: Updated Registration Resource.
-        :rtype: `.RegistrationResource`
-
-        """
-        return self.update_registration(
-            regr.update(body=regr.body.update(agreement=regr.terms_of_service)))
-
-    def request_challenges(self, identifier, new_authzr_uri=None):
-        """Request challenges.
-
-        :param .messages.Identifier identifier: Identifier to be challenged.
-        :param str new_authzr_uri: Deprecated. Do not use.
-
-        :returns: Authorization Resource.
-        :rtype: `.AuthorizationResource`
-
-        :raises errors.WildcardUnsupportedError: if a wildcard is requested
-
-        """
-        if new_authzr_uri is not None:
-            logger.debug("request_challenges with new_authzr_uri deprecated.")
-
-        if identifier.value.startswith("*"):
-            raise errors.WildcardUnsupportedError(
-                "Requesting an authorization for a wildcard name is"
-                " forbidden by this version of the ACME protocol.")
-
-        new_authz = messages.NewAuthorization(identifier=identifier)
-        response = self._post(self.directory.new_authz, new_authz)
-        # TODO: handle errors
-        assert response.status_code == http_client.CREATED
-        return self._authzr_from_response(response, identifier)
-
-    def request_domain_challenges(self, domain, new_authzr_uri=None):
-        """Request challenges for domain names.
-
-        This is simply a convenience function that wraps around
-        `request_challenges`, but works with domain names instead of
-        generic identifiers. See ``request_challenges`` for more
-        documentation.
-
-        :param str domain: Domain name to be challenged.
-        :param str new_authzr_uri: Deprecated. Do not use.
-
-        :returns: Authorization Resource.
-        :rtype: `.AuthorizationResource`
-
-        :raises errors.WildcardUnsupportedError: if a wildcard is requested
-
-        """
-        return self.request_challenges(messages.Identifier(
-            typ=messages.IDENTIFIER_FQDN, value=domain), new_authzr_uri)
+        response = self.net.get(authzr.uri)
+        updated_authzr = self._authzr_from_response(
+            response, authzr.body.identifier, authzr.uri)
+        return updated_authzr, response
 
     def request_issuance(self, csr, authzrs):
         """Request issuance.
@@ -371,7 +306,7 @@ class Client(ClientBase):
         req = messages.CertificateRequest(csr=csr)
 
         content_type = DER_CONTENT_TYPE  # TODO: add 'cert_type 'argument
-        response = self._post(
+        response = self.net.post(
             self.directory.new_cert,
             req,
             content_type=content_type,
@@ -389,22 +324,6 @@ class Client(ClientBase):
             body=jose.ComparableX509(OpenSSL.crypto.load_certificate(
                 OpenSSL.crypto.FILETYPE_ASN1, response.content)))
 
-    def poll(self, authzr):
-        """Poll Authorization Resource for status.
-
-        :param authzr: Authorization Resource
-        :type authzr: `.AuthorizationResource`
-
-        :returns: Updated Authorization Resource and HTTP response.
-
-        :rtype: (`.AuthorizationResource`, `requests.Response`)
-
-        """
-        response = self.net.get(authzr.uri)
-        updated_authzr = self._authzr_from_response(
-            response, authzr.body.identifier, authzr.uri)
-        return updated_authzr, response
-
     def poll_and_request_issuance(
             self, csr, authzrs, mintime=5, max_attempts=10):
         """Poll and request issuance.
@@ -434,8 +353,9 @@ class Client(ClientBase):
             was marked by the CA as invalid
 
         """
+        # pylint: disable=too-many-locals
         assert max_attempts > 0
-        attempts = collections.defaultdict(int) # type: Dict[messages.AuthorizationResource, int]
+        attempts = collections.defaultdict(int)
         exhausted = set()
 
         # priority queue with datetime.datetime (based on Retry-After) as key,
@@ -487,7 +407,7 @@ class Client(ClientBase):
         :param str uri: URI of certificate
 
         :returns: tuple of the form
-            (response, :class:`josepy.util.ComparableX509`)
+            (response, :class:`acme.jose.ComparableX509`)
         :rtype: tuple
 
         """
@@ -549,7 +469,7 @@ class Client(ClientBase):
         :rtype: `list` of `OpenSSL.crypto.X509` wrapped in `.ComparableX509`
 
         """
-        chain = [] # type: List[jose.ComparableX509]
+        chain = []
         uri = certr.cert_chain_uri
         while uri is not None and len(chain) < max_length:
             response, cert = self._get_cert(uri)
@@ -571,455 +491,51 @@ class Client(ClientBase):
         :raises .ClientError: If revocation is unsuccessful.
 
         """
-        return self._revoke(cert, rsn, self.directory[messages.Revocation])
-
-
-class ClientV2(ClientBase):
-    """ACME client for a v2 API.
-
-    :ivar messages.Directory directory:
-    :ivar .ClientNetwork net: Client network.
-    """
-
-    def __init__(self, directory, net):
-        """Initialize.
-
-        :param .messages.Directory directory: Directory Resource
-        :param .ClientNetwork net: Client network.
-        """
-        super(ClientV2, self).__init__(directory=directory,
-            net=net, acme_version=2)
-
-    def new_account(self, new_account):
-        """Register.
-
-        :param .NewRegistration new_account:
-
-        :raises .ConflictError: in case the account already exists
-
-        :returns: Registration Resource.
-        :rtype: `.RegistrationResource`
-        """
-        response = self._post(self.directory['newAccount'], new_account)
-        # if account already exists
-        if response.status_code == 200 and 'Location' in response.headers:
-            raise errors.ConflictError(response.headers.get('Location'))
-        # "Instance of 'Field' has no key/contact member" bug:
-        # pylint: disable=no-member
-        regr = self._regr_from_response(response)
-        self.net.account = regr
-        return regr
-
-    def query_registration(self, regr):
-        """Query server about registration.
-
-        :param messages.RegistrationResource: Existing Registration
-            Resource.
-
-        """
-        self.net.account = regr  # See certbot/certbot#6258
-        # ACME v2 requires to use a POST-as-GET request (POST an empty JWS) here.
-        # This is done by passing None instead of an empty UpdateRegistration to _post().
-        response = self._post(regr.uri, None)
-        self.net.account = self._regr_from_response(response, uri=regr.uri,
-                                                    terms_of_service=regr.terms_of_service)
-        return self.net.account
-
-    def update_registration(self, regr, update=None):
-        """Update registration.
-
-        :param messages.RegistrationResource regr: Registration Resource.
-        :param messages.Registration update: Updated body of the
-            resource. If not provided, body will be taken from `regr`.
-
-        :returns: Updated Registration Resource.
-        :rtype: `.RegistrationResource`
-
-        """
-        # https://github.com/certbot/certbot/issues/6155
-        new_regr = self._get_v2_account(regr)
-        return super(ClientV2, self).update_registration(new_regr, update)
-
-    def _get_v2_account(self, regr):
-        self.net.account = None
-        only_existing_reg = regr.body.update(only_return_existing=True)
-        response = self._post(self.directory['newAccount'], only_existing_reg)
-        updated_uri = response.headers['Location']
-        new_regr = regr.update(uri=updated_uri)
-        self.net.account = new_regr
-        return new_regr
-
-    def new_order(self, csr_pem):
-        """Request a new Order object from the server.
-
-        :param str csr_pem: A CSR in PEM format.
-
-        :returns: The newly created order.
-        :rtype: OrderResource
-        """
-        csr = OpenSSL.crypto.load_certificate_request(OpenSSL.crypto.FILETYPE_PEM, csr_pem)
-        # pylint: disable=protected-access
-        dnsNames = crypto_util._pyopenssl_cert_or_req_all_names(csr)
-
-        identifiers = []
-        for name in dnsNames:
-            identifiers.append(messages.Identifier(typ=messages.IDENTIFIER_FQDN,
-                value=name))
-        order = messages.NewOrder(identifiers=identifiers)
-        response = self._post(self.directory['newOrder'], order)
-        body = messages.Order.from_json(response.json())
-        authorizations = []
-        for url in body.authorizations:  # pylint: disable=not-an-iterable
-            authorizations.append(self._authzr_from_response(self._post_as_get(url), uri=url))
-        return messages.OrderResource(
-            body=body,
-            uri=response.headers.get('Location'),
-            authorizations=authorizations,
-            csr_pem=csr_pem)
-
-    def poll(self, authzr):
-        """Poll Authorization Resource for status.
-
-        :param authzr: Authorization Resource
-        :type authzr: `.AuthorizationResource`
-
-        :returns: Updated Authorization Resource and HTTP response.
-
-        :rtype: (`.AuthorizationResource`, `requests.Response`)
-
-        """
-        response = self._post_as_get(authzr.uri)
-        updated_authzr = self._authzr_from_response(
-            response, authzr.body.identifier, authzr.uri)
-        return updated_authzr, response
-
-    def poll_and_finalize(self, orderr, deadline=None):
-        """Poll authorizations and finalize the order.
-
-        If no deadline is provided, this method will timeout after 90
-        seconds.
-
-        :param messages.OrderResource orderr: order to finalize
-        :param datetime.datetime deadline: when to stop polling and timeout
-
-        :returns: finalized order
-        :rtype: messages.OrderResource
-
-        """
-        if deadline is None:
-            deadline = datetime.datetime.now() + datetime.timedelta(seconds=90)
-        orderr = self.poll_authorizations(orderr, deadline)
-        return self.finalize_order(orderr, deadline)
-
-    def poll_authorizations(self, orderr, deadline):
-        """Poll Order Resource for status."""
-        responses = []
-        for url in orderr.body.authorizations:
-            while datetime.datetime.now() < deadline:
-                authzr = self._authzr_from_response(self._post_as_get(url), uri=url)
-                if authzr.body.status != messages.STATUS_PENDING:
-                    responses.append(authzr)
-                    break
-                time.sleep(1)
-        # If we didn't get a response for every authorization, we fell through
-        # the bottom of the loop due to hitting the deadline.
-        if len(responses) < len(orderr.body.authorizations):
-            raise errors.TimeoutError()
-        failed = []
-        for authzr in responses:
-            if authzr.body.status != messages.STATUS_VALID:
-                for chall in authzr.body.challenges:
-                    if chall.error != None:
-                        failed.append(authzr)
-        if failed:
-            raise errors.ValidationError(failed)
-        return orderr.update(authorizations=responses)
-
-    def finalize_order(self, orderr, deadline):
-        """Finalize an order and obtain a certificate.
-
-        :param messages.OrderResource orderr: order to finalize
-        :param datetime.datetime deadline: when to stop polling and timeout
-
-        :returns: finalized order
-        :rtype: messages.OrderResource
-
-        """
-        csr = OpenSSL.crypto.load_certificate_request(
-            OpenSSL.crypto.FILETYPE_PEM, orderr.csr_pem)
-        wrapped_csr = messages.CertificateRequest(csr=jose.ComparableX509(csr))
-        self._post(orderr.body.finalize, wrapped_csr)
-        while datetime.datetime.now() < deadline:
-            time.sleep(1)
-            response = self._post_as_get(orderr.uri)
-            body = messages.Order.from_json(response.json())
-            if body.error is not None:
-                raise errors.IssuanceError(body.error)
-            if body.certificate is not None:
-                certificate_response = self._post_as_get(body.certificate).text
-                return orderr.update(body=body, fullchain_pem=certificate_response)
-        raise errors.TimeoutError()
-
-    def revoke(self, cert, rsn):
-        """Revoke certificate.
-
-        :param .ComparableX509 cert: `OpenSSL.crypto.X509` wrapped in
-            `.ComparableX509`
-
-        :param int rsn: Reason code for certificate revocation.
-
-        :raises .ClientError: If revocation is unsuccessful.
-
-        """
-        return self._revoke(cert, rsn, self.directory['revokeCert'])
-
-    def external_account_required(self):
-        """Checks if ACME server requires External Account Binding authentication."""
-        return hasattr(self.directory, 'meta') and self.directory.meta.external_account_required
-
-    def _post_as_get(self, *args, **kwargs):
-        """
-        Send GET request using the POST-as-GET protocol if needed.
-        The request will be first issued using POST-as-GET for ACME v2. If the ACME CA servers do
-        not support this yet and return an error, request will be retried using GET.
-        For ACME v1, only GET request will be tried, as POST-as-GET is not supported.
-        :param args:
-        :param kwargs:
-        :return:
-        """
-        if self.acme_version >= 2:
-            # We add an empty payload for POST-as-GET requests
-            new_args = args[:1] + (None,) + args[1:]
-            try:
-                return self._post(*new_args, **kwargs)
-            except messages.Error as error:
-                if error.code == 'malformed':
-                    logger.debug('Error during a POST-as-GET request, '
-                                 'your ACME CA server may not support it:\n%s', error)
-                    logger.debug('Retrying request with GET.')
-                else:  # pragma: no cover
-                    raise
-
-        # If POST-as-GET is not supported yet, we use a GET instead.
-        return self.net.get(*args, **kwargs)
-
-
-class BackwardsCompatibleClientV2(object):
-    """ACME client wrapper that tends towards V2-style calls, but
-    supports V1 servers.
-
-    .. note:: While this class handles the majority of the differences
-        between versions of the ACME protocol, if you need to support an
-        ACME server based on version 3 or older of the IETF ACME draft
-        that uses combinations in authorizations (or lack thereof) to
-        signal that the client needs to complete something other than
-        any single challenge in the authorization to make it valid, the
-        user of this class needs to understand and handle these
-        differences themselves.  This does not apply to either of Let's
-        Encrypt's endpoints where successfully completing any challenge
-        in an authorization will make it valid.
-
-    :ivar int acme_version: 1 or 2, corresponding to the Let's Encrypt endpoint
-    :ivar .ClientBase client: either Client or ClientV2
-    """
-
-    def __init__(self, net, key, server):
-        directory = messages.Directory.from_json(net.get(server).json())
-        self.acme_version = self._acme_version_from_directory(directory)
-        if self.acme_version == 1:
-            self.client = Client(directory, key=key, net=net)
-        else:
-            self.client = ClientV2(directory, net=net)
-
-    def __getattr__(self, name):
-        return getattr(self.client, name)
-
-    def new_account_and_tos(self, regr, check_tos_cb=None):
-        """Combined register and agree_tos for V1, new_account for V2
-
-        :param .NewRegistration regr:
-        :param callable check_tos_cb: callback that raises an error if
-            the check does not work
-        """
-        def _assess_tos(tos):
-            if check_tos_cb is not None:
-                check_tos_cb(tos)
-        if self.acme_version == 1:
-            regr = self.client.register(regr)
-            if regr.terms_of_service is not None:
-                _assess_tos(regr.terms_of_service)
-                return self.client.agree_to_tos(regr)
-            return regr
-        else:
-            if "terms_of_service" in self.client.directory.meta:
-                _assess_tos(self.client.directory.meta.terms_of_service)
-                regr = regr.update(terms_of_service_agreed=True)
-            return self.client.new_account(regr)
-
-    def new_order(self, csr_pem):
-        """Request a new Order object from the server.
-
-        If using ACMEv1, returns a dummy OrderResource with only
-        the authorizations field filled in.
-
-        :param str csr_pem: A CSR in PEM format.
-
-        :returns: The newly created order.
-        :rtype: OrderResource
-
-        :raises errors.WildcardUnsupportedError: if a wildcard domain is
-            requested but unsupported by the ACME version
-
-        """
-        if self.acme_version == 1:
-            csr = OpenSSL.crypto.load_certificate_request(OpenSSL.crypto.FILETYPE_PEM, csr_pem)
-            # pylint: disable=protected-access
-            dnsNames = crypto_util._pyopenssl_cert_or_req_all_names(csr)
-            authorizations = []
-            for domain in dnsNames:
-                authorizations.append(self.client.request_domain_challenges(domain))
-            return messages.OrderResource(authorizations=authorizations, csr_pem=csr_pem)
-        return self.client.new_order(csr_pem)
-
-    def finalize_order(self, orderr, deadline):
-        """Finalize an order and obtain a certificate.
-
-        :param messages.OrderResource orderr: order to finalize
-        :param datetime.datetime deadline: when to stop polling and timeout
-
-        :returns: finalized order
-        :rtype: messages.OrderResource
-
-        """
-        if self.acme_version == 1:
-            csr_pem = orderr.csr_pem
-            certr = self.client.request_issuance(
-                jose.ComparableX509(
-                    OpenSSL.crypto.load_certificate_request(OpenSSL.crypto.FILETYPE_PEM, csr_pem)),
-                    orderr.authorizations)
-
-            chain = None
-            while datetime.datetime.now() < deadline:
-                try:
-                    chain = self.client.fetch_chain(certr)
-                    break
-                except errors.Error:
-                    time.sleep(1)
-
-            if chain is None:
-                raise errors.TimeoutError(
-                    'Failed to fetch chain. You should not deploy the generated '
-                    'certificate, please rerun the command for a new one.')
-
-            cert = OpenSSL.crypto.dump_certificate(
-                    OpenSSL.crypto.FILETYPE_PEM, certr.body.wrapped).decode()
-            chain = crypto_util.dump_pyopenssl_chain(chain).decode()
-
-            return orderr.update(fullchain_pem=(cert + chain))
-        return self.client.finalize_order(orderr, deadline)
-
-    def revoke(self, cert, rsn):
-        """Revoke certificate.
-
-        :param .ComparableX509 cert: `OpenSSL.crypto.X509` wrapped in
-            `.ComparableX509`
-
-        :param int rsn: Reason code for certificate revocation.
-
-        :raises .ClientError: If revocation is unsuccessful.
-
-        """
-        return self.client.revoke(cert, rsn)
-
-    def _acme_version_from_directory(self, directory):
-        if hasattr(directory, 'newNonce'):
-            return 2
-        return 1
-
-    def external_account_required(self):
-        """Checks if the server requires an external account for ACMEv2 servers.
-
-        Always return False for ACMEv1 servers, as it doesn't use External Account Binding."""
-        if self.acme_version == 1:
-            return False
-        return self.client.external_account_required()
-
-
-class ClientNetwork(object):
-    """Wrapper around requests that signs POSTs for authentication.
-
-    Also adds user agent, and handles Content-Type.
-    """
+        response = self.net.post(self.directory[messages.Revocation],
+                                 messages.Revocation(
+                                     certificate=cert,
+                                     reason=rsn),
+                                 content_type=None)
+        if response.status_code != http_client.OK:
+            raise errors.ClientError(
+                'Successful revocation must return HTTP OK status')
+
+
+class ClientNetwork(object):  # pylint: disable=too-many-instance-attributes
+    """Client network."""
     JSON_CONTENT_TYPE = 'application/json'
     JOSE_CONTENT_TYPE = 'application/jose+json'
     JSON_ERROR_CONTENT_TYPE = 'application/problem+json'
     REPLAY_NONCE_HEADER = 'Replay-Nonce'
 
-    """Initialize.
-
-    :param josepy.JWK key: Account private key
-    :param messages.RegistrationResource account: Account object. Required if you are
-            planning to use .post() with acme_version=2 for anything other than
-            creating a new account; may be set later after registering.
-    :param josepy.JWASignature alg: Algoritm to use in signing JWS.
-    :param bool verify_ssl: Whether to verify certificates on SSL connections.
-    :param str user_agent: String to send as User-Agent header.
-    :param float timeout: Timeout for requests.
-    :param source_address: Optional source address to bind to when making requests.
-    :type source_address: str or tuple(str, int)
-    """
-    def __init__(self, key, account=None, alg=jose.RS256, verify_ssl=True,
-                 user_agent='acme-python', timeout=DEFAULT_NETWORK_TIMEOUT,
-                 source_address=None):
+    def __init__(self, key, alg=jose.RS256, verify_ssl=True,
+                 user_agent='acme-python', timeout=DEFAULT_NETWORK_TIMEOUT):
         self.key = key
-        self.account = account
         self.alg = alg
         self.verify_ssl = verify_ssl
-        self._nonces = set() # type: Set[Text]
+        self._nonces = set()
         self.user_agent = user_agent
         self.session = requests.Session()
         self._default_timeout = timeout
-        adapter = HTTPAdapter()
-
-        if source_address is not None:
-            adapter = SourceAddressAdapter(source_address)
-
-        self.session.mount("http://", adapter)
-        self.session.mount("https://", adapter)
 
     def __del__(self):
-        # Try to close the session, but don't show exceptions to the
-        # user if the call to close() fails. See #4840.
-        try:
-            self.session.close()
-        except Exception:  # pylint: disable=broad-except
-            pass
+        self.session.close()
 
-    def _wrap_in_jws(self, obj, nonce, url, acme_version):
+    def _wrap_in_jws(self, obj, nonce):
         """Wrap `JSONDeSerializable` object in JWS.
 
         .. todo:: Implement ``acmePath``.
 
-        :param josepy.JSONDeSerializable obj:
-        :param str url: The URL to which this object will be POSTed
+        :param .JSONDeSerializable obj:
         :param bytes nonce:
-        :rtype: `josepy.JWS`
+        :rtype: `.JWS`
 
         """
-        jobj = obj.json_dumps(indent=2).encode() if obj else b''
+        jobj = obj.json_dumps(indent=2).encode()
         logger.debug('JWS payload:\n%s', jobj)
-        kwargs = {
-            "alg": self.alg,
-            "nonce": nonce
-        }
-        if acme_version == 2:
-            kwargs["url"] = url
-            # newAccount and revokeCert work without the kid
-            # newAccount must not have kid
-            if self.account is not None:
-                kwargs["kid"] = self.account["uri"]
-        kwargs["key"] = self.key
-        return jws.JWS.sign(jobj, **kwargs).json_dumps(indent=2)
+        return jws.JWS.sign(
+            payload=jobj, key=self.key, alg=self.alg,
+            nonce=nonce).json_dumps(indent=2)
 
     @classmethod
     def _check_response(cls, response, content_type=None):
@@ -1103,38 +619,13 @@ class ClientNetwork(object):
         kwargs.setdefault('headers', {})
         kwargs['headers'].setdefault('User-Agent', self.user_agent)
         kwargs.setdefault('timeout', self._default_timeout)
-        try:
-            response = self.session.request(method, url, *args, **kwargs)
-        except requests.exceptions.RequestException as e:
-            # pylint: disable=pointless-string-statement
-            """Requests response parsing
-
-            The requests library emits exceptions with a lot of extra text.
-            We parse them with a regexp to raise a more readable exceptions.
-
-            Example:
-            HTTPSConnectionPool(host='acme-v01.api.letsencrypt.org',
-            port=443): Max retries exceeded with url: /directory
-            (Caused by NewConnectionError('
-            <requests.packages.urllib3.connection.VerifiedHTTPSConnection
-            object at 0x108356c50>: Failed to establish a new connection:
-            [Errno 65] No route to host',))"""
-
-            # pylint: disable=line-too-long
-            err_regex = r".*host='(\S*)'.*Max retries exceeded with url\: (\/\w*).*(\[Errno \d+\])([A-Za-z ]*)"
-            m = re.match(err_regex, str(e))
-            if m is None:
-                raise # pragma: no cover
-            else:
-                host, path, _err_no, err_msg = m.groups()
-                raise ValueError("Requesting {0}{1}:{2}".format(host, path, err_msg))
-
+        response = self.session.request(method, url, *args, **kwargs)
         # If content is DER, log the base64 of it instead of raw bytes, to keep
         # binary data out of the logs.
         if response.headers.get("Content-Type") == DER_CONTENT_TYPE:
             debug_content = base64.b64encode(response.content)
         else:
-            debug_content = response.content.decode("utf-8")
+            debug_content = response.content
         logger.debug('Received response:\nHTTP %d\n%s\n\n%s',
                      response.status_code,
                      "\n".join(["{0}: {1}".format(k, v)
@@ -1169,15 +660,10 @@ class ClientNetwork(object):
         else:
             raise errors.MissingNonce(response)
 
-    def _get_nonce(self, url, new_nonce_url):
+    def _get_nonce(self, url):
         if not self._nonces:
             logger.debug('Requesting fresh nonce')
-            if new_nonce_url is None:
-                response = self.head(url)
-            else:
-                # request a new nonce from the acme newNonce endpoint
-                response = self._check_response(self.head(new_nonce_url), content_type=None)
-            self._add_nonce(response)
+            self._add_nonce(self.head(url))
         return self._nonces.pop()
 
     def post(self, *args, **kwargs):
@@ -1196,12 +682,9 @@ class ClientNetwork(object):
             else:
                 raise
 
-    def _post_once(self, url, obj, content_type=JOSE_CONTENT_TYPE,
-            acme_version=1, **kwargs):
-        new_nonce_url = kwargs.pop('new_nonce_url', None)
-        data = self._wrap_in_jws(obj, self._get_nonce(url, new_nonce_url), url, acme_version)
+    def _post_once(self, url, obj, content_type=JOSE_CONTENT_TYPE, **kwargs):
+        data = self._wrap_in_jws(obj, self._get_nonce(url))
         kwargs.setdefault('headers', {'Content-Type': content_type})
         response = self._send_request('POST', url, data=data, **kwargs)
-        response = self._check_response(response, content_type=content_type)
         self._add_nonce(response)
-        return response
+        return self._check_response(response, content_type=content_type)

acme/acme/client_test.py

@@ -1,53 +1,30 @@
 """Tests for acme.client."""
-# pylint: disable=too-many-lines
-import copy
 import datetime
 import json
 import unittest
 
 from six.moves import http_client  # pylint: disable=import-error
 
-import josepy as jose
 import mock
-import OpenSSL
 import requests
 
 from acme import challenges
 from acme import errors
+from acme import jose
 from acme import jws as acme_jws
 from acme import messages
 from acme import messages_test
 from acme import test_util
-from acme.magic_typing import Dict # pylint: disable=unused-import, no-name-in-module
 
 
 CERT_DER = test_util.load_vector('cert.der')
-CERT_SAN_PEM = test_util.load_vector('cert-san.pem')
-CSR_SAN_PEM = test_util.load_vector('csr-san.pem')
 KEY = jose.JWKRSA.load(test_util.load_vector('rsa512_key.pem'))
 KEY2 = jose.JWKRSA.load(test_util.load_vector('rsa256_key.pem'))
 
-DIRECTORY_V1 = messages.Directory({
-    messages.NewRegistration:
-        'https://www.letsencrypt-demo.org/acme/new-reg',
-    messages.Revocation:
-        'https://www.letsencrypt-demo.org/acme/revoke-cert',
-    messages.NewAuthorization:
-        'https://www.letsencrypt-demo.org/acme/new-authz',
-    messages.CertificateRequest:
-        'https://www.letsencrypt-demo.org/acme/new-cert',
-})
 
-DIRECTORY_V2 = messages.Directory({
-    'newAccount': 'https://www.letsencrypt-demo.org/acme/new-account',
-    'newNonce': 'https://www.letsencrypt-demo.org/acme/new-nonce',
-    'newOrder': 'https://www.letsencrypt-demo.org/acme/new-order',
-    'revokeCert': 'https://www.letsencrypt-demo.org/acme/revoke-cert',
-})
-
-
-class ClientTestBase(unittest.TestCase):
-    """Base for tests in acme.client."""
+class ClientTest(unittest.TestCase):
+    """Tests for  acme.client.Client."""
+    # pylint: disable=too-many-instance-attributes,too-many-public-methods
 
     def setUp(self):
         self.response = mock.MagicMock(
@@ -56,6 +33,21 @@ class ClientTestBase(unittest.TestCase):
         self.net.post.return_value = self.response
         self.net.get.return_value = self.response
 
+        self.directory = messages.Directory({
+            messages.NewRegistration:
+                'https://www.letsencrypt-demo.org/acme/new-reg',
+            messages.Revocation:
+                'https://www.letsencrypt-demo.org/acme/revoke-cert',
+            messages.NewAuthorization:
+                'https://www.letsencrypt-demo.org/acme/new-authz',
+            messages.CertificateRequest:
+                'https://www.letsencrypt-demo.org/acme/new-cert',
+        })
+
+        from acme.client import Client
+        self.client = Client(
+            directory=self.directory, key=KEY, alg=jose.RS256, net=self.net)
+
         self.identifier = messages.Identifier(
             typ=messages.IDENTIFIER_FQDN, value='example.com')
 
@@ -63,10 +55,10 @@ class ClientTestBase(unittest.TestCase):
         self.contact = ('mailto:cert-admin@example.com', 'tel:+12025551212')
         reg = messages.Registration(
             contact=self.contact, key=KEY.public_key())
-        the_arg = dict(reg) # type: Dict
-        self.new_reg = messages.NewRegistration(**the_arg)
+        self.new_reg = messages.NewRegistration(**dict(reg))
         self.regr = messages.RegistrationResource(
-            body=reg, uri='https://www.letsencrypt-demo.org/acme/reg/1')
+            body=reg, uri='https://www.letsencrypt-demo.org/acme/reg/1',
+            terms_of_service='https://www.letsencrypt-demo.org/tos')
 
         # Authorization
         authzr_uri = 'https://www.letsencrypt-demo.org/acme/authz/1'
@@ -83,260 +75,14 @@ class ClientTestBase(unittest.TestCase):
         self.authzr = messages.AuthorizationResource(
             body=self.authz, uri=authzr_uri)
 
-        # Reason code for revocation
-        self.rsn = 1
-
-
-class BackwardsCompatibleClientV2Test(ClientTestBase):
-    """Tests for  acme.client.BackwardsCompatibleClientV2."""
-
-    def setUp(self):
-        super(BackwardsCompatibleClientV2Test, self).setUp()
-        # contains a loaded cert
-        self.certr = messages.CertificateResource(
-            body=messages_test.CERT)
-
-        loaded = OpenSSL.crypto.load_certificate(
-            OpenSSL.crypto.FILETYPE_PEM, CERT_SAN_PEM)
-        wrapped = jose.ComparableX509(loaded)
-        self.chain = [wrapped, wrapped]
-
-        self.cert_pem = OpenSSL.crypto.dump_certificate(
-            OpenSSL.crypto.FILETYPE_PEM, messages_test.CERT.wrapped).decode()
-
-        single_chain = OpenSSL.crypto.dump_certificate(
-            OpenSSL.crypto.FILETYPE_PEM, loaded).decode()
-        self.chain_pem = single_chain + single_chain
-
-        self.fullchain_pem = self.cert_pem + self.chain_pem
-
-        self.orderr = messages.OrderResource(
-            csr_pem=CSR_SAN_PEM)
-
-    def _init(self):
-        uri = 'http://www.letsencrypt-demo.org/directory'
-        from acme.client import BackwardsCompatibleClientV2
-        return BackwardsCompatibleClientV2(net=self.net,
-            key=KEY, server=uri)
-
-    def test_init_downloads_directory(self):
-        uri = 'http://www.letsencrypt-demo.org/directory'
-        from acme.client import BackwardsCompatibleClientV2
-        BackwardsCompatibleClientV2(net=self.net,
-            key=KEY, server=uri)
-        self.net.get.assert_called_once_with(uri)
-
-    def test_init_acme_version(self):
-        self.response.json.return_value = DIRECTORY_V1.to_json()
-        client = self._init()
-        self.assertEqual(client.acme_version, 1)
-
-        self.response.json.return_value = DIRECTORY_V2.to_json()
-        client = self._init()
-        self.assertEqual(client.acme_version, 2)
-
-    def test_query_registration_client_v2(self):
-        self.response.json.return_value = DIRECTORY_V2.to_json()
-        client = self._init()
-        self.response.json.return_value = self.regr.body.to_json()
-        self.assertEqual(self.regr, client.query_registration(self.regr))
-
-    def test_forwarding(self):
-        self.response.json.return_value = DIRECTORY_V1.to_json()
-        client = self._init()
-        self.assertEqual(client.directory, client.client.directory)
-        self.assertEqual(client.key, KEY)
-        self.assertEqual(client.deactivate_registration, client.client.deactivate_registration)
-        self.assertRaises(AttributeError, client.__getattr__, 'nonexistent')
-        self.assertRaises(AttributeError, client.__getattr__, 'new_account_and_tos')
-        self.assertRaises(AttributeError, client.__getattr__, 'new_account')
-
-    def test_new_account_and_tos(self):
-        # v2 no tos
-        self.response.json.return_value = DIRECTORY_V2.to_json()
-        with mock.patch('acme.client.ClientV2') as mock_client:
-            client = self._init()
-            client.new_account_and_tos(self.new_reg)
-            mock_client().new_account.assert_called_with(self.new_reg)
-
-        # v2 tos good
-        with mock.patch('acme.client.ClientV2') as mock_client:
-            mock_client().directory.meta.__contains__.return_value = True
-            client = self._init()
-            client.new_account_and_tos(self.new_reg, lambda x: True)
-            mock_client().new_account.assert_called_with(
-                self.new_reg.update(terms_of_service_agreed=True))
-
-        # v2 tos bad
-        with mock.patch('acme.client.ClientV2') as mock_client:
-            mock_client().directory.meta.__contains__.return_value = True
-            client = self._init()
-            def _tos_cb(tos):
-                raise errors.Error
-            self.assertRaises(errors.Error, client.new_account_and_tos,
-                self.new_reg, _tos_cb)
-            mock_client().new_account.assert_not_called()
-
-        # v1 yes tos
-        self.response.json.return_value = DIRECTORY_V1.to_json()
-        with mock.patch('acme.client.Client') as mock_client:
-            regr = mock.MagicMock(terms_of_service="TOS")
-            mock_client().register.return_value = regr
-            client = self._init()
-            client.new_account_and_tos(self.new_reg)
-            mock_client().register.assert_called_once_with(self.new_reg)
-            mock_client().agree_to_tos.assert_called_once_with(regr)
-
-        # v1 no tos
-        with mock.patch('acme.client.Client') as mock_client:
-            regr = mock.MagicMock(terms_of_service=None)
-            mock_client().register.return_value = regr
-            client = self._init()
-            client.new_account_and_tos(self.new_reg)
-            mock_client().register.assert_called_once_with(self.new_reg)
-            mock_client().agree_to_tos.assert_not_called()
-
-    @mock.patch('OpenSSL.crypto.load_certificate_request')
-    @mock.patch('acme.crypto_util._pyopenssl_cert_or_req_all_names')
-    def test_new_order_v1(self, mock__pyopenssl_cert_or_req_all_names,
-        unused_mock_load_certificate_request):
-        self.response.json.return_value = DIRECTORY_V1.to_json()
-        mock__pyopenssl_cert_or_req_all_names.return_value = ['example.com', 'www.example.com']
-        mock_csr_pem = mock.MagicMock()
-        with mock.patch('acme.client.Client') as mock_client:
-            mock_client().request_domain_challenges.return_value = mock.sentinel.auth
-            client = self._init()
-            orderr = client.new_order(mock_csr_pem)
-            self.assertEqual(orderr.authorizations, [mock.sentinel.auth, mock.sentinel.auth])
-
-    def test_new_order_v2(self):
-        self.response.json.return_value = DIRECTORY_V2.to_json()
-        mock_csr_pem = mock.MagicMock()
-        with mock.patch('acme.client.ClientV2') as mock_client:
-            client = self._init()
-            client.new_order(mock_csr_pem)
-            mock_client().new_order.assert_called_once_with(mock_csr_pem)
-
-    @mock.patch('acme.client.Client')
-    def test_finalize_order_v1_success(self, mock_client):
-        self.response.json.return_value = DIRECTORY_V1.to_json()
-
-        mock_client().request_issuance.return_value = self.certr
-        mock_client().fetch_chain.return_value = self.chain
-
-        deadline = datetime.datetime(9999, 9, 9)
-        client = self._init()
-        result = client.finalize_order(self.orderr, deadline)
-        self.assertEqual(result.fullchain_pem, self.fullchain_pem)
-        mock_client().fetch_chain.assert_called_once_with(self.certr)
-
-    @mock.patch('acme.client.Client')
-    def test_finalize_order_v1_fetch_chain_error(self, mock_client):
-        self.response.json.return_value = DIRECTORY_V1.to_json()
-
-        mock_client().request_issuance.return_value = self.certr
-        mock_client().fetch_chain.return_value = self.chain
-        mock_client().fetch_chain.side_effect = [errors.Error, self.chain]
-
-        deadline = datetime.datetime(9999, 9, 9)
-        client = self._init()
-        result = client.finalize_order(self.orderr, deadline)
-        self.assertEqual(result.fullchain_pem, self.fullchain_pem)
-        self.assertEqual(mock_client().fetch_chain.call_count, 2)
-
-    @mock.patch('acme.client.Client')
-    def test_finalize_order_v1_timeout(self, mock_client):
-        self.response.json.return_value = DIRECTORY_V1.to_json()
-
-        mock_client().request_issuance.return_value = self.certr
-
-        deadline = deadline = datetime.datetime.now() - datetime.timedelta(seconds=60)
-        client = self._init()
-        self.assertRaises(errors.TimeoutError, client.finalize_order,
-            self.orderr, deadline)
-
-    def test_finalize_order_v2(self):
-        self.response.json.return_value = DIRECTORY_V2.to_json()
-        mock_orderr = mock.MagicMock()
-        mock_deadline = mock.MagicMock()
-        with mock.patch('acme.client.ClientV2') as mock_client:
-            client = self._init()
-            client.finalize_order(mock_orderr, mock_deadline)
-            mock_client().finalize_order.assert_called_once_with(mock_orderr, mock_deadline)
-
-    def test_revoke(self):
-        self.response.json.return_value = DIRECTORY_V1.to_json()
-        with mock.patch('acme.client.Client') as mock_client:
-            client = self._init()
-            client.revoke(messages_test.CERT, self.rsn)
-        mock_client().revoke.assert_called_once_with(messages_test.CERT, self.rsn)
-
-        self.response.json.return_value = DIRECTORY_V2.to_json()
-        with mock.patch('acme.client.ClientV2') as mock_client:
-            client = self._init()
-            client.revoke(messages_test.CERT, self.rsn)
-        mock_client().revoke.assert_called_once_with(messages_test.CERT, self.rsn)
-
-    def test_update_registration(self):
-        self.response.json.return_value = DIRECTORY_V1.to_json()
-        with mock.patch('acme.client.Client') as mock_client:
-            client = self._init()
-            client.update_registration(mock.sentinel.regr, None)
-        mock_client().update_registration.assert_called_once_with(mock.sentinel.regr, None)
-
-    # newNonce present means it will pick acme_version 2
-    def test_external_account_required_true(self):
-        self.response.json.return_value = messages.Directory({
-            'newNonce': 'http://letsencrypt-test.com/acme/new-nonce',
-            'meta': messages.Directory.Meta(external_account_required=True),
-        }).to_json()
-
-        client = self._init()
-
-        self.assertTrue(client.external_account_required())
-
-    # newNonce present means it will pick acme_version 2
-    def test_external_account_required_false(self):
-        self.response.json.return_value = messages.Directory({
-            'newNonce': 'http://letsencrypt-test.com/acme/new-nonce',
-            'meta': messages.Directory.Meta(external_account_required=False),
-        }).to_json()
-
-        client = self._init()
-
-        self.assertFalse(client.external_account_required())
-
-    def test_external_account_required_false_v1(self):
-        self.response.json.return_value = messages.Directory({
-            'meta': messages.Directory.Meta(external_account_required=False),
-        }).to_json()
-
-        client = self._init()
-
-        self.assertFalse(client.external_account_required())
-
-
-class ClientTest(ClientTestBase):
-    """Tests for acme.client.Client."""
-
-    def setUp(self):
-        super(ClientTest, self).setUp()
-
-        self.directory = DIRECTORY_V1
-
-        # Registration
-        self.regr = self.regr.update(
-            terms_of_service='https://www.letsencrypt-demo.org/tos')
-
         # Request issuance
         self.certr = messages.CertificateResource(
             body=messages_test.CERT, authzrs=(self.authzr,),
             uri='https://www.letsencrypt-demo.org/acme/cert/1',
             cert_chain_uri='https://www.letsencrypt-demo.org/ca')
 
-        from acme.client import Client
-        self.client = Client(
-            directory=self.directory, key=KEY, alg=jose.RS256, net=self.net)
+        # Reason code for revocation
+        self.rsn = 1
 
     def test_init_downloads_directory(self):
         uri = 'http://www.letsencrypt-demo.org/directory'
@@ -345,18 +91,9 @@ class ClientTest(ClientTestBase):
             directory=uri, key=KEY, alg=jose.RS256, net=self.net)
         self.net.get.assert_called_once_with(uri)
 
-    @mock.patch('acme.client.ClientNetwork')
-    def test_init_without_net(self, mock_net):
-        mock_net.return_value = mock.sentinel.net
-        alg = jose.RS256
-        from acme.client import Client
-        self.client = Client(
-            directory=self.directory, key=KEY, alg=alg)
-        mock_net.called_once_with(KEY, alg=alg, verify_ssl=True)
-        self.assertEqual(self.client.net, mock.sentinel.net)
-
     def test_register(self):
         # "Instance of 'Field' has no to_json/update member" bug:
+        # pylint: disable=no-member
         self.response.status_code = http_client.CREATED
         self.response.json.return_value = self.regr.body.to_json()
         self.response.headers['Location'] = self.regr.uri
@@ -369,6 +106,7 @@ class ClientTest(ClientTestBase):
 
     def test_update_registration(self):
         # "Instance of 'Field' has no to_json/update member" bug:
+        # pylint: disable=no-member
         self.response.headers['Location'] = self.regr.uri
         self.response.json.return_value = self.regr.body.to_json()
         self.assertEqual(self.regr, self.client.update_registration(self.regr))
@@ -404,23 +142,20 @@ class ClientTest(ClientTestBase):
         self.client.request_challenges(self.identifier)
         self.net.post.assert_called_once_with(
             self.directory.new_authz,
-            messages.NewAuthorization(identifier=self.identifier),
-            acme_version=1)
+            messages.NewAuthorization(identifier=self.identifier))
 
     def test_request_challenges_deprecated_arg(self):
         self._prepare_response_for_request_challenges()
         self.client.request_challenges(self.identifier, new_authzr_uri="hi")
         self.net.post.assert_called_once_with(
             self.directory.new_authz,
-            messages.NewAuthorization(identifier=self.identifier),
-            acme_version=1)
+            messages.NewAuthorization(identifier=self.identifier))
 
     def test_request_challenges_custom_uri(self):
         self._prepare_response_for_request_challenges()
         self.client.request_challenges(self.identifier)
         self.net.post.assert_called_once_with(
-            'https://www.letsencrypt-demo.org/acme/new-authz', mock.ANY,
-            acme_version=1)
+            'https://www.letsencrypt-demo.org/acme/new-authz', mock.ANY)
 
     def test_request_challenges_unexpected_update(self):
         self._prepare_response_for_request_challenges()
@@ -430,13 +165,6 @@ class ClientTest(ClientTestBase):
             errors.UnexpectedUpdate, self.client.request_challenges,
             self.identifier)
 
-    def test_request_challenges_wildcard(self):
-        wildcard_identifier = messages.Identifier(
-            typ=messages.IDENTIFIER_FQDN, value='*.example.org')
-        self.assertRaises(
-            errors.WildcardUnsupportedError, self.client.request_challenges,
-            wildcard_identifier)
-
     def test_request_domain_challenges(self):
         self.client.request_challenges = mock.MagicMock()
         self.assertEqual(
@@ -636,14 +364,6 @@ class ClientTest(ClientTestBase):
             errors.PollError, self.client.poll_and_request_issuance,
             csr, authzrs, mintime=mintime, max_attempts=2)
 
-    def test_deactivate_authorization(self):
-        authzb = self.authzr.body.update(status=messages.STATUS_DEACTIVATED)
-        self.response.json.return_value = authzb.to_json()
-        authzr = self.client.deactivate_authorization(self.authzr)
-        self.assertEqual(authzb, authzr.body)
-        self.assertEqual(self.client.net.post.call_count, 1)
-        self.assertTrue(self.authzr.uri in self.net.post.call_args_list[0][0])
-
     def test_check_cert(self):
         self.response.headers['Location'] = self.certr.uri
         self.response.content = CERT_DER
@@ -697,12 +417,12 @@ class ClientTest(ClientTestBase):
     def test_revoke(self):
         self.client.revoke(self.certr.body, self.rsn)
         self.net.post.assert_called_once_with(
-            self.directory[messages.Revocation], mock.ANY, acme_version=1)
+            self.directory[messages.Revocation], mock.ANY, content_type=None)
 
     def test_revocation_payload(self):
         obj = messages.Revocation(certificate=self.certr.body, reason=self.rsn)
         self.assertTrue('reason' in obj.to_partial_json().keys())
-        self.assertEqual(self.rsn, obj.to_partial_json()['reason'])
+        self.assertEquals(self.rsn, obj.to_partial_json()['reason'])
 
     def test_revoke_bad_status_raises_error(self):
         self.response.status_code = http_client.METHOD_NOT_ALLOWED
@@ -713,207 +433,6 @@ class ClientTest(ClientTestBase):
             self.rsn)
 
 
-class ClientV2Test(ClientTestBase):
-    """Tests for acme.client.ClientV2."""
-
-    def setUp(self):
-        super(ClientV2Test, self).setUp()
-
-        self.directory = DIRECTORY_V2
-
-        from acme.client import ClientV2
-        self.client = ClientV2(self.directory, self.net)
-
-        self.new_reg = self.new_reg.update(terms_of_service_agreed=True)
-
-        self.authzr_uri2 = 'https://www.letsencrypt-demo.org/acme/authz/2'
-        self.authz2 = self.authz.update(identifier=messages.Identifier(
-            typ=messages.IDENTIFIER_FQDN, value='www.example.com'),
-            status=messages.STATUS_PENDING)
-        self.authzr2 = messages.AuthorizationResource(
-            body=self.authz2, uri=self.authzr_uri2)
-
-        self.order = messages.Order(
-            identifiers=(self.authz.identifier, self.authz2.identifier),
-            status=messages.STATUS_PENDING,
-            authorizations=(self.authzr.uri, self.authzr_uri2),
-            finalize='https://www.letsencrypt-demo.org/acme/acct/1/order/1/finalize')
-        self.orderr = messages.OrderResource(
-            body=self.order,
-            uri='https://www.letsencrypt-demo.org/acme/acct/1/order/1',
-            authorizations=[self.authzr, self.authzr2], csr_pem=CSR_SAN_PEM)
-
-    def test_new_account(self):
-        self.response.status_code = http_client.CREATED
-        self.response.json.return_value = self.regr.body.to_json()
-        self.response.headers['Location'] = self.regr.uri
-
-        self.assertEqual(self.regr, self.client.new_account(self.new_reg))
-
-    def test_new_account_conflict(self):
-        self.response.status_code = http_client.OK
-        self.response.headers['Location'] = self.regr.uri
-        self.assertRaises(errors.ConflictError, self.client.new_account, self.new_reg)
-
-    def test_new_order(self):
-        order_response = copy.deepcopy(self.response)
-        order_response.status_code = http_client.CREATED
-        order_response.json.return_value = self.order.to_json()
-        order_response.headers['Location'] = self.orderr.uri
-        self.net.post.return_value = order_response
-
-        authz_response = copy.deepcopy(self.response)
-        authz_response.json.return_value = self.authz.to_json()
-        authz_response.headers['Location'] = self.authzr.uri
-        authz_response2 = self.response
-        authz_response2.json.return_value = self.authz2.to_json()
-        authz_response2.headers['Location'] = self.authzr2.uri
-
-        with mock.patch('acme.client.ClientV2._post_as_get') as mock_post_as_get:
-            mock_post_as_get.side_effect = (authz_response, authz_response2)
-            self.assertEqual(self.client.new_order(CSR_SAN_PEM), self.orderr)
-
-    @mock.patch('acme.client.datetime')
-    def test_poll_and_finalize(self, mock_datetime):
-        mock_datetime.datetime.now.return_value = datetime.datetime(2018, 2, 15)
-        mock_datetime.timedelta = datetime.timedelta
-        expected_deadline = mock_datetime.datetime.now() + datetime.timedelta(seconds=90)
-
-        self.client.poll_authorizations = mock.Mock(return_value=self.orderr)
-        self.client.finalize_order = mock.Mock(return_value=self.orderr)
-
-        self.assertEqual(self.client.poll_and_finalize(self.orderr), self.orderr)
-        self.client.poll_authorizations.assert_called_once_with(self.orderr, expected_deadline)
-        self.client.finalize_order.assert_called_once_with(self.orderr, expected_deadline)
-
-    @mock.patch('acme.client.datetime')
-    def test_poll_authorizations_timeout(self, mock_datetime):
-        now_side_effect = [datetime.datetime(2018, 2, 15),
-                           datetime.datetime(2018, 2, 16),
-                           datetime.datetime(2018, 2, 17)]
-        mock_datetime.datetime.now.side_effect = now_side_effect
-        self.response.json.side_effect = [
-            self.authz.to_json(), self.authz2.to_json(), self.authz2.to_json()]
-
-        self.assertRaises(
-            errors.TimeoutError, self.client.poll_authorizations, self.orderr, now_side_effect[1])
-
-    def test_poll_authorizations_failure(self):
-        deadline = datetime.datetime(9999, 9, 9)
-        challb = self.challr.body.update(status=messages.STATUS_INVALID,
-                                         error=messages.Error.with_code('unauthorized'))
-        authz = self.authz.update(status=messages.STATUS_INVALID, challenges=(challb,))
-        self.response.json.return_value = authz.to_json()
-
-        self.assertRaises(
-            errors.ValidationError, self.client.poll_authorizations, self.orderr, deadline)
-
-    def test_poll_authorizations_success(self):
-        deadline = datetime.datetime(9999, 9, 9)
-        updated_authz2 = self.authz2.update(status=messages.STATUS_VALID)
-        updated_authzr2 = messages.AuthorizationResource(
-            body=updated_authz2, uri=self.authzr_uri2)
-        updated_orderr = self.orderr.update(authorizations=[self.authzr, updated_authzr2])
-
-        self.response.json.side_effect = (
-            self.authz.to_json(), self.authz2.to_json(), updated_authz2.to_json())
-        self.assertEqual(self.client.poll_authorizations(self.orderr, deadline), updated_orderr)
-
-    def test_finalize_order_success(self):
-        updated_order = self.order.update(
-            certificate='https://www.letsencrypt-demo.org/acme/cert/')
-        updated_orderr = self.orderr.update(body=updated_order, fullchain_pem=CERT_SAN_PEM)
-
-        self.response.json.return_value = updated_order.to_json()
-        self.response.text = CERT_SAN_PEM
-
-        deadline = datetime.datetime(9999, 9, 9)
-        self.assertEqual(self.client.finalize_order(self.orderr, deadline), updated_orderr)
-
-    def test_finalize_order_error(self):
-        updated_order = self.order.update(error=messages.Error.with_code('unauthorized'))
-        self.response.json.return_value = updated_order.to_json()
-
-        deadline = datetime.datetime(9999, 9, 9)
-        self.assertRaises(errors.IssuanceError, self.client.finalize_order, self.orderr, deadline)
-
-    def test_finalize_order_timeout(self):
-        deadline = datetime.datetime.now() - datetime.timedelta(seconds=60)
-        self.assertRaises(errors.TimeoutError, self.client.finalize_order, self.orderr, deadline)
-
-    def test_revoke(self):
-        self.client.revoke(messages_test.CERT, self.rsn)
-        self.net.post.assert_called_once_with(
-            self.directory["revokeCert"], mock.ANY, acme_version=2,
-            new_nonce_url=DIRECTORY_V2['newNonce'])
-
-    def test_update_registration(self):
-        # "Instance of 'Field' has no to_json/update member" bug:
-        self.response.headers['Location'] = self.regr.uri
-        self.response.json.return_value = self.regr.body.to_json()
-        self.assertEqual(self.regr, self.client.update_registration(self.regr))
-        self.assertNotEqual(self.client.net.account, None)
-        self.assertEqual(self.client.net.post.call_count, 2)
-        self.assertTrue(DIRECTORY_V2.newAccount in self.net.post.call_args_list[0][0])
-
-        self.response.json.return_value = self.regr.body.update(
-            contact=()).to_json()
-
-    def test_external_account_required_true(self):
-        self.client.directory = messages.Directory({
-            'meta': messages.Directory.Meta(external_account_required=True)
-        })
-
-        self.assertTrue(self.client.external_account_required())
-
-    def test_external_account_required_false(self):
-        self.client.directory = messages.Directory({
-            'meta': messages.Directory.Meta(external_account_required=False)
-        })
-
-        self.assertFalse(self.client.external_account_required())
-
-    def test_external_account_required_default(self):
-        self.assertFalse(self.client.external_account_required())
-
-    def test_post_as_get(self):
-        with mock.patch('acme.client.ClientV2._authzr_from_response') as mock_client:
-            mock_client.return_value = self.authzr2
-
-            self.client.poll(self.authzr2)  # pylint: disable=protected-access
-
-            self.client.net.post.assert_called_once_with(
-                self.authzr2.uri, None, acme_version=2,
-                new_nonce_url='https://www.letsencrypt-demo.org/acme/new-nonce')
-            self.client.net.get.assert_not_called()
-
-            class FakeError(messages.Error):
-                """Fake error to reproduce a malformed request ACME error"""
-                def __init__(self):  # pylint: disable=super-init-not-called
-                    pass
-                @property
-                def code(self):
-                    return 'malformed'
-            self.client.net.post.side_effect = FakeError()
-
-            self.client.poll(self.authzr2)  # pylint: disable=protected-access
-
-            self.client.net.get.assert_called_once_with(self.authzr2.uri)
-
-
-class MockJSONDeSerializable(jose.JSONDeSerializable):
-    # pylint: disable=missing-docstring
-    def __init__(self, value):
-        self.value = value
-
-    def to_partial_json(self):
-        return {'foo': self.value}
-
-    @classmethod
-    def from_json(cls, jobj):
-        pass  # pragma: no cover
-
-
 class ClientNetworkTest(unittest.TestCase):
     """Tests for acme.client.ClientNetwork."""
 
@@ -934,25 +453,24 @@ class ClientNetworkTest(unittest.TestCase):
         self.assertTrue(self.net.verify_ssl is self.verify_ssl)
 
     def test_wrap_in_jws(self):
-        # pylint: disable=protected-access
-        jws_dump = self.net._wrap_in_jws(
-            MockJSONDeSerializable('foo'), nonce=b'Tg', url="url",
-            acme_version=1)
-        jws = acme_jws.JWS.json_loads(jws_dump)
-        self.assertEqual(json.loads(jws.payload.decode()), {'foo': 'foo'})
-        self.assertEqual(jws.signature.combined.nonce, b'Tg')
+        class MockJSONDeSerializable(jose.JSONDeSerializable):
+            # pylint: disable=missing-docstring
+            def __init__(self, value):
+                self.value = value
+
+            def to_partial_json(self):
+                return {'foo': self.value}
+
+            @classmethod
+            def from_json(cls, value):
+                pass  # pragma: no cover
 
-    def test_wrap_in_jws_v2(self):
-        self.net.account = {'uri': 'acct-uri'}
         # pylint: disable=protected-access
         jws_dump = self.net._wrap_in_jws(
-            MockJSONDeSerializable('foo'), nonce=b'Tg', url="url",
-            acme_version=2)
+            MockJSONDeSerializable('foo'), nonce=b'Tg')
         jws = acme_jws.JWS.json_loads(jws_dump)
         self.assertEqual(json.loads(jws.payload.decode()), {'foo': 'foo'})
         self.assertEqual(jws.signature.combined.nonce, b'Tg')
-        self.assertEqual(jws.signature.combined.kid, u'acct-uri')
-        self.assertEqual(jws.signature.combined.url, u'url')
 
     def test_check_response_not_ok_jobj_no_error(self):
         self.response.ok = False
@@ -1082,19 +600,12 @@ class ClientNetworkTest(unittest.TestCase):
             mock.ANY, mock.ANY, verify=mock.ANY, headers=mock.ANY,
             timeout=45)
 
-    def test_del(self, close_exception=None):
+    def test_del(self):
         sess = mock.MagicMock()
-
-        if close_exception is not None:
-            sess.close.side_effect = close_exception
-
         self.net.session = sess
         del self.net
         sess.close.assert_called_once_with()
 
-    def test_del_error(self):
-        self.test_del(ReferenceError)
-
     @mock.patch('acme.client.requests')
     def test_requests_error_passthrough(self, mock_requests):
         mock_requests.exceptions = requests.exceptions
@@ -1103,24 +614,10 @@ class ClientNetworkTest(unittest.TestCase):
         self.assertRaises(requests.exceptions.RequestException,
                           self.net._send_request, 'GET', 'uri')
 
-    def test_urllib_error(self):
-        # Using a connection error to test a properly formatted error message
-        try:
-            # pylint: disable=protected-access
-            self.net._send_request('GET', "http://localhost:19123/nonexistent.txt")
-
-        # Value Error Generated Exceptions
-        except ValueError as y:
-            self.assertEqual("Requesting localhost/nonexistent: "
-                             "Connection refused", str(y))
-
-        # Requests Library Exceptions
-        except requests.exceptions.ConnectionError as z: #pragma: no cover
-            self.assertTrue("'Connection aborted.'" in str(z) or "[WinError 10061]" in str(z))
-
 
 class ClientNetworkWithMockedResponseTest(unittest.TestCase):
     """Tests for acme.client.ClientNetwork which mock out response."""
+    # pylint: disable=too-many-instance-attributes
 
     def setUp(self):
         from acme.client import ClientNetwork
@@ -1129,10 +626,7 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase):
         self.response = mock.MagicMock(ok=True, status_code=http_client.OK)
         self.response.headers = {}
         self.response.links = {}
-        self.response.checked = False
-        self.acmev1_nonce_response = mock.MagicMock(ok=False,
-            status_code=http_client.METHOD_NOT_ALLOWED)
-        self.acmev1_nonce_response.headers = {}
+        self.checked_response = mock.MagicMock()
         self.obj = mock.MagicMock()
         self.wrapped_obj = mock.MagicMock()
         self.content_type = mock.sentinel.content_type
@@ -1144,21 +638,13 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase):
 
         def send_request(*args, **kwargs):
             # pylint: disable=unused-argument,missing-docstring
-            self.assertFalse("new_nonce_url" in kwargs)
-            method = args[0]
-            uri = args[1]
-            if method == 'HEAD' and uri != "new_nonce_uri":
-                response = self.acmev1_nonce_response
-            else:
-                response = self.response
-
             if self.available_nonces:
-                response.headers = {
+                self.response.headers = {
                     self.net.REPLAY_NONCE_HEADER:
                     self.available_nonces.pop().decode()}
             else:
-                response.headers = {}
-            return response
+                self.response.headers = {}
+            return self.response
 
         # pylint: disable=protected-access
         self.net._send_request = self.send_request = mock.MagicMock(
@@ -1170,47 +656,36 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase):
         # pylint: disable=missing-docstring
         self.assertEqual(self.response, response)
         self.assertEqual(self.content_type, content_type)
-        self.assertTrue(self.response.ok)
-        self.response.checked = True
-        return self.response
+        return self.checked_response
 
     def test_head(self):
-        self.assertEqual(self.acmev1_nonce_response, self.net.head(
+        self.assertEqual(self.response, self.net.head(
             'http://example.com/', 'foo', bar='baz'))
         self.send_request.assert_called_once_with(
             'HEAD', 'http://example.com/', 'foo', bar='baz')
 
-    def test_head_v2(self):
-        self.assertEqual(self.response, self.net.head(
-            'new_nonce_uri', 'foo', bar='baz'))
-        self.send_request.assert_called_once_with(
-            'HEAD', 'new_nonce_uri', 'foo', bar='baz')
-
     def test_get(self):
-        self.assertEqual(self.response, self.net.get(
+        self.assertEqual(self.checked_response, self.net.get(
             'http://example.com/', content_type=self.content_type, bar='baz'))
-        self.assertTrue(self.response.checked)
         self.send_request.assert_called_once_with(
             'GET', 'http://example.com/', bar='baz')
 
     def test_post_no_content_type(self):
         self.content_type = self.net.JOSE_CONTENT_TYPE
-        self.assertEqual(self.response, self.net.post('uri', self.obj))
-        self.assertTrue(self.response.checked)
+        self.assertEqual(self.checked_response, self.net.post('uri', self.obj))
 
     def test_post(self):
         # pylint: disable=protected-access
-        self.assertEqual(self.response, self.net.post(
+        self.assertEqual(self.checked_response, self.net.post(
             'uri', self.obj, content_type=self.content_type))
-        self.assertTrue(self.response.checked)
         self.net._wrap_in_jws.assert_called_once_with(
-            self.obj, jose.b64decode(self.all_nonces.pop()), "uri", 1)
+            self.obj, jose.b64decode(self.all_nonces.pop()))
 
         self.available_nonces = []
         self.assertRaises(errors.MissingNonce, self.net.post,
                           'uri', self.obj, content_type=self.content_type)
         self.net._wrap_in_jws.assert_called_with(
-            self.obj, jose.b64decode(self.all_nonces.pop()), "uri", 1)
+            self.obj, jose.b64decode(self.all_nonces.pop()))
 
     def test_post_wrong_initial_nonce(self):  # HEAD
         self.available_nonces = [b'f', jose.b64encode(b'good')]
@@ -1234,7 +709,7 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase):
     def test_post_not_retried(self):
         check_response = mock.MagicMock()
         check_response.side_effect = [messages.Error.with_code('malformed'),
-                                      self.response]
+                                      self.checked_response]
 
         # pylint: disable=protected-access
         self.net._check_response = check_response
@@ -1242,12 +717,13 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase):
                           self.obj, content_type=self.content_type)
 
     def test_post_successful_retry(self):
-        post_once = mock.MagicMock()
-        post_once.side_effect = [messages.Error.with_code('badNonce'),
-                                      self.response]
+        check_response = mock.MagicMock()
+        check_response.side_effect = [messages.Error.with_code('badNonce'),
+                                      self.checked_response]
 
         # pylint: disable=protected-access
-        self.assertEqual(self.response, self.net.post(
+        self.net._check_response = check_response
+        self.assertEqual(self.checked_response, self.net.post(
             'uri', self.obj, content_type=self.content_type))
 
     def test_head_get_post_error_passthrough(self):
@@ -1258,51 +734,6 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase):
         self.assertRaises(requests.exceptions.RequestException,
                           self.net.post, 'uri', obj=self.obj)
 
-    def test_post_bad_nonce_head(self):
-        # pylint: disable=protected-access
-        # regression test for https://github.com/certbot/certbot/issues/6092
-        bad_response = mock.MagicMock(ok=False, status_code=http_client.SERVICE_UNAVAILABLE)
-        self.net._send_request = mock.MagicMock()
-        self.net._send_request.return_value = bad_response
-        self.content_type = None
-        check_response = mock.MagicMock()
-        self.net._check_response = check_response
-        self.assertRaises(errors.ClientError, self.net.post, 'uri',
-                          self.obj, content_type=self.content_type, acme_version=2,
-                          new_nonce_url='new_nonce_uri')
-        self.assertEqual(check_response.call_count, 1)
-
-    def test_new_nonce_uri_removed(self):
-        self.content_type = None
-        self.net.post('uri', self.obj, content_type=None,
-            acme_version=2, new_nonce_url='new_nonce_uri')
-
-
-class ClientNetworkSourceAddressBindingTest(unittest.TestCase):
-    """Tests that if ClientNetwork has a source IP set manually, the underlying library has
-    used the provided source address."""
-
-    def setUp(self):
-        self.source_address = "8.8.8.8"
-
-    def test_source_address_set(self):
-        from acme.client import ClientNetwork
-        net = ClientNetwork(key=None, alg=None, source_address=self.source_address)
-        for adapter in net.session.adapters.values():
-            self.assertTrue(self.source_address in adapter.source_address)
-
-    def test_behavior_assumption(self):
-        """This is a test that guardrails the HTTPAdapter behavior so that if the default for
-        a Session() changes, the assumptions here aren't violated silently."""
-        from acme.client import ClientNetwork
-        # Source address not specified, so the default adapter type should be bound -- this
-        # test should fail if the default adapter type is changed by requests
-        net = ClientNetwork(key=None, alg=None)
-        session = requests.Session()
-        for scheme in session.adapters.keys():
-            client_network_adapter = net.session.adapters.get(scheme)
-            default_adapter = session.adapters.get(scheme)
-            self.assertEqual(client_network_adapter.__class__, default_adapter.__class__)
 
 if __name__ == '__main__':
     unittest.main()  # pragma: no cover

acme/acme/crypto_util.py

@@ -2,33 +2,31 @@
 import binascii
 import contextlib
 import logging
-import os
 import re
 import socket
+import sys
 
-from OpenSSL import crypto
-from OpenSSL import SSL # type: ignore # https://github.com/python/typeshed/issues/2052
-import josepy as jose
+import OpenSSL
 
 from acme import errors
-# pylint: disable=unused-import, no-name-in-module
-from acme.magic_typing import Callable, Union, Tuple, Optional
-# pylint: enable=unused-import, no-name-in-module
 
 
 logger = logging.getLogger(__name__)
 
-# Default SSL method selected here is the most compatible, while secure
-# SSL method: TLSv1_METHOD is only compatible with
+# TLSSNI01 certificate serving and probing is not affected by SSL
+# vulnerabilities: prober needs to check certificate for expected
+# contents anyway. Working SNI is the only thing that's necessary for
+# the challenge and thus scoping down SSL/TLS method (version) would
+# cause interoperability issues: TLSv1_METHOD is only compatible with
 # TLSv1_METHOD, while SSLv23_METHOD is compatible with all other
 # methods, including TLSv2_METHOD (read more at
 # https://www.openssl.org/docs/ssl/SSLv23_method.html). _serve_sni
 # should be changed to use "set_options" to disable SSLv2 and SSLv3,
 # in case it's used for things other than probing/serving!
-_DEFAULT_SSL_METHOD = SSL.SSLv23_METHOD  # type: ignore
+_DEFAULT_TLSSNI01_SSL_METHOD = OpenSSL.SSL.SSLv23_METHOD  # type: ignore
 
 
-class SSLSocket(object):
+class SSLSocket(object):  # pylint: disable=too-few-public-methods
     """SSL wrapper for sockets.
 
     :ivar socket sock: Original wrapped socket.
@@ -37,7 +35,7 @@ class SSLSocket(object):
     :ivar method: See `OpenSSL.SSL.Context` for allowed values.
 
     """
-    def __init__(self, sock, certs, method=_DEFAULT_SSL_METHOD):
+    def __init__(self, sock, certs, method=_DEFAULT_TLSSNI01_SSL_METHOD):
         self.sock = sock
         self.certs = certs
         self.method = method
@@ -64,9 +62,9 @@ class SSLSocket(object):
             logger.debug("Server name (%s) not recognized, dropping SSL",
                          server_name)
             return
-        new_context = SSL.Context(self.method)
-        new_context.set_options(SSL.OP_NO_SSLv2)
-        new_context.set_options(SSL.OP_NO_SSLv3)
+        new_context = OpenSSL.SSL.Context(self.method)
+        new_context.set_options(OpenSSL.SSL.OP_NO_SSLv2)
+        new_context.set_options(OpenSSL.SSL.OP_NO_SSLv3)
         new_context.use_privatekey(key)
         new_context.use_certificate(cert)
         connection.set_context(new_context)
@@ -74,7 +72,7 @@ class SSLSocket(object):
     class FakeConnection(object):
         """Fake OpenSSL.SSL.Connection."""
 
-        # pylint: disable=missing-docstring
+        # pylint: disable=too-few-public-methods,missing-docstring
 
         def __init__(self, connection):
             self._wrapped = connection
@@ -89,18 +87,18 @@ class SSLSocket(object):
     def accept(self):  # pylint: disable=missing-docstring
         sock, addr = self.sock.accept()
 
-        context = SSL.Context(self.method)
-        context.set_options(SSL.OP_NO_SSLv2)
-        context.set_options(SSL.OP_NO_SSLv3)
+        context = OpenSSL.SSL.Context(self.method)
+        context.set_options(OpenSSL.SSL.OP_NO_SSLv2)
+        context.set_options(OpenSSL.SSL.OP_NO_SSLv3)
         context.set_tlsext_servername_callback(self._pick_certificate_cb)
 
-        ssl_sock = self.FakeConnection(SSL.Connection(context, sock))
+        ssl_sock = self.FakeConnection(OpenSSL.SSL.Connection(context, sock))
         ssl_sock.set_accept_state()
 
         logger.debug("Performing handshake with %s", addr)
         try:
             ssl_sock.do_handshake()
-        except SSL.Error as error:
+        except OpenSSL.SSL.Error as error:
             # _pick_certificate_cb might have returned without
             # creating SSL context (wrong server name)
             raise socket.error(error)
@@ -109,7 +107,7 @@ class SSLSocket(object):
 
 
 def probe_sni(name, host, port=443, timeout=300,
-              method=_DEFAULT_SSL_METHOD, source_address=('', 0)):
+              method=_DEFAULT_TLSSNI01_SSL_METHOD, source_address=('', 0)):
     """Probe SNI server for SSL certificate.
 
     :param bytes name: Byte string to send as the server name in the
@@ -128,32 +126,31 @@ def probe_sni(name, host, port=443, timeout=300,
     :rtype: OpenSSL.crypto.X509
 
     """
-    context = SSL.Context(method)
+    context = OpenSSL.SSL.Context(method)
     context.set_timeout(timeout)
 
-    socket_kwargs = {'source_address': source_address}
+    socket_kwargs = {} if sys.version_info < (2, 7) else {
+        'source_address': source_address}
+
+    host_protocol_agnostic = None if host == '::' or host == '0' else host
 
     try:
-        logger.debug(
-            "Attempting to connect to %s:%d%s.", host, port,
-            " from {0}:{1}".format(
-                source_address[0],
-                source_address[1]
-            ) if socket_kwargs else ""
-        )
-        socket_tuple = (host, port)  # type: Tuple[str, int]
-        sock = socket.create_connection(socket_tuple, **socket_kwargs)  # type: ignore
+        # pylint: disable=star-args
+        logger.debug("Attempting to connect to %s:%d%s.", host_protocol_agnostic, port,
+            " from {0}:{1}".format(source_address[0], source_address[1]) if \
+            socket_kwargs else "")
+        sock = socket.create_connection((host_protocol_agnostic, port), **socket_kwargs)
     except socket.error as error:
         raise errors.Error(error)
 
     with contextlib.closing(sock) as client:
-        client_ssl = SSL.Connection(context, client)
+        client_ssl = OpenSSL.SSL.Connection(context, client)
         client_ssl.set_connect_state()
         client_ssl.set_tlsext_host_name(name)  # pyOpenSSL>=0.13
         try:
             client_ssl.do_handshake()
             client_ssl.shutdown()
-        except SSL.Error as error:
+        except OpenSSL.SSL.Error as error:
             raise errors.Error(error)
     return client_ssl.get_peer_certificate()
 
@@ -166,18 +163,18 @@ def make_csr(private_key_pem, domains, must_staple=False):
         OCSP Must Staple: https://tools.ietf.org/html/rfc7633).
     :returns: buffer PEM-encoded Certificate Signing Request.
     """
-    private_key = crypto.load_privatekey(
-        crypto.FILETYPE_PEM, private_key_pem)
-    csr = crypto.X509Req()
+    private_key = OpenSSL.crypto.load_privatekey(
+        OpenSSL.crypto.FILETYPE_PEM, private_key_pem)
+    csr = OpenSSL.crypto.X509Req()
     extensions = [
-        crypto.X509Extension(
+        OpenSSL.crypto.X509Extension(
             b'subjectAltName',
             critical=False,
             value=', '.join('DNS:' + d for d in domains).encode('ascii')
         ),
     ]
     if must_staple:
-        extensions.append(crypto.X509Extension(
+        extensions.append(OpenSSL.crypto.X509Extension(
             b"1.3.6.1.5.5.7.1.24",
             critical=False,
             value=b"DER:30:03:02:01:05"))
@@ -185,16 +182,8 @@ def make_csr(private_key_pem, domains, must_staple=False):
     csr.set_pubkey(private_key)
     csr.set_version(2)
     csr.sign(private_key, 'sha256')
-    return crypto.dump_certificate_request(
-        crypto.FILETYPE_PEM, csr)
-
-def _pyopenssl_cert_or_req_all_names(loaded_cert_or_req):
-    common_name = loaded_cert_or_req.get_subject().CN
-    sans = _pyopenssl_cert_or_req_san(loaded_cert_or_req)
-
-    if common_name is None:
-        return sans
-    return [common_name] + [d for d in sans if d != common_name]
+    return OpenSSL.crypto.dump_certificate_request(
+        OpenSSL.crypto.FILETYPE_PEM, csr)
 
 def _pyopenssl_cert_or_req_san(cert_or_req):
     """Get Subject Alternative Names from certificate or CSR using pyOpenSSL.
@@ -222,15 +211,14 @@ def _pyopenssl_cert_or_req_san(cert_or_req):
     parts_separator = ", "
     prefix = "DNS" + part_separator
 
-    if isinstance(cert_or_req, crypto.X509):
-        # pylint: disable=line-too-long
-        func = crypto.dump_certificate # type: Union[Callable[[int, crypto.X509Req], bytes], Callable[[int, crypto.X509], bytes]]
+    if isinstance(cert_or_req, OpenSSL.crypto.X509):
+        func = OpenSSL.crypto.dump_certificate
     else:
-        func = crypto.dump_certificate_request
-    text = func(crypto.FILETYPE_TEXT, cert_or_req).decode("utf-8")
+        func = OpenSSL.crypto.dump_certificate_request
+    text = func(OpenSSL.crypto.FILETYPE_TEXT, cert_or_req).decode("utf-8")
     # WARNING: this function does not support multiple SANs extensions.
     # Multiple X509v3 extensions of the same type is disallowed by RFC 5280.
-    match = re.search(r"X509v3 Subject Alternative Name:(?: critical)?\s*(.*)", text)
+    match = re.search(r"X509v3 Subject Alternative Name:\s*(.*)", text)
     # WARNING: this function assumes that no SAN can include
     # parts_separator, hence the split!
     sans_parts = [] if match is None else match.group(1).split(parts_separator)
@@ -254,12 +242,12 @@ def gen_ss_cert(key, domains, not_before=None,
 
     """
     assert domains, "Must provide one or more hostnames for the cert."
-    cert = crypto.X509()
-    cert.set_serial_number(int(binascii.hexlify(os.urandom(16)), 16))
+    cert = OpenSSL.crypto.X509()
+    cert.set_serial_number(int(binascii.hexlify(OpenSSL.rand.bytes(16)), 16))
     cert.set_version(2)
 
     extensions = [
-        crypto.X509Extension(
+        OpenSSL.crypto.X509Extension(
             b"basicConstraints", True, b"CA:TRUE, pathlen:0"),
     ]
 
@@ -268,7 +256,7 @@ def gen_ss_cert(key, domains, not_before=None,
     cert.set_issuer(cert.get_subject())
 
     if force_san or len(domains) > 1:
-        extensions.append(crypto.X509Extension(
+        extensions.append(OpenSSL.crypto.X509Extension(
             b"subjectAltName",
             critical=False,
             value=b", ".join(b"DNS:" + d.encode() for d in domains)
@@ -282,26 +270,3 @@ def gen_ss_cert(key, domains, not_before=None,
     cert.set_pubkey(key)
     cert.sign(key, "sha256")
     return cert
-
-def dump_pyopenssl_chain(chain, filetype=crypto.FILETYPE_PEM):
-    """Dump certificate chain into a bundle.
-
-    :param list chain: List of `OpenSSL.crypto.X509` (or wrapped in
-        :class:`josepy.util.ComparableX509`).
-
-    :returns: certificate chain bundle
-    :rtype: bytes
-
-    """
-    # XXX: returns empty string when no chain is available, which
-    # shuts up RenewableCert, but might not be the best solution...
-
-    def _dump_cert(cert):
-        if isinstance(cert, jose.ComparableX509):
-            # pylint: disable=protected-access
-            cert = cert.wrapped
-        return crypto.dump_certificate(filetype, cert)
-
-    # assumes that OpenSSL.crypto.dump_certificate includes ending
-    # newline character
-    return b"".join(_dump_cert(cert) for cert in chain)

acme/acme/crypto_util_test.py

@@ -8,17 +8,17 @@ import unittest
 import six
 from six.moves import socketserver  #type: ignore  # pylint: disable=import-error
 
-import josepy as jose
 import OpenSSL
 
 from acme import errors
+from acme import jose
 from acme import test_util
-from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
 
 
 class SSLSocketAndProbeSNITest(unittest.TestCase):
     """Tests for acme.crypto_util.SSLSocket/probe_sni."""
 
+    _multiprocess_can_split_ = True
 
     def setUp(self):
         self.cert = test_util.load_comparable_cert('rsa2048_cert.pem')
@@ -30,6 +30,7 @@ class SSLSocketAndProbeSNITest(unittest.TestCase):
 
         class _TestServer(socketserver.TCPServer):
 
+            # pylint: disable=too-few-public-methods
             # six.moves.* | pylint: disable=attribute-defined-outside-init,no-init
 
             def server_bind(self):  # pylint: disable=missing-docstring
@@ -41,67 +42,34 @@ class SSLSocketAndProbeSNITest(unittest.TestCase):
         self.server_thread = threading.Thread(
             # pylint: disable=no-member
             target=self.server.handle_request)
+        self.server_thread.start()
+        time.sleep(1)  # TODO: avoid race conditions in other way
 
     def tearDown(self):
-        if self.server_thread.is_alive():
-            # The thread may have already terminated.
-            self.server_thread.join()  # pragma: no cover
+        self.server_thread.join()
 
     def _probe(self, name):
         from acme.crypto_util import probe_sni
         return jose.ComparableX509(probe_sni(
             name, host='127.0.0.1', port=self.port))
 
-    def _start_server(self):
-        self.server_thread.start()
-        time.sleep(1)  # TODO: avoid race conditions in other way
-
     def test_probe_ok(self):
-        self._start_server()
         self.assertEqual(self.cert, self._probe(b'foo'))
 
     def test_probe_not_recognized_name(self):
-        self._start_server()
         self.assertRaises(errors.Error, self._probe, b'bar')
 
-    def test_probe_connection_error(self):
-        # pylint has a hard time with six
-        self.server.server_close()  # pylint: disable=no-member
-        original_timeout = socket.getdefaulttimeout()
-        try:
-            socket.setdefaulttimeout(1)
-            self.assertRaises(errors.Error, self._probe, b'bar')
-        finally:
-            socket.setdefaulttimeout(original_timeout)
-
-
-class PyOpenSSLCertOrReqAllNamesTest(unittest.TestCase):
-    """Test for acme.crypto_util._pyopenssl_cert_or_req_all_names."""
-
-    @classmethod
-    def _call(cls, loader, name):
-        # pylint: disable=protected-access
-        from acme.crypto_util import _pyopenssl_cert_or_req_all_names
-        return _pyopenssl_cert_or_req_all_names(loader(name))
-
-    def _call_cert(self, name):
-        return self._call(test_util.load_cert, name)
-
-    def test_cert_one_san_no_common(self):
-        self.assertEqual(self._call_cert('cert-nocn.der'),
-                         ['no-common-name.badssl.com'])
-
-    def test_cert_no_sans_yes_common(self):
-        self.assertEqual(self._call_cert('cert.pem'), ['example.com'])
-
-    def test_cert_two_sans_yes_common(self):
-        self.assertEqual(self._call_cert('cert-san.pem'),
-                         ['example.com', 'www.example.com'])
+    # TODO: py33/py34 tox hangs forever on do_handshake in second probe
+    #def probe_connection_error(self):
+    #    self._probe(b'foo')
+    #    #time.sleep(1)  # TODO: avoid race conditions in other way
+    #    self.assertRaises(errors.Error, self._probe, b'bar')
 
 
 class PyOpenSSLCertOrReqSANTest(unittest.TestCase):
     """Test for acme.crypto_util._pyopenssl_cert_or_req_san."""
 
+    _multiprocess_can_split_ = True
 
     @classmethod
     def _call(cls, loader, name):
@@ -163,19 +131,15 @@ class PyOpenSSLCertOrReqSANTest(unittest.TestCase):
         self.assertEqual(self._call_csr('csr-idnsans.pem'),
                          self._get_idn_names())
 
-    def test_critical_san(self):
-        self.assertEqual(self._call_cert('critical-san.pem'),
-                         ['chicago-cubs.venafi.example', 'cubs.venafi.example'])
-
-
 
 class RandomSnTest(unittest.TestCase):
     """Test for random certificate serial numbers."""
 
+    _multiprocess_can_split_ = True
 
     def setUp(self):
         self.cert_count = 5
-        self.serial_num = [] # type: List[int]
+        self.serial_num = []
         self.key = OpenSSL.crypto.PKey()
         self.key.generate_key(OpenSSL.crypto.TYPE_RSA, 2048)
 
@@ -204,12 +168,12 @@ class MakeCSRTest(unittest.TestCase):
         self.assertTrue(b'--END CERTIFICATE REQUEST--' in csr_pem)
         csr = OpenSSL.crypto.load_certificate_request(
             OpenSSL.crypto.FILETYPE_PEM, csr_pem)
-        # In pyopenssl 0.13 (used with TOXENV=py27-oldest), csr objects don't
-        # have a get_extensions() method, so we skip this test if the method
-        # isn't available.
+        # In pyopenssl 0.13 (used with TOXENV=py26-oldest and py27-oldest), csr
+        # objects don't have a get_extensions() method, so we skip this test if
+        # the method isn't available.
         if hasattr(csr, 'get_extensions'):
-            self.assertEqual(len(csr.get_extensions()), 1)
-            self.assertEqual(csr.get_extensions()[0].get_data(),
+            self.assertEquals(len(csr.get_extensions()), 1)
+            self.assertEquals(csr.get_extensions()[0].get_data(),
                 OpenSSL.crypto.X509Extension(
                     b'subjectAltName',
                     critical=False,
@@ -222,11 +186,11 @@ class MakeCSRTest(unittest.TestCase):
         csr = OpenSSL.crypto.load_certificate_request(
             OpenSSL.crypto.FILETYPE_PEM, csr_pem)
 
-        # In pyopenssl 0.13 (used with TOXENV=py27-oldest), csr objects don't
-        # have a get_extensions() method, so we skip this test if the method
-        # isn't available.
+        # In pyopenssl 0.13 (used with TOXENV=py26-oldest and py27-oldest), csr
+        # objects don't have a get_extensions() method, so we skip this test if
+        # the method isn't available.
         if hasattr(csr, 'get_extensions'):
-            self.assertEqual(len(csr.get_extensions()), 2)
+            self.assertEquals(len(csr.get_extensions()), 2)
             # NOTE: Ideally we would filter by the TLS Feature OID, but
             # OpenSSL.crypto.X509Extension doesn't give us the extension's raw OID,
             # and the shortname field is just "UNDEF"
@@ -235,33 +199,5 @@ class MakeCSRTest(unittest.TestCase):
             self.assertEqual(len(must_staple_exts), 1,
                 "Expected exactly one Must Staple extension")
 
-
-class DumpPyopensslChainTest(unittest.TestCase):
-    """Test for dump_pyopenssl_chain."""
-
-    @classmethod
-    def _call(cls, loaded):
-        # pylint: disable=protected-access
-        from acme.crypto_util import dump_pyopenssl_chain
-        return dump_pyopenssl_chain(loaded)
-
-    def test_dump_pyopenssl_chain(self):
-        names = ['cert.pem', 'cert-san.pem', 'cert-idnsans.pem']
-        loaded = [test_util.load_cert(name) for name in names]
-        length = sum(
-            len(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert))
-            for cert in loaded)
-        self.assertEqual(len(self._call(loaded)), length)
-
-    def test_dump_pyopenssl_chain_wrapped(self):
-        names = ['cert.pem', 'cert-san.pem', 'cert-idnsans.pem']
-        loaded = [test_util.load_cert(name) for name in names]
-        wrap_func = jose.ComparableX509
-        wrapped = [wrap_func(cert) for cert in loaded]
-        dump_func = OpenSSL.crypto.dump_certificate
-        length = sum(len(dump_func(OpenSSL.crypto.FILETYPE_PEM, cert)) for cert in loaded)
-        self.assertEqual(len(self._call(wrapped)), length)
-
-
 if __name__ == '__main__':
     unittest.main()  # pragma: no cover

acme/acme/errors.py

@@ -1,5 +1,5 @@
 """ACME errors."""
-from josepy import errors as jose_errors
+from acme.jose import errors as jose_errors
 
 
 class Error(Exception):
@@ -83,40 +83,13 @@ class PollError(ClientError):
         return '{0}(exhausted={1!r}, updated={2!r})'.format(
             self.__class__.__name__, self.exhausted, self.updated)
 
-class ValidationError(Error):
-    """Error for authorization failures. Contains a list of authorization
-    resources, each of which is invalid and should have an error field.
-    """
-    def __init__(self, failed_authzrs):
-        self.failed_authzrs = failed_authzrs
-        super(ValidationError, self).__init__()
-
-class TimeoutError(Error):
-    """Error for when polling an authorization or an order times out."""
-
-class IssuanceError(Error):
-    """Error sent by the server after requesting issuance of a certificate."""
-
-    def __init__(self, error):
-        """Initialize.
-
-        :param messages.Error error: The error provided by the server.
-        """
-        self.error = error
-        super(IssuanceError, self).__init__()
-
 class ConflictError(ClientError):
     """Error for when the server returns a 409 (Conflict) HTTP status.
 
     In the version of ACME implemented by Boulder, this is used to find an
     account if you only have the private key, but don't know the account URL.
-
-    Also used in V2 of the ACME client for the same purpose.
     """
     def __init__(self, location):
         self.location = location
         super(ConflictError, self).__init__()
 
-
-class WildcardUnsupportedError(Error):
-    """Error for when a wildcard is requested but is unsupported by ACME CA."""

acme/acme/fields.py

@@ -1,9 +1,10 @@
 """ACME JSON fields."""
 import logging
 
-import josepy as jose
 import pyrfc3339
 
+from acme import jose
+
 
 logger = logging.getLogger(__name__)
 

acme/acme/fields_test.py

@@ -2,9 +2,10 @@
 import datetime
 import unittest
 
-import josepy as jose
 import pytz
 
+from acme import jose
+
 
 class FixedTest(unittest.TestCase):
     """Tests for acme.fields.Fixed."""

acme/acme/jose/__init__.py

@@ -0,0 +1,82 @@
+"""Javascript Object Signing and Encryption (jose).
+
+This package is a Python implementation of the standards developed by
+IETF `Javascript Object Signing and Encryption (Active WG)`_, in
+particular the following RFCs:
+
+  - `JSON Web Algorithms (JWA)`_
+  - `JSON Web Key (JWK)`_
+  - `JSON Web Signature (JWS)`_
+
+
+.. _`Javascript Object Signing and Encryption (Active WG)`:
+  https://tools.ietf.org/wg/jose/
+
+.. _`JSON Web Algorithms (JWA)`:
+  https://datatracker.ietf.org/doc/draft-ietf-jose-json-web-algorithms/
+
+.. _`JSON Web Key (JWK)`:
+  https://datatracker.ietf.org/doc/draft-ietf-jose-json-web-key/
+
+.. _`JSON Web Signature (JWS)`:
+  https://datatracker.ietf.org/doc/draft-ietf-jose-json-web-signature/
+
+"""
+from acme.jose.b64 import (
+    b64decode,
+    b64encode,
+)
+
+from acme.jose.errors import (
+    DeserializationError,
+    SerializationError,
+    Error,
+    UnrecognizedTypeError,
+)
+
+from acme.jose.interfaces import JSONDeSerializable
+
+from acme.jose.json_util import (
+    Field,
+    JSONObjectWithFields,
+    TypedJSONObjectWithFields,
+    decode_b64jose,
+    decode_cert,
+    decode_csr,
+    decode_hex16,
+    encode_b64jose,
+    encode_cert,
+    encode_csr,
+    encode_hex16,
+)
+
+from acme.jose.jwa import (
+    HS256,
+    HS384,
+    HS512,
+    JWASignature,
+    PS256,
+    PS384,
+    PS512,
+    RS256,
+    RS384,
+    RS512,
+)
+
+from acme.jose.jwk import (
+    JWK,
+    JWKRSA,
+)
+
+from acme.jose.jws import (
+    Header,
+    JWS,
+    Signature,
+)
+
+from acme.jose.util import (
+    ComparableX509,
+    ComparableKey,
+    ComparableRSAKey,
+    ImmutableMap,
+)

acme/acme/jose/b64.py

@@ -0,0 +1,61 @@
+"""JOSE Base64.
+
+`JOSE Base64`_ is defined as:
+
+  - URL-safe Base64
+  - padding stripped
+
+
+.. _`JOSE Base64`:
+    https://tools.ietf.org/html/draft-ietf-jose-json-web-signature-37#appendix-C
+
+.. Do NOT try to call this module "base64", as it will "shadow" the
+   standard library.
+
+"""
+import base64
+
+import six
+
+
+def b64encode(data):
+    """JOSE Base64 encode.
+
+    :param data: Data to be encoded.
+    :type data: `bytes`
+
+    :returns: JOSE Base64 string.
+    :rtype: bytes
+
+    :raises TypeError: if `data` is of incorrect type
+
+    """
+    if not isinstance(data, six.binary_type):
+        raise TypeError('argument should be {0}'.format(six.binary_type))
+    return base64.urlsafe_b64encode(data).rstrip(b'=')
+
+
+def b64decode(data):
+    """JOSE Base64 decode.
+
+    :param data: Base64 string to be decoded. If it's unicode, then
+                 only ASCII characters are allowed.
+    :type data: `bytes` or `unicode`
+
+    :returns: Decoded data.
+    :rtype: bytes
+
+    :raises TypeError: if input is of incorrect type
+    :raises ValueError: if input is unicode with non-ASCII characters
+
+    """
+    if isinstance(data, six.string_types):
+        try:
+            data = data.encode('ascii')
+        except UnicodeEncodeError:
+            raise ValueError(
+                'unicode argument should contain only ASCII characters')
+    elif not isinstance(data, six.binary_type):
+        raise TypeError('argument should be a str or unicode')
+
+    return base64.urlsafe_b64decode(data + b'=' * (4 - (len(data) % 4)))

acme/acme/jose/b64_test.py

@@ -0,0 +1,77 @@
+"""Tests for acme.jose.b64."""
+import unittest
+
+import six
+
+
+# https://en.wikipedia.org/wiki/Base64#Examples
+B64_PADDING_EXAMPLES = {
+    b'any carnal pleasure.': (b'YW55IGNhcm5hbCBwbGVhc3VyZS4', b'='),
+    b'any carnal pleasure': (b'YW55IGNhcm5hbCBwbGVhc3VyZQ', b'=='),
+    b'any carnal pleasur': (b'YW55IGNhcm5hbCBwbGVhc3Vy', b''),
+    b'any carnal pleasu': (b'YW55IGNhcm5hbCBwbGVhc3U', b'='),
+    b'any carnal pleas': (b'YW55IGNhcm5hbCBwbGVhcw', b'=='),
+}
+
+
+B64_URL_UNSAFE_EXAMPLES = {
+    six.int2byte(251) + six.int2byte(239): b'--8',
+    six.int2byte(255) * 2: b'__8',
+}
+
+
+class B64EncodeTest(unittest.TestCase):
+    """Tests for acme.jose.b64.b64encode."""
+
+    @classmethod
+    def _call(cls, data):
+        from acme.jose.b64 import b64encode
+        return b64encode(data)
+
+    def test_empty(self):
+        self.assertEqual(self._call(b''), b'')
+
+    def test_unsafe_url(self):
+        for text, b64 in six.iteritems(B64_URL_UNSAFE_EXAMPLES):
+            self.assertEqual(self._call(text), b64)
+
+    def test_different_paddings(self):
+        for text, (b64, _) in six.iteritems(B64_PADDING_EXAMPLES):
+            self.assertEqual(self._call(text), b64)
+
+    def test_unicode_fails_with_type_error(self):
+        self.assertRaises(TypeError, self._call, u'some unicode')
+
+
+class B64DecodeTest(unittest.TestCase):
+    """Tests for acme.jose.b64.b64decode."""
+
+    @classmethod
+    def _call(cls, data):
+        from acme.jose.b64 import b64decode
+        return b64decode(data)
+
+    def test_unsafe_url(self):
+        for text, b64 in six.iteritems(B64_URL_UNSAFE_EXAMPLES):
+            self.assertEqual(self._call(b64), text)
+
+    def test_input_without_padding(self):
+        for text, (b64, _) in six.iteritems(B64_PADDING_EXAMPLES):
+            self.assertEqual(self._call(b64), text)
+
+    def test_input_with_padding(self):
+        for text, (b64, pad) in six.iteritems(B64_PADDING_EXAMPLES):
+            self.assertEqual(self._call(b64 + pad), text)
+
+    def test_unicode_with_ascii(self):
+        self.assertEqual(self._call(u'YQ'), b'a')
+
+    def test_non_ascii_unicode_fails(self):
+        self.assertRaises(ValueError, self._call, u'\u0105')
+
+    def test_type_error_no_unicode_or_bytes(self):
+        self.assertRaises(TypeError, self._call, object())
+
+
+if __name__ == '__main__':
+    unittest.main()  # pragma: no cover

acme/acme/jose/errors.py

@@ -0,0 +1,35 @@
+"""JOSE errors."""
+
+
+class Error(Exception):
+    """Generic JOSE Error."""
+
+
+class DeserializationError(Error):
+    """JSON deserialization error."""
+
+    def __str__(self):
+        return "Deserialization error: {0}".format(
+            super(DeserializationError, self).__str__())
+
+
+class SerializationError(Error):
+    """JSON serialization error."""
+
+
+class UnrecognizedTypeError(DeserializationError):
+    """Unrecognized type error.
+
+    :ivar str typ: The unrecognized type of the JSON object.
+    :ivar jobj: Full JSON object.
+
+    """
+
+    def __init__(self, typ, jobj):
+        self.typ = typ
+        self.jobj = jobj
+        super(UnrecognizedTypeError, self).__init__(str(self))
+
+    def __str__(self):
+        return '{0} was not recognized, full message: {1}'.format(
+            self.typ, self.jobj)

acme/acme/jose/errors_test.py

@@ -0,0 +1,17 @@
+"""Tests for acme.jose.errors."""
+import unittest
+
+
+class UnrecognizedTypeErrorTest(unittest.TestCase):
+    def setUp(self):
+        from acme.jose.errors import UnrecognizedTypeError
+        self.error = UnrecognizedTypeError('foo', {'type': 'foo'})
+
+    def test_str(self):
+        self.assertEqual(
+            "foo was not recognized, full message: {'type': 'foo'}",
+            str(self.error))
+
+
+if __name__ == '__main__':
+    unittest.main()  # pragma: no cover

acme/acme/jose/interfaces.py

@@ -0,0 +1,216 @@
+"""JOSE interfaces."""
+import abc
+import collections
+import json
+
+import six
+
+from acme.jose import errors
+from acme.jose import util
+
+# pylint: disable=no-self-argument,no-method-argument,no-init,inherit-non-class
+# pylint: disable=too-few-public-methods
+
+
+@six.add_metaclass(abc.ABCMeta)
+class JSONDeSerializable(object):
+    # pylint: disable=too-few-public-methods
+    """Interface for (de)serializable JSON objects.
+
+    Please recall, that standard Python library implements
+    :class:`json.JSONEncoder` and :class:`json.JSONDecoder` that perform
+    translations based on respective :ref:`conversion tables
+    <conversion-table>` that look pretty much like the one below (for
+    complete tables see relevant Python documentation):
+
+    .. _conversion-table:
+
+    ======  ======
+     JSON   Python
+    ======  ======
+    object  dict
+    ...     ...
+    ======  ======
+
+    While the above **conversion table** is about translation of JSON
+    documents to/from the basic Python types only,
+    :class:`JSONDeSerializable` introduces the following two concepts:
+
+      serialization
+        Turning an arbitrary Python object into Python object that can
+        be encoded into a JSON document. **Full serialization** produces
+        a Python object composed of only basic types as required by the
+        :ref:`conversion table <conversion-table>`. **Partial
+        serialization** (accomplished by :meth:`to_partial_json`)
+        produces a Python object that might also be built from other
+        :class:`JSONDeSerializable` objects.
+
+      deserialization
+        Turning a decoded Python object (necessarily one of the basic
+        types as required by the :ref:`conversion table
+        <conversion-table>`) into an arbitrary Python object.
+
+    Serialization produces **serialized object** ("partially serialized
+    object" or "fully serialized object" for partial and full
+    serialization respectively) and deserialization produces
+    **deserialized object**, both usually denoted in the source code as
+    ``jobj``.
+
+    Wording in the official Python documentation might be confusing
+    after reading the above, but in the light of those definitions, one
+    can view :meth:`json.JSONDecoder.decode` as decoder and
+    deserializer of basic types, :meth:`json.JSONEncoder.default` as
+    serializer of basic types, :meth:`json.JSONEncoder.encode`  as
+    serializer and encoder of basic types.
+
+    One could extend :mod:`json` to support arbitrary object
+    (de)serialization either by:
+
+      - overriding :meth:`json.JSONDecoder.decode` and
+        :meth:`json.JSONEncoder.default` in subclasses
+
+      - or passing ``object_hook`` argument (or ``object_hook_pairs``)
+        to :func:`json.load`/:func:`json.loads` or ``default`` argument
+        for :func:`json.dump`/:func:`json.dumps`.
+
+    Interestingly, ``default`` is required to perform only partial
+    serialization, as :func:`json.dumps` applies ``default``
+    recursively. This is the idea behind making :meth:`to_partial_json`
+    produce only partial serialization, while providing custom
+    :meth:`json_dumps` that dumps with ``default`` set to
+    :meth:`json_dump_default`.
+
+    To make further documentation a bit more concrete, please, consider
+    the following imaginatory implementation example::
+
+      class Foo(JSONDeSerializable):
+          def to_partial_json(self):
+              return 'foo'
+
+          @classmethod
+          def from_json(cls, jobj):
+              return Foo()
+
+      class Bar(JSONDeSerializable):
+          def to_partial_json(self):
+              return [Foo(), Foo()]
+
+          @classmethod
+          def from_json(cls, jobj):
+              return Bar()
+
+    """
+
+    @abc.abstractmethod
+    def to_partial_json(self):  # pragma: no cover
+        """Partially serialize.
+
+        Following the example, **partial serialization** means the following::
+
+          assert isinstance(Bar().to_partial_json()[0], Foo)
+          assert isinstance(Bar().to_partial_json()[1], Foo)
+
+          # in particular...
+          assert Bar().to_partial_json() != ['foo', 'foo']
+
+        :raises acme.jose.errors.SerializationError:
+            in case of any serialization error.
+        :returns: Partially serializable object.
+
+        """
+        raise NotImplementedError()
+
+    def to_json(self):
+        """Fully serialize.
+
+        Again, following the example from before, **full serialization**
+        means the following::
+
+          assert Bar().to_json() == ['foo', 'foo']
+
+        :raises acme.jose.errors.SerializationError:
+            in case of any serialization error.
+        :returns: Fully serialized object.
+
+        """
+        def _serialize(obj):
+            if isinstance(obj, JSONDeSerializable):
+                return _serialize(obj.to_partial_json())
+            if isinstance(obj, six.string_types):  # strings are Sequence
+                return obj
+            elif isinstance(obj, list):
+                return [_serialize(subobj) for subobj in obj]
+            elif isinstance(obj, collections.Sequence):
+                # default to tuple, otherwise Mapping could get
+                # unhashable list
+                return tuple(_serialize(subobj) for subobj in obj)
+            elif isinstance(obj, collections.Mapping):
+                return dict((_serialize(key), _serialize(value))
+                            for key, value in six.iteritems(obj))
+            else:
+                return obj
+
+        return _serialize(self)
+
+    @util.abstractclassmethod
+    def from_json(cls, jobj):  # pylint: disable=unused-argument
+        """Deserialize a decoded JSON document.
+
+        :param jobj: Python object, composed of only other basic data
+            types, as decoded from JSON document. Not necessarily
+            :class:`dict` (as decoded from "JSON object" document).
+
+        :raises acme.jose.errors.DeserializationError:
+            if decoding was unsuccessful, e.g. in case of unparseable
+            X509 certificate, or wrong padding in JOSE base64 encoded
+            string, etc.
+
+        """
+        # TypeError: Can't instantiate abstract class <cls> with
+        # abstract methods from_json, to_partial_json
+        return cls()  # pylint: disable=abstract-class-instantiated
+
+    @classmethod
+    def json_loads(cls, json_string):
+        """Deserialize from JSON document string."""
+        try:
+            loads = json.loads(json_string)
+        except ValueError as error:
+            raise errors.DeserializationError(error)
+        return cls.from_json(loads)
+
+    def json_dumps(self, **kwargs):
+        """Dump to JSON string using proper serializer.
+
+        :returns: JSON document string.
+        :rtype: str
+
+        """
+        return json.dumps(self, default=self.json_dump_default, **kwargs)
+
+    def json_dumps_pretty(self):
+        """Dump the object to pretty JSON document string.
+
+        :rtype: str
+
+        """
+        return self.json_dumps(sort_keys=True, indent=4, separators=(',', ': '))
+
+    @classmethod
+    def json_dump_default(cls, python_object):
+        """Serialize Python object.
+
+        This function is meant to be passed as ``default`` to
+        :func:`json.dump` or :func:`json.dumps`. They call
+        ``default(python_object)`` only for non-basic Python types, so
+        this function necessarily raises :class:`TypeError` if
+        ``python_object`` is not an instance of
+        :class:`IJSONSerializable`.
+
+        Please read the class docstring for more information.
+
+        """
+        if isinstance(python_object, JSONDeSerializable):
+            return python_object.to_partial_json()
+        else:  # this branch is necessary, cannot just "return"
+            raise TypeError(repr(python_object) + ' is not JSON serializable')

acme/acme/jose/interfaces_test.py

@@ -0,0 +1,114 @@
+"""Tests for acme.jose.interfaces."""
+import unittest
+
+
+class JSONDeSerializableTest(unittest.TestCase):
+    # pylint: disable=too-many-instance-attributes
+
+    def setUp(self):
+        from acme.jose.interfaces import JSONDeSerializable
+
+        # pylint: disable=missing-docstring,invalid-name
+
+        class Basic(JSONDeSerializable):
+            def __init__(self, v):
+                self.v = v
+
+            def to_partial_json(self):
+                return self.v
+
+            @classmethod
+            def from_json(cls, jobj):
+                return cls(jobj)
+
+        class Sequence(JSONDeSerializable):
+            def __init__(self, x, y):
+                self.x = x
+                self.y = y
+
+            def to_partial_json(self):
+                return [self.x, self.y]
+
+            @classmethod
+            def from_json(cls, jobj):
+                return cls(
+                    Basic.from_json(jobj[0]), Basic.from_json(jobj[1]))
+
+        class Mapping(JSONDeSerializable):
+            def __init__(self, x, y):
+                self.x = x
+                self.y = y
+
+            def to_partial_json(self):
+                return {self.x: self.y}
+
+            @classmethod
+            def from_json(cls, jobj):
+                pass  # pragma: no cover
+
+        self.basic1 = Basic('foo1')
+        self.basic2 = Basic('foo2')
+        self.seq = Sequence(self.basic1, self.basic2)
+        self.mapping = Mapping(self.basic1, self.basic2)
+        self.nested = Basic([[self.basic1]])
+        self.tuple = Basic(('foo',))
+
+        # pylint: disable=invalid-name
+        self.Basic = Basic
+        self.Sequence = Sequence
+        self.Mapping = Mapping
+
+    def test_to_json_sequence(self):
+        self.assertEqual(self.seq.to_json(), ['foo1', 'foo2'])
+
+    def test_to_json_mapping(self):
+        self.assertEqual(self.mapping.to_json(), {'foo1': 'foo2'})
+
+    def test_to_json_other(self):
+        mock_value = object()
+        self.assertTrue(self.Basic(mock_value).to_json() is mock_value)
+
+    def test_to_json_nested(self):
+        self.assertEqual(self.nested.to_json(), [['foo1']])
+
+    def test_to_json(self):
+        self.assertEqual(self.tuple.to_json(), (('foo', )))
+
+    def test_from_json_not_implemented(self):
+        from acme.jose.interfaces import JSONDeSerializable
+        self.assertRaises(TypeError, JSONDeSerializable.from_json, 'xxx')
+
+    def test_json_loads(self):
+        seq = self.Sequence.json_loads('["foo1", "foo2"]')
+        self.assertTrue(isinstance(seq, self.Sequence))
+        self.assertTrue(isinstance(seq.x, self.Basic))
+        self.assertTrue(isinstance(seq.y, self.Basic))
+        self.assertEqual(seq.x.v, 'foo1')
+        self.assertEqual(seq.y.v, 'foo2')
+
+    def test_json_dumps(self):
+        self.assertEqual('["foo1", "foo2"]', self.seq.json_dumps())
+
+    def test_json_dumps_pretty(self):
+        self.assertEqual(self.seq.json_dumps_pretty(),
+                         '[\n    "foo1",\n    "foo2"\n]')
+
+    def test_json_dump_default(self):
+        from acme.jose.interfaces import JSONDeSerializable
+
+        self.assertEqual(
+            'foo1', JSONDeSerializable.json_dump_default(self.basic1))
+
+        jobj = JSONDeSerializable.json_dump_default(self.seq)
+        self.assertEqual(len(jobj), 2)
+        self.assertTrue(jobj[0] is self.basic1)
+        self.assertTrue(jobj[1] is self.basic2)
+
+    def test_json_dump_default_type_error(self):
+        from acme.jose.interfaces import JSONDeSerializable
+        self.assertRaises(
+            TypeError, JSONDeSerializable.json_dump_default, object())
+
+
+if __name__ == '__main__':
+    unittest.main()  # pragma: no cover

acme/acme/jose/json_util.py

@@ -0,0 +1,485 @@
+"""JSON (de)serialization framework.
+
+The framework presented here is somewhat based on `Go's "json" package`_
+(especially the ``omitempty`` functionality).
+
+.. _`Go's "json" package`: http://golang.org/pkg/encoding/json/
+
+"""
+import abc
+import binascii
+import logging
+
+import OpenSSL
+import six
+
+from acme.jose import b64
+from acme.jose import errors
+from acme.jose import interfaces
+from acme.jose import util
+
+
+logger = logging.getLogger(__name__)
+
+
+class Field(object):
+    """JSON object field.
+
+    :class:`Field` is meant to be used together with
+    :class:`JSONObjectWithFields`.
+
+    ``encoder`` (``decoder``) is a callable that accepts a single
+    parameter, i.e. a value to be encoded (decoded), and returns the
+    serialized (deserialized) value. In case of errors it should raise
+    :class:`~acme.jose.errors.SerializationError`
+    (:class:`~acme.jose.errors.DeserializationError`).
+
+    Note, that ``decoder`` should perform partial serialization only.
+
+    :ivar str json_name: Name of the field when encoded to JSON.
+    :ivar default: Default value (used when not present in JSON object).
+    :ivar bool omitempty: If ``True`` and the field value is empty, then
+        it will not be included in the serialized JSON object, and
+        ``default`` will be used for deserialization. Otherwise, if ``False``,
+        field is considered as required, value will always be included in the
+        serialized JSON objected, and it must also be present when
+        deserializing.
+
+    """
+    __slots__ = ('json_name', 'default', 'omitempty', 'fdec', 'fenc')
+
+    def __init__(self, json_name, default=None, omitempty=False,
+                 decoder=None, encoder=None):
+        # pylint: disable=too-many-arguments
+        self.json_name = json_name
+        self.default = default
+        self.omitempty = omitempty
+
+        self.fdec = self.default_decoder if decoder is None else decoder
+        self.fenc = self.default_encoder if encoder is None else encoder
+
+    @classmethod
+    def _empty(cls, value):
+        """Is the provided value considered "empty" for this field?
+
+        This is useful for subclasses that might want to override the
+        definition of being empty, e.g. for some more exotic data types.
+
+        """
+        return not isinstance(value, bool) and not value
+
+    def omit(self, value):
+        """Omit the value in output?"""
+        return self._empty(value) and self.omitempty
+
+    def _update_params(self, **kwargs):
+        current = dict(json_name=self.json_name, default=self.default,
+                       omitempty=self.omitempty,
+                       decoder=self.fdec, encoder=self.fenc)
+        current.update(kwargs)
+        return type(self)(**current)  # pylint: disable=star-args
+
+    def decoder(self, fdec):
+        """Descriptor to change the decoder on JSON object field."""
+        return self._update_params(decoder=fdec)
+
+    def encoder(self, fenc):
+        """Descriptor to change the encoder on JSON object field."""
+        return self._update_params(encoder=fenc)
+
+    def decode(self, value):
+        """Decode a value, optionally with context JSON object."""
+        return self.fdec(value)
+
+    def encode(self, value):
+        """Encode a value, optionally with context JSON object."""
+        return self.fenc(value)
+
+    @classmethod
+    def default_decoder(cls, value):
+        """Default decoder.
+
+        Recursively deserialize into immutable types (
+        :class:`acme.jose.util.frozendict` instead of
+        :func:`dict`, :func:`tuple` instead of :func:`list`).
+
+        """
+        # bases cases for different types returned by json.loads
+        if isinstance(value, list):
+            return tuple(cls.default_decoder(subvalue) for subvalue in value)
+        elif isinstance(value, dict):
+            return util.frozendict(
+                dict((cls.default_decoder(key), cls.default_decoder(value))
+                     for key, value in six.iteritems(value)))
+        else:  # integer or string
+            return value
+
+    @classmethod
+    def default_encoder(cls, value):
+        """Default (passthrough) encoder."""
+        # field.to_partial_json() is no good as encoder has to do partial
+        # serialization only
+        return value
+
+
+class JSONObjectWithFieldsMeta(abc.ABCMeta):
+    """Metaclass for :class:`JSONObjectWithFields` and its subclasses.
+
+    It makes sure that, for any class ``cls`` with ``__metaclass__``
+    set to ``JSONObjectWithFieldsMeta``:
+
+    1. All fields (attributes of type :class:`Field`) in the class
+       definition are moved to the ``cls._fields`` dictionary, where
+       keys are field attribute names and values are fields themselves.
+
+    2. ``cls.__slots__`` is extended by all field attribute names
+       (i.e. not :attr:`Field.json_name`). Original ``cls.__slots__``
+       are stored in ``cls._orig_slots``.
+
+    In a consequence, for a field attribute name ``some_field``,
+    ``cls.some_field`` will be a slot descriptor and not an instance
+    of :class:`Field`. For example::
+
+      some_field = Field('someField', default=())
+
+      class Foo(object):
+          __metaclass__ = JSONObjectWithFieldsMeta
+          __slots__ = ('baz',)
+          some_field = some_field
+
+      assert Foo.__slots__ == ('some_field', 'baz')
+      assert Foo._orig_slots == ()
+      assert Foo.some_field is not Field
+
+      assert Foo._fields.keys() == ['some_field']
+      assert Foo._fields['some_field'] is some_field
+
+    As an implementation note, this metaclass inherits from
+    :class:`abc.ABCMeta` (and not the usual :class:`type`) to mitigate
+    the metaclass conflict (:class:`ImmutableMap` and
+    :class:`JSONDeSerializable`, parents of :class:`JSONObjectWithFields`,
+    use :class:`abc.ABCMeta` as its metaclass).
+
+    """
+
+    def __new__(mcs, name, bases, dikt):
+        fields = {}
+
+        for base in bases:
+            fields.update(getattr(base, '_fields', {}))
+        # Do not reorder, this class might override fields from base classes!
+        for key, value in tuple(six.iteritems(dikt)):
+            # not six.iterkeys() (in-place edit!)
+            if isinstance(value, Field):
+                fields[key] = dikt.pop(key)
+
+        dikt['_orig_slots'] = dikt.get('__slots__', ())
+        dikt['__slots__'] = tuple(
+            list(dikt['_orig_slots']) + list(six.iterkeys(fields)))
+        dikt['_fields'] = fields
+
+        return abc.ABCMeta.__new__(mcs, name, bases, dikt)
+
+
+@six.add_metaclass(JSONObjectWithFieldsMeta)
+class JSONObjectWithFields(util.ImmutableMap, interfaces.JSONDeSerializable):
+    # pylint: disable=too-few-public-methods
+    """JSON object with fields.
+
+    Example::
+
+      class Foo(JSONObjectWithFields):
+          bar = Field('Bar')
+          empty = Field('Empty', omitempty=True)
+
+          @bar.encoder
+          def bar(value):
+              return value + 'bar'
+
+          @bar.decoder
+          def bar(value):
+              if not value.endswith('bar'):
+                  raise errors.DeserializationError('No bar suffix!')
+              return value[:-3]
+
+      assert Foo(bar='baz').to_partial_json() == {'Bar': 'bazbar'}
+      assert Foo.from_json({'Bar': 'bazbar'}) == Foo(bar='baz')
+      assert (Foo.from_json({'Bar': 'bazbar', 'Empty': '!'})
+              == Foo(bar='baz', empty='!'))
+      assert Foo(bar='baz').bar == 'baz'
+
+    """
+
+    @classmethod
+    def _defaults(cls):
+        """Get default fields values."""
+        return dict([(slot, field.default) for slot, field
+                     in six.iteritems(cls._fields)])
+
+    def __init__(self, **kwargs):
+        # pylint: disable=star-args
+        super(JSONObjectWithFields, self).__init__(
+            **(dict(self._defaults(), **kwargs)))
+
+    def encode(self, name):
+        """Encode a single field.
+
+        :param str name: Name of the field to be encoded.
+
+        :raises errors.SerializationError: if field cannot be serialized
+        :raises errors.Error: if field could not be found
+
+        """
+        try:
+            field = self._fields[name]
+        except KeyError:
+            raise errors.Error("Field not found: {0}".format(name))
+
+        return field.encode(getattr(self, name))
+
+    def fields_to_partial_json(self):
+        """Serialize fields to JSON."""
+        jobj = {}
+        omitted = set()
+        for slot, field in six.iteritems(self._fields):
+            value = getattr(self, slot)
+
+            if field.omit(value):
+                omitted.add((slot, value))
+            else:
+                try:
+                    jobj[field.json_name] = field.encode(value)
+                except errors.SerializationError as error:
+                    raise errors.SerializationError(
+                        'Could not encode {0} ({1}): {2}'.format(
+                            slot, value, error))
+        return jobj
+
+    def to_partial_json(self):
+        return self.fields_to_partial_json()
+
+    @classmethod
+    def _check_required(cls, jobj):
+        missing = set()
+        for _, field in six.iteritems(cls._fields):
+            if not field.omitempty and field.json_name not in jobj:
+                missing.add(field.json_name)
+
+        if missing:
+            raise errors.DeserializationError(
+                'The following fields are required: {0}'.format(
+                    ','.join(missing)))
+
+    @classmethod
+    def fields_from_json(cls, jobj):
+        """Deserialize fields from JSON."""
+        cls._check_required(jobj)
+        fields = {}
+        for slot, field in six.iteritems(cls._fields):
+            if field.json_name not in jobj and field.omitempty:
+                fields[slot] = field.default
+            else:
+                value = jobj[field.json_name]
+                try:
+                    fields[slot] = field.decode(value)
+                except errors.DeserializationError as error:
+                    raise errors.DeserializationError(
+                        'Could not decode {0!r} ({1!r}): {2}'.format(
+                            slot, value, error))
+        return fields
+
+    @classmethod
+    def from_json(cls, jobj):
+        return cls(**cls.fields_from_json(jobj))
+
+
+def encode_b64jose(data):
+    """Encode JOSE Base-64 field.
+
+    :param bytes data:
+    :rtype: `unicode`
+
+    """
+    # b64encode produces ASCII characters only
+    return b64.b64encode(data).decode('ascii')
+
+
+def decode_b64jose(data, size=None, minimum=False):
+    """Decode JOSE Base-64 field.
+
+    :param unicode data:
+    :param int size: Required length (after decoding).
+    :param bool minimum: If ``True``, then `size` will be treated as
+        minimum required length, as opposed to exact equality.
+
+    :rtype: bytes
+
+    """
+    error_cls = TypeError if six.PY2 else binascii.Error
+    try:
+        decoded = b64.b64decode(data.encode())
+    except error_cls as error:
+        raise errors.DeserializationError(error)
+
+    if size is not None and ((not minimum and len(decoded) != size) or
+                             (minimum and len(decoded) < size)):
+        raise errors.DeserializationError(
+            "Expected at least or exactly {0} bytes".format(size))
+
+    return decoded
+
+
+def encode_hex16(value):
+    """Hexlify.
+
+    :param bytes value:
+    :rtype: unicode
+
+    """
+    return binascii.hexlify(value).decode()
+
+
+def decode_hex16(value, size=None, minimum=False):
+    """Decode hexlified field.
+
+    :param unicode value:
+    :param int size: Required length (after decoding).
+    :param bool minimum: If ``True``, then `size` will be treated as
+        minimum required length, as opposed to exact equality.
+
+    :rtype: bytes
+
+    """
+    value = value.encode()
+    if size is not None and ((not minimum and len(value) != size * 2) or
+                             (minimum and len(value) < size * 2)):
+        raise errors.DeserializationError()
+    error_cls = TypeError if six.PY2 else binascii.Error
+    try:
+        return binascii.unhexlify(value)
+    except error_cls as error:
+        raise errors.DeserializationError(error)
+
+
+def encode_cert(cert):
+    """Encode certificate as JOSE Base-64 DER.
+
+    :type cert: `OpenSSL.crypto.X509` wrapped in `.ComparableX509`
+    :rtype: unicode
+
+    """
+    return encode_b64jose(OpenSSL.crypto.dump_certificate(
+        OpenSSL.crypto.FILETYPE_ASN1, cert.wrapped))
+
+
+def decode_cert(b64der):
+    """Decode JOSE Base-64 DER-encoded certificate.
+
+    :param unicode b64der:
+    :rtype: `OpenSSL.crypto.X509` wrapped in `.ComparableX509`
+
+    """
+    try:
+        return util.ComparableX509(OpenSSL.crypto.load_certificate(
+            OpenSSL.crypto.FILETYPE_ASN1, decode_b64jose(b64der)))
+    except OpenSSL.crypto.Error as error:
+        raise errors.DeserializationError(error)
+
+
+def encode_csr(csr):
+    """Encode CSR as JOSE Base-64 DER.
+
+    :type csr: `OpenSSL.crypto.X509Req` wrapped in `.ComparableX509`
+    :rtype: unicode
+
+    """
+    return encode_b64jose(OpenSSL.crypto.dump_certificate_request(
+        OpenSSL.crypto.FILETYPE_ASN1, csr.wrapped))
+
+
+def decode_csr(b64der):
+    """Decode JOSE Base-64 DER-encoded CSR.
+
+    :param unicode b64der:
+    :rtype: `OpenSSL.crypto.X509Req` wrapped in `.ComparableX509`
+
+    """
+    try:
+        return util.ComparableX509(OpenSSL.crypto.load_certificate_request(
+            OpenSSL.crypto.FILETYPE_ASN1, decode_b64jose(b64der)))
+    except OpenSSL.crypto.Error as error:
+        raise errors.DeserializationError(error)
+
+
+class TypedJSONObjectWithFields(JSONObjectWithFields):
+    """JSON object with type."""
+
+    typ = NotImplemented
+    """Type of the object. Subclasses must override."""
+
+    type_field_name = "type"
+    """Field name used to distinguish different object types.
+
+    Subclasses will probably have to override this.
+
+    """
+
+    TYPES = NotImplemented
+    """Types registered for JSON deserialization"""
+
+    @classmethod
+    def register(cls, type_cls, typ=None):
+        """Register class for JSON deserialization."""
+        typ = type_cls.typ if typ is None else typ
+        cls.TYPES[typ] = type_cls
+        return type_cls
+
+    @classmethod
+    def get_type_cls(cls, jobj):
+        """Get the registered class for ``jobj``."""
+        if cls in six.itervalues(cls.TYPES):
+            if cls.type_field_name not in jobj:
+                raise errors.DeserializationError(
+                    "Missing type field ({0})".format(cls.type_field_name))
+            # cls is already registered type_cls, force to use it
+            # so that, e.g Revocation.from_json(jobj) fails if
+            # jobj["type"] != "revocation".
+            return cls
+
+        if not isinstance(jobj, dict):
+            raise errors.DeserializationError(
+                "{0} is not a dictionary object".format(jobj))
+        try:
+            typ = jobj[cls.type_field_name]
+        except KeyError:
+            raise errors.DeserializationError("missing type field")
+
+        try:
+            return cls.TYPES[typ]
+        except KeyError:
+            raise errors.UnrecognizedTypeError(typ, jobj)
+
+    def to_partial_json(self):
+        """Get JSON serializable object.
+
+        :returns: Serializable JSON object representing ACME typed object.
+            :meth:`validate` will almost certainly not work, due to reasons
+            explained in :class:`acme.interfaces.IJSONSerializable`.
+        :rtype: dict
+
+        """
+        jobj = self.fields_to_partial_json()
+        jobj[self.type_field_name] = self.typ
+        return jobj
+
+    @classmethod
+    def from_json(cls, jobj):
+        """Deserialize ACME object from valid JSON object.
+
+        :raises acme.errors.UnrecognizedTypeError: if type
+            of the ACME object has not been registered.
+
+        """
+        # make sure subclasses don't cause infinite recursive from_json calls
+        type_cls = cls.get_type_cls(jobj)
+        return type_cls(**type_cls.fields_from_json(jobj))

acme/acme/jose/json_util_test.py

@@ -0,0 +1,381 @@
+"""Tests for acme.jose.json_util."""
+import itertools
+import unittest
+
+import mock
+import six
+
+from acme import test_util
+
+from acme.jose import errors
+from acme.jose import interfaces
+from acme.jose import util
+
+
+CERT = test_util.load_comparable_cert('cert.pem')
+CSR = test_util.load_comparable_csr('csr.pem')
+
+
+class FieldTest(unittest.TestCase):
+    """Tests for acme.jose.json_util.Field."""
+
+    def test_no_omit_boolean(self):
+        from acme.jose.json_util import Field
+        for default, omitempty, value in itertools.product(
+                [True, False], [True, False], [True, False]):
+            self.assertFalse(
+                Field("foo", default=default, omitempty=omitempty).omit(value))
+
+    def test_descriptors(self):
+        mock_value = mock.MagicMock()
+
+        # pylint: disable=missing-docstring
+
+        def decoder(unused_value):
+            return 'd'
+
+        def encoder(unused_value):
+            return 'e'
+
+        from acme.jose.json_util import Field
+        field = Field('foo')
+
+        field = field.encoder(encoder)
+        self.assertEqual('e', field.encode(mock_value))
+
+        field = field.decoder(decoder)
+        self.assertEqual('e', field.encode(mock_value))
+        self.assertEqual('d', field.decode(mock_value))
+
+    def test_default_encoder_is_partial(self):
+        class MockField(interfaces.JSONDeSerializable):
+            # pylint: disable=missing-docstring
+            def to_partial_json(self):
+                return 'foo'  # pragma: no cover
+
+            @classmethod
+            def from_json(cls, jobj):
+                pass  # pragma: no cover
+        mock_field = MockField()
+
+        from acme.jose.json_util import Field
+        self.assertTrue(Field.default_encoder(mock_field) is mock_field)
+        # in particular...
+        self.assertNotEqual('foo', Field.default_encoder(mock_field))
+
+    def test_default_encoder_passthrough(self):
+        mock_value = mock.MagicMock()
+        from acme.jose.json_util import Field
+        self.assertTrue(Field.default_encoder(mock_value) is mock_value)
+
+    def test_default_decoder_list_to_tuple(self):
+        from acme.jose.json_util import Field
+        self.assertEqual((1, 2, 3), Field.default_decoder([1, 2, 3]))
+
+    def test_default_decoder_dict_to_frozendict(self):
+        from acme.jose.json_util import Field
+        obj = Field.default_decoder({'x': 2})
+        self.assertTrue(isinstance(obj, util.frozendict))
+        self.assertEqual(obj, util.frozendict(x=2))
+
+    def test_default_decoder_passthrough(self):
+        mock_value = mock.MagicMock()
+        from acme.jose.json_util import Field
+        self.assertTrue(Field.default_decoder(mock_value) is mock_value)
+
+
+class JSONObjectWithFieldsMetaTest(unittest.TestCase):
+    """Tests for acme.jose.json_util.JSONObjectWithFieldsMeta."""
+
+    def setUp(self):
+        from acme.jose.json_util import Field
+        from acme.jose.json_util import JSONObjectWithFieldsMeta
+        self.field = Field('Baz')
+        self.field2 = Field('Baz2')
+        # pylint: disable=invalid-name,missing-docstring,too-few-public-methods
+        # pylint: disable=blacklisted-name
+
+        @six.add_metaclass(JSONObjectWithFieldsMeta)
+        class A(object):
+            __slots__ = ('bar',)
+            baz = self.field
+
+        class B(A):
+            pass
+
+        class C(A):
+            baz = self.field2
+
+        self.a_cls = A
+        self.b_cls = B
+        self.c_cls = C
+
+    def test_fields(self):
+        # pylint: disable=protected-access,no-member
+        self.assertEqual({'baz': self.field}, self.a_cls._fields)
+        self.assertEqual({'baz': self.field}, self.b_cls._fields)
+
+    def test_fields_inheritance(self):
+        # pylint: disable=protected-access,no-member
+        self.assertEqual({'baz': self.field2}, self.c_cls._fields)
+
+    def test_slots(self):
+        self.assertEqual(('bar', 'baz'), self.a_cls.__slots__)
+        self.assertEqual(('baz',), self.b_cls.__slots__)
+
+    def test_orig_slots(self):
+        # pylint: disable=protected-access,no-member
+        self.assertEqual(('bar',), self.a_cls._orig_slots)
+        self.assertEqual((), self.b_cls._orig_slots)
+
+
+class JSONObjectWithFieldsTest(unittest.TestCase):
+    """Tests for acme.jose.json_util.JSONObjectWithFields."""
+    # pylint: disable=protected-access
+
+    def setUp(self):
+        from acme.jose.json_util import JSONObjectWithFields
+        from acme.jose.json_util import Field
+
+        class MockJSONObjectWithFields(JSONObjectWithFields):
+            # pylint: disable=invalid-name,missing-docstring,no-self-argument
+            # pylint: disable=too-few-public-methods
+            x = Field('x', omitempty=True,
+                      encoder=(lambda x: x * 2),
+                      decoder=(lambda x: x / 2))
+            y = Field('y')
+            z = Field('Z')  # on purpose uppercase
+
+            @y.encoder
+            def y(value):
+                if value == 500:
+                    raise errors.SerializationError()
+                return value
+
+            @y.decoder
+            def y(value):
+                if value == 500:
+                    raise errors.DeserializationError()
+                return value
+
+        # pylint: disable=invalid-name
+        self.MockJSONObjectWithFields = MockJSONObjectWithFields
+        self.mock = MockJSONObjectWithFields(x=None, y=2, z=3)
+
+    def test_init_defaults(self):
+        self.assertEqual(self.mock, self.MockJSONObjectWithFields(y=2, z=3))
+
+    def test_encode(self):
+        self.assertEqual(10, self.MockJSONObjectWithFields(
+            x=5, y=0, z=0).encode("x"))
+
+    def test_encode_wrong_field(self):
+        self.assertRaises(errors.Error, self.mock.encode, 'foo')
+
+    def test_encode_serialization_error_passthrough(self):
+        self.assertRaises(
+            errors.SerializationError,
+            self.MockJSONObjectWithFields(y=500, z=None).encode, "y")
+
+    def test_fields_to_partial_json_omits_empty(self):
+        self.assertEqual(self.mock.fields_to_partial_json(), {'y': 2, 'Z': 3})
+
+    def test_fields_from_json_fills_default_for_empty(self):
+        self.assertEqual(
+            {'x': None, 'y': 2, 'z': 3},
+            self.MockJSONObjectWithFields.fields_from_json({'y': 2, 'Z': 3}))
+
+    def test_fields_from_json_fails_on_missing(self):
+        self.assertRaises(
+            errors.DeserializationError,
+            self.MockJSONObjectWithFields.fields_from_json, {'y': 0})
+        self.assertRaises(
+            errors.DeserializationError,
+            self.MockJSONObjectWithFields.fields_from_json, {'Z': 0})
+        self.assertRaises(
+            errors.DeserializationError,
+            self.MockJSONObjectWithFields.fields_from_json, {'x': 0, 'y': 0})
+        self.assertRaises(
+            errors.DeserializationError,
+            self.MockJSONObjectWithFields.fields_from_json, {'x': 0, 'Z': 0})
+
+    def test_fields_to_partial_json_encoder(self):
+        self.assertEqual(
+            self.MockJSONObjectWithFields(x=1, y=2, z=3).to_partial_json(),
+            {'x': 2, 'y': 2, 'Z': 3})
+
+    def test_fields_from_json_decoder(self):
+        self.assertEqual(
+            {'x': 2, 'y': 2, 'z': 3},
+            self.MockJSONObjectWithFields.fields_from_json(
+                {'x': 4, 'y': 2, 'Z': 3}))
+
+    def test_fields_to_partial_json_error_passthrough(self):
+        self.assertRaises(
+            errors.SerializationError, self.MockJSONObjectWithFields(
+                x=1, y=500, z=3).to_partial_json)
+
+    def test_fields_from_json_error_passthrough(self):
+        self.assertRaises(
+            errors.DeserializationError,
+            self.MockJSONObjectWithFields.from_json,
+            {'x': 4, 'y': 500, 'Z': 3})
+
+
+class DeEncodersTest(unittest.TestCase):
+    def setUp(self):
+        self.b64_cert = (
+            u'MIIB3jCCAYigAwIBAgICBTkwDQYJKoZIhvcNAQELBQAwdzELMAkGA1UEBhM'
+            u'CVVMxETAPBgNVBAgMCE1pY2hpZ2FuMRIwEAYDVQQHDAlBbm4gQXJib3IxKz'
+            u'ApBgNVBAoMIlVuaXZlcnNpdHkgb2YgTWljaGlnYW4gYW5kIHRoZSBFRkYxF'
+            u'DASBgNVBAMMC2V4YW1wbGUuY29tMB4XDTE0MTIxMTIyMzQ0NVoXDTE0MTIx'
+            u'ODIyMzQ0NVowdzELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE1pY2hpZ2FuMRI'
+            u'wEAYDVQQHDAlBbm4gQXJib3IxKzApBgNVBAoMIlVuaXZlcnNpdHkgb2YgTW'
+            u'ljaGlnYW4gYW5kIHRoZSBFRkYxFDASBgNVBAMMC2V4YW1wbGUuY29tMFwwD'
+            u'QYJKoZIhvcNAQEBBQADSwAwSAJBAKx1c7RR7R_drnBSQ_zfx1vQLHUbFLh1'
+            u'AQQQ5R8DZUXd36efNK79vukFhN9HFoHZiUvOjm0c-pVE6K-EdE_twuUCAwE'
+            u'AATANBgkqhkiG9w0BAQsFAANBAC24z0IdwIVKSlntksllvr6zJepBH5fMnd'
+            u'fk3XJp10jT6VE-14KNtjh02a56GoraAvJAT5_H67E8GvJ_ocNnB_o'
+        )
+        self.b64_csr = (
+            u'MIIBXTCCAQcCAQAweTELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE1pY2hpZ2F'
+            u'uMRIwEAYDVQQHDAlBbm4gQXJib3IxDDAKBgNVBAoMA0VGRjEfMB0GA1UECw'
+            u'wWVW5pdmVyc2l0eSBvZiBNaWNoaWdhbjEUMBIGA1UEAwwLZXhhbXBsZS5jb'
+            u'20wXDANBgkqhkiG9w0BAQEFAANLADBIAkEArHVztFHtH92ucFJD_N_HW9As'
+            u'dRsUuHUBBBDlHwNlRd3fp580rv2-6QWE30cWgdmJS86ObRz6lUTor4R0T-3'
+            u'C5QIDAQABoCkwJwYJKoZIhvcNAQkOMRowGDAWBgNVHREEDzANggtleGFtcG'
+            u'xlLmNvbTANBgkqhkiG9w0BAQsFAANBAHJH_O6BtC9aGzEVCMGOZ7z9iIRHW'
+            u'Szr9x_bOzn7hLwsbXPAgO1QxEwL-X-4g20Gn9XBE1N9W6HCIEut2d8wACg'
+        )
+
+    def test_encode_b64jose(self):
+        from acme.jose.json_util import encode_b64jose
+        encoded = encode_b64jose(b'x')
+        self.assertTrue(isinstance(encoded, six.string_types))
+        self.assertEqual(u'eA', encoded)
+
+    def test_decode_b64jose(self):
+        from acme.jose.json_util import decode_b64jose
+        decoded = decode_b64jose(u'eA')
+        self.assertTrue(isinstance(decoded, six.binary_type))
+        self.assertEqual(b'x', decoded)
+
+    def test_decode_b64jose_padding_error(self):
+        from acme.jose.json_util import decode_b64jose
+        self.assertRaises(errors.DeserializationError, decode_b64jose, u'x')
+
+    def test_decode_b64jose_size(self):
+        from acme.jose.json_util import decode_b64jose
+        self.assertEqual(b'foo', decode_b64jose(u'Zm9v', size=3))
+        self.assertRaises(
+            errors.DeserializationError, decode_b64jose, u'Zm9v', size=2)
+        self.assertRaises(
+            errors.DeserializationError, decode_b64jose, u'Zm9v', size=4)
+
+    def test_decode_b64jose_minimum_size(self):
+        from acme.jose.json_util import decode_b64jose
+        self.assertEqual(b'foo', decode_b64jose(u'Zm9v', size=3, minimum=True))
+        self.assertEqual(b'foo', decode_b64jose(u'Zm9v', size=2, minimum=True))
+        self.assertRaises(errors.DeserializationError, decode_b64jose,
+                          u'Zm9v', size=4, minimum=True)
+
+    def test_encode_hex16(self):
+        from acme.jose.json_util import encode_hex16
+        encoded = encode_hex16(b'foo')
+        self.assertEqual(u'666f6f', encoded)
+        self.assertTrue(isinstance(encoded, six.string_types))
+
+    def test_decode_hex16(self):
+        from acme.jose.json_util import decode_hex16
+        decoded = decode_hex16(u'666f6f')
+        self.assertEqual(b'foo', decoded)
+        self.assertTrue(isinstance(decoded, six.binary_type))
+
+    def test_decode_hex16_minimum_size(self):
+        from acme.jose.json_util import decode_hex16
+        self.assertEqual(b'foo', decode_hex16(u'666f6f', size=3, minimum=True))
+        self.assertEqual(b'foo', decode_hex16(u'666f6f', size=2, minimum=True))
+        self.assertRaises(errors.DeserializationError, decode_hex16,
+                          u'666f6f', size=4, minimum=True)
+
+    def test_decode_hex16_odd_length(self):
+        from acme.jose.json_util import decode_hex16
+        self.assertRaises(errors.DeserializationError, decode_hex16, u'x')
+
+    def test_encode_cert(self):
+        from acme.jose.json_util import encode_cert
+        self.assertEqual(self.b64_cert, encode_cert(CERT))
+
+    def test_decode_cert(self):
+        from acme.jose.json_util import decode_cert
+        cert = decode_cert(self.b64_cert)
+        self.assertTrue(isinstance(cert, util.ComparableX509))
+        self.assertEqual(cert, CERT)
+        self.assertRaises(errors.DeserializationError, decode_cert, u'')
+
+    def test_encode_csr(self):
+        from acme.jose.json_util import encode_csr
+        self.assertEqual(self.b64_csr, encode_csr(CSR))
+
+    def test_decode_csr(self):
+        from acme.jose.json_util import decode_csr
+        csr = decode_csr(self.b64_csr)
+        self.assertTrue(isinstance(csr, util.ComparableX509))
+        self.assertEqual(csr, CSR)
+        self.assertRaises(errors.DeserializationError, decode_csr, u'')
+
+
+class TypedJSONObjectWithFieldsTest(unittest.TestCase):
+
+    def setUp(self):
+        from acme.jose.json_util import TypedJSONObjectWithFields
+
+        # pylint: disable=missing-docstring,abstract-method
+        # pylint: disable=too-few-public-methods
+
+        class MockParentTypedJSONObjectWithFields(TypedJSONObjectWithFields):
+            TYPES = {}
+            type_field_name = 'type'
+
+        @MockParentTypedJSONObjectWithFields.register
+        class MockTypedJSONObjectWithFields(
+                MockParentTypedJSONObjectWithFields):
+            typ = 'test'
+            __slots__ = ('foo',)
+
+            @classmethod
+            def fields_from_json(cls, jobj):
+                return {'foo': jobj['foo']}
+
+            def fields_to_partial_json(self):
+                return {'foo': self.foo}
+
+        self.parent_cls = MockParentTypedJSONObjectWithFields
+        self.msg = MockTypedJSONObjectWithFields(foo='bar')
+
+    def test_to_partial_json(self):
+        self.assertEqual(self.msg.to_partial_json(), {
+            'type': 'test',
+            'foo': 'bar',
+        })
+
+    def test_from_json_non_dict_fails(self):
+        for value in [[], (), 5, "asd"]:  # all possible input types
+            self.assertRaises(
+                errors.DeserializationError, self.parent_cls.from_json, value)
+
+    def test_from_json_dict_no_type_fails(self):
+        self.assertRaises(
+            errors.DeserializationError, self.parent_cls.from_json, {})
+
+    def test_from_json_unknown_type_fails(self):
+        self.assertRaises(errors.UnrecognizedTypeError,
+                          self.parent_cls.from_json, {'type': 'bar'})
+
+    def test_from_json_returns_obj(self):
+        self.assertEqual({'foo': 'bar'}, self.parent_cls.from_json(
+            {'type': 'test', 'foo': 'bar'}))
+
+
+if __name__ == '__main__':
+    unittest.main()  # pragma: no cover

acme/acme/jose/jwa.py

@@ -0,0 +1,180 @@
+"""JSON Web Algorithm.
+
+https://tools.ietf.org/html/draft-ietf-jose-json-web-algorithms-40
+
+"""
+import abc
+import collections
+import logging
+
+import cryptography.exceptions
+from cryptography.hazmat.backends import default_backend
+from cryptography.hazmat.primitives import hashes  # type: ignore
+from cryptography.hazmat.primitives import hmac  # type: ignore
+from cryptography.hazmat.primitives.asymmetric import padding  # type: ignore
+
+from acme.jose import errors
+from acme.jose import interfaces
+from acme.jose import jwk
+
+
+logger = logging.getLogger(__name__)
+
+
+class JWA(interfaces.JSONDeSerializable):  # pylint: disable=abstract-method
+    # pylint: disable=too-few-public-methods
+    # for some reason disable=abstract-method has to be on the line
+    # above...
+    """JSON Web Algorithm."""
+
+
+class JWASignature(JWA, collections.Hashable):  # type: ignore
+    """JSON Web Signature Algorithm."""
+    SIGNATURES = {}  # type: dict
+
+    def __init__(self, name):
+        self.name = name
+
+    def __eq__(self, other):
+        if not isinstance(other, JWASignature):
+            return NotImplemented
+        return self.name == other.name
+
+    def __hash__(self):
+        return hash((self.__class__, self.name))
+
+    def __ne__(self, other):
+        return not self == other
+
+    @classmethod
+    def register(cls, signature_cls):
+        """Register class for JSON deserialization."""
+        cls.SIGNATURES[signature_cls.name] = signature_cls
+        return signature_cls
+
+    def to_partial_json(self):
+        return self.name
+
+    @classmethod
+    def from_json(cls, jobj):
+        return cls.SIGNATURES[jobj]
+
+    @abc.abstractmethod
+    def sign(self, key, msg):  # pragma: no cover
+        """Sign the ``msg`` using ``key``."""
+        raise NotImplementedError()
+
+    @abc.abstractmethod
+    def verify(self, key, msg, sig):  # pragma: no cover
+        """Verify the ``msg` and ``sig`` using ``key``."""
+        raise NotImplementedError()
+
+    def __repr__(self):
+        return self.name
+
+
+class _JWAHS(JWASignature):
+
+    kty = jwk.JWKOct
+
+    def __init__(self, name, hash_):
+        super(_JWAHS, self).__init__(name)
+        self.hash = hash_()
+
+    def sign(self, key, msg):
+        signer = hmac.HMAC(key, self.hash, backend=default_backend())
+        signer.update(msg)
+        return signer.finalize()
+
+    def verify(self, key, msg, sig):
+        verifier = hmac.HMAC(key, self.hash, backend=default_backend())
+        verifier.update(msg)
+        try:
+            verifier.verify(sig)
+        except cryptography.exceptions.InvalidSignature as error:
+            logger.debug(error, exc_info=True)
+            return False
+        else:
+            return True
+
+
+class _JWARSA(object):
+
+    kty = jwk.JWKRSA
+    padding = NotImplemented
+    hash = NotImplemented
+
+    def sign(self, key, msg):
+        """Sign the ``msg`` using ``key``."""
+        try:
+            signer = key.signer(self.padding, self.hash)
+        except AttributeError as error:
+            logger.debug(error, exc_info=True)
+            raise errors.Error("Public key cannot be used for signing")
+        except ValueError as error:  # digest too large
+            logger.debug(error, exc_info=True)
+            raise errors.Error(str(error))
+        signer.update(msg)
+        try:
+            return signer.finalize()
+        except ValueError as error:
+            logger.debug(error, exc_info=True)
+            raise errors.Error(str(error))
+
+    def verify(self, key, msg, sig):
+        """Verify the ``msg` and ``sig`` using ``key``."""
+        verifier = key.verifier(sig, self.padding, self.hash)
+        verifier.update(msg)
+        try:
+            verifier.verify()
+        except cryptography.exceptions.InvalidSignature as error:
+            logger.debug(error, exc_info=True)
+            return False
+        else:
+            return True
+
+
+class _JWARS(_JWARSA, JWASignature):
+
+    def __init__(self, name, hash_):
+        super(_JWARS, self).__init__(name)
+        self.padding = padding.PKCS1v15()
+        self.hash = hash_()
+
+
+class _JWAPS(_JWARSA, JWASignature):
+
+    def __init__(self, name, hash_):
+        super(_JWAPS, self).__init__(name)
+        self.padding = padding.PSS(
+            mgf=padding.MGF1(hash_()),
+            salt_length=padding.PSS.MAX_LENGTH)
+        self.hash = hash_()
+
+
+class _JWAES(JWASignature):  # pylint: disable=abstract-class-not-used
+
+    # TODO: implement ES signatures
+
+    def sign(self, key, msg):  # pragma: no cover
+        raise NotImplementedError()
+
+    def verify(self, key, msg, sig):  # pragma: no cover
+        raise NotImplementedError()
+
+
+HS256 = JWASignature.register(_JWAHS('HS256', hashes.SHA256))
+HS384 = JWASignature.register(_JWAHS('HS384', hashes.SHA384))
+HS512 = JWASignature.register(_JWAHS('HS512', hashes.SHA512))
+
+RS256 = JWASignature.register(_JWARS('RS256', hashes.SHA256))
+RS384 = JWASignature.register(_JWARS('RS384', hashes.SHA384))
+RS512 = JWASignature.register(_JWARS('RS512', hashes.SHA512))
+
+PS256 = JWASignature.register(_JWAPS('PS256', hashes.SHA256))
+PS384 = JWASignature.register(_JWAPS('PS384', hashes.SHA384))
+PS512 = JWASignature.register(_JWAPS('PS512', hashes.SHA512))
+
+ES256 = JWASignature.register(_JWAES('ES256'))
+ES384 = JWASignature.register(_JWAES('ES384'))
+ES512 = JWASignature.register(_JWAES('ES512'))

acme/acme/jose/jwa_test.py

@@ -0,0 +1,104 @@
+"""Tests for acme.jose.jwa."""
+import unittest
+
+from acme import test_util
+
+from acme.jose import errors
+
+
+RSA256_KEY = test_util.load_rsa_private_key('rsa256_key.pem')
+RSA512_KEY = test_util.load_rsa_private_key('rsa512_key.pem')
+RSA1024_KEY = test_util.load_rsa_private_key('rsa1024_key.pem')
+
+
+class JWASignatureTest(unittest.TestCase):
+    """Tests for acme.jose.jwa.JWASignature."""
+
+    def setUp(self):
+        from acme.jose.jwa import JWASignature
+
+        class MockSig(JWASignature):
+            # pylint: disable=missing-docstring,too-few-public-methods
+            # pylint: disable=abstract-class-not-used
+            def sign(self, key, msg):
+                raise NotImplementedError()  # pragma: no cover
+
+            def verify(self, key, msg, sig):
+                raise NotImplementedError()  # pragma: no cover
+
+        # pylint: disable=invalid-name
+        self.Sig1 = MockSig('Sig1')
+        self.Sig2 = MockSig('Sig2')
+
+    def test_eq(self):
+        self.assertEqual(self.Sig1, self.Sig1)
+
+    def test_ne(self):
+        self.assertNotEqual(self.Sig1, self.Sig2)
+
+    def test_ne_other_type(self):
+        self.assertNotEqual(self.Sig1, 5)
+
+    def test_repr(self):
+        self.assertEqual('Sig1', repr(self.Sig1))
+        self.assertEqual('Sig2', repr(self.Sig2))
+
+    def test_to_partial_json(self):
+        self.assertEqual(self.Sig1.to_partial_json(), 'Sig1')
+        self.assertEqual(self.Sig2.to_partial_json(), 'Sig2')
+
+    def test_from_json(self):
+        from acme.jose.jwa import JWASignature
+        from acme.jose.jwa import RS256
+        self.assertTrue(JWASignature.from_json('RS256') is RS256)
+
+
+class JWAHSTest(unittest.TestCase):  # pylint: disable=too-few-public-methods
+
+    def test_it(self):
+        from acme.jose.jwa import HS256
+        sig = (
+            b"\xceR\xea\xcd\x94\xab\xcf\xfb\xe0\xacA.:\x1a'\x08i\xe2\xc4"
+            b"\r\x85+\x0e\x85\xaeUZ\xd4\xb3\x97zO"
+        )
+        self.assertEqual(HS256.sign(b'some key', b'foo'), sig)
+        self.assertTrue(HS256.verify(b'some key', b'foo', sig) is True)
+        self.assertTrue(HS256.verify(b'some key', b'foo', sig + b'!') is False)
+
+
+class JWARSTest(unittest.TestCase):
+
+    def test_sign_no_private_part(self):
+        from acme.jose.jwa import RS256
+        self.assertRaises(
+            errors.Error, RS256.sign, RSA512_KEY.public_key(), b'foo')
+
+    def test_sign_key_too_small(self):
+        from acme.jose.jwa import RS256
+        from acme.jose.jwa import PS256
+        self.assertRaises(errors.Error, RS256.sign, RSA256_KEY, b'foo')
+        self.assertRaises(errors.Error, PS256.sign, RSA256_KEY, b'foo')
+
+    def test_rs(self):
+        from acme.jose.jwa import RS256
+        sig = (
+            b'|\xc6\xb2\xa4\xab(\x87\x99\xfa*:\xea\xf8\xa0N&}\x9f\x0f\xc0O'
+            b'\xc6t\xa3\xe6\xfa\xbb"\x15Y\x80Y\xe0\x81\xb8\x88)\xba\x0c\x9c'
+            b'\xa4\x99\x1e\x19&\xd8\xc7\x99S\x97\xfc\x85\x0cOV\xe6\x07\x99'
+            b'\xd2\xb9.>}\xfd'
+        )
+        self.assertEqual(RS256.sign(RSA512_KEY, b'foo'), sig)
+        self.assertTrue(RS256.verify(RSA512_KEY.public_key(), b'foo', sig))
+        self.assertFalse(RS256.verify(
+            RSA512_KEY.public_key(), b'foo', sig + b'!'))
+
+    def test_ps(self):
+        from acme.jose.jwa import PS256
+        sig = PS256.sign(RSA1024_KEY, b'foo')
+        self.assertTrue(PS256.verify(RSA1024_KEY.public_key(), b'foo', sig))
+        self.assertFalse(PS256.verify(
+            RSA1024_KEY.public_key(), b'foo', sig + b'!'))
+
+
+if __name__ == '__main__':
+    unittest.main()  # pragma: no cover

acme/acme/jose/jwk.py

@@ -0,0 +1,281 @@
+"""JSON Web Key."""
+import abc
+import binascii
+import json
+import logging
+
+import cryptography.exceptions
+from cryptography.hazmat.backends import default_backend
+from cryptography.hazmat.primitives import hashes  # type: ignore
+from cryptography.hazmat.primitives import serialization
+from cryptography.hazmat.primitives.asymmetric import ec  # type: ignore
+from cryptography.hazmat.primitives.asymmetric import rsa
+
+import six
+
+from acme.jose import errors
+from acme.jose import json_util
+from acme.jose import util
+
+
+logger = logging.getLogger(__name__)
+
+
+class JWK(json_util.TypedJSONObjectWithFields):
+    # pylint: disable=too-few-public-methods
+    """JSON Web Key."""
+    type_field_name = 'kty'
+    TYPES = {}  # type: dict
+    cryptography_key_types = ()  # type: tuple
+    """Subclasses should override."""
+
+    required = NotImplemented
+    """Required members of public key's representation as defined by JWK/JWA."""
+
+    _thumbprint_json_dumps_params = {
+        # "no whitespace or line breaks before or after any syntactic
+        # elements"
+        'indent': None,
+        'separators': (',', ':'),
+        # "members ordered lexicographically by the Unicode [UNICODE]
+        # code points of the member names"
+        'sort_keys': True,
+    }
+
+    def thumbprint(self, hash_function=hashes.SHA256):
+        """Compute JWK Thumbprint.
+
+        https://tools.ietf.org/html/rfc7638
+
+        :returns bytes:
+
+        """
+        digest = hashes.Hash(hash_function(), backend=default_backend())
+        digest.update(json.dumps(
+            dict((k, v) for k, v in six.iteritems(self.to_json())
+                 if k in self.required),
+            **self._thumbprint_json_dumps_params).encode())
+        return digest.finalize()
+
+    @abc.abstractmethod
+    def public_key(self):  # pragma: no cover
+        """Generate JWK with public key.
+
+        For symmetric cryptosystems, this would return ``self``.
+
+        """
+        raise NotImplementedError()
+
+    @classmethod
+    def _load_cryptography_key(cls, data, password=None, backend=None):
+        backend = default_backend() if backend is None else backend
+        exceptions = {}
+
+        # private key?
+        for loader in (serialization.load_pem_private_key,
+                       serialization.load_der_private_key):
+            try:
+                return loader(data, password, backend)
+            except (ValueError, TypeError,
+                    cryptography.exceptions.UnsupportedAlgorithm) as error:
+                exceptions[loader] = error
+
+        # public key?
+        for loader in (serialization.load_pem_public_key,
+                       serialization.load_der_public_key):
+            try:
+                return loader(data, backend)
+            except (ValueError,
+                    cryptography.exceptions.UnsupportedAlgorithm) as error:
+                exceptions[loader] = error
+
+        # no luck
+        raise errors.Error('Unable to deserialize key: {0}'.format(exceptions))
+
+    @classmethod
+    def load(cls, data, password=None, backend=None):
+        """Load serialized key as JWK.
+
+        :param str data: Public or private key serialized as PEM or DER.
+        :param str password: Optional password.
+        :param backend: A `.PEMSerializationBackend` and
+            `.DERSerializationBackend` provider.
+
+        :raises errors.Error: if unable to deserialize, or unsupported
+            JWK algorithm
+
+        :returns: JWK of an appropriate type.
+        :rtype: `JWK`
+
+        """
+        try:
+            key = cls._load_cryptography_key(data, password, backend)
+        except errors.Error as error:
+            logger.debug('Loading symmetric key, asymmetric failed: %s', error)
+            return JWKOct(key=data)
+
+        if cls.typ is not NotImplemented and not isinstance(
+                key, cls.cryptography_key_types):
+            raise errors.Error('Unable to deserialize {0} into {1}'.format(
+                key.__class__, cls.__class__))
+        for jwk_cls in six.itervalues(cls.TYPES):
+            if isinstance(key, jwk_cls.cryptography_key_types):
+                return jwk_cls(key=key)
+        raise errors.Error('Unsupported algorithm: {0}'.format(key.__class__))
+
+
+@JWK.register
+class JWKES(JWK):  # pragma: no cover
+    # pylint: disable=abstract-class-not-used
+    """ES JWK.
+
+    .. warning:: This is not yet implemented!
+
+    """
+    typ = 'ES'
+    cryptography_key_types = (
+        ec.EllipticCurvePublicKey, ec.EllipticCurvePrivateKey)
+    required = ('crv', JWK.type_field_name, 'x', 'y')
+
+    def fields_to_partial_json(self):
+        raise NotImplementedError()
+
+    @classmethod
+    def fields_from_json(cls, jobj):
+        raise NotImplementedError()
+
+    def public_key(self):
+        raise NotImplementedError()
+
+
+@JWK.register
+class JWKOct(JWK):
+    """Symmetric JWK."""
+    typ = 'oct'
+    __slots__ = ('key',)
+    required = ('k', JWK.type_field_name)
+
+    def fields_to_partial_json(self):
+        # TODO: An "alg" member SHOULD also be present to identify the
+        # algorithm intended to be used with the key, unless the
+        # application uses another means or convention to determine
+        # the algorithm used.
+        return {'k': json_util.encode_b64jose(self.key)}
+
+    @classmethod
+    def fields_from_json(cls, jobj):
+        return cls(key=json_util.decode_b64jose(jobj['k']))
+
+    def public_key(self):
+        return self
+
+
+@JWK.register
+class JWKRSA(JWK):
+    """RSA JWK.
+
+    :ivar key: `cryptography.hazmat.primitives.rsa.RSAPrivateKey`
+        or `cryptography.hazmat.primitives.rsa.RSAPublicKey` wrapped
+        in `.ComparableRSAKey`
+
+    """
+    typ = 'RSA'
+    cryptography_key_types = (rsa.RSAPublicKey, rsa.RSAPrivateKey)
+    __slots__ = ('key',)
+    required = ('e', JWK.type_field_name, 'n')
+
+    def __init__(self, *args, **kwargs):
+        if 'key' in kwargs and not isinstance(
+                kwargs['key'], util.ComparableRSAKey):
+            kwargs['key'] = util.ComparableRSAKey(kwargs['key'])
+        super(JWKRSA, self).__init__(*args, **kwargs)
+
+    @classmethod
+    def _encode_param(cls, data):
+        """Encode Base64urlUInt.
+
+        :type data: long
+        :rtype: unicode
+
+        """
+        def _leading_zeros(arg):
+            if len(arg) % 2:
+                return '0' + arg
+            return arg
+
+        return json_util.encode_b64jose(binascii.unhexlify(
+            _leading_zeros(hex(data)[2:].rstrip('L'))))
+
+    @classmethod
+    def _decode_param(cls, data):
+        """Decode Base64urlUInt."""
+        try:
+            return int(binascii.hexlify(json_util.decode_b64jose(data)), 16)
+        except ValueError:  # invalid literal for long() with base 16
+            raise errors.DeserializationError()
+
+    def public_key(self):
+        return type(self)(key=self.key.public_key())
+
+    @classmethod
+    def fields_from_json(cls, jobj):
+        # pylint: disable=invalid-name
+        n, e = (cls._decode_param(jobj[x]) for x in ('n', 'e'))
+        public_numbers = rsa.RSAPublicNumbers(e=e, n=n)
+        if 'd' not in jobj:  # public key
+            key = public_numbers.public_key(default_backend())
+        else:  # private key
+            d = cls._decode_param(jobj['d'])
+            if ('p' in jobj or 'q' in jobj or 'dp' in jobj or
+                    'dq' in jobj or 'qi' in jobj or 'oth' in jobj):
+                # "If the producer includes any of the other private
+                # key parameters, then all of the others MUST be
+                # present, with the exception of "oth", which MUST
+                # only be present when more than two prime factors
+                # were used."
+                p, q, dp, dq, qi, = all_params = tuple(
+                    jobj.get(x) for x in ('p', 'q', 'dp', 'dq', 'qi'))
+                if tuple(param for param in all_params if param is None):
+                    raise errors.Error(
+                        'Some private parameters are missing: {0}'.format(
+                            all_params))
+                p, q, dp, dq, qi = tuple(
+                    cls._decode_param(x) for x in all_params)
+
+                # TODO: check for oth
+            else:
+                # cryptography>=0.8
+                p, q = rsa.rsa_recover_prime_factors(n, e, d)
+                dp = rsa.rsa_crt_dmp1(d, p)
+                dq = rsa.rsa_crt_dmq1(d, q)
+                qi = rsa.rsa_crt_iqmp(p, q)
+
+            key = rsa.RSAPrivateNumbers(
+                p, q, d, dp, dq, qi, public_numbers).private_key(
+                    default_backend())
+
+        return cls(key=key)
+
+    def fields_to_partial_json(self):
+        # pylint: disable=protected-access
+        if isinstance(self.key._wrapped, rsa.RSAPublicKey):
+            numbers = self.key.public_numbers()
+            params = {
+                'n': numbers.n,
+                'e': numbers.e,
+            }
+        else:  # rsa.RSAPrivateKey
+            private = self.key.private_numbers()
+            public = self.key.public_key().public_numbers()
+            params = {
+                'n': public.n,
+                'e': public.e,
+                'd': private.d,
+                'p': private.p,
+                'q': private.q,
+                'dp': private.dmp1,
+                'dq': private.dmq1,
+                'qi': private.iqmp,
+            }
+        return dict((key, self._encode_param(value))
+                    for key, value in six.iteritems(params))

acme/acme/jose/jwk_test.py

@@ -0,0 +1,191 @@
+"""Tests for acme.jose.jwk."""
+import binascii
+import unittest
+
+from acme import test_util
+
+from acme.jose import errors
+from acme.jose import json_util
+from acme.jose import util
+
+
+DSA_PEM = test_util.load_vector('dsa512_key.pem')
+RSA256_KEY = test_util.load_rsa_private_key('rsa256_key.pem')
+RSA512_KEY = test_util.load_rsa_private_key('rsa512_key.pem')
+
+
+class JWKTest(unittest.TestCase):
+    """Tests for acme.jose.jwk.JWK."""
+
+    def test_load(self):
+        from acme.jose.jwk import JWK
+        self.assertRaises(errors.Error, JWK.load, DSA_PEM)
+
+    def test_load_subclass_wrong_type(self):
+        from acme.jose.jwk import JWKRSA
+        self.assertRaises(errors.Error, JWKRSA.load, DSA_PEM)
+
+
+class JWKTestBaseMixin(object):
+    """Mixin test for JWK subclass tests."""
+
+    thumbprint = NotImplemented
+
+    def test_thumbprint_private(self):
+        self.assertEqual(self.thumbprint, self.jwk.thumbprint())
+
+    def test_thumbprint_public(self):
+        self.assertEqual(self.thumbprint, self.jwk.public_key().thumbprint())
+
+
+class JWKOctTest(unittest.TestCase, JWKTestBaseMixin):
+    """Tests for acme.jose.jwk.JWKOct."""
+
+    thumbprint = (b"\xf3\xe7\xbe\xa8`\xd2\xdap\xe9}\x9c\xce>"
+                  b"\xd0\xfcI\xbe\xcd\x92'\xd4o\x0e\xf41\xea"
+                  b"\x8e(\x8a\xb2i\x1c")
+
+    def setUp(self):
+        from acme.jose.jwk import JWKOct
+        self.jwk = JWKOct(key=b'foo')
+        self.jobj = {'kty': 'oct', 'k': json_util.encode_b64jose(b'foo')}
+
+    def test_to_partial_json(self):
+        self.assertEqual(self.jwk.to_partial_json(), self.jobj)
+
+    def test_from_json(self):
+        from acme.jose.jwk import JWKOct
+        self.assertEqual(self.jwk, JWKOct.from_json(self.jobj))
+
+    def test_from_json_hashable(self):
+        from acme.jose.jwk import JWKOct
+        hash(JWKOct.from_json(self.jobj))
+
+    def test_load(self):
+        from acme.jose.jwk import JWKOct
+        self.assertEqual(self.jwk, JWKOct.load(b'foo'))
+
+    def test_public_key(self):
+        self.assertTrue(self.jwk.public_key() is self.jwk)
+
+
+class JWKRSATest(unittest.TestCase, JWKTestBaseMixin):
+    """Tests for acme.jose.jwk.JWKRSA."""
+    # pylint: disable=too-many-instance-attributes
+
+    thumbprint = (b'\x83K\xdc#3\x98\xca\x98\xed\xcb\x80\x80<\x0c'
+                  b'\xf0\x95\xb9H\xb2*l\xbd$\xe5&|O\x91\xd4 \xb0Y')
+
+    def setUp(self):
+        from acme.jose.jwk import JWKRSA
+        self.jwk256 = JWKRSA(key=RSA256_KEY.public_key())
+        self.jwk256json = {
+            'kty': 'RSA',
+            'e': 'AQAB',
+            'n': 'm2Fylv-Uz7trgTW8EBHP3FQSMeZs2GNQ6VRo1sIVJEk',
+        }
+        # pylint: disable=protected-access
+        self.jwk256_not_comparable = JWKRSA(
+            key=RSA256_KEY.public_key()._wrapped)
+        self.jwk512 = JWKRSA(key=RSA512_KEY.public_key())
+        self.jwk512json = {
+            'kty': 'RSA',
+            'e': 'AQAB',
+            'n': 'rHVztFHtH92ucFJD_N_HW9AsdRsUuHUBBBDlHwNlRd3fp5'
+                 '80rv2-6QWE30cWgdmJS86ObRz6lUTor4R0T-3C5Q',
+        }
+        self.private = JWKRSA(key=RSA256_KEY)
+        self.private_json_small = self.jwk256json.copy()
+        self.private_json_small['d'] = (
+            'lPQED_EPTV0UIBfNI3KP2d9Jlrc2mrMllmf946bu-CE')
+        self.private_json = self.jwk256json.copy()
+        self.private_json.update({
+            'd': 'lPQED_EPTV0UIBfNI3KP2d9Jlrc2mrMllmf946bu-CE',
+            'p': 'zUVNZn4lLLBD1R6NE8TKNQ',
+            'q': 'wcfKfc7kl5jfqXArCRSURQ',
+            'dp': 'CWJFq43QvT5Bm5iN8n1okQ',
+            'dq': 'bHh2u7etM8LKKCF2pY2UdQ',
+            'qi': 'oi45cEkbVoJjAbnQpFY87Q',
+        })
+        self.jwk = self.private
+
+    def test_init_auto_comparable(self):
+        self.assertTrue(isinstance(
+            self.jwk256_not_comparable.key, util.ComparableRSAKey))
+        self.assertEqual(self.jwk256, self.jwk256_not_comparable)
+
+    def test_encode_param_zero(self):
+        from acme.jose.jwk import JWKRSA
+        # pylint: disable=protected-access
+        # TODO: move encode/decode _param to separate class
+        self.assertEqual('AA', JWKRSA._encode_param(0))
+
+    def test_equals(self):
+        self.assertEqual(self.jwk256, self.jwk256)
+        self.assertEqual(self.jwk512, self.jwk512)
+
+    def test_not_equals(self):
+        self.assertNotEqual(self.jwk256, self.jwk512)
+        self.assertNotEqual(self.jwk512, self.jwk256)
+
+    def test_load(self):
+        from acme.jose.jwk import JWKRSA
+        self.assertEqual(self.private, JWKRSA.load(
+            test_util.load_vector('rsa256_key.pem')))
+
+    def test_public_key(self):
+        self.assertEqual(self.jwk256, self.private.public_key())
+
+    def test_to_partial_json(self):
+        self.assertEqual(self.jwk256.to_partial_json(), self.jwk256json)
+        self.assertEqual(self.jwk512.to_partial_json(), self.jwk512json)
+        self.assertEqual(self.private.to_partial_json(), self.private_json)
+
+    def test_from_json(self):
+        from acme.jose.jwk import JWK
+        self.assertEqual(
+            self.jwk256, JWK.from_json(self.jwk256json))
+        self.assertEqual(
+            self.jwk512, JWK.from_json(self.jwk512json))
+        self.assertEqual(self.private, JWK.from_json(self.private_json))
+
+    def test_from_json_private_small(self):
+        from acme.jose.jwk import JWK
+        self.assertEqual(self.private, JWK.from_json(self.private_json_small))
+
+    def test_from_json_missing_one_additional(self):
+        from acme.jose.jwk import JWK
+        del self.private_json['q']
+        self.assertRaises(errors.Error, JWK.from_json, self.private_json)
+
+    def test_from_json_hashable(self):
+        from acme.jose.jwk import JWK
+        hash(JWK.from_json(self.jwk256json))
+
+    def test_from_json_non_schema_errors(self):
+        # valid against schema, but still failing
+        from acme.jose.jwk import JWK
+        self.assertRaises(errors.DeserializationError, JWK.from_json,
+                          {'kty': 'RSA', 'e': 'AQAB', 'n': ''})
+        self.assertRaises(errors.DeserializationError, JWK.from_json,
+                          {'kty': 'RSA', 'e': 'AQAB', 'n': '1'})
+
+    def test_thumbprint_go_jose(self):
+        # https://github.com/square/go-jose/blob/4ddd71883fa547d37fbf598071f04512d8bafee3/jwk.go#L155
+        # https://github.com/square/go-jose/blob/4ddd71883fa547d37fbf598071f04512d8bafee3/jwk_test.go#L331-L344
+        # https://github.com/square/go-jose/blob/4ddd71883fa547d37fbf598071f04512d8bafee3/jwk_test.go#L384
+        from acme.jose.jwk import JWKRSA
+        key = JWKRSA.json_loads("""{
+    "kty": "RSA",
+    "kid": "bilbo.baggins@hobbiton.example",
+    "use": "sig",
+    "n": "n4EPtAOCc9AlkeQHPzHStgAbgs7bTZLwUBZdR8_KuKPEHLd4rHVTeT-O-XV2jRojdNhxJWTDvNd7nqQ0VEiZQHz_AJmSCpMaJMRBSFKrKb2wqVwGU_NsYOYL-QtiWN2lbzcEe6XC0dApr5ydQLrHqkHHig3RBordaZ6Aj-oBHqFEHYpPe7Tpe-OfVfHd1E6cS6M1FZcD1NNLYD5lFHpPI9bTwJlsde3uhGqC0ZCuEHg8lhzwOHrtIQbS0FVbb9k3-tVTU4fg_3L_vniUFAKwuCLqKnS2BYwdq_mzSnbLY7h_qixoR7jig3__kRhuaxwUkRz5iaiQkqgc5gHdrNP5zw",
+    "e": "AQAB"
+}""")
+        self.assertEqual(
+            binascii.hexlify(key.thumbprint()),
+            b"f63838e96077ad1fc01c3f8405774dedc0641f558ebb4b40dccf5f9b6d66a932")
+
+
+if __name__ == '__main__':
+    unittest.main()  # pragma: no cover

acme/acme/jose/jws.py

@@ -0,0 +1,433 @@
+"""JOSE Web Signature."""
+import argparse
+import base64
+import sys
+
+import OpenSSL
+import six
+
+from acme.jose import b64
+from acme.jose import errors
+from acme.jose import json_util
+from acme.jose import jwa
+from acme.jose import jwk
+from acme.jose import util
+
+
+class MediaType(object):
+    """MediaType field encoder/decoder."""
+
+    PREFIX = 'application/'
+    """MIME Media Type and Content Type prefix."""
+
+    @classmethod
+    def decode(cls, value):
+        """Decoder."""
+        # 4.1.10
+        if '/' not in value:
+            if ';' in value:
+                raise errors.DeserializationError('Unexpected semi-colon')
+            return cls.PREFIX + value
+        return value
+
+    @classmethod
+    def encode(cls, value):
+        """Encoder."""
+        # 4.1.10
+        if ';' not in value:
+            assert value.startswith(cls.PREFIX)
+            return value[len(cls.PREFIX):]
+        return value
+
+
+class Header(json_util.JSONObjectWithFields):
+    """JOSE Header.
+
+    .. warning:: This class supports **only** Registered Header
+        Parameter Names (as defined in section 4.1 of the
+        protocol). If you need Public Header Parameter Names (4.2)
+        or Private Header Parameter Names (4.3), you must subclass
+        and override :meth:`from_json` and :meth:`to_partial_json`
+        appropriately.
+
+    .. warning:: This class does not support any extensions through
+        the "crit" (Critical) Header Parameter (4.1.11) and as a
+        conforming implementation, :meth:`from_json` treats its
+        occurrence as an error. Please subclass if you seek for
+        a different behaviour.
+
+    :ivar x5tS256: "x5t#S256"
+    :ivar str typ: MIME Media Type, inc. :const:`MediaType.PREFIX`.
+    :ivar str cty: Content-Type, inc. :const:`MediaType.PREFIX`.
+
+    """
+    alg = json_util.Field(
+        'alg', decoder=jwa.JWASignature.from_json, omitempty=True)
+    jku = json_util.Field('jku', omitempty=True)
+    jwk = json_util.Field('jwk', decoder=jwk.JWK.from_json, omitempty=True)
+    kid = json_util.Field('kid', omitempty=True)
+    x5u = json_util.Field('x5u', omitempty=True)
+    x5c = json_util.Field('x5c', omitempty=True, default=())
+    x5t = json_util.Field(
+        'x5t', decoder=json_util.decode_b64jose, omitempty=True)
+    x5tS256 = json_util.Field(
+        'x5t#S256', decoder=json_util.decode_b64jose, omitempty=True)
+    typ = json_util.Field('typ', encoder=MediaType.encode,
+                          decoder=MediaType.decode, omitempty=True)
+    cty = json_util.Field('cty', encoder=MediaType.encode,
+                          decoder=MediaType.decode, omitempty=True)
+    crit = json_util.Field('crit', omitempty=True, default=())
+
+    def not_omitted(self):
+        """Fields that would not be omitted in the JSON object."""
+        return dict((name, getattr(self, name))
+                    for name, field in six.iteritems(self._fields)
+                    if not field.omit(getattr(self, name)))
+
+    def __add__(self, other):
+        if not isinstance(other, type(self)):
+            raise TypeError('Header cannot be added to: {0}'.format(
+                type(other)))
+
+        not_omitted_self = self.not_omitted()
+        not_omitted_other = other.not_omitted()
+
+        if set(not_omitted_self).intersection(not_omitted_other):
+            raise TypeError('Addition of overlapping headers not defined')
+
+        not_omitted_self.update(not_omitted_other)
+        return type(self)(**not_omitted_self)  # pylint: disable=star-args
+
+    def find_key(self):
+        """Find key based on header.
+
+        .. todo:: Supports only "jwk" header parameter lookup.
+
+        :returns: (Public) key found in the header.
+        :rtype: .JWK
+
+        :raises acme.jose.errors.Error: if key could not be found
+
+        """
+        if self.jwk is None:
+            raise errors.Error('No key found')
+        return self.jwk
+
+    @crit.decoder
+    def crit(unused_value):
+        # pylint: disable=missing-docstring,no-self-argument,no-self-use
+        raise errors.DeserializationError(
+            '"crit" is not supported, please subclass')
+
+    # x5c does NOT use JOSE Base64 (4.1.6)
+
+    @x5c.encoder  # type: ignore
+    def x5c(value):  # pylint: disable=missing-docstring,no-self-argument
+        return [base64.b64encode(OpenSSL.crypto.dump_certificate(
+            OpenSSL.crypto.FILETYPE_ASN1, cert.wrapped)) for cert in value]
+
+    @x5c.decoder  # type: ignore
+    def x5c(value):  # pylint: disable=missing-docstring,no-self-argument
+        try:
+            return tuple(util.ComparableX509(OpenSSL.crypto.load_certificate(
+                OpenSSL.crypto.FILETYPE_ASN1,
+                base64.b64decode(cert))) for cert in value)
+        except OpenSSL.crypto.Error as error:
+            raise errors.DeserializationError(error)
+
+
+class Signature(json_util.JSONObjectWithFields):
+    """JWS Signature.
+
+    :ivar combined: Combined Header (protected and unprotected,
+        :class:`Header`).
+    :ivar unicode protected: JWS protected header (Jose Base-64 decoded).
+    :ivar header: JWS Unprotected Header (:class:`Header`).
+    :ivar str signature: The signature.
+
+    """
+    header_cls = Header
+
+    __slots__ = ('combined',)
+    protected = json_util.Field('protected', omitempty=True, default='')
+    header = json_util.Field(
+        'header', omitempty=True, default=header_cls(),
+        decoder=header_cls.from_json)
+    signature = json_util.Field(
+        'signature', decoder=json_util.decode_b64jose,
+        encoder=json_util.encode_b64jose)
+
+    @protected.encoder  # type: ignore
+    def protected(value):  # pylint: disable=missing-docstring,no-self-argument
+        # wrong type guess (Signature, not bytes) | pylint: disable=no-member
+        return json_util.encode_b64jose(value.encode('utf-8'))
+
+    @protected.decoder  # type: ignore
+    def protected(value):  # pylint: disable=missing-docstring,no-self-argument
+        return json_util.decode_b64jose(value).decode('utf-8')
+
+    def __init__(self, **kwargs):
+        if 'combined' not in kwargs:
+            kwargs = self._with_combined(kwargs)
+        super(Signature, self).__init__(**kwargs)
+        assert self.combined.alg is not None
+
+    @classmethod
+    def _with_combined(cls, kwargs):
+        assert 'combined' not in kwargs
+        header = kwargs.get('header', cls._fields['header'].default)
+        protected = kwargs.get('protected', cls._fields['protected'].default)
+
+        if protected:
+            combined = header + cls.header_cls.json_loads(protected)
+        else:
+            combined = header
+
+        kwargs['combined'] = combined
+        return kwargs
+
+    @classmethod
+    def _msg(cls, protected, payload):
+        return (b64.b64encode(protected.encode('utf-8')) + b'.' +
+                b64.b64encode(payload))
+
+    def verify(self, payload, key=None):
+        """Verify.
+
+        :param JWK key: Key used for verification.
+
+        """
+        key = self.combined.find_key() if key is None else key
+        return self.combined.alg.verify(
+            key=key.key, sig=self.signature,
+            msg=self._msg(self.protected, payload))
+
+    @classmethod
+    def sign(cls, payload, key, alg, include_jwk=True,
+             protect=frozenset(), **kwargs):
+        """Sign.
+
+        :param JWK key: Key for signature.
+
+        """
+        assert isinstance(key, alg.kty)
+
+        header_params = kwargs
+        header_params['alg'] = alg
+        if include_jwk:
+            header_params['jwk'] = key.public_key()
+
+        assert set(header_params).issubset(cls.header_cls._fields)
+        assert protect.issubset(cls.header_cls._fields)
+
+        protected_params = {}
+        for header in protect:
+            if header in header_params:
+                protected_params[header] = header_params.pop(header)
+        if protected_params:
+            # pylint: disable=star-args
+            protected = cls.header_cls(**protected_params).json_dumps()
+        else:
+            protected = ''
+
+        header = cls.header_cls(**header_params)  # pylint: disable=star-args
+        signature = alg.sign(key.key, cls._msg(protected, payload))
+
+        return cls(protected=protected, header=header, signature=signature)
+
+    def fields_to_partial_json(self):
+        fields = super(Signature, self).fields_to_partial_json()
+        if not fields['header'].not_omitted():
+            del fields['header']
+        return fields
+
+    @classmethod
+    def fields_from_json(cls, jobj):
+        fields = super(Signature, cls).fields_from_json(jobj)
+        fields_with_combined = cls._with_combined(fields)
+        if 'alg' not in fields_with_combined['combined'].not_omitted():
+            raise errors.DeserializationError('alg not present')
+        return fields_with_combined
+
+
+class JWS(json_util.JSONObjectWithFields):
+    """JSON Web Signature.
+
+    :ivar str payload: JWS Payload.
+    :ivar str signature: JWS Signatures.
+
+    """
+    __slots__ = ('payload', 'signatures')
+
+    signature_cls = Signature
+
+    def verify(self, key=None):
+        """Verify."""
+        return all(sig.verify(self.payload, key) for sig in self.signatures)
+
+    @classmethod
+    def sign(cls, payload, **kwargs):
+        """Sign."""
+        return cls(payload=payload, signatures=(
+            cls.signature_cls.sign(payload=payload, **kwargs),))
+
+    @property
+    def signature(self):
+        """Get a singleton signature.
+
+        :rtype: `signature_cls`
+
+        """
+        assert len(self.signatures) == 1
+        return self.signatures[0]
+
+    def to_compact(self):
+        """Compact serialization.
+
+        :rtype: bytes
+
+        """
+        assert len(self.signatures) == 1
+
+        assert 'alg' not in self.signature.header.not_omitted()
+        # ... it must be in protected
+
+        return (
+            b64.b64encode(self.signature.protected.encode('utf-8')) +
+            b'.' +
+            b64.b64encode(self.payload) +
+            b'.' +
+            b64.b64encode(self.signature.signature))
+
+    @classmethod
+    def from_compact(cls, compact):
+        """Compact deserialization.
+
+        :param bytes compact:
+
+        """
+        try:
+            protected, payload, signature = compact.split(b'.')
+        except ValueError:
+            raise errors.DeserializationError(
+                'Compact JWS serialization should comprise of exactly'
+                ' 3 dot-separated components')
+
+        sig = cls.signature_cls(
+            protected=b64.b64decode(protected).decode('utf-8'),
+            signature=b64.b64decode(signature))
+        return cls(payload=b64.b64decode(payload), signatures=(sig,))
+
+    def to_partial_json(self, flat=True):  # pylint: disable=arguments-differ
+        assert self.signatures
+        payload = json_util.encode_b64jose(self.payload)
+
+        if flat and len(self.signatures) == 1:
+            ret = self.signatures[0].to_partial_json()
+            ret['payload'] = payload
+            return ret
+        else:
+            return {
+                'payload': payload,
+                'signatures': self.signatures,
+            }
+
+    @classmethod
+    def from_json(cls, jobj):
+        if 'signature' in jobj and 'signatures' in jobj:
+            raise errors.DeserializationError('Flat mixed with non-flat')
+        elif 'signature' in jobj:  # flat
+            return cls(payload=json_util.decode_b64jose(jobj.pop('payload')),
+                       signatures=(cls.signature_cls.from_json(jobj),))
+        else:
+            return cls(payload=json_util.decode_b64jose(jobj['payload']),
+                       signatures=tuple(cls.signature_cls.from_json(sig)
+                                        for sig in jobj['signatures']))
+
+
+class CLI(object):
+    """JWS CLI."""
+
+    @classmethod
+    def sign(cls, args):
+        """Sign."""
+        key = args.alg.kty.load(args.key.read())
+        args.key.close()
+        if args.protect is None:
+            args.protect = []
+        if args.compact:
+            args.protect.append('alg')
+
+        sig = JWS.sign(payload=sys.stdin.read().encode(), key=key, alg=args.alg,
+                       protect=set(args.protect))
+
+        if args.compact:
+            six.print_(sig.to_compact().decode('utf-8'))
+        else:  # JSON
+            six.print_(sig.json_dumps_pretty())
+
+    @classmethod
+    def verify(cls, args):
+        """Verify."""
+        if args.compact:
+            sig = JWS.from_compact(sys.stdin.read().encode())
+        else:  # JSON
+            try:
+                sig = JWS.json_loads(sys.stdin.read())
+            except errors.Error as error:
+                six.print_(error)
+                return -1
+
+        if args.key is not None:
+            assert args.kty is not None
+            key = args.kty.load(args.key.read()).public_key()
+            args.key.close()
+        else:
+            key = None
+
+        sys.stdout.write(sig.payload)
+        return not sig.verify(key=key)
+
+    @classmethod
+    def _alg_type(cls, arg):
+        return jwa.JWASignature.from_json(arg)
+
+    @classmethod
+    def _header_type(cls, arg):
+        assert arg in Signature.header_cls._fields
+        return arg
+
+    @classmethod
+    def _kty_type(cls, arg):
+        assert arg in jwk.JWK.TYPES
+        return jwk.JWK.TYPES[arg]
+
+    @classmethod
+    def run(cls, args=sys.argv[1:]):
+        """Parse arguments and sign/verify."""
+        parser = argparse.ArgumentParser()
+        parser.add_argument('--compact', action='store_true')
+
+        subparsers = parser.add_subparsers()
+        parser_sign = subparsers.add_parser('sign')
+        parser_sign.set_defaults(func=cls.sign)
+        parser_sign.add_argument(
+            '-k', '--key', type=argparse.FileType('rb'), required=True)
+        parser_sign.add_argument(
+            '-a', '--alg', type=cls._alg_type, default=jwa.RS256)
+        parser_sign.add_argument(
+            '-p', '--protect', action='append', type=cls._header_type)
+
+        parser_verify = subparsers.add_parser('verify')
+        parser_verify.set_defaults(func=cls.verify)
+        parser_verify.add_argument(
+            '-k', '--key', type=argparse.FileType('rb'), required=False)
+        parser_verify.add_argument(
+            '--kty', type=cls._kty_type, required=False)
+
+        parsed = parser.parse_args(args)
+        return parsed.func(parsed)
+
+
+if __name__ == '__main__':
+    exit(CLI.run())  # pragma: no cover

acme/acme/jose/jws_test.py

@@ -0,0 +1,239 @@
+"""Tests for acme.jose.jws."""
+import base64
+import unittest
+
+import mock
+import OpenSSL
+
+from acme import test_util
+
+from acme.jose import errors
+from acme.jose import json_util
+from acme.jose import jwa
+from acme.jose import jwk
+
+
+CERT = test_util.load_comparable_cert('cert.pem')
+KEY = jwk.JWKRSA.load(test_util.load_vector('rsa512_key.pem'))
+
+
+class MediaTypeTest(unittest.TestCase):
+    """Tests for acme.jose.jws.MediaType."""
+
+    def test_decode(self):
+        from acme.jose.jws import MediaType
+        self.assertEqual('application/app', MediaType.decode('application/app'))
+        self.assertEqual('application/app', MediaType.decode('app'))
+        self.assertRaises(
+            errors.DeserializationError, MediaType.decode, 'app;foo')
+
+    def test_encode(self):
+        from acme.jose.jws import MediaType
+        self.assertEqual('app', MediaType.encode('application/app'))
+        self.assertEqual('application/app;foo',
+                         MediaType.encode('application/app;foo'))
+
+
+class HeaderTest(unittest.TestCase):
+    """Tests for acme.jose.jws.Header."""
+
+    def setUp(self):
+        from acme.jose.jws import Header
+        self.header1 = Header(jwk='foo')
+        self.header2 = Header(jwk='bar')
+        self.crit = Header(crit=('a', 'b'))
+        self.empty = Header()
+
+    def test_add_non_empty(self):
+        from acme.jose.jws import Header
+        self.assertEqual(Header(jwk='foo', crit=('a', 'b')),
+                         self.header1 + self.crit)
+
+    def test_add_empty(self):
+        self.assertEqual(self.header1, self.header1 + self.empty)
+        self.assertEqual(self.header1, self.empty + self.header1)
+
+    def test_add_overlapping_error(self):
+        self.assertRaises(TypeError, self.header1.__add__, self.header2)
+
+    def test_add_wrong_type_error(self):
+        self.assertRaises(TypeError, self.header1.__add__, 'xxx')
+
+    def test_crit_decode_always_errors(self):
+        from acme.jose.jws import Header
+        self.assertRaises(errors.DeserializationError, Header.from_json,
+                          {'crit': ['a', 'b']})
+
+    def test_x5c_decoding(self):
+        from acme.jose.jws import Header
+        header = Header(x5c=(CERT, CERT))
+        jobj = header.to_partial_json()
+        cert_asn1 = OpenSSL.crypto.dump_certificate(
+            OpenSSL.crypto.FILETYPE_ASN1, CERT.wrapped)
+        cert_b64 = base64.b64encode(cert_asn1)
+        self.assertEqual(jobj, {'x5c': [cert_b64, cert_b64]})
+        self.assertEqual(header, Header.from_json(jobj))
+        jobj['x5c'][0] = base64.b64encode(b'xxx' + cert_asn1)
+        self.assertRaises(errors.DeserializationError, Header.from_json, jobj)
+
+    def test_find_key(self):
+        self.assertEqual('foo', self.header1.find_key())
+        self.assertEqual('bar', self.header2.find_key())
+        self.assertRaises(errors.Error, self.crit.find_key)
+
+
+class SignatureTest(unittest.TestCase):
+    """Tests for acme.jose.jws.Signature."""
+
+    def test_from_json(self):
+        from acme.jose.jws import Header
+        from acme.jose.jws import Signature
+        self.assertEqual(
+            Signature(signature=b'foo', header=Header(alg=jwa.RS256)),
+            Signature.from_json(
+                {'signature': 'Zm9v', 'header': {'alg': 'RS256'}}))
+
+    def test_from_json_no_alg_error(self):
+        from acme.jose.jws import Signature
+        self.assertRaises(errors.DeserializationError,
+                          Signature.from_json, {'signature': 'foo'})
+
+
+class JWSTest(unittest.TestCase):
+    """Tests for acme.jose.jws.JWS."""
+
+    def setUp(self):
+        self.privkey = KEY
+        self.pubkey = self.privkey.public_key()
+
+        from acme.jose.jws import JWS
+        self.unprotected = JWS.sign(
+            payload=b'foo', key=self.privkey, alg=jwa.RS256)
+        self.protected = JWS.sign(
+            payload=b'foo', key=self.privkey, alg=jwa.RS256,
+            protect=frozenset(['jwk', 'alg']))
+        self.mixed = JWS.sign(
+            payload=b'foo', key=self.privkey, alg=jwa.RS256,
+            protect=frozenset(['alg']))
+
+    def test_pubkey_jwk(self):
+        self.assertEqual(self.unprotected.signature.combined.jwk, self.pubkey)
+        self.assertEqual(self.protected.signature.combined.jwk, self.pubkey)
+        self.assertEqual(self.mixed.signature.combined.jwk, self.pubkey)
+
+    def test_sign_unprotected(self):
+        self.assertTrue(self.unprotected.verify())
+
+    def test_sign_protected(self):
+        self.assertTrue(self.protected.verify())
+
+    def test_sign_mixed(self):
+        self.assertTrue(self.mixed.verify())
+
+    def test_compact_lost_unprotected(self):
+        compact = self.mixed.to_compact()
+        self.assertEqual(
+            b'eyJhbGciOiAiUlMyNTYifQ.Zm9v.OHdxFVj73l5LpxbFp1AmYX4yJM0Pyb'
+            b'_893n1zQjpim_eLS5J1F61lkvrCrCDErTEJnBGOGesJ72M7b6Ve1cAJA',
+            compact)
+
+        from acme.jose.jws import JWS
+        mixed = JWS.from_compact(compact)
+
+        self.assertNotEqual(self.mixed, mixed)
+        self.assertEqual(
+            set(['alg']), set(mixed.signature.combined.not_omitted()))
+
+    def test_from_compact_missing_components(self):
+        from acme.jose.jws import JWS
+        self.assertRaises(errors.DeserializationError, JWS.from_compact, b'.')
+
+    def test_json_omitempty(self):
+        protected_jobj = self.protected.to_partial_json(flat=True)
+        unprotected_jobj = self.unprotected.to_partial_json(flat=True)
+
+        self.assertTrue('protected' not in unprotected_jobj)
+        self.assertTrue('header' not in protected_jobj)
+
+        unprotected_jobj['header'] = unprotected_jobj['header'].to_json()
+
+        from acme.jose.jws import JWS
+        self.assertEqual(JWS.from_json(protected_jobj), self.protected)
+        self.assertEqual(JWS.from_json(unprotected_jobj), self.unprotected)
+
+    def test_json_flat(self):
+        jobj_to = {
+            'signature': json_util.encode_b64jose(
+                self.mixed.signature.signature),
+            'payload': json_util.encode_b64jose(b'foo'),
+            'header': self.mixed.signature.header,
+            'protected': json_util.encode_b64jose(
+                self.mixed.signature.protected.encode('utf-8')),
+        }
+        jobj_from = jobj_to.copy()
+        jobj_from['header'] = jobj_from['header'].to_json()
+
+        self.assertEqual(self.mixed.to_partial_json(flat=True), jobj_to)
+        from acme.jose.jws import JWS
+        self.assertEqual(self.mixed, JWS.from_json(jobj_from))
+
+    def test_json_not_flat(self):
+        jobj_to = {
+            'signatures': (self.mixed.signature,),
+            'payload': json_util.encode_b64jose(b'foo'),
+        }
+        jobj_from = jobj_to.copy()
+        jobj_from['signatures'] = [jobj_to['signatures'][0].to_json()]
+
+        self.assertEqual(self.mixed.to_partial_json(flat=False), jobj_to)
+        from acme.jose.jws import JWS
+        self.assertEqual(self.mixed, JWS.from_json(jobj_from))
+
+    def test_from_json_mixed_flat(self):
+        from acme.jose.jws import JWS
+        self.assertRaises(errors.DeserializationError, JWS.from_json,
+                          {'signatures': (), 'signature': 'foo'})
+
+    def test_from_json_hashable(self):
+        from acme.jose.jws import JWS
+        hash(JWS.from_json(self.mixed.to_json()))
+
+
+class CLITest(unittest.TestCase):
+
+    def setUp(self):
+        self.key_path = test_util.vector_path('rsa512_key.pem')
+
+    def test_unverified(self):
+        from acme.jose.jws import CLI
+        with mock.patch('sys.stdin') as sin:
+            sin.read.return_value = '{"payload": "foo", "signature": "xxx"}'
+            with mock.patch('sys.stdout'):
+                self.assertEqual(-1, CLI.run(['verify']))
+
+    def test_json(self):
+        from acme.jose.jws import CLI
+
+        with mock.patch('sys.stdin') as sin:
+            sin.read.return_value = 'foo'
+            with mock.patch('sys.stdout') as sout:
+                CLI.run(['sign', '-k', self.key_path, '-a', 'RS256',
+                         '-p', 'jwk'])
+                sin.read.return_value = sout.write.mock_calls[0][1][0]
+                self.assertEqual(0, CLI.run(['verify']))
+
+    def test_compact(self):
+        from acme.jose.jws import CLI
+
+        with mock.patch('sys.stdin') as sin:
+            sin.read.return_value = 'foo'
+            with mock.patch('sys.stdout') as sout:
+                CLI.run(['--compact', 'sign', '-k', self.key_path])
+                sin.read.return_value = sout.write.mock_calls[0][1][0]
+                self.assertEqual(0, CLI.run([
+                    '--compact', 'verify', '--kty', 'RSA',
+                    '-k', self.key_path]))
+
+
+if __name__ == '__main__':
+    unittest.main()  # pragma: no cover

acme/acme/jose/util.py

@@ -0,0 +1,226 @@
+"""JOSE utilities."""
+import collections
+
+from cryptography.hazmat.primitives.asymmetric import rsa
+import OpenSSL
+import six
+
+
+class abstractclassmethod(classmethod):
+    # pylint: disable=invalid-name,too-few-public-methods
+    """Descriptor for an abstract classmethod.
+
+    It augments the :mod:`abc` framework with an abstract
+    classmethod. This is implemented as :class:`abc.abstractclassmethod`
+    in the standard Python library starting with version 3.2.
+
+    This particular implementation, allegedly based on Python 3.3 source
+    code, is stolen from
+    http://stackoverflow.com/questions/11217878/python-2-7-combine-abc-abstractmethod-and-classmethod.
+
+    """
+    __isabstractmethod__ = True
+
+    def __init__(self, target):
+        target.__isabstractmethod__ = True
+        super(abstractclassmethod, self).__init__(target)
+
+
+class ComparableX509(object):  # pylint: disable=too-few-public-methods
+    """Wrapper for OpenSSL.crypto.X509** objects that supports __eq__.
+
+    :ivar wrapped: Wrapped certificate or certificate request.
+    :type wrapped: `OpenSSL.crypto.X509` or `OpenSSL.crypto.X509Req`.
+
+    """
+    def __init__(self, wrapped):
+        assert isinstance(wrapped, OpenSSL.crypto.X509) or isinstance(
+            wrapped, OpenSSL.crypto.X509Req)
+        self.wrapped = wrapped
+
+    def __getattr__(self, name):
+        return getattr(self.wrapped, name)
+
+    def _dump(self, filetype=OpenSSL.crypto.FILETYPE_ASN1):
+        """Dumps the object into a buffer with the specified encoding.
+
+        :param int filetype: The desired encoding. Should be one of
+            `OpenSSL.crypto.FILETYPE_ASN1`,
+            `OpenSSL.crypto.FILETYPE_PEM`, or
+            `OpenSSL.crypto.FILETYPE_TEXT`.
+
+        :returns: Encoded X509 object.
+        :rtype: str
+
+        """
+        if isinstance(self.wrapped, OpenSSL.crypto.X509):
+            func = OpenSSL.crypto.dump_certificate
+        else:  # assert in __init__ makes sure this is X509Req
+            func = OpenSSL.crypto.dump_certificate_request
+        return func(filetype, self.wrapped)
+
+    def __eq__(self, other):
+        if not isinstance(other, self.__class__):
+            return NotImplemented
+        # pylint: disable=protected-access
+        return self._dump() == other._dump()
+
+    def __hash__(self):
+        return hash((self.__class__, self._dump()))
+
+    def __ne__(self, other):
+        return not self == other
+
+    def __repr__(self):
+        return '<{0}({1!r})>'.format(self.__class__.__name__, self.wrapped)
+
+
+class ComparableKey(object):  # pylint: disable=too-few-public-methods
+    """Comparable wrapper for `cryptography` keys.
+
+    See https://github.com/pyca/cryptography/issues/2122.
+
+    """
+    __hash__ = NotImplemented
+
+    def __init__(self, wrapped):
+        self._wrapped = wrapped
+
+    def __getattr__(self, name):
+        return getattr(self._wrapped, name)
+
+    def __eq__(self, other):
+        # pylint: disable=protected-access
+        if (not isinstance(other, self.__class__) or
+                self._wrapped.__class__ is not other._wrapped.__class__):
+            return NotImplemented
+        elif hasattr(self._wrapped, 'private_numbers'):
+            return self.private_numbers() == other.private_numbers()
+        elif hasattr(self._wrapped, 'public_numbers'):
+            return self.public_numbers() == other.public_numbers()
+        else:
+            return NotImplemented
+
+    def __ne__(self, other):
+        return not self == other
+
+    def __repr__(self):
+        return '<{0}({1!r})>'.format(self.__class__.__name__, self._wrapped)
+
+    def public_key(self):
+        """Get wrapped public key."""
+        return self.__class__(self._wrapped.public_key())
+
+
+class ComparableRSAKey(ComparableKey):  # pylint: disable=too-few-public-methods
+    """Wrapper for `cryptography` RSA keys.
+
+    Wraps around:
+    - `cryptography.hazmat.primitives.asymmetric.RSAPrivateKey`
+    - `cryptography.hazmat.primitives.asymmetric.RSAPublicKey`
+
+    """
+
+    def __hash__(self):
+        # public_numbers() hasn't got stable hash!
+        # https://github.com/pyca/cryptography/issues/2143
+        if isinstance(self._wrapped, rsa.RSAPrivateKeyWithSerialization):
+            priv = self.private_numbers()
+            pub = priv.public_numbers
+            return hash((self.__class__, priv.p, priv.q, priv.dmp1,
+                         priv.dmq1, priv.iqmp, pub.n, pub.e))
+        elif isinstance(self._wrapped, rsa.RSAPublicKeyWithSerialization):
+            pub = self.public_numbers()
+            return hash((self.__class__, pub.n, pub.e))
+
+
+class ImmutableMap(collections.Mapping, collections.Hashable):  # type: ignore
+    # pylint: disable=too-few-public-methods
+    """Immutable key to value mapping with attribute access."""
+
+    __slots__ = ()
+    """Must be overridden in subclasses."""
+
+    def __init__(self, **kwargs):
+        if set(kwargs) != set(self.__slots__):
+            raise TypeError(
+                '__init__() takes exactly the following arguments: {0} '
+                '({1} given)'.format(', '.join(self.__slots__),
+                                     ', '.join(kwargs) if kwargs else 'none'))
+        for slot in self.__slots__:
+            object.__setattr__(self, slot, kwargs.pop(slot))
+
+    def update(self, **kwargs):
+        """Return updated map."""
+        items = dict(self)
+        items.update(kwargs)
+        return type(self)(**items)  # pylint: disable=star-args
+
+    def __getitem__(self, key):
+        try:
+            return getattr(self, key)
+        except AttributeError:
+            raise KeyError(key)
+
+    def __iter__(self):
+        return iter(self.__slots__)
+
+    def __len__(self):
+        return len(self.__slots__)
+
+    def __hash__(self):
+        return hash(tuple(getattr(self, slot) for slot in self.__slots__))
+
+    def __setattr__(self, name, value):
+        raise AttributeError("can't set attribute")
+
+    def __repr__(self):
+        return '{0}({1})'.format(self.__class__.__name__, ', '.join(
+            '{0}={1!r}'.format(key, value)
+            for key, value in six.iteritems(self)))
+
+
+class frozendict(collections.Mapping, collections.Hashable):  # type: ignore
+    # pylint: disable=invalid-name,too-few-public-methods
+    """Frozen dictionary."""
+    __slots__ = ('_items', '_keys')
+
+    def __init__(self, *args, **kwargs):
+        if kwargs and not args:
+            items = dict(kwargs)
+        elif len(args) == 1 and isinstance(args[0], collections.Mapping):
+            items = args[0]
+        else:
+            raise TypeError()
+        # TODO: support generators/iterators
+
+        object.__setattr__(self, '_items', items)
+        object.__setattr__(self, '_keys', tuple(sorted(six.iterkeys(items))))
+
+    def __getitem__(self, key):
+        return self._items[key]
+
+    def __iter__(self):
+        return iter(self._keys)
+
+    def __len__(self):
+        return len(self._items)
+
+    def _sorted_items(self):
+        return tuple((key, self[key]) for key in self._keys)
+
+    def __hash__(self):
+        return hash(self._sorted_items())
+
+    def __getattr__(self, name):
+        try:
+            return self._items[name]
+        except KeyError:
+            raise AttributeError(name)
+
+    def __setattr__(self, name, value):
+        raise AttributeError("can't set attribute")
+
+    def __repr__(self):
+        return 'frozendict({0})'.format(', '.join('{0}={1!r}'.format(
+            key, value) for key, value in self._sorted_items()))

acme/acme/jose/util_test.py

@@ -0,0 +1,199 @@
+"""Tests for acme.jose.util."""
+import functools
+import unittest
+
+import six
+
+from acme import test_util
+
+
+class ComparableX509Test(unittest.TestCase):
+    """Tests for acme.jose.util.ComparableX509."""
+
+    def setUp(self):
+        # test_util.load_comparable_{csr,cert} return ComparableX509
+        self.req1 = test_util.load_comparable_csr('csr.pem')
+        self.req2 = test_util.load_comparable_csr('csr.pem')
+        self.req_other = test_util.load_comparable_csr('csr-san.pem')
+
+        self.cert1 = test_util.load_comparable_cert('cert.pem')
+        self.cert2 = test_util.load_comparable_cert('cert.pem')
+        self.cert_other = test_util.load_comparable_cert('cert-san.pem')
+
+    def test_getattr_proxy(self):
+        self.assertTrue(self.cert1.has_expired())
+
+    def test_eq(self):
+        self.assertEqual(self.req1, self.req2)
+        self.assertEqual(self.cert1, self.cert2)
+
+    def test_ne(self):
+        self.assertNotEqual(self.req1, self.req_other)
+        self.assertNotEqual(self.cert1, self.cert_other)
+
+    def test_ne_wrong_types(self):
+        self.assertNotEqual(self.req1, 5)
+        self.assertNotEqual(self.cert1, 5)
+
+    def test_hash(self):
+        self.assertEqual(hash(self.req1), hash(self.req2))
+        self.assertNotEqual(hash(self.req1), hash(self.req_other))
+
+        self.assertEqual(hash(self.cert1), hash(self.cert2))
+        self.assertNotEqual(hash(self.cert1), hash(self.cert_other))
+
+    def test_repr(self):
+        for x509 in self.req1, self.cert1:
+            self.assertEqual(repr(x509),
+                             '<ComparableX509({0!r})>'.format(x509.wrapped))
+
+
+class ComparableRSAKeyTest(unittest.TestCase):
+    """Tests for acme.jose.util.ComparableRSAKey."""
+
+    def setUp(self):
+        # test_utl.load_rsa_private_key return ComparableRSAKey
+        self.key = test_util.load_rsa_private_key('rsa256_key.pem')
+        self.key_same = test_util.load_rsa_private_key('rsa256_key.pem')
+        self.key2 = test_util.load_rsa_private_key('rsa512_key.pem')
+
+    def test_getattr_proxy(self):
+        self.assertEqual(256, self.key.key_size)
+
+    def test_eq(self):
+        self.assertEqual(self.key, self.key_same)
+
+    def test_ne(self):
+        self.assertNotEqual(self.key, self.key2)
+
+    def test_ne_different_types(self):
+        self.assertNotEqual(self.key, 5)
+
+    def test_ne_not_wrapped(self):
+        # pylint: disable=protected-access
+        self.assertNotEqual(self.key, self.key_same._wrapped)
+
+    def test_ne_no_serialization(self):
+        from acme.jose.util import ComparableRSAKey
+        self.assertNotEqual(ComparableRSAKey(5), ComparableRSAKey(5))
+
+    def test_hash(self):
+        self.assertTrue(isinstance(hash(self.key), int))
+        self.assertEqual(hash(self.key), hash(self.key_same))
+        self.assertNotEqual(hash(self.key), hash(self.key2))
+
+    def test_repr(self):
+        self.assertTrue(repr(self.key).startswith(
+            '<ComparableRSAKey(<cryptography.hazmat.'))
+
+    def test_public_key(self):
+        from acme.jose.util import ComparableRSAKey
+        self.assertTrue(isinstance(self.key.public_key(), ComparableRSAKey))
+
+
+class ImmutableMapTest(unittest.TestCase):
+    """Tests for acme.jose.util.ImmutableMap."""
+
+    def setUp(self):
+        # pylint: disable=invalid-name,too-few-public-methods
+        # pylint: disable=missing-docstring
+        from acme.jose.util import ImmutableMap
+
+        class A(ImmutableMap):
+            __slots__ = ('x', 'y')
+
+        class B(ImmutableMap):
+            __slots__ = ('x', 'y')
+
+        self.A = A
+        self.B = B
+
+        self.a1 = self.A(x=1, y=2)
+        self.a1_swap = self.A(y=2, x=1)
+        self.a2 = self.A(x=3, y=4)
+        self.b = self.B(x=1, y=2)
+
+    def test_update(self):
+        self.assertEqual(self.A(x=2, y=2), self.a1.update(x=2))
+        self.assertEqual(self.a2, self.a1.update(x=3, y=4))
+
+    def test_get_missing_item_raises_key_error(self):
+        self.assertRaises(KeyError, self.a1.__getitem__, 'z')
+
+    def test_order_of_args_does_not_matter(self):
+        self.assertEqual(self.a1, self.a1_swap)
+
+    def test_type_error_on_missing(self):
+        self.assertRaises(TypeError, self.A, x=1)
+        self.assertRaises(TypeError, self.A, y=2)
+
+    def test_type_error_on_unrecognized(self):
+        self.assertRaises(TypeError, self.A, x=1, z=2)
+        self.assertRaises(TypeError, self.A, x=1, y=2, z=3)
+
+    def test_get_attr(self):
+        self.assertEqual(1, self.a1.x)
+        self.assertEqual(2, self.a1.y)
+        self.assertEqual(1, self.a1_swap.x)
+        self.assertEqual(2, self.a1_swap.y)
+
+    def test_set_attr_raises_attribute_error(self):
+        self.assertRaises(
+            AttributeError, functools.partial(self.a1.__setattr__, 'x'), 10)
+
+    def test_equal(self):
+        self.assertEqual(self.a1, self.a1)
+        self.assertEqual(self.a2, self.a2)
+        self.assertNotEqual(self.a1, self.a2)
+
+    def test_hash(self):
+        self.assertEqual(hash((1, 2)), hash(self.a1))
+
+    def test_unhashable(self):
+        self.assertRaises(TypeError, self.A(x=1, y={}).__hash__)
+
+    def test_repr(self):
+        self.assertEqual('A(x=1, y=2)', repr(self.a1))
+        self.assertEqual('A(x=1, y=2)', repr(self.a1_swap))
+        self.assertEqual('B(x=1, y=2)', repr(self.b))
+        self.assertEqual("B(x='foo', y='bar')", repr(self.B(x='foo', y='bar')))
+
+
+class frozendictTest(unittest.TestCase):  # pylint: disable=invalid-name
+    """Tests for acme.jose.util.frozendict."""
+
+    def setUp(self):
+        from acme.jose.util import frozendict
+        self.fdict = frozendict(x=1, y='2')
+
+    def test_init_dict(self):
+        from acme.jose.util import frozendict
+        self.assertEqual(self.fdict, frozendict({'x': 1, 'y': '2'}))
+
+    def test_init_other_raises_type_error(self):
+        from acme.jose.util import frozendict
+        # specifically fail for generators...
+        self.assertRaises(TypeError, frozendict, six.iteritems({'a': 'b'}))
+
+    def test_len(self):
+        self.assertEqual(2, len(self.fdict))
+
+    def test_hash(self):
+        self.assertTrue(isinstance(hash(self.fdict), int))
+
+    def test_getattr_proxy(self):
+        self.assertEqual(1, self.fdict.x)
+        self.assertEqual('2', self.fdict.y)
+
+    def test_getattr_raises_attribute_error(self):
+        self.assertRaises(AttributeError, self.fdict.__getattr__, 'z')
+
+    def test_setattr_immutable(self):
+        self.assertRaises(AttributeError, self.fdict.__setattr__, 'z', 3)
+
+    def test_repr(self):
+        self.assertEqual("frozendict(x=1, y='2')", repr(self.fdict))
+
+
+if __name__ == '__main__':
+    unittest.main()  # pragma: no cover

acme/acme/jose_test.py

@@ -1,53 +0,0 @@
-"""Tests for acme.jose shim."""
-import importlib
-import unittest
-
-class JoseTest(unittest.TestCase):
-    """Tests for acme.jose shim."""
-
-    def _test_it(self, submodule, attribute):
-        if submodule:
-            acme_jose_path = 'acme.jose.' + submodule
-            josepy_path = 'josepy.' + submodule
-        else:
-            acme_jose_path = 'acme.jose'
-            josepy_path = 'josepy'
-        acme_jose_mod = importlib.import_module(acme_jose_path)
-        josepy_mod = importlib.import_module(josepy_path)
-
-        self.assertIs(acme_jose_mod, josepy_mod)
-        self.assertIs(getattr(acme_jose_mod, attribute), getattr(josepy_mod, attribute))
-
-        # We use the imports below with eval, but pylint doesn't
-        # understand that.
-        # pylint: disable=eval-used,unused-variable
-        import acme
-        import josepy
-        acme_jose_mod = eval(acme_jose_path)
-        josepy_mod = eval(josepy_path)
-        self.assertIs(acme_jose_mod, josepy_mod)
-        self.assertIs(getattr(acme_jose_mod, attribute), getattr(josepy_mod, attribute))
-
-    def test_top_level(self):
-        self._test_it('', 'RS512')
-
-    def test_submodules(self):
-        # This test ensures that the modules in josepy that were
-        # available at the time it was moved into its own package are
-        # available under acme.jose. Backwards compatibility with new
-        # modules or testing code is not maintained.
-        mods_and_attrs = [('b64', 'b64decode',),
-                          ('errors', 'Error',),
-                          ('interfaces', 'JSONDeSerializable',),
-                          ('json_util', 'Field',),
-                          ('jwa', 'HS256',),
-                          ('jwk', 'JWK',),
-                          ('jws', 'JWS',),
-                          ('util', 'ImmutableMap',),]
-
-        for mod, attr in mods_and_attrs:
-            self._test_it(mod, attr)
-
-
-if __name__ == '__main__':
-    unittest.main()  # pragma: no cover

acme/acme/jws.py

@@ -1,10 +1,10 @@
 """ACME-specific JWS.
 
-The JWS implementation in josepy only implements the base JOSE standard. In
+The JWS implementation in acme.jose only implements the base JOSE standard. In
 order to support the new header fields defined in ACME, this module defines some
-ACME-specific classes that layer on top of josepy.
+ACME-specific classes that layer on top of acme.jose.
 """
-import josepy as jose
+from acme import jose
 
 
 class Header(jose.Header):
@@ -43,7 +43,7 @@ class JWS(jose.JWS):
     __slots__ = jose.JWS._orig_slots  # pylint: disable=no-member
 
     @classmethod
-    # pylint: disable=arguments-differ
+    # pylint: disable=arguments-differ,too-many-arguments
     def sign(cls, payload, key, alg, nonce, url=None, kid=None):
         # Per ACME spec, jwk and kid are mutually exclusive, so only include a
         # jwk field if kid is not provided.

acme/acme/jws_test.py

@@ -1,8 +1,7 @@
 """Tests for acme.jws."""
 import unittest
 
-import josepy as jose
-
+from acme import jose
 from acme import test_util
 
 

acme/acme/magic_typing.py

@@ -1,16 +0,0 @@
-"""Shim class to not have to depend on typing module in prod."""
-import sys
-
-class TypingClass(object):
-    """Ignore import errors by getting anything"""
-    def __getattr__(self, name):
-        return None
-
-try:
-    # mypy doesn't respect modifying sys.modules
-    from typing import *  # pylint: disable=wildcard-import, unused-wildcard-import
-    # pylint: disable=unused-import
-    from typing import Collection, IO  # type: ignore
-    # pylint: enable=unused-import
-except ImportError:
-    sys.modules[__name__] = TypingClass()

acme/acme/magic_typing_test.py

@@ -1,41 +0,0 @@
-"""Tests for acme.magic_typing."""
-import sys
-import unittest
-
-import mock
-
-
-class MagicTypingTest(unittest.TestCase):
-    """Tests for acme.magic_typing."""
-    def test_import_success(self):
-        try:
-            import typing as temp_typing
-        except ImportError: # pragma: no cover
-            temp_typing = None # pragma: no cover
-        typing_class_mock = mock.MagicMock()
-        text_mock = mock.MagicMock()
-        typing_class_mock.Text = text_mock
-        sys.modules['typing'] = typing_class_mock
-        if 'acme.magic_typing' in sys.modules:
-            del sys.modules['acme.magic_typing'] # pragma: no cover
-        from acme.magic_typing import Text # pylint: disable=no-name-in-module
-        self.assertEqual(Text, text_mock)
-        del sys.modules['acme.magic_typing']
-        sys.modules['typing'] = temp_typing
-
-    def test_import_failure(self):
-        try:
-            import typing as temp_typing
-        except ImportError: # pragma: no cover
-            temp_typing = None # pragma: no cover
-        sys.modules['typing'] = None
-        if 'acme.magic_typing' in sys.modules:
-            del sys.modules['acme.magic_typing'] # pragma: no cover
-        from acme.magic_typing import Text # pylint: disable=no-name-in-module
-        self.assertTrue(Text is None)
-        del sys.modules['acme.magic_typing']
-        sys.modules['typing'] = temp_typing
-
-
-if __name__ == '__main__':
-    unittest.main()  # pragma: no cover

acme/acme/messages.py

@@ -1,53 +1,30 @@
 """ACME protocol messages."""
-import json
-import six
-try:
-    from collections.abc import Hashable  # pylint: disable=no-name-in-module
-except ImportError:  # pragma: no cover
-    from collections import Hashable
-
-import josepy as jose
+import collections
 
 from acme import challenges
 from acme import errors
 from acme import fields
+from acme import jose
 from acme import util
-from acme import jws
 
 OLD_ERROR_PREFIX = "urn:acme:error:"
 ERROR_PREFIX = "urn:ietf:params:acme:error:"
 
 ERROR_CODES = {
-    'accountDoesNotExist': 'The request specified an account that does not exist',
-    'alreadyRevoked': 'The request specified a certificate to be revoked that has' \
-    ' already been revoked',
     'badCSR': 'The CSR is unacceptable (e.g., due to a short key)',
     'badNonce': 'The client sent an unacceptable anti-replay nonce',
-    'badPublicKey': 'The JWS was signed by a public key the server does not support',
-    'badRevocationReason': 'The revocation reason provided is not allowed by the server',
-    'badSignatureAlgorithm': 'The JWS was signed with an algorithm the server does not support',
-    'caa': 'Certification Authority Authorization (CAA) records forbid the CA from issuing' \
-    ' a certificate',
-    'compound': 'Specific error conditions are indicated in the "subproblems" array',
     'connection': ('The server could not connect to the client to verify the'
                    ' domain'),
-    'dns': 'There was a problem with a DNS query during identifier validation',
     'dnssec': 'The server could not validate a DNSSEC signed domain',
-    'incorrectResponse': 'Response recieved didn\'t match the challenge\'s requirements',
     # deprecate invalidEmail
     'invalidEmail': 'The provided email for a registration was invalid',
     'invalidContact': 'The provided contact URI was invalid',
     'malformed': 'The request message was malformed',
-    'rejectedIdentifier': 'The server will not issue certificates for the identifier',
-    'orderNotReady': 'The request attempted to finalize an order that is not ready to be finalized',
     'rateLimited': 'There were too many requests of a given type',
     'serverInternal': 'The server experienced an internal error',
     'tls': 'The server experienced a TLS error during domain verification',
     'unauthorized': 'The client lacks sufficient authorization',
-    'unsupportedContact': 'A contact URL for an account used an unsupported protocol scheme',
     'unknownHost': 'The server could not resolve a domain name',
-    'unsupportedIdentifier': 'An identifier is of an unsupported type',
-    'externalAccountRequired': 'The server requires external account binding',
 }
 
 ERROR_TYPE_DESCRIPTIONS = dict(
@@ -59,12 +36,9 @@ ERROR_TYPE_DESCRIPTIONS.update(dict(  # add errors with old prefix, deprecate me
 
 def is_acme_error(err):
     """Check if argument is an ACME error."""
-    if isinstance(err, Error) and (err.typ is not None):
-        return (ERROR_PREFIX in err.typ) or (OLD_ERROR_PREFIX in err.typ)
-    return False
+    return (ERROR_PREFIX in str(err)) or (OLD_ERROR_PREFIX in str(err))
 
 
-@six.python_2_unicode_compatible
 class Error(jose.JSONObjectWithFields, errors.Error):
     """ACME error.
 
@@ -116,34 +90,32 @@ class Error(jose.JSONObjectWithFields, errors.Error):
         code = str(self.typ).split(':')[-1]
         if code in ERROR_CODES:
             return code
-        return None
 
     def __str__(self):
-        return b' :: '.join(
-            part.encode('ascii', 'backslashreplace') for part in
+        return ' :: '.join(
+            part for part in
             (self.typ, self.description, self.detail, self.title)
-            if part is not None).decode()
+            if part is not None)
 
 
-class _Constant(jose.JSONDeSerializable, Hashable):  # type: ignore
+class _Constant(jose.JSONDeSerializable, collections.Hashable):  # type: ignore
     """ACME constant."""
     __slots__ = ('name',)
     POSSIBLE_NAMES = NotImplemented
 
     def __init__(self, name):
-        super(_Constant, self).__init__()
-        self.POSSIBLE_NAMES[name] = self  # pylint: disable=unsupported-assignment-operation
+        self.POSSIBLE_NAMES[name] = self
         self.name = name
 
     def to_partial_json(self):
         return self.name
 
     @classmethod
-    def from_json(cls, jobj):
-        if jobj not in cls.POSSIBLE_NAMES:  # pylint: disable=unsupported-membership-test
+    def from_json(cls, value):
+        if value not in cls.POSSIBLE_NAMES:
             raise jose.DeserializationError(
                 '{0} not recognized'.format(cls.__name__))
-        return cls.POSSIBLE_NAMES[jobj]  # pylint: disable=unsubscriptable-object
+        return cls.POSSIBLE_NAMES[value]
 
     def __repr__(self):
         return '{0}({1})'.format(self.__class__.__name__, self.name)
@@ -167,8 +139,6 @@ STATUS_PROCESSING = Status('processing')
 STATUS_VALID = Status('valid')
 STATUS_INVALID = Status('invalid')
 STATUS_REVOKED = Status('revoked')
-STATUS_READY = Status('ready')
-STATUS_DEACTIVATED = Status('deactivated')
 
 
 class IdentifierType(_Constant):
@@ -195,30 +165,9 @@ class Directory(jose.JSONDeSerializable):
 
     class Meta(jose.JSONObjectWithFields):
         """Directory Meta."""
-        _terms_of_service = jose.Field('terms-of-service', omitempty=True)
-        _terms_of_service_v2 = jose.Field('termsOfService', omitempty=True)
+        terms_of_service = jose.Field('terms-of-service', omitempty=True)
         website = jose.Field('website', omitempty=True)
-        caa_identities = jose.Field('caaIdentities', omitempty=True)
-        external_account_required = jose.Field('externalAccountRequired', omitempty=True)
-
-        def __init__(self, **kwargs):
-            kwargs = dict((self._internal_name(k), v) for k, v in kwargs.items())
-            super(Directory.Meta, self).__init__(**kwargs)
-
-        @property
-        def terms_of_service(self):
-            """URL for the CA TOS"""
-            return self._terms_of_service or self._terms_of_service_v2
-
-        def __iter__(self):
-            # When iterating over fields, use the external name 'terms_of_service' instead of
-            # the internal '_terms_of_service'.
-            for name in super(Directory.Meta, self).__iter__():
-                yield name[1:] if name == '_terms_of_service' else name
-
-        def _internal_name(self, name):
-            return '_' + name if name == 'terms_of_service' else name
-
+        caa_identities = jose.Field('caa-identities', omitempty=True)
 
     @classmethod
     def _canon_key(cls, key):
@@ -281,28 +230,10 @@ class ResourceBody(jose.JSONObjectWithFields):
     """ACME Resource Body."""
 
 
-class ExternalAccountBinding(object):
-    """ACME External Account Binding"""
-
-    @classmethod
-    def from_data(cls, account_public_key, kid, hmac_key, directory):
-        """Create External Account Binding Resource from contact details, kid and hmac."""
-
-        key_json = json.dumps(account_public_key.to_partial_json()).encode()
-        decoded_hmac_key = jose.b64.b64decode(hmac_key)
-        url = directory["newAccount"]
-
-        eab = jws.JWS.sign(key_json, jose.jwk.JWKOct(key=decoded_hmac_key),
-                           jose.jwa.HS256, None,
-                           url, kid)
-
-        return eab.to_partial_json()
-
-
 class Registration(ResourceBody):
     """Registration Resource Body.
 
-    :ivar josepy.jwk.JWK key: Public key.
+    :ivar acme.jose.jwk.JWK key: Public key.
     :ivar tuple contact: Contact information following ACME spec,
         `tuple` of `unicode`.
     :ivar unicode agreement:
@@ -314,31 +245,24 @@ class Registration(ResourceBody):
     contact = jose.Field('contact', omitempty=True, default=())
     agreement = jose.Field('agreement', omitempty=True)
     status = jose.Field('status', omitempty=True)
-    terms_of_service_agreed = jose.Field('termsOfServiceAgreed', omitempty=True)
-    only_return_existing = jose.Field('onlyReturnExisting', omitempty=True)
-    external_account_binding = jose.Field('externalAccountBinding', omitempty=True)
 
     phone_prefix = 'tel:'
     email_prefix = 'mailto:'
 
     @classmethod
-    def from_data(cls, phone=None, email=None, external_account_binding=None, **kwargs):
+    def from_data(cls, phone=None, email=None, **kwargs):
         """Create registration resource from contact details."""
         details = list(kwargs.pop('contact', ()))
         if phone is not None:
             details.append(cls.phone_prefix + phone)
         if email is not None:
-            details.extend([cls.email_prefix + mail for mail in email.split(',')])
+            details.append(cls.email_prefix + email)
         kwargs['contact'] = tuple(details)
-
-        if external_account_binding:
-            kwargs['external_account_binding'] = external_account_binding
-
         return cls(**kwargs)
 
     def _filter_contact(self, prefix):
         return tuple(
-            detail[len(prefix):] for detail in self.contact  # pylint: disable=not-an-iterable
+            detail[len(prefix):] for detail in self.contact
             if detail.startswith(prefix))
 
     @property
@@ -396,25 +320,13 @@ class ChallengeBody(ResourceBody):
 
     """
     __slots__ = ('chall',)
-    # ACMEv1 has a "uri" field in challenges. ACMEv2 has a "url" field. This
-    # challenge object supports either one, but should be accessed through the
-    # name "uri". In Client.answer_challenge, whichever one is set will be
-    # used.
-    _uri = jose.Field('uri', omitempty=True, default=None)
-    _url = jose.Field('url', omitempty=True, default=None)
+    uri = jose.Field('uri')
     status = jose.Field('status', decoder=Status.from_json,
                         omitempty=True, default=STATUS_PENDING)
     validated = fields.RFC3339Field('validated', omitempty=True)
     error = jose.Field('error', decoder=Error.from_json,
                        omitempty=True, default=None)
 
-    def __init__(self, **kwargs):
-        kwargs = dict((self._internal_name(k), v) for k, v in kwargs.items())
-        super(ChallengeBody, self).__init__(**kwargs)
-
-    def encode(self, name):
-        return super(ChallengeBody, self).encode(self._internal_name(name))
-
     def to_partial_json(self):
         jobj = super(ChallengeBody, self).to_partial_json()
         jobj.update(self.chall.to_partial_json())
@@ -426,23 +338,9 @@ class ChallengeBody(ResourceBody):
         jobj_fields['chall'] = challenges.Challenge.from_json(jobj)
         return jobj_fields
 
-    @property
-    def uri(self):
-        """The URL of this challenge."""
-        return self._url or self._uri
-
     def __getattr__(self, name):
         return getattr(self.chall, name)
 
-    def __iter__(self):
-        # When iterating over fields, use the external name 'uri' instead of
-        # the internal '_uri'.
-        for name in super(ChallengeBody, self).__iter__():
-            yield name[1:] if name == '_uri' else name
-
-    def _internal_name(self, name):
-        return '_' + name if name == 'uri' else name
-
 
 class ChallengeResource(Resource):
     """Challenge Resource.
@@ -455,10 +353,10 @@ class ChallengeResource(Resource):
     authzr_uri = jose.Field('authzr_uri')
 
     @property
-    def uri(self):
-        """The URL of the challenge body."""
-        # pylint: disable=function-redefined,no-member
-        return self.body.uri
+    def uri(self):  # pylint: disable=missing-docstring,no-self-argument
+        # bug? 'method already defined line None'
+        # pylint: disable=function-redefined
+        return self.body.uri  # pylint: disable=no-member
 
 
 class Authorization(ResourceBody):
@@ -472,7 +370,7 @@ class Authorization(ResourceBody):
     :ivar datetime.datetime expires:
 
     """
-    identifier = jose.Field('identifier', decoder=Identifier.from_json, omitempty=True)
+    identifier = jose.Field('identifier', decoder=Identifier.from_json)
     challenges = jose.Field('challenges', omitempty=True)
     combinations = jose.Field('combinations', omitempty=True)
 
@@ -482,7 +380,6 @@ class Authorization(ResourceBody):
     # be absent'... then acme-spec gives example with 'expires'
     # present... That's confusing!
     expires = fields.RFC3339Field('expires', omitempty=True)
-    wildcard = jose.Field('wildcard', omitempty=True)
 
     @challenges.decoder
     def challenges(value):  # pylint: disable=missing-docstring,no-self-argument
@@ -492,7 +389,7 @@ class Authorization(ResourceBody):
     def resolved_combinations(self):
         """Combinations with challenges instead of indices."""
         return tuple(tuple(self.challenges[idx] for idx in combo)
-                     for combo in self.combinations)  # pylint: disable=not-an-iterable
+                     for combo in self.combinations)
 
 
 @Directory.register
@@ -502,12 +399,6 @@ class NewAuthorization(Authorization):
     resource = fields.Resource(resource_type)
 
 
-class UpdateAuthorization(Authorization):
-    """Update authorization."""
-    resource_type = 'authz'
-    resource = fields.Resource(resource_type)
-
-
 class AuthorizationResource(ResourceWithURI):
     """Authorization Resource.
 
@@ -523,7 +414,7 @@ class AuthorizationResource(ResourceWithURI):
 class CertificateRequest(jose.JSONObjectWithFields):
     """ACME new-cert request.
 
-    :ivar josepy.util.ComparableX509 csr:
+    :ivar acme.jose.util.ComparableX509 csr:
         `OpenSSL.crypto.X509Req` wrapped in `.ComparableX509`
 
     """
@@ -535,7 +426,7 @@ class CertificateRequest(jose.JSONObjectWithFields):
 class CertificateResource(ResourceWithURI):
     """Certificate Resource.
 
-    :ivar josepy.util.ComparableX509 body:
+    :ivar acme.jose.util.ComparableX509 body:
         `OpenSSL.crypto.X509` wrapped in `.ComparableX509`
     :ivar unicode cert_chain_uri: URI found in the 'up' ``Link`` header
     :ivar tuple authzrs: `tuple` of `AuthorizationResource`.
@@ -558,49 +449,3 @@ class Revocation(jose.JSONObjectWithFields):
     certificate = jose.Field(
         'certificate', decoder=jose.decode_cert, encoder=jose.encode_cert)
     reason = jose.Field('reason')
-
-
-class Order(ResourceBody):
-    """Order Resource Body.
-
-    :ivar list of .Identifier: List of identifiers for the certificate.
-    :ivar acme.messages.Status status:
-    :ivar list of str authorizations: URLs of authorizations.
-    :ivar str certificate: URL to download certificate as a fullchain PEM.
-    :ivar str finalize: URL to POST to to request issuance once all
-        authorizations have "valid" status.
-    :ivar datetime.datetime expires: When the order expires.
-    :ivar .Error error: Any error that occurred during finalization, if applicable.
-    """
-    identifiers = jose.Field('identifiers', omitempty=True)
-    status = jose.Field('status', decoder=Status.from_json,
-                        omitempty=True)
-    authorizations = jose.Field('authorizations', omitempty=True)
-    certificate = jose.Field('certificate', omitempty=True)
-    finalize = jose.Field('finalize', omitempty=True)
-    expires = fields.RFC3339Field('expires', omitempty=True)
-    error = jose.Field('error', omitempty=True, decoder=Error.from_json)
-
-    @identifiers.decoder
-    def identifiers(value):  # pylint: disable=missing-docstring,no-self-argument
-        return tuple(Identifier.from_json(identifier) for identifier in value)
-
-class OrderResource(ResourceWithURI):
-    """Order Resource.
-
-    :ivar acme.messages.Order body:
-    :ivar str csr_pem: The CSR this Order will be finalized with.
-    :ivar list of acme.messages.AuthorizationResource authorizations:
-        Fully-fetched AuthorizationResource objects.
-    :ivar str fullchain_pem: The fetched contents of the certificate URL
-        produced once the order was finalized, if it's present.
-    """
-    body = jose.Field('body', decoder=Order.from_json)
-    csr_pem = jose.Field('csr_pem', omitempty=True)
-    authorizations = jose.Field('authorizations')
-    fullchain_pem = jose.Field('fullchain_pem', omitempty=True)
-
-@Directory.register
-class NewOrder(Order):
-    """New order."""
-    resource_type = 'new-order'

acme/acme/messages_test.py

@@ -1,12 +1,11 @@
 """Tests for acme.messages."""
 import unittest
 
-import josepy as jose
 import mock
 
 from acme import challenges
+from acme import jose
 from acme import test_util
-from acme.magic_typing import Dict # pylint: disable=unused-import, no-name-in-module
 
 
 CERT = test_util.load_comparable_cert('cert.der')
@@ -27,7 +26,6 @@ class ErrorTest(unittest.TestCase):
             'type': ERROR_PREFIX + 'malformed',
         }
         self.error_custom = Error(typ='custom', detail='bar')
-        self.empty_error = Error()
         self.jobj_custom = {'type': 'custom', 'detail': 'bar'}
 
     def test_default_typ(self):
@@ -47,6 +45,12 @@ class ErrorTest(unittest.TestCase):
             'The request message was malformed', self.error.description)
         self.assertTrue(self.error_custom.description is None)
 
+    def test_str(self):
+        self.assertEqual(
+            'urn:ietf:params:acme:error:malformed :: The request message was '
+            'malformed :: foo :: title', str(self.error))
+        self.assertEqual('custom :: bar', str(self.error_custom))
+
     def test_code(self):
         from acme.messages import Error
         self.assertEqual('malformed', self.error.code)
@@ -56,28 +60,14 @@ class ErrorTest(unittest.TestCase):
     def test_is_acme_error(self):
         from acme.messages import is_acme_error
         self.assertTrue(is_acme_error(self.error))
+        self.assertTrue(is_acme_error(str(self.error)))
         self.assertFalse(is_acme_error(self.error_custom))
-        self.assertFalse(is_acme_error(self.empty_error))
-        self.assertFalse(is_acme_error("must pet all the {dogs|rabbits}"))
-
-    def test_unicode_error(self):
-        from acme.messages import Error, ERROR_PREFIX, is_acme_error
-        arabic_error = Error(
-                detail=u'\u0639\u062f\u0627\u0644\u0629', typ=ERROR_PREFIX + 'malformed',
-            title='title')
-        self.assertTrue(is_acme_error(arabic_error))
 
     def test_with_code(self):
         from acme.messages import Error, is_acme_error
         self.assertTrue(is_acme_error(Error.with_code('badCSR')))
         self.assertRaises(ValueError, Error.with_code, 'not an ACME error code')
 
-    def test_str(self):
-        self.assertEqual(
-            str(self.error),
-            u"{0.typ} :: {0.description} :: {0.detail} :: {0.title}"
-            .format(self.error))
-
 
 class ConstantTest(unittest.TestCase):
     """Tests for acme.messages._Constant."""
@@ -86,7 +76,7 @@ class ConstantTest(unittest.TestCase):
         from acme.messages import _Constant
 
         class MockConstant(_Constant):  # pylint: disable=missing-docstring
-            POSSIBLE_NAMES = {} # type: Dict
+            POSSIBLE_NAMES = {}
 
         self.MockConstant = MockConstant  # pylint: disable=invalid-name
         self.const_a = MockConstant('a')
@@ -158,7 +148,7 @@ class DirectoryTest(unittest.TestCase):
             'meta': {
                 'terms-of-service': 'https://example.com/acme/terms',
                 'website': 'https://www.example.com/',
-                'caaIdentities': ['example.com'],
+                'caa-identities': ['example.com'],
             },
         })
 
@@ -166,31 +156,6 @@ class DirectoryTest(unittest.TestCase):
         from acme.messages import Directory
         Directory.from_json({'foo': 'bar'})
 
-    def test_iter_meta(self):
-        result = False
-        for k in self.dir.meta:
-            if k == 'terms_of_service':
-                result = self.dir.meta[k] == 'https://example.com/acme/terms'
-        self.assertTrue(result)
-
-
-class ExternalAccountBindingTest(unittest.TestCase):
-    def setUp(self):
-        from acme.messages import Directory
-        self.key = jose.jwk.JWKRSA(key=KEY.public_key())
-        self.kid = "kid-for-testing"
-        self.hmac_key = "hmac-key-for-testing"
-        self.dir = Directory({
-            'newAccount': 'http://url/acme/new-account',
-        })
-
-    def test_from_data(self):
-        from acme.messages import ExternalAccountBinding
-        eab = ExternalAccountBinding.from_data(self.key, self.kid, self.hmac_key, self.dir)
-
-        self.assertEqual(len(eab), 3)
-        self.assertEqual(sorted(eab.keys()), sorted(['protected', 'payload', 'signature']))
-
 
 class RegistrationTest(unittest.TestCase):
     """Tests for acme.messages.Registration."""
@@ -223,22 +188,6 @@ class RegistrationTest(unittest.TestCase):
             'mailto:admin@foo.com',
         ))
 
-    def test_new_registration_from_data_with_eab(self):
-        from acme.messages import NewRegistration, ExternalAccountBinding, Directory
-        key = jose.jwk.JWKRSA(key=KEY.public_key())
-        kid = "kid-for-testing"
-        hmac_key = "hmac-key-for-testing"
-        directory = Directory({
-            'newAccount': 'http://url/acme/new-account',
-        })
-        eab = ExternalAccountBinding.from_data(key, kid, hmac_key, directory)
-        reg = NewRegistration.from_data(email='admin@foo.com', external_account_binding=eab)
-        self.assertEqual(reg.contact, (
-            'mailto:admin@foo.com',
-        ))
-        self.assertEqual(sorted(reg.external_account_binding.keys()),
-                         sorted(['protected', 'payload', 'signature']))
-
     def test_phones(self):
         self.assertEqual(('1234',), self.reg.phones)
 
@@ -325,9 +274,6 @@ class ChallengeBodyTest(unittest.TestCase):
             'detail': 'Unable to communicate with DNS server',
         }
 
-    def test_encode(self):
-        self.assertEqual(self.challb.encode('uri'), self.challb.uri)
-
     def test_to_partial_json(self):
         self.assertEqual(self.jobj_to, self.challb.to_partial_json())
 
@@ -443,34 +389,5 @@ class RevocationTest(unittest.TestCase):
         hash(Revocation.from_json(self.rev.to_json()))
 
 
-class OrderResourceTest(unittest.TestCase):
-    """Tests for acme.messages.OrderResource."""
-
-    def setUp(self):
-        from acme.messages import OrderResource
-        self.regr = OrderResource(
-            body=mock.sentinel.body, uri=mock.sentinel.uri)
-
-    def test_to_partial_json(self):
-        self.assertEqual(self.regr.to_json(), {
-            'body': mock.sentinel.body,
-            'uri': mock.sentinel.uri,
-            'authorizations': None,
-        })
-
-class NewOrderTest(unittest.TestCase):
-    """Tests for acme.messages.NewOrder."""
-
-    def setUp(self):
-        from acme.messages import NewOrder
-        self.reg = NewOrder(
-            identifiers=mock.sentinel.identifiers)
-
-    def test_to_partial_json(self):
-        self.assertEqual(self.reg.to_json(), {
-            'identifiers': mock.sentinel.identifiers,
-        })
-
-
 if __name__ == '__main__':
     unittest.main()  # pragma: no cover

acme/acme/standalone.py

@@ -1,23 +1,27 @@
 """Support for standalone client challenge solvers. """
+import argparse
 import collections
 import functools
 import logging
+import os
 import socket
+import sys
 import threading
 
 from six.moves import BaseHTTPServer  # type: ignore  # pylint: disable=import-error
 from six.moves import http_client  # pylint: disable=import-error
 from six.moves import socketserver  # type: ignore  # pylint: disable=import-error
 
+import OpenSSL
+
 from acme import challenges
 from acme import crypto_util
-from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
 
 
 logger = logging.getLogger(__name__)
 
 # six.moves.* | pylint: disable=no-member,attribute-defined-outside-init
-# pylint: disable=no-init
+# pylint: disable=too-few-public-methods,no-init
 
 
 class TLSServer(socketserver.TCPServer):
@@ -32,7 +36,7 @@ class TLSServer(socketserver.TCPServer):
         self.certs = kwargs.pop("certs", {})
         self.method = kwargs.pop(
             # pylint: disable=protected-access
-            "method", crypto_util._DEFAULT_SSL_METHOD)
+            "method", crypto_util._DEFAULT_TLSSNI01_SSL_METHOD)
         self.allow_reuse_address = kwargs.pop("allow_reuse_address", True)
         socketserver.TCPServer.__init__(self, *args, **kwargs)
 
@@ -62,8 +66,8 @@ class BaseDualNetworkedServers(object):
 
     def __init__(self, ServerClass, server_address, *remaining_args, **kwargs):
         port = server_address[1]
-        self.threads = [] # type: List[threading.Thread]
-        self.servers = [] # type: List[ACMEServerMixin]
+        self.threads = []
+        self.servers = []
 
         # Must try True first.
         # Ubuntu, for example, will fail to bind to IPv4 if we've already bound
@@ -77,29 +81,16 @@ class BaseDualNetworkedServers(object):
                 kwargs["ipv6"] = ip_version
                 new_address = (server_address[0],) + (port,) + server_address[2:]
                 new_args = (new_address,) + remaining_args
-                server = ServerClass(*new_args, **kwargs)
-                logger.debug(
-                    "Successfully bound to %s:%s using %s", new_address[0],
-                    new_address[1], "IPv6" if ip_version else "IPv4")
+                server = ServerClass(*new_args, **kwargs) # pylint: disable=star-args
             except socket.error:
-                if self.servers:
-                    # Already bound using IPv6.
-                    logger.debug(
-                        "Certbot wasn't able to bind to %s:%s using %s, this "
-                        "is often expected due to the dual stack nature of "
-                        "IPv6 socket implementations.",
-                        new_address[0], new_address[1],
-                        "IPv6" if ip_version else "IPv4")
-                else:
-                    logger.debug(
-                        "Failed to bind to %s:%s using %s", new_address[0],
-                        new_address[1], "IPv6" if ip_version else "IPv4")
+                logger.debug("Failed to bind to %s:%s using %s", new_address[0],
+                    new_address[1], "IPv6" if ip_version else "IPv4")
             else:
                 self.servers.append(server)
                 # If two servers are set up and port 0 was passed in, ensure we always
                 # bind to the same port for both servers.
                 port = server.socket.getsockname()[1]
-        if not self.servers:
+        if len(self.servers) == 0:
             raise socket.error("Could not bind to IPv4 or IPv6.")
 
     def serve_forever(self):
@@ -126,6 +117,35 @@ class BaseDualNetworkedServers(object):
         self.threads = []
 
 
+class TLSSNI01Server(TLSServer, ACMEServerMixin):
+    """TLSSNI01 Server."""
+
+    def __init__(self, server_address, certs, ipv6=False):
+        TLSServer.__init__(
+            self, server_address, BaseRequestHandlerWithLogging, certs=certs, ipv6=ipv6)
+
+
+class TLSSNI01DualNetworkedServers(BaseDualNetworkedServers):
+    """TLSSNI01Server Wrapper. Tries everything for both. Failures for one don't
+       affect the other."""
+
+    def __init__(self, *args, **kwargs):
+        BaseDualNetworkedServers.__init__(self, TLSSNI01Server, *args, **kwargs)
+
+
+class BaseRequestHandlerWithLogging(socketserver.BaseRequestHandler):
+    """BaseRequestHandler with logging."""
+
+    def log_message(self, format, *args):  # pylint: disable=redefined-builtin
+        """Log arbitrary message."""
+        logger.debug("%s - - %s", self.client_address[0], format % args)
+
+    def handle(self):
+        """Handle request."""
+        self.log_message("Incoming request")
+        socketserver.BaseRequestHandler.handle(self)
+
+
 class HTTPServer(BaseHTTPServer.HTTPServer):
     """Generic HTTP Server."""
 
@@ -169,7 +189,7 @@ class HTTP01RequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
 
     def __init__(self, *args, **kwargs):
         self.simple_http_resources = kwargs.pop("simple_http_resources", set())
-        BaseHTTPServer.BaseHTTPRequestHandler.__init__(self, *args, **kwargs)
+        socketserver.BaseRequestHandler.__init__(self, *args, **kwargs)
 
     def log_message(self, format, *args):  # pylint: disable=redefined-builtin
         """Log arbitrary message."""
@@ -228,3 +248,39 @@ class HTTP01RequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
         """
         return functools.partial(
             cls, simple_http_resources=simple_http_resources)
+
+
+def simple_tls_sni_01_server(cli_args, forever=True):
+    """Run simple standalone TLSSNI01 server."""
+    logging.basicConfig(level=logging.DEBUG)
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "-p", "--port", default=0, help="Port to serve at. By default "
+        "picks random free port.")
+    args = parser.parse_args(cli_args[1:])
+
+    certs = {}
+
+    _, hosts, _ = next(os.walk('.'))
+    for host in hosts:
+        with open(os.path.join(host, "cert.pem")) as cert_file:
+            cert_contents = cert_file.read()
+        with open(os.path.join(host, "key.pem")) as key_file:
+            key_contents = key_file.read()
+        certs[host.encode()] = (
+            OpenSSL.crypto.load_privatekey(
+                OpenSSL.crypto.FILETYPE_PEM, key_contents),
+            OpenSSL.crypto.load_certificate(
+                OpenSSL.crypto.FILETYPE_PEM, cert_contents))
+
+    server = TLSSNI01Server(('', int(args.port)), certs=certs)
+    logger.info("Serving at https://%s:%s...", *server.socket.getsockname()[:2])
+    if forever:  # pragma: no cover
+        server.serve_forever()
+    else:
+        server.handle_request()
+
+
+if __name__ == "__main__":
+    sys.exit(simple_tls_sni_01_server(sys.argv))  # pragma: no cover

acme/acme/standalone_test.py

@@ -1,56 +1,92 @@
 """Tests for acme.standalone."""
+import os
+import shutil
 import socket
 import threading
+import tempfile
+import time
 import unittest
 
 from six.moves import http_client  # pylint: disable=import-error
 from six.moves import socketserver  # type: ignore  # pylint: disable=import-error
 
-import josepy as jose
 import mock
 import requests
 
 from acme import challenges
+from acme import crypto_util
+from acme import errors
+from acme import jose
 from acme import test_util
-from acme.magic_typing import Set # pylint: disable=unused-import, no-name-in-module
 
 
 class TLSServerTest(unittest.TestCase):
     """Tests for acme.standalone.TLSServer."""
 
+    _multiprocess_can_split_ = True
 
     def test_bind(self):  # pylint: disable=no-self-use
         from acme.standalone import TLSServer
         server = TLSServer(
             ('', 0), socketserver.BaseRequestHandler, bind_and_activate=True)
-        server.server_close()
+        server.server_close()  # pylint: disable=no-member
 
     def test_ipv6(self):
         if socket.has_ipv6:
             from acme.standalone import TLSServer
             server = TLSServer(
                 ('', 0), socketserver.BaseRequestHandler, bind_and_activate=True, ipv6=True)
-            server.server_close()
+            server.server_close()  # pylint: disable=no-member
+
+
+class TLSSNI01ServerTest(unittest.TestCase):
+    """Test for acme.standalone.TLSSNI01Server."""
+
+    _multiprocess_can_split_ = True
+
+    def setUp(self):
+        self.certs = {b'localhost': (
+            test_util.load_pyopenssl_private_key('rsa2048_key.pem'),
+            test_util.load_cert('rsa2048_cert.pem'),
+        )}
+        from acme.standalone import TLSSNI01Server
+        self.server = TLSSNI01Server(("", 0), certs=self.certs)
+        # pylint: disable=no-member
+        self.thread = threading.Thread(target=self.server.serve_forever)
+        self.thread.start()
+
+    def tearDown(self):
+        self.server.shutdown()  # pylint: disable=no-member
+        self.thread.join()
+
+    def test_it(self):
+        host, port = self.server.socket.getsockname()[:2]
+        cert = crypto_util.probe_sni(
+            b'localhost', host=host, port=port, timeout=1)
+        self.assertEqual(jose.ComparableX509(cert),
+                         jose.ComparableX509(self.certs[b'localhost'][1]))
 
 
 class HTTP01ServerTest(unittest.TestCase):
     """Tests for acme.standalone.HTTP01Server."""
 
+    _multiprocess_can_split_ = True
 
     def setUp(self):
         self.account_key = jose.JWK.load(
             test_util.load_vector('rsa1024_key.pem'))
-        self.resources = set() # type: Set
+        self.resources = set()
 
         from acme.standalone import HTTP01Server
         self.server = HTTP01Server(('', 0), resources=self.resources)
 
+        # pylint: disable=no-member
         self.port = self.server.socket.getsockname()[1]
         self.thread = threading.Thread(target=self.server.serve_forever)
         self.thread.start()
 
     def tearDown(self):
-        self.server.shutdown()
+        self.server.shutdown()  # pylint: disable=no-member
         self.thread.join()
 
     def test_index(self):
@@ -88,6 +124,7 @@ class HTTP01ServerTest(unittest.TestCase):
 class BaseDualNetworkedServersTest(unittest.TestCase):
     """Test for acme.standalone.BaseDualNetworkedServers."""
 
+    _multiprocess_can_split_ = True
 
     class SingleProtocolServer(socketserver.TCPServer):
         """Server that only serves on a single protocol. FreeBSD has this behavior for AF_INET6."""
@@ -100,10 +137,8 @@ class BaseDualNetworkedServersTest(unittest.TestCase):
                 self.address_family = socket.AF_INET
             socketserver.TCPServer.__init__(self, *args, **kwargs)
             if ipv6:
-                # NB: On Windows, socket.IPPROTO_IPV6 constant may be missing.
-                # We use the corresponding value (41) instead.
-                level = getattr(socket, "IPPROTO_IPV6", 41)
-                self.socket.setsockopt(level, socket.IPV6_V6ONLY, 1)
+                # pylint: disable=no-member
+                self.socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)
                 try:
                     self.server_bind()
                     self.server_activate()
@@ -116,15 +151,15 @@ class BaseDualNetworkedServersTest(unittest.TestCase):
         mock_bind.side_effect = socket.error
         from acme.standalone import BaseDualNetworkedServers
         self.assertRaises(socket.error, BaseDualNetworkedServers,
-                          BaseDualNetworkedServersTest.SingleProtocolServer,
-                          ('', 0),
-                          socketserver.BaseRequestHandler)
+            BaseDualNetworkedServersTest.SingleProtocolServer,
+            ("", 0),
+            socketserver.BaseRequestHandler)
 
     def test_ports_equal(self):
         from acme.standalone import BaseDualNetworkedServers
         servers = BaseDualNetworkedServers(
             BaseDualNetworkedServersTest.SingleProtocolServer,
-            ('', 0),
+            ("", 0),
             socketserver.BaseRequestHandler)
         socknames = servers.getsocknames()
         prev_port = None
@@ -136,18 +171,48 @@ class BaseDualNetworkedServersTest(unittest.TestCase):
             prev_port = port
 
 
+class TLSSNI01DualNetworkedServersTest(unittest.TestCase):
+    """Test for acme.standalone.TLSSNI01DualNetworkedServers."""
+
+    _multiprocess_can_split_ = True
+
+    def setUp(self):
+        self.certs = {b'localhost': (
+            test_util.load_pyopenssl_private_key('rsa2048_key.pem'),
+            test_util.load_cert('rsa2048_cert.pem'),
+        )}
+        from acme.standalone import TLSSNI01DualNetworkedServers
+        self.servers = TLSSNI01DualNetworkedServers(("", 0), certs=self.certs)
+        self.servers.serve_forever()
+
+    def tearDown(self):
+        self.servers.shutdown_and_server_close()
+
+    def test_connect(self):
+        socknames = self.servers.getsocknames()
+        # connect to all addresses
+        for sockname in socknames:
+            host, port = sockname[:2]
+            cert = crypto_util.probe_sni(
+                b'localhost', host=host, port=port, timeout=1)
+            self.assertEqual(jose.ComparableX509(cert),
+                             jose.ComparableX509(self.certs[b'localhost'][1]))
+
+
 class HTTP01DualNetworkedServersTest(unittest.TestCase):
     """Tests for acme.standalone.HTTP01DualNetworkedServers."""
 
+    _multiprocess_can_split_ = True
 
     def setUp(self):
         self.account_key = jose.JWK.load(
             test_util.load_vector('rsa1024_key.pem'))
-        self.resources = set() # type: Set
+        self.resources = set()
 
         from acme.standalone import HTTP01DualNetworkedServers
         self.servers = HTTP01DualNetworkedServers(('', 0), resources=self.resources)
 
+        # pylint: disable=no-member
         self.port = self.servers.getsocknames()[0][1]
         self.servers.serve_forever()
 
@@ -186,5 +251,57 @@ class HTTP01DualNetworkedServersTest(unittest.TestCase):
         self.assertFalse(self._test_http01(add=False))
 
 
+class TestSimpleTLSSNI01Server(unittest.TestCase):
+    """Tests for acme.standalone.simple_tls_sni_01_server."""
+
+    _multiprocess_can_split_ = True
+
+    def setUp(self):
+        # mirror ../examples/standalone
+        self.test_cwd = tempfile.mkdtemp()
+        localhost_dir = os.path.join(self.test_cwd, 'localhost')
+        os.makedirs(localhost_dir)
+        shutil.copy(test_util.vector_path('rsa2048_cert.pem'),
+                    os.path.join(localhost_dir, 'cert.pem'))
+        shutil.copy(test_util.vector_path('rsa2048_key.pem'),
+                    os.path.join(localhost_dir, 'key.pem'))
+
+        from acme.standalone import simple_tls_sni_01_server
+        self.port = 1234
+        self.thread = threading.Thread(
+            target=simple_tls_sni_01_server, kwargs={
+                'cli_args': ('xxx', '--port', str(self.port)),
+                'forever': False,
+            },
+        )
+        self.old_cwd = os.getcwd()
+        os.chdir(self.test_cwd)
+
+    def tearDown(self):
+        os.chdir(self.old_cwd)
+        self.thread.join()
+        shutil.rmtree(self.test_cwd)
+
+    def test_it(self):
+        max_attempts = 5
+        for attempt in range(max_attempts):
+            try:
+                cert = crypto_util.probe_sni(
+                    b'localhost', b'0.0.0.0', self.port)
+            except errors.Error:
+                self.assertTrue(attempt + 1 < max_attempts, "Timeout!")
+                time.sleep(1)  # wait until thread starts
+            else:
+                self.assertEqual(jose.ComparableX509(cert),
+                                 test_util.load_comparable_cert(
+                                     'rsa2048_cert.pem'))
+                break
+
+            if attempt == 0:
+                # the first attempt is always meant to fail, so we can test
+                # the socket failure code-path for probe_sni, as well
+                self.thread.start()
+
+
 if __name__ == "__main__":
     unittest.main()  # pragma: no cover

acme/acme/test_util.py

@@ -5,11 +5,19 @@
 """
 import os
 import pkg_resources
+import unittest
 
 from cryptography.hazmat.backends import default_backend
 from cryptography.hazmat.primitives import serialization
-import josepy as jose
-from OpenSSL import crypto
+import OpenSSL
+
+from acme import jose
+
+
+def vector_path(*names):
+    """Path to a test vector."""
+    return pkg_resources.resource_filename(
+        __name__, os.path.join('testdata', *names))
 
 
 def load_vector(*names):
@@ -32,8 +40,8 @@ def _guess_loader(filename, loader_pem, loader_der):
 def load_cert(*names):
     """Load certificate."""
     loader = _guess_loader(
-        names[-1], crypto.FILETYPE_PEM, crypto.FILETYPE_ASN1)
-    return crypto.load_certificate(loader, load_vector(*names))
+        names[-1], OpenSSL.crypto.FILETYPE_PEM, OpenSSL.crypto.FILETYPE_ASN1)
+    return OpenSSL.crypto.load_certificate(loader, load_vector(*names))
 
 
 def load_comparable_cert(*names):
@@ -44,8 +52,8 @@ def load_comparable_cert(*names):
 def load_csr(*names):
     """Load certificate request."""
     loader = _guess_loader(
-        names[-1], crypto.FILETYPE_PEM, crypto.FILETYPE_ASN1)
-    return crypto.load_certificate_request(loader, load_vector(*names))
+        names[-1], OpenSSL.crypto.FILETYPE_PEM, OpenSSL.crypto.FILETYPE_ASN1)
+    return OpenSSL.crypto.load_certificate_request(loader, load_vector(*names))
 
 
 def load_comparable_csr(*names):
@@ -64,5 +72,26 @@ def load_rsa_private_key(*names):
 def load_pyopenssl_private_key(*names):
     """Load pyOpenSSL private key."""
     loader = _guess_loader(
-        names[-1], crypto.FILETYPE_PEM, crypto.FILETYPE_ASN1)
-    return crypto.load_privatekey(loader, load_vector(*names))
+        names[-1], OpenSSL.crypto.FILETYPE_PEM, OpenSSL.crypto.FILETYPE_ASN1)
+    return OpenSSL.crypto.load_privatekey(loader, load_vector(*names))
+
+
+def skip_unless(condition, reason):  # pragma: no cover
+    """Skip tests unless a condition holds.
+
+    This implements the basic functionality of unittest.skipUnless
+    which is only available on Python 2.7+.
+
+    :param bool condition: If ``False``, the test will be skipped
+    :param str reason: the reason for skipping the test
+
+    :rtype: callable
+    :returns: decorator that hides tests unless condition is ``True``
+
+    """
+    if hasattr(unittest, "skipUnless"):
+        return unittest.skipUnless(condition, reason)
+    elif condition:
+        return lambda cls: cls
+    else:
+        return lambda cls: None

acme/acme/testdata/critical-san.pem

@@ -1,28 +0,0 @@
------BEGIN CERTIFICATE-----
-MIIErTCCA5WgAwIBAgIKETb7VQAAAAAdGTANBgkqhkiG9w0BAQsFADCBkTELMAkG
-A1UEBhMCVVMxDTALBgNVBAgTBFV0YWgxFzAVBgNVBAcTDlNhbHQgTGFrZSBDaXR5
-MRUwEwYDVQQKEwxWZW5hZmksIEluYy4xHzAdBgNVBAsTFkRlbW9uc3RyYXRpb24g
-U2VydmljZXMxIjAgBgNVBAMTGVZlbmFmaSBFeGFtcGxlIElzc3VpbmcgQ0EwHhcN
-MTcwNzEwMjMxNjA1WhcNMTcwODA5MjMxNjA1WjAAMIIBIjANBgkqhkiG9w0BAQEF
-AAOCAQ8AMIIBCgKCAQEA7CU5qRIzCs9hCRiSUvLZ8r81l4zIYbx1V1vZz6x1cS4M
-0keNfFJ1wB+zuvx80KaMYkWPYlg4Rsm9Ok3ZapakXDlaWtrfg78lxtHuPw1o7AYV
-EXDwwPkNugLMJfYw5hWYSr8PCLcOJoY00YQ0fJ44L+kVsUyGjN4UTRRZmOh/yNVU
-0W12dTCz4X7BAW01OuY6SxxwewnW3sBEep+APfr2jd/oIx7fgZmVB8aRCDPj4AFl
-XINWIwxmptOwnKPbwLN/vhCvJRUkO6rA8lpYwQkedFf6fHhqi2Sq/NCEOg4RvMCF
-fKbMpncOXxz+f4/i43SVLrPz/UyhjNbKGJZ+zFrQowIDAQABo4IBlTCCAZEwPgYD
-VR0RAQH/BDQwMoIbY2hpY2Fnby1jdWJzLnZlbmFmaS5leGFtcGxlghNjdWJzLnZl
-bmFmaS5leGFtcGxlMB0GA1UdDgQWBBTgKZXVSFNyPHHtO/phtIALPcCF5DAfBgNV
-HSMEGDAWgBT/JJ6Wei/pzf+9DRHuv6Wgdk2HsjBSBgNVHR8ESzBJMEegRaBDhkFo
-dHRwOi8vcGtpLnZlbmFmaS5leGFtcGxlL2NybC9WZW5hZmklMjBFeGFtcGxlJTIw
-SXNzdWluZyUyMENBLmNybDA6BggrBgEFBQcBAQQuMCwwKgYIKwYBBQUHMAGGHmh0
-dHA6Ly9wa2kudmVuYWZpLmV4YW1wbGUvb2NzcDAOBgNVHQ8BAf8EBAMCBaAwPQYJ
-KwYBBAGCNxUHBDAwLgYmKwYBBAGCNxUIhIDLGYTvsSSEnZ8ehvD5UofP4hMEgobv
-DIGy4mcCAWQCAQIwEwYDVR0lBAwwCgYIKwYBBQUHAwEwGwYJKwYBBAGCNxUKBA4w
-DDAKBggrBgEFBQcDATANBgkqhkiG9w0BAQsFAAOCAQEA3YW4t1AzxEn384OqdU6L
-ny8XkMhWpRM0W0Z9ZC3gRZKbVUu49nG/KB5hbVn/de33zdX9HOZJKc0vXzkGZQUs
-OUCCsKX4VKzV5naGXOuGRbvV4CJh5P0kPlDzyb5t312S49nJdcdBf0Y/uL5Qzhst
-bXy8qNfFNG3SIKKRAUpqE9OVIl+F+JBwexa+v/4dFtUOqMipfXxB3TaxnDqvU1dS
-yO34ZTvIMGXJIZ5nn/d/LNc3N3vBg2SHkMpladqw0Hr7mL0bFOe0b+lJgkDP06Be
-n08fikhz1j2AW4/ZHa9w4DUz7J21+RtHMhh+Vd1On0EAeZ563svDe7Z+yrg6zOVv
-KA==
------END CERTIFICATE-----

acme/docs/api/jose.rst

@@ -1,7 +1,10 @@
 JOSE
 ----
 
-The ``acme.jose`` module was moved to its own package "josepy_".
-Please refer to its documentation there.
+.. automodule:: acme.jose
+   :members:
 
-.. _josepy: https://josepy.readthedocs.io/
+.. toctree::
+   :glob:
+
+   jose/*

acme/docs/api/jose/base64.rst

@@ -0,0 +1,5 @@
+JOSE Base64
+-----------
+
+.. automodule:: acme.jose.b64
+   :members:

acme/docs/api/jose/errors.rst

@@ -0,0 +1,5 @@
+Errors
+------
+
+.. automodule:: acme.jose.errors
+   :members:

acme/docs/api/jose/interfaces.rst

@@ -0,0 +1,5 @@
+Interfaces
+----------
+
+.. automodule:: acme.jose.interfaces
+   :members:

acme/docs/api/jose/json_util.rst

@@ -0,0 +1,5 @@
+JSON utilities
+--------------
+
+.. automodule:: acme.jose.json_util
+   :members:

acme/docs/api/jose/jwa.rst

@@ -0,0 +1,5 @@
+JSON Web Algorithms
+-------------------
+
+.. automodule:: acme.jose.jwa
+   :members:

acme/docs/api/jose/jwk.rst

@@ -0,0 +1,5 @@
+JSON Web Key
+------------
+
+.. automodule:: acme.jose.jwk
+   :members:

acme/docs/api/jose/jws.rst

@@ -0,0 +1,5 @@
+JSON Web Signature
+------------------
+
+.. automodule:: acme.jose.jws
+   :members:

acme/docs/api/jose/util.rst

@@ -0,0 +1,5 @@
+Utilities
+---------
+
+.. automodule:: acme.jose.util
+   :members:

acme/docs/api/other.rst

@@ -0,0 +1,5 @@
+Other ACME objects
+------------------
+
+.. automodule:: acme.other
+   :members:

acme/docs/conf.py

@@ -308,5 +308,4 @@ texinfo_documents = [
 
 intersphinx_mapping = {
     'python': ('https://docs.python.org/', None),
-    'josepy': ('https://josepy.readthedocs.io/en/latest/', None),
 }

acme/docs/index.rst

@@ -16,6 +16,13 @@ Contents:
 .. automodule:: acme
    :members:
 
+
+Example client:
+
+.. include:: ../examples/example_client.py
+   :code: python
+
+
 Indices and tables
 ==================
 

acme/examples/example_client.py

@@ -0,0 +1,47 @@
+"""Example script showing how to use acme client API."""
+import logging
+import os
+import pkg_resources
+
+from cryptography.hazmat.backends import default_backend
+from cryptography.hazmat.primitives.asymmetric import rsa
+import OpenSSL
+
+from acme import client
+from acme import messages
+from acme import jose
+
+
+logging.basicConfig(level=logging.DEBUG)
+
+
+DIRECTORY_URL = 'https://acme-staging.api.letsencrypt.org/directory'
+BITS = 2048  # minimum for Boulder
+DOMAIN = 'example1.com'  # example.com is ignored by Boulder
+
+# generate_private_key requires cryptography>=0.5
+key = jose.JWKRSA(key=rsa.generate_private_key(
+    public_exponent=65537,
+    key_size=BITS,
+    backend=default_backend()))
+acme = client.Client(DIRECTORY_URL, key)
+
+regr = acme.register()
+logging.info('Auto-accepting TOS: %s', regr.terms_of_service)
+acme.agree_to_tos(regr)
+logging.debug(regr)
+
+authzr = acme.request_challenges(
+    identifier=messages.Identifier(typ=messages.IDENTIFIER_FQDN, value=DOMAIN))
+logging.debug(authzr)
+
+authzr, authzr_response = acme.poll(authzr)
+
+csr = OpenSSL.crypto.load_certificate_request(
+    OpenSSL.crypto.FILETYPE_ASN1, pkg_resources.resource_string(
+        'acme', os.path.join('testdata', 'csr.der')))
+try:
+    acme.request_issuance(jose.util.ComparableX509(csr), (authzr,))
+except messages.Error as error:
+    print ("This script is doomed to fail as no authorization "
+           "challenges are ever solved. Error from server: {0}".format(error))

acme/examples/http01_example.py

@@ -1,240 +0,0 @@
-"""Example ACME-V2 API for HTTP-01 challenge.
-
-Brief:
-
-This a complete usage example of the python-acme API.
-
-Limitations of this example:
-    - Works for only one Domain name
-    - Performs only HTTP-01 challenge
-    - Uses ACME-v2
-
-Workflow:
-    (Account creation)
-    - Create account key
-    - Register account and accept TOS
-    (Certificate actions)
-    - Select HTTP-01 within offered challenges by the CA server
-    - Set up http challenge resource
-    - Set up standalone web server
-    - Create domain private key and CSR
-    - Issue certificate
-    - Renew certificate
-    - Revoke certificate
-    (Account update actions)
-    - Change contact information
-    - Deactivate Account
-"""
-from contextlib import contextmanager
-from cryptography.hazmat.backends import default_backend
-from cryptography.hazmat.primitives.asymmetric import rsa
-import OpenSSL
-
-from acme import challenges
-from acme import client
-from acme import crypto_util
-from acme import errors
-from acme import messages
-from acme import standalone
-import josepy as jose
-
-# Constants:
-
-# This is the staging point for ACME-V2 within Let's Encrypt.
-DIRECTORY_URL = 'https://acme-staging-v02.api.letsencrypt.org/directory'
-
-USER_AGENT = 'python-acme-example'
-
-# Account key size
-ACC_KEY_BITS = 2048
-
-# Certificate private key size
-CERT_PKEY_BITS = 2048
-
-# Domain name for the certificate.
-DOMAIN = 'client.example.com'
-
-# If you are running Boulder locally, it is possible to configure any port
-# number to execute the challenge, but real CA servers will always use port
-# 80, as described in the ACME specification.
-PORT = 80
-
-
-# Useful methods and classes:
-
-
-def new_csr_comp(domain_name, pkey_pem=None):
-    """Create certificate signing request."""
-    if pkey_pem is None:
-        # Create private key.
-        pkey = OpenSSL.crypto.PKey()
-        pkey.generate_key(OpenSSL.crypto.TYPE_RSA, CERT_PKEY_BITS)
-        pkey_pem = OpenSSL.crypto.dump_privatekey(OpenSSL.crypto.FILETYPE_PEM,
-                                                  pkey)
-    csr_pem = crypto_util.make_csr(pkey_pem, [domain_name])
-    return pkey_pem, csr_pem
-
-
-def select_http01_chall(orderr):
-    """Extract authorization resource from within order resource."""
-    # Authorization Resource: authz.
-    # This object holds the offered challenges by the server and their status.
-    authz_list = orderr.authorizations
-
-    for authz in authz_list:
-        # Choosing challenge.
-        # authz.body.challenges is a set of ChallengeBody objects.
-        for i in authz.body.challenges:
-            # Find the supported challenge.
-            if isinstance(i.chall, challenges.HTTP01):
-                return i
-
-    raise Exception('HTTP-01 challenge was not offered by the CA server.')
-
-
-@contextmanager
-def challenge_server(http_01_resources):
-    """Manage standalone server set up and shutdown."""
-
-    # Setting up a fake server that binds at PORT and any address.
-    address = ('', PORT)
-    try:
-        servers = standalone.HTTP01DualNetworkedServers(address,
-                                                        http_01_resources)
-        # Start client standalone web server.
-        servers.serve_forever()
-        yield servers
-    finally:
-        # Shutdown client web server and unbind from PORT
-        servers.shutdown_and_server_close()
-
-
-def perform_http01(client_acme, challb, orderr):
-    """Set up standalone webserver and perform HTTP-01 challenge."""
-
-    response, validation = challb.response_and_validation(client_acme.net.key)
-
-    resource = standalone.HTTP01RequestHandler.HTTP01Resource(
-        chall=challb.chall, response=response, validation=validation)
-
-    with challenge_server({resource}):
-        # Let the CA server know that we are ready for the challenge.
-        client_acme.answer_challenge(challb, response)
-
-        # Wait for challenge status and then issue a certificate.
-        # It is possible to set a deadline time.
-        finalized_orderr = client_acme.poll_and_finalize(orderr)
-
-    return finalized_orderr.fullchain_pem
-
-
-# Main examples:
-
-
-def example_http():
-    """This example executes the whole process of fulfilling a HTTP-01
-    challenge for one specific domain.
-
-    The workflow consists of:
-    (Account creation)
-    - Create account key
-    - Register account and accept TOS
-    (Certificate actions)
-    - Select HTTP-01 within offered challenges by the CA server
-    - Set up http challenge resource
-    - Set up standalone web server
-    - Create domain private key and CSR
-    - Issue certificate
-    - Renew certificate
-    - Revoke certificate
-    (Account update actions)
-    - Change contact information
-    - Deactivate Account
-
-    """
-    # Create account key
-
-    acc_key = jose.JWKRSA(
-        key=rsa.generate_private_key(public_exponent=65537,
-                                     key_size=ACC_KEY_BITS,
-                                     backend=default_backend()))
-
-    # Register account and accept TOS
-
-    net = client.ClientNetwork(acc_key, user_agent=USER_AGENT)
-    directory = messages.Directory.from_json(net.get(DIRECTORY_URL).json())
-    client_acme = client.ClientV2(directory, net=net)
-
-    # Terms of Service URL is in client_acme.directory.meta.terms_of_service
-    # Registration Resource: regr
-    # Creates account with contact information.
-    email = ('fake@example.com')
-    regr = client_acme.new_account(
-        messages.NewRegistration.from_data(
-            email=email, terms_of_service_agreed=True))
-
-    # Create domain private key and CSR
-    pkey_pem, csr_pem = new_csr_comp(DOMAIN)
-
-    # Issue certificate
-
-    orderr = client_acme.new_order(csr_pem)
-
-    # Select HTTP-01 within offered challenges by the CA server
-    challb = select_http01_chall(orderr)
-
-    # The certificate is ready to be used in the variable "fullchain_pem".
-    fullchain_pem = perform_http01(client_acme, challb, orderr)
-
-    # Renew certificate
-
-    _, csr_pem = new_csr_comp(DOMAIN, pkey_pem)
-
-    orderr = client_acme.new_order(csr_pem)
-
-    challb = select_http01_chall(orderr)
-
-    # Performing challenge
-    fullchain_pem = perform_http01(client_acme, challb, orderr)
-
-    # Revoke certificate
-
-    fullchain_com = jose.ComparableX509(
-        OpenSSL.crypto.load_certificate(
-            OpenSSL.crypto.FILETYPE_PEM, fullchain_pem))
-
-    try:
-        client_acme.revoke(fullchain_com, 0)  # revocation reason = 0
-    except errors.ConflictError:
-        # Certificate already revoked.
-        pass
-
-    # Query registration status.
-    client_acme.net.account = regr
-    try:
-        regr = client_acme.query_registration(regr)
-    except errors.Error as err:
-        if err.typ == messages.OLD_ERROR_PREFIX + 'unauthorized' \
-                or err.typ == messages.ERROR_PREFIX + 'unauthorized':
-            # Status is deactivated.
-            pass
-        raise
-
-    # Change contact information
-
-    email = 'newfake@example.com'
-    regr = client_acme.update_registration(
-        regr.update(
-            body=regr.body.update(
-                contact=('mailto:' + email,)
-            )
-        )
-    )
-
-    # Deactivate account/registration
-
-    regr = client_acme.deactivate_registration(regr)
-
-
-if __name__ == "__main__":
-    example_http()

acme/pytest.ini

@@ -1,2 +0,0 @@
-[pytest]
-norecursedirs = .* build dist CVS _darcs {arch} *.egg

acme/setup.py

@@ -1,33 +1,41 @@
-from setuptools import setup
-from setuptools import find_packages
-from setuptools.command.test import test as TestCommand
 import sys
 
-version = '1.0.0.dev0'
+from setuptools import setup
+from setuptools import find_packages
+
+
+version = '0.16.0.dev0'
 
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [
     # load_pem_private/public_key (>=0.6)
     # rsa_recover_prime_factors (>=0.8)
-    'cryptography>=1.2.3',
-    # formerly known as acme.jose:
-    # 1.1.0+ is required to avoid the warnings described at
-    # https://github.com/certbot/josepy/issues/13.
-    'josepy>=1.1.0',
+    'cryptography>=0.8',
     # Connection.set_tlsext_host_name (>=0.13)
     'mock',
-    'PyOpenSSL>=0.13.1',
+    'PyOpenSSL>=0.13',
     'pyrfc3339',
     'pytz',
-    'requests[security]>=2.6.0',  # security extras added in 2.4.1
-    'requests-toolbelt>=0.3.0',
-    'setuptools',
-    'six>=1.9.0',  # needed for python_2_unicode_compatible
+    # requests>=2.10 is required to fix
+    # https://github.com/shazow/urllib3/issues/556. This requirement can be
+    # relaxed to 'requests[security]>=2.4.1', however, less useful errors
+    # will be raised for some network/SSL errors.
+    'requests[security]>=2.10',
+    # For pkg_resources. >=1.0 so pip resolves it to a version cryptography
+    # will tolerate; see #2599:
+    'setuptools>=1.0',
+    'six',
 ]
 
+# env markers cause problems with older pip and setuptools
+if sys.version_info < (2, 7):
+    install_requires.extend([
+        'argparse',
+        'ordereddict',
+    ])
+
 dev_extras = [
-    'pytest',
-    'pytest-xdist',
+    'nose',
     'tox',
 ]
 
@@ -37,21 +45,6 @@ docs_extras = [
 ]
 
 
-class PyTest(TestCommand):
-    user_options = []
-
-    def initialize_options(self):
-        TestCommand.initialize_options(self)
-        self.pytest_args = ''
-
-    def run_tests(self):
-        import shlex
-        # import here, cause outside the eggs aren't loaded
-        import pytest
-        errno = pytest.main(shlex.split(self.pytest_args))
-        sys.exit(errno)
-
-
 setup(
     name='acme',
     version=version,
@@ -60,20 +53,19 @@ setup(
     author="Certbot Project",
     author_email='client-dev@letsencrypt.org',
     license='Apache License 2.0',
-    python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*',
     classifiers=[
-        'Development Status :: 5 - Production/Stable',
+        'Development Status :: 3 - Alpha',
         'Intended Audience :: Developers',
         'License :: OSI Approved :: Apache Software License',
         'Programming Language :: Python',
         'Programming Language :: Python :: 2',
+        'Programming Language :: Python :: 2.6',
         'Programming Language :: Python :: 2.7',
         'Programming Language :: Python :: 3',
+        'Programming Language :: Python :: 3.3',
         'Programming Language :: Python :: 3.4',
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
-        'Programming Language :: Python :: 3.7',
-        'Programming Language :: Python :: 3.8',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Security',
     ],
@@ -85,7 +77,10 @@ setup(
         'dev': dev_extras,
         'docs': docs_extras,
     },
+    entry_points={
+        'console_scripts': [
+            'jws = acme.jose.jws:CLI.run',
+        ],
+    },
     test_suite='acme',
-    tests_require=["pytest"],
-    cmdclass={"test": PyTest},
 )

certbot-apache/certbot_apache/apache_util.py

@@ -1,107 +0,0 @@
-""" Utility functions for certbot-apache plugin """
-import binascii
-
-from certbot import util
-from certbot.compat import os
-
-
-def get_mod_deps(mod_name):
-    """Get known module dependencies.
-
-    .. note:: This does not need to be accurate in order for the client to
-        run.  This simply keeps things clean if the user decides to revert
-        changes.
-    .. warning:: If all deps are not included, it may cause incorrect parsing
-        behavior, due to enable_mod's shortcut for updating the parser's
-        currently defined modules (`.ApacheParser.add_mod`)
-        This would only present a major problem in extremely atypical
-        configs that use ifmod for the missing deps.
-
-    """
-    deps = {
-        "ssl": ["setenvif", "mime"]
-    }
-    return deps.get(mod_name, [])
-
-
-def get_file_path(vhost_path):
-    """Get file path from augeas_vhost_path.
-
-    Takes in Augeas path and returns the file name
-
-    :param str vhost_path: Augeas virtual host path
-
-    :returns: filename of vhost
-    :rtype: str
-
-    """
-    if not vhost_path or not vhost_path.startswith("/files/"):
-        return None
-
-    return _split_aug_path(vhost_path)[0]
-
-
-def get_internal_aug_path(vhost_path):
-    """Get the Augeas path for a vhost with the file path removed.
-
-    :param str vhost_path: Augeas virtual host path
-
-    :returns: Augeas path to vhost relative to the containing file
-    :rtype: str
-
-    """
-    return _split_aug_path(vhost_path)[1]
-
-
-def _split_aug_path(vhost_path):
-    """Splits an Augeas path into a file path and an internal path.
-
-    After removing "/files", this function splits vhost_path into the
-    file path and the remaining Augeas path.
-
-    :param str vhost_path: Augeas virtual host path
-
-    :returns: file path and internal Augeas path
-    :rtype: `tuple` of `str`
-
-    """
-    # Strip off /files
-    file_path = vhost_path[6:]
-    internal_path = []
-
-    # Remove components from the end of file_path until it becomes valid
-    while not os.path.exists(file_path):
-        file_path, _, internal_path_part = file_path.rpartition("/")
-        internal_path.append(internal_path_part)
-
-    return file_path, "/".join(reversed(internal_path))
-
-
-def parse_define_file(filepath, varname):
-    """ Parses Defines from a variable in configuration file
-
-    :param str filepath: Path of file to parse
-    :param str varname: Name of the variable
-
-    :returns: Dict of Define:Value pairs
-    :rtype: `dict`
-
-    """
-    return_vars = {}
-    # Get list of words in the variable
-    a_opts = util.get_var_from_file(varname, filepath).split()
-    for i, v in enumerate(a_opts):
-        # Handle Define statements and make sure it has an argument
-        if v == "-D" and len(a_opts) >= i+2:
-            var_parts = a_opts[i+1].partition("=")
-            return_vars[var_parts[0]] = var_parts[2]
-        elif len(v) > 2 and v.startswith("-D"):
-            # Found var with no whitespace separator
-            var_parts = v[2:].partition("=")
-            return_vars[var_parts[0]] = var_parts[2]
-    return return_vars
-
-
-def unique_id():
-    """ Returns an unique id to be used as a VirtualHost identifier"""
-    return binascii.hexlify(os.urandom(16)).decode("utf-8")

certbot-apache/certbot_apache/augeas_configurator.py

@@ -0,0 +1,222 @@
+"""Class of Augeas Configurators."""
+import logging
+
+
+from certbot import errors
+from certbot import reverter
+from certbot.plugins import common
+
+from certbot_apache import constants
+
+logger = logging.getLogger(__name__)
+
+
+class AugeasConfigurator(common.Plugin):
+    """Base Augeas Configurator class.
+
+    :ivar config: Configuration.
+    :type config: :class:`~certbot.interfaces.IConfig`
+
+    :ivar aug: Augeas object
+    :type aug: :class:`augeas.Augeas`
+
+    :ivar str save_notes: Human-readable configuration change notes
+    :ivar reverter: saves and reverts checkpoints
+    :type reverter: :class:`certbot.reverter.Reverter`
+
+    """
+    def __init__(self, *args, **kwargs):
+        super(AugeasConfigurator, self).__init__(*args, **kwargs)
+
+        # Placeholder for augeas
+        self.aug = None
+
+        self.save_notes = ""
+
+        # See if any temporary changes need to be recovered
+        # This needs to occur before VirtualHost objects are setup...
+        # because this will change the underlying configuration and potential
+        # vhosts
+        self.reverter = reverter.Reverter(self.config)
+
+    def init_augeas(self):
+        """ Initialize the actual Augeas instance """
+        import augeas
+        self.aug = augeas.Augeas(
+            # specify a directory to load our preferred lens from
+            loadpath=constants.AUGEAS_LENS_DIR,
+            # Do not save backup (we do it ourselves), do not load
+            # anything by default
+            flags=(augeas.Augeas.NONE |
+                   augeas.Augeas.NO_MODL_AUTOLOAD |
+                   augeas.Augeas.ENABLE_SPAN))
+        self.recovery_routine()
+
+    def check_parsing_errors(self, lens):
+        """Verify Augeas can parse all of the lens files.
+
+        :param str lens: lens to check for errors
+
+        :raises .errors.PluginError: If there has been an error in parsing with
+            the specified lens.
+
+        """
+        error_files = self.aug.match("/augeas//error")
+
+        for path in error_files:
+            # Check to see if it was an error resulting from the use of
+            # the httpd lens
+            lens_path = self.aug.get(path + "/lens")
+            # As aug.get may return null
+            if lens_path and lens in lens_path:
+                msg = (
+                    "There has been an error in parsing the file {0} on line {1}: "
+                    "{2}".format(
+                    # Strip off /augeas/files and /error
+                    path[13:len(path) - 6],
+                    self.aug.get(path + "/line"),
+                    self.aug.get(path + "/message")))
+                raise errors.PluginError(msg)
+
+    # TODO: Cleanup this function
+    def save(self, title=None, temporary=False):
+        """Saves all changes to the configuration files.
+
+        This function first checks for save errors, if none are found,
+        all configuration changes made will be saved. According to the
+        function parameters. If an exception is raised, a new checkpoint
+        was not created.
+
+        :param str title: The title of the save. If a title is given, the
+            configuration will be saved as a new checkpoint and put in a
+            timestamped directory.
+
+        :param bool temporary: Indicates whether the changes made will
+            be quickly reversed in the future (ie. challenges)
+
+        :raises .errors.PluginError: If there was an error in Augeas, in
+            an attempt to save the configuration, or an error creating a
+            checkpoint
+
+        """
+        save_state = self.aug.get("/augeas/save")
+        self.aug.set("/augeas/save", "noop")
+        # Existing Errors
+        ex_errs = self.aug.match("/augeas//error")
+        try:
+            # This is a noop save
+            self.aug.save()
+        except (RuntimeError, IOError):
+            self._log_save_errors(ex_errs)
+            # Erase Save Notes
+            self.save_notes = ""
+            raise errors.PluginError(
+                "Error saving files, check logs for more info.")
+
+        # Retrieve list of modified files
+        # Note: Noop saves can cause the file to be listed twice, I used a
+        # set to remove this possibility. This is a known augeas 0.10 error.
+        save_paths = self.aug.match("/augeas/events/saved")
+
+        # If the augeas tree didn't change, no files were saved and a backup
+        # should not be created
+        save_files = set()
+        if save_paths:
+            for path in save_paths:
+                save_files.add(self.aug.get(path)[6:])
+
+            try:
+                # Create Checkpoint
+                if temporary:
+                    self.reverter.add_to_temp_checkpoint(
+                        save_files, self.save_notes)
+                else:
+                    self.reverter.add_to_checkpoint(save_files,
+                                                    self.save_notes)
+            except errors.ReverterError as err:
+                raise errors.PluginError(str(err))
+
+        self.aug.set("/augeas/save", save_state)
+        self.save_notes = ""
+        self.aug.save()
+
+        # Force reload if files were modified
+        # This is needed to recalculate augeas directive span
+        if save_files:
+            for sf in save_files:
+                self.aug.remove("/files/"+sf)
+            self.aug.load()
+        if title and not temporary:
+            try:
+                self.reverter.finalize_checkpoint(title)
+            except errors.ReverterError as err:
+                raise errors.PluginError(str(err))
+
+    def _log_save_errors(self, ex_errs):
+        """Log errors due to bad Augeas save.
+
+        :param list ex_errs: Existing errors before save
+
+        """
+        # Check for the root of save problems
+        new_errs = self.aug.match("/augeas//error")
+        # logger.error("During Save - %s", mod_conf)
+        logger.error("Unable to save files: %s. Attempted Save Notes: %s",
+                     ", ".join(err[13:len(err) - 6] for err in new_errs
+                               # Only new errors caused by recent save
+                               if err not in ex_errs), self.save_notes)
+
+    # Wrapper functions for Reverter class
+    def recovery_routine(self):
+        """Revert all previously modified files.
+
+        Reverts all modified files that have not been saved as a checkpoint
+
+        :raises .errors.PluginError: If unable to recover the configuration
+
+        """
+        try:
+            self.reverter.recovery_routine()
+        except errors.ReverterError as err:
+            raise errors.PluginError(str(err))
+        # Need to reload configuration after these changes take effect
+        self.aug.load()
+
+    def revert_challenge_config(self):
+        """Used to cleanup challenge configurations.
+
+        :raises .errors.PluginError: If unable to revert the challenge config.
+
+        """
+        try:
+            self.reverter.revert_temporary_config()
+        except errors.ReverterError as err:
+            raise errors.PluginError(str(err))
+        self.aug.load()
+
+    def rollback_checkpoints(self, rollback=1):
+        """Rollback saved checkpoints.
+
+        :param int rollback: Number of checkpoints to revert
+
+        :raises .errors.PluginError: If there is a problem with the input or
+            the function is unable to correctly revert the configuration
+
+        """
+        try:
+            self.reverter.rollback_checkpoints(rollback)
+        except errors.ReverterError as err:
+            raise errors.PluginError(str(err))
+        self.aug.load()
+
+    def view_config_changes(self):
+        """Show all of the configuration changes that have taken place.
+
+        :raises .errors.PluginError: If there is a problem while processing
+            the checkpoints directories.
+
+        """
+        try:
+            self.reverter.view_config_changes()
+        except errors.ReverterError as err:
+            raise errors.PluginError(str(err))

certbot-apache/certbot_apache/augeas_lens/httpd.aug

@@ -44,134 +44,67 @@ autoload xfm
  *****************************************************************)
 let dels (s:string)     = del s s
 
-(* The continuation sequence that indicates that we should consider the
- * next line part of the current line *)
-let cont = /\\\\\r?\n/
-
-(* Whitespace within a line: space, tab, and the continuation sequence *)
-let ws = /[ \t]/ | cont
-
-(* Any possible character - '.' does not match \n *)
-let any = /(.|\n)/
-
-(* Any character preceded by a backslash *)
-let esc_any = /\\\\(.|\n)/
-
-(* Newline sequence - both for Unix and DOS newlines *)
-let nl = /\r?\n/
-
-(* Whitespace at the end of a line *)
-let eol = del (ws* . nl) "\n"
-
 (* deal with continuation lines *)
-let sep_spc             = del ws+ " "
-let sep_osp             = del ws* ""
-let sep_eq              = del (ws* . "=" . ws*) "="
+let sep_spc             = del /([ \t]+|[ \t]*\\\\\r?\n[ \t]*)+/ " "
+let sep_osp             = del /([ \t]*|[ \t]*\\\\\r?\n[ \t]*)*/ ""
+let sep_eq              = del /[ \t]*=[ \t]*/ "="
 
 let nmtoken             = /[a-zA-Z:_][a-zA-Z0-9:_.-]*/
 let word                = /[a-z][a-z0-9._-]*/i
 
-(* A complete line that is either just whitespace or a comment that only
- * contains whitespace *)
-let empty = [ del (ws* . /#?/ . ws* . nl) "\n" ]
-
+let eol                 = Util.doseol
+let empty               = Util.empty_dos
 let indent              = Util.indent
 
-(* A comment that is not just whitespace. We define it in terms of the
- * things that are not allowed as part of such a comment:
- *   1) Starts with whitespace
- *   2) Ends with whitespace, a backslash or \r
- *   3) Unescaped newlines
- *)
-let comment =
-  let comment_start = del (ws* . "#" . ws* ) "# " in
-  let unesc_eol = /[^\]?/ . nl in
-  let w = /[^\t\n\r \\]/ in
-  let r = /[\r\\]/ in
-  let s = /[\t\r ]/ in
-  (*
-   * we'd like to write
-   * let b = /\\\\/ in
-   * let t = /[\t\n\r ]/ in
-   * let x = b . (t? . (s|w)* ) in
-   * but the definition of b depends on commit 244c0edd in 1.9.0 and
-   * would make the lens unusable with versions before 1.9.0. So we write
-   * x out which works in older versions, too
-   *)
-  let x = /\\\\[\t\n\r ]?[^\n\\]*/ in
-  let line = ((r . s* . w|w|r) . (s|w)* . x*|(r.s* )?).w.(s*.w)* in
-  [ label "#comment" . comment_start . store line . eol ]
+let comment_val_re      = /([^ \t\r\n](.|\\\\\r?\n)*[^ \\\t\r\n]|[^ \t\r\n])/
+let comment             = [ label "#comment" . del /[ \t]*#[ \t]*/ "# "
+                          . store comment_val_re . eol ]
 
 (* borrowed from shellvars.aug *)
+let char_arg_dir  = /([^\\ '"{\t\r\n]|[^ '"{\t\r\n]+[^\\ \t\r\n])|\\\\"|\\\\'|\\\\ /
 let char_arg_sec  = /([^\\ '"\t\r\n>]|[^ '"\t\r\n>]+[^\\ \t\r\n>])|\\\\"|\\\\'|\\\\ /
 let char_arg_wl   = /([^\\ '"},\t\r\n]|[^ '"},\t\r\n]+[^\\ '"},\t\r\n])/
 
+let cdot = /\\\\./
+let cl = /\\\\\n/
 let dquot =
      let no_dquot = /[^"\\\r\n]/
-  in /"/ . (no_dquot|esc_any)* . /"/
+  in /"/ . (no_dquot|cdot|cl)* . /"/
 let dquot_msg =
      let no_dquot = /([^ \t"\\\r\n]|[^"\\\r\n]+[^ \t"\\\r\n])/
-  in /"/ . (no_dquot|esc_any)* . no_dquot
-
+  in /"/ . (no_dquot|cdot|cl)*
 let squot =
      let no_squot = /[^'\\\r\n]/
-  in /'/ . (no_squot|esc_any)* . /'/
+  in /'/ . (no_squot|cdot|cl)* . /'/
 let comp = /[<>=]?=/
 
 (******************************************************************
  *                            Attributes
  *****************************************************************)
 
-(* The arguments for a directive come in two flavors: quoted with single or
- * double quotes, or bare. Bare arguments may not start with a single or
- * double quote; since we also treat "word lists" special, i.e. lists
- * enclosed in curly braces, bare arguments may not start with those,
- * either.
- *
- * Bare arguments may not contain unescaped spaces, but we allow escaping
- * with '\\'. Quoted arguments can contain anything, though the quote must
- * be escaped with '\\'.
- *)
-let bare = /([^{"' \t\n\r]|\\\\.)([^ \t\n\r]|\\\\.)*[^ \t\n\r\\]|[^{"' \t\n\r\\]/
-
-let arg_quoted = [ label "arg" . store (dquot|squot) ]
-let arg_bare = [ label "arg" . store bare ]
-
+let arg_dir = [ label "arg" . store (char_arg_dir+|dquot|squot) ]
 (* message argument starts with " but ends at EOL *)
 let arg_dir_msg = [ label "arg" . store dquot_msg ]
+let arg_sec = [ label "arg" . store (char_arg_sec+|comp|dquot|squot) ]
 let arg_wl  = [ label "arg" . store (char_arg_wl+|dquot|squot) ]
 
 (* comma-separated wordlist as permitted in the SSLRequire directive *)
 let arg_wordlist =
-     let wl_start = dels "{" in
-     let wl_end   = dels "}" in
+     let wl_start = Util.del_str "{" in
+     let wl_end   = Util.del_str "}" in
      let wl_sep   = del /[ \t]*,[ \t]*/ ", "
   in [ label "wordlist" . wl_start . arg_wl . (wl_sep . arg_wl)* . wl_end ]
 
 let argv (l:lens) = l . (sep_spc . l)*
 
-(* the arguments of a directive. We use this once we have parsed the name
- * of the directive, and the space right after it. When dir_args is used,
- * we also know that we have at least one argument. We need to be careful
- * with the spacing between arguments: quoted arguments and word lists do
- * not need to have space between them, but bare arguments do.
- *
- * Apache apparently is also happy if the last argument starts with a double
- * quote, but has no corresponding closing duoble quote, which is what
- * arg_dir_msg handles
- *)
-let dir_args =
-  let arg_nospc = arg_quoted|arg_wordlist in
-  (arg_bare . sep_spc | arg_nospc . sep_osp)* . (arg_bare|arg_nospc|arg_dir_msg)
-
 let directive =
-  [ indent . label "directive" . store word .  (sep_spc . dir_args)? . eol ]
-
-let arg_sec = [ label "arg" . store (char_arg_sec+|comp|dquot|squot) ]
+    (* arg_dir_msg may be the last or only argument *)
+     let dir_args = (argv (arg_dir|arg_wordlist) . (sep_spc . arg_dir_msg)?) | arg_dir_msg
+  in [ indent . label "directive" . store word .  (sep_spc . dir_args)? . eol ]
 
 let section (body:lens) =
     (* opt_eol includes empty lines *)
-    let opt_eol = del /([ \t]*#?[ \t]*\r?\n)*/ "\n" in
+    let opt_eol = del /([ \t]*#?\r?\n)*/ "\n" in
     let inner = (sep_spc . argv arg_sec)? . sep_osp .
              dels ">" . opt_eol . ((body|comment) . (body|empty|comment)*)? .
              indent . dels "</" in
@@ -200,7 +133,6 @@ let filter = (incl "/etc/apache2/apache2.conf") .
              (incl "/etc/httpd/conf.d/*.conf") .
              (incl "/etc/httpd/httpd.conf") .
              (incl "/etc/httpd/conf/httpd.conf") .
-             (incl "/etc/httpd/conf.modules.d/*.conf") .
              Util.stdexcl
 
 let xfm = transform lns filter

certbot-apache/certbot_apache/centos-options-ssl-apache.conf

@@ -8,7 +8,7 @@ SSLEngine on
 
 # Intermediate configuration, tweak to your needs
 SSLProtocol             all -SSLv2 -SSLv3
-SSLCipherSuite          ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS
+SSLCipherSuite          ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
 SSLHonorCipherOrder     on
 
 SSLOptions +StrictRequire

certbot-apache/certbot_apache/constants.py

@@ -1,6 +1,151 @@
 """Apache plugin constants."""
 import pkg_resources
+from certbot import util
 
+CLI_DEFAULTS_DEFAULT = dict(
+    server_root="/etc/apache2",
+    vhost_root="/etc/apache2/sites-available",
+    vhost_files="*",
+    logs_root="/var/log/apache2",
+    version_cmd=['apache2ctl', '-v'],
+    define_cmd=['apache2ctl', '-t', '-D', 'DUMP_RUN_CFG'],
+    restart_cmd=['apache2ctl', 'graceful'],
+    conftest_cmd=['apache2ctl', 'configtest'],
+    enmod=None,
+    dismod=None,
+    le_vhost_ext="-le-ssl.conf",
+    handle_mods=False,
+    handle_sites=False,
+    challenge_location="/etc/apache2",
+    MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
+        "certbot_apache", "options-ssl-apache.conf")
+)
+CLI_DEFAULTS_DEBIAN = dict(
+    server_root="/etc/apache2",
+    vhost_root="/etc/apache2/sites-available",
+    vhost_files="*",
+    logs_root="/var/log/apache2",
+    version_cmd=['apache2ctl', '-v'],
+    define_cmd=['apache2ctl', '-t', '-D', 'DUMP_RUN_CFG'],
+    restart_cmd=['apache2ctl', 'graceful'],
+    conftest_cmd=['apache2ctl', 'configtest'],
+    enmod="a2enmod",
+    dismod="a2dismod",
+    le_vhost_ext="-le-ssl.conf",
+    handle_mods=True,
+    handle_sites=True,
+    challenge_location="/etc/apache2",
+    MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
+        "certbot_apache", "options-ssl-apache.conf")
+)
+CLI_DEFAULTS_CENTOS = dict(
+    server_root="/etc/httpd",
+    vhost_root="/etc/httpd/conf.d",
+    vhost_files="*.conf",
+    logs_root="/var/log/httpd",
+    version_cmd=['apachectl', '-v'],
+    define_cmd=['apachectl', '-t', '-D', 'DUMP_RUN_CFG'],
+    restart_cmd=['apachectl', 'graceful'],
+    conftest_cmd=['apachectl', 'configtest'],
+    enmod=None,
+    dismod=None,
+    le_vhost_ext="-le-ssl.conf",
+    handle_mods=False,
+    handle_sites=False,
+    challenge_location="/etc/httpd/conf.d",
+    MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
+        "certbot_apache", "centos-options-ssl-apache.conf")
+)
+CLI_DEFAULTS_GENTOO = dict(
+    server_root="/etc/apache2",
+    vhost_root="/etc/apache2/vhosts.d",
+    vhost_files="*.conf",
+    logs_root="/var/log/apache2",
+    version_cmd=['/usr/sbin/apache2', '-v'],
+    define_cmd=['apache2ctl', 'virtualhosts'],
+    restart_cmd=['apache2ctl', 'graceful'],
+    conftest_cmd=['apache2ctl', 'configtest'],
+    enmod=None,
+    dismod=None,
+    le_vhost_ext="-le-ssl.conf",
+    handle_mods=False,
+    handle_sites=False,
+    challenge_location="/etc/apache2/vhosts.d",
+    MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
+        "certbot_apache", "options-ssl-apache.conf")
+)
+CLI_DEFAULTS_DARWIN = dict(
+    server_root="/etc/apache2",
+    vhost_root="/etc/apache2/other",
+    vhost_files="*.conf",
+    logs_root="/var/log/apache2",
+    version_cmd=['/usr/sbin/httpd', '-v'],
+    define_cmd=['/usr/sbin/httpd', '-t', '-D', 'DUMP_RUN_CFG'],
+    restart_cmd=['apachectl', 'graceful'],
+    conftest_cmd=['apachectl', 'configtest'],
+    enmod=None,
+    dismod=None,
+    le_vhost_ext="-le-ssl.conf",
+    handle_mods=False,
+    handle_sites=False,
+    challenge_location="/etc/apache2/other",
+    MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
+        "certbot_apache", "options-ssl-apache.conf")
+)
+CLI_DEFAULTS_SUSE = dict(
+    server_root="/etc/apache2",
+    vhost_root="/etc/apache2/vhosts.d",
+    vhost_files="*.conf",
+    logs_root="/var/log/apache2",
+    version_cmd=['apache2ctl', '-v'],
+    define_cmd=['apache2ctl', '-t', '-D', 'DUMP_RUN_CFG'],
+    restart_cmd=['apache2ctl', 'graceful'],
+    conftest_cmd=['apache2ctl', 'configtest'],
+    enmod="a2enmod",
+    dismod="a2dismod",
+    le_vhost_ext="-le-ssl.conf",
+    handle_mods=False,
+    handle_sites=False,
+    challenge_location="/etc/apache2/vhosts.d",
+    MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
+        "certbot_apache", "options-ssl-apache.conf")
+)
+CLI_DEFAULTS_ARCH = dict(
+    server_root="/etc/httpd",
+    vhost_root="/etc/httpd/conf",
+    vhost_files="*.conf",
+    logs_root="/var/log/httpd",
+    version_cmd=['apachectl', '-v'],
+    define_cmd=['apachectl', '-t', '-D', 'DUMP_RUN_CFG'],
+    restart_cmd=['apachectl', 'graceful'],
+    conftest_cmd=['apachectl', 'configtest'],
+    enmod=None,
+    dismod=None,
+    le_vhost_ext="-le-ssl.conf",
+    handle_mods=False,
+    handle_sites=False,
+    challenge_location="/etc/httpd/conf",
+    MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
+        "certbot_apache", "options-ssl-apache.conf")
+)
+CLI_DEFAULTS = {
+    "default": CLI_DEFAULTS_DEFAULT,
+    "debian": CLI_DEFAULTS_DEBIAN,
+    "ubuntu": CLI_DEFAULTS_DEBIAN,
+    "centos": CLI_DEFAULTS_CENTOS,
+    "centos linux": CLI_DEFAULTS_CENTOS,
+    "fedora": CLI_DEFAULTS_CENTOS,
+    "red hat enterprise linux server": CLI_DEFAULTS_CENTOS,
+    "rhel": CLI_DEFAULTS_CENTOS,
+    "amazon": CLI_DEFAULTS_CENTOS,
+    "gentoo": CLI_DEFAULTS_GENTOO,
+    "gentoo base system": CLI_DEFAULTS_GENTOO,
+    "darwin": CLI_DEFAULTS_DARWIN,
+    "opensuse": CLI_DEFAULTS_SUSE,
+    "suse": CLI_DEFAULTS_SUSE,
+    "arch": CLI_DEFAULTS_ARCH,
+}
+"""CLI defaults."""
 
 MOD_SSL_CONF_DEST = "options-ssl-apache.conf"
 """Name of the mod_ssl config file as saved in `IConfig.config_dir`."""
@@ -9,7 +154,6 @@ MOD_SSL_CONF_DEST = "options-ssl-apache.conf"
 UPDATED_MOD_SSL_CONF_DIGEST = ".updated-options-ssl-apache-conf-digest.txt"
 """Name of the hash of the updated or informed mod_ssl_conf as saved in `IConfig.config_dir`."""
 
-# NEVER REMOVE A SINGLE HASH FROM THIS LIST UNLESS YOU KNOW EXACTLY WHAT YOU ARE DOING!
 ALL_SSL_OPTIONS_HASHES = [
     '2086bca02db48daf93468332543c60ac6acdb6f0b58c7bfdf578a5d47092f82a',
     '4844d36c9a0f587172d9fa10f4f1c9518e3bcfa1947379f155e16a70a728c21a',
@@ -17,12 +161,6 @@ ALL_SSL_OPTIONS_HASHES = [
     '4066b90268c03c9ba0201068eaa39abbc02acf9558bb45a788b630eb85dadf27',
     'f175e2e7c673bd88d0aff8220735f385f916142c44aa83b09f1df88dd4767a88',
     'cfdd7c18d2025836ea3307399f509cfb1ebf2612c87dd600a65da2a8e2f2797b',
-    '80720bd171ccdc2e6b917ded340defae66919e4624962396b992b7218a561791',
-    'c0c022ea6b8a51ecc8f1003d0a04af6c3f2bc1c3ce506b3c2dfc1f11ef931082',
-    '717b0a89f5e4c39b09a42813ac6e747cfbdeb93439499e73f4f70a1fe1473f20',
-    '0fcdc81280cd179a07ec4d29d3595068b9326b455c488de4b09f585d5dafc137',
-    '86cc09ad5415cd6d5f09a947fe2501a9344328b1e8a8b458107ea903e80baa6c',
-    '06675349e457eae856120cdebb564efe546f0b87399f2264baeb41e442c724c7',
 ]
 """SHA256 hashes of the contents of previous versions of all versions of MOD_SSL_CONF_SRC"""
 
@@ -54,15 +192,38 @@ UIR_ARGS = ["always", "set", "Content-Security-Policy",
 HEADER_ARGS = {"Strict-Transport-Security": HSTS_ARGS,
                "Upgrade-Insecure-Requests": UIR_ARGS}
 
-AUTOHSTS_STEPS = [60, 300, 900, 3600, 21600, 43200, 86400]
-"""AutoHSTS increase steps: 1min, 5min, 15min, 1h, 6h, 12h, 24h"""
 
-AUTOHSTS_PERMANENT = 31536000
-"""Value for the last max-age of HSTS"""
+def os_constant(key):
+    """
+    Get a constant value for operating system
 
-AUTOHSTS_FREQ = 172800
-"""Minimum time since last increase to perform a new one: 48h"""
+    :param key: name of cli constant
+    :return: value of constant for active os
+    """
 
-MANAGED_COMMENT = "DO NOT REMOVE - Managed by Certbot"
-MANAGED_COMMENT_ID = MANAGED_COMMENT+", VirtualHost id: {0}"
-"""Managed by Certbot comments and the VirtualHost identification template"""
+    os_info = util.get_os_info()
+    try:
+        constants = CLI_DEFAULTS[os_info[0].lower()]
+    except KeyError:
+        constants = os_like_constants()
+        if not constants:
+            constants = CLI_DEFAULTS["default"]
+    return constants[key]
+
+
+def os_like_constants():
+    """
+    Try to get constants for distribution with
+    similar layout and configuration, indicated by
+    /etc/os-release variable "LIKE"
+
+    :returns: Constants dictionary
+    :rtype: `dict`
+    """
+
+    os_like = util.get_systemd_os_like()
+    if os_like:
+        for os_name in os_like:
+            if os_name in CLI_DEFAULTS.keys():
+                return CLI_DEFAULTS[os_name]
+    return {}

certbot-apache/certbot_apache/display_ops.py

@@ -1,54 +1,22 @@
 """Contains UI methods for Apache operations."""
 import logging
+import os
 
 import zope.component
 
-import certbot.display.util as display_util
 from certbot import errors
 from certbot import interfaces
-from certbot.compat import os
+
+import certbot.display.util as display_util
+
 
 logger = logging.getLogger(__name__)
 
 
-def select_vhost_multiple(vhosts):
-    """Select multiple Vhosts to install the certificate for
-
-    :param vhosts: Available Apache VirtualHosts
-    :type vhosts: :class:`list` of type `~obj.Vhost`
-
-    :returns: List of VirtualHosts
-    :rtype: :class:`list`of type `~obj.Vhost`
-    """
-    if not vhosts:
-        return list()
-    tags_list = [vhost.display_repr()+"\n" for vhost in vhosts]
-    # Remove the extra newline from the last entry
-    if tags_list:
-        tags_list[-1] = tags_list[-1][:-1]
-    code, names = zope.component.getUtility(interfaces.IDisplay).checklist(
-        "Which VirtualHosts would you like to install the wildcard certificate for?",
-        tags=tags_list, force_interactive=True)
-    if code == display_util.OK:
-        return_vhosts = _reversemap_vhosts(names, vhosts)
-        return return_vhosts
-    return []
-
-def _reversemap_vhosts(names, vhosts):
-    """Helper function for select_vhost_multiple for mapping string
-    representations back to actual vhost objects"""
-    return_vhosts = list()
-
-    for selection in names:
-        for vhost in vhosts:
-            if vhost.display_repr().strip() == selection.strip():
-                return_vhosts.append(vhost)
-    return return_vhosts
-
 def select_vhost(domain, vhosts):
     """Select an appropriate Apache Vhost.
 
-    :param vhosts: Available Apache VirtualHosts
+    :param vhosts: Available Apache Virtual Hosts
     :type vhosts: :class:`list` of type `~obj.Vhost`
 
     :returns: VirtualHost or `None`
@@ -57,10 +25,13 @@ def select_vhost(domain, vhosts):
     """
     if not vhosts:
         return None
-    code, tag = _vhost_menu(domain, vhosts)
-    if code == display_util.OK:
-        return vhosts[tag]
-    return None
+    while True:
+        code, tag = _vhost_menu(domain, vhosts)
+        if code == display_util.OK:
+            return vhosts[tag]
+        else:
+            return None
+
 
 def _vhost_menu(domain, vhosts):
     """Select an appropriate Apache Vhost.
@@ -90,7 +61,7 @@ def _vhost_menu(domain, vhosts):
     for vhost in vhosts:
         if len(vhost.get_names()) == 1:
             disp_name = next(iter(vhost.get_names()))
-        elif not vhost.get_names():
+        elif len(vhost.get_names()) == 0:
             disp_name = ""
         else:
             disp_name = "Multiple Names"
@@ -110,15 +81,14 @@ def _vhost_menu(domain, vhosts):
         code, tag = zope.component.getUtility(interfaces.IDisplay).menu(
             "We were unable to find a vhost with a ServerName "
             "or Address of {0}.{1}Which virtual host would you "
-            "like to choose?".format(domain, os.linesep),
+            "like to choose?\n(note: conf files with multiple "
+            "vhosts are not yet supported)".format(domain, os.linesep),
             choices, force_interactive=True)
     except errors.MissingCommandlineFlag:
-        msg = (
-            "Encountered vhost ambiguity when trying to find a vhost for "
-            "{0} but was unable to ask for user "
-            "guidance in non-interactive mode. Certbot may need "
-            "vhosts to be explicitly labelled with ServerName or "
-            "ServerAlias directives.".format(domain))
+        msg = ("Encountered vhost ambiguity but unable to ask for user guidance in "
+               "non-interactive mode. Currently Certbot needs each vhost to be "
+               "in its own conf file, and may need vhosts to be explicitly "
+               "labelled with ServerName or ServerAlias directives.")
         logger.warning(msg)
         raise errors.MissingCommandlineFlag(msg)
 

certbot-apache/certbot_apache/entrypoint.py

@@ -1,69 +0,0 @@
-""" Entry point for Apache Plugin """
-# Pylint does not like disutils.version when running inside a venv.
-# See: https://github.com/PyCQA/pylint/issues/73
-from distutils.version import LooseVersion  # pylint: disable=no-name-in-module,import-error
-
-from certbot import util
-
-from certbot_apache import configurator
-from certbot_apache import override_arch
-from certbot_apache import override_fedora
-from certbot_apache import override_darwin
-from certbot_apache import override_debian
-from certbot_apache import override_centos
-from certbot_apache import override_gentoo
-from certbot_apache import override_suse
-
-OVERRIDE_CLASSES = {
-    "arch": override_arch.ArchConfigurator,
-    "cloudlinux": override_centos.CentOSConfigurator,
-    "darwin": override_darwin.DarwinConfigurator,
-    "debian": override_debian.DebianConfigurator,
-    "ubuntu": override_debian.DebianConfigurator,
-    "centos": override_centos.CentOSConfigurator,
-    "centos linux": override_centos.CentOSConfigurator,
-    "fedora_old": override_centos.CentOSConfigurator,
-    "fedora": override_fedora.FedoraConfigurator,
-    "linuxmint": override_debian.DebianConfigurator,
-    "ol": override_centos.CentOSConfigurator,
-    "oracle": override_centos.CentOSConfigurator,
-    "redhatenterpriseserver": override_centos.CentOSConfigurator,
-    "red hat enterprise linux server": override_centos.CentOSConfigurator,
-    "rhel": override_centos.CentOSConfigurator,
-    "amazon": override_centos.CentOSConfigurator,
-    "gentoo": override_gentoo.GentooConfigurator,
-    "gentoo base system": override_gentoo.GentooConfigurator,
-    "opensuse": override_suse.OpenSUSEConfigurator,
-    "suse": override_suse.OpenSUSEConfigurator,
-    "sles": override_suse.OpenSUSEConfigurator,
-    "scientific": override_centos.CentOSConfigurator,
-    "scientific linux": override_centos.CentOSConfigurator,
-}
-
-
-def get_configurator():
-    """ Get correct configurator class based on the OS fingerprint """
-    os_name, os_version = util.get_os_info()
-    os_name = os_name.lower()
-    override_class = None
-
-    # Special case for older Fedora versions
-    if os_name == 'fedora' and LooseVersion(os_version) < LooseVersion('29'):
-        os_name = 'fedora_old'
-
-    try:
-        override_class = OVERRIDE_CLASSES[os_name]
-    except KeyError:
-        # OS not found in the list
-        os_like = util.get_systemd_os_like()
-        if os_like:
-            for os_name in os_like:
-                if os_name in OVERRIDE_CLASSES.keys():
-                    override_class = OVERRIDE_CLASSES[os_name]
-        if not override_class:
-            # No override class found, return the generic configurator
-            override_class = configurator.ApacheConfigurator
-    return override_class
-
-
-ENTRYPOINT = get_configurator()

certbot-apache/certbot_apache/http_01.py

@@ -1,210 +0,0 @@
-"""A class that performs HTTP-01 challenges for Apache"""
-import logging
-
-from acme.magic_typing import List, Set  # pylint: disable=unused-import, no-name-in-module
-
-from certbot import errors
-from certbot.compat import os
-from certbot.compat import filesystem
-from certbot.plugins import common
-
-from certbot_apache.obj import VirtualHost  # pylint: disable=unused-import
-from certbot_apache.parser import get_aug_path
-
-logger = logging.getLogger(__name__)
-
-
-class ApacheHttp01(common.ChallengePerformer):
-    """Class that performs HTTP-01 challenges within the Apache configurator."""
-
-    CONFIG_TEMPLATE22_PRE = """\
-        RewriteEngine on
-        RewriteRule ^/\\.well-known/acme-challenge/([A-Za-z0-9-_=]+)$ {0}/$1 [L]
-
-    """
-    CONFIG_TEMPLATE22_POST = """\
-        <Directory {0}>
-            Order Allow,Deny
-            Allow from all
-        </Directory>
-        <Location /.well-known/acme-challenge>
-            Order Allow,Deny
-            Allow from all
-        </Location>
-    """
-
-    CONFIG_TEMPLATE24_PRE = """\
-        RewriteEngine on
-        RewriteRule ^/\\.well-known/acme-challenge/([A-Za-z0-9-_=]+)$ {0}/$1 [END]
-    """
-    CONFIG_TEMPLATE24_POST = """\
-        <Directory {0}>
-            Require all granted
-        </Directory>
-        <Location /.well-known/acme-challenge>
-            Require all granted
-        </Location>
-    """
-
-    def __init__(self, *args, **kwargs):
-        super(ApacheHttp01, self).__init__(*args, **kwargs)
-        self.challenge_conf_pre = os.path.join(
-            self.configurator.conf("challenge-location"),
-            "le_http_01_challenge_pre.conf")
-        self.challenge_conf_post = os.path.join(
-            self.configurator.conf("challenge-location"),
-            "le_http_01_challenge_post.conf")
-        self.challenge_dir = os.path.join(
-            self.configurator.config.work_dir,
-            "http_challenges")
-        self.moded_vhosts = set()  # type: Set[VirtualHost]
-
-    def perform(self):
-        """Perform all HTTP-01 challenges."""
-        if not self.achalls:
-            return []
-        # Save any changes to the configuration as a precaution
-        # About to make temporary changes to the config
-        self.configurator.save("Changes before challenge setup", True)
-
-        self.configurator.ensure_listen(str(
-            self.configurator.config.http01_port))
-        self.prepare_http01_modules()
-
-        responses = self._set_up_challenges()
-
-        self._mod_config()
-        # Save reversible changes
-        self.configurator.save("HTTP Challenge", True)
-
-        return responses
-
-    def prepare_http01_modules(self):
-        """Make sure that we have the needed modules available for http01"""
-
-        if self.configurator.conf("handle-modules"):
-            needed_modules = ["rewrite"]
-            if self.configurator.version < (2, 4):
-                needed_modules.append("authz_host")
-            else:
-                needed_modules.append("authz_core")
-            for mod in needed_modules:
-                if mod + "_module" not in self.configurator.parser.modules:
-                    self.configurator.enable_mod(mod, temp=True)
-
-    def _mod_config(self):
-        selected_vhosts = []  # type: List[VirtualHost]
-        http_port = str(self.configurator.config.http01_port)
-        for chall in self.achalls:
-            # Search for matching VirtualHosts
-            for vh in self._matching_vhosts(chall.domain):
-                selected_vhosts.append(vh)
-
-        # Ensure that we have one or more VirtualHosts that we can continue
-        # with. (one that listens to port configured with --http-01-port)
-        found = False
-        for vhost in selected_vhosts:
-            if any(a.is_wildcard() or a.get_port() == http_port for a in vhost.addrs):
-                found = True
-
-        if not found:
-            for vh in self._relevant_vhosts():
-                selected_vhosts.append(vh)
-
-        # Add the challenge configuration
-        for vh in selected_vhosts:
-            self._set_up_include_directives(vh)
-
-        self.configurator.reverter.register_file_creation(
-            True, self.challenge_conf_pre)
-        self.configurator.reverter.register_file_creation(
-            True, self.challenge_conf_post)
-
-        if self.configurator.version < (2, 4):
-            config_template_pre = self.CONFIG_TEMPLATE22_PRE
-            config_template_post = self.CONFIG_TEMPLATE22_POST
-        else:
-            config_template_pre = self.CONFIG_TEMPLATE24_PRE
-            config_template_post = self.CONFIG_TEMPLATE24_POST
-
-        config_text_pre = config_template_pre.format(self.challenge_dir)
-        config_text_post = config_template_post.format(self.challenge_dir)
-
-        logger.debug("writing a pre config file with text:\n %s", config_text_pre)
-        with open(self.challenge_conf_pre, "w") as new_conf:
-            new_conf.write(config_text_pre)
-        logger.debug("writing a post config file with text:\n %s", config_text_post)
-        with open(self.challenge_conf_post, "w") as new_conf:
-            new_conf.write(config_text_post)
-
-    def _matching_vhosts(self, domain):
-        """Return all VirtualHost objects that have the requested domain name or
-        a wildcard name that would match the domain in ServerName or ServerAlias
-        directive.
-        """
-        matching_vhosts = []
-        for vhost in self.configurator.vhosts:
-            if self.configurator.domain_in_names(vhost.get_names(), domain):
-                # domain_in_names also matches the exact names, so no need
-                # to check "domain in vhost.get_names()" explicitly here
-                matching_vhosts.append(vhost)
-
-        return matching_vhosts
-
-    def _relevant_vhosts(self):
-        http01_port = str(self.configurator.config.http01_port)
-        relevant_vhosts = []
-        for vhost in self.configurator.vhosts:
-            if any(a.is_wildcard() or a.get_port() == http01_port for a in vhost.addrs):
-                if not vhost.ssl:
-                    relevant_vhosts.append(vhost)
-        if not relevant_vhosts:
-            raise errors.PluginError(
-                "Unable to find a virtual host listening on port {0} which is"
-                " currently needed for Certbot to prove to the CA that you"
-                " control your domain. Please add a virtual host for port"
-                " {0}.".format(http01_port))
-
-        return relevant_vhosts
-
-    def _set_up_challenges(self):
-        if not os.path.isdir(self.challenge_dir):
-            filesystem.makedirs(self.challenge_dir, 0o755)
-
-        responses = []
-        for achall in self.achalls:
-            responses.append(self._set_up_challenge(achall))
-
-        return responses
-
-    def _set_up_challenge(self, achall):
-        response, validation = achall.response_and_validation()
-
-        name = os.path.join(self.challenge_dir, achall.chall.encode("token"))
-
-        self.configurator.reverter.register_file_creation(True, name)
-        with open(name, 'wb') as f:
-            f.write(validation.encode())
-        filesystem.chmod(name, 0o644)
-
-        return response
-
-    def _set_up_include_directives(self, vhost):
-        """Includes override configuration to the beginning and to the end of
-        VirtualHost. Note that this include isn't added to Augeas search tree"""
-
-        if vhost not in self.moded_vhosts:
-            logger.debug(
-                "Adding a temporary challenge validation Include for name: %s " +
-                "in: %s", vhost.name, vhost.filep)
-            self.configurator.parser.add_dir_beginning(
-                vhost.path, "Include", self.challenge_conf_pre)
-            self.configurator.parser.add_dir(
-                vhost.path, "Include", self.challenge_conf_post)
-
-            if not vhost.enabled:
-                self.configurator.parser.add_dir(
-                    get_aug_path(self.configurator.parser.loc["default"]),
-                    "Include", vhost.filep)
-
-            self.moded_vhosts.add(vhost)

certbot-apache/certbot_apache/obj.py

@@ -1,7 +1,6 @@
 """Module contains classes used by the Apache Configurator."""
 import re
 
-from acme.magic_typing import Set # pylint: disable=unused-import, no-name-in-module
 from certbot.plugins import common
 
 
@@ -26,7 +25,7 @@ class Addr(common.Addr):
     def __repr__(self):
         return "certbot_apache.obj.Addr(" + repr(self.tup) + ")"
 
-    def __hash__(self):  # pylint: disable=useless-super-delegation
+    def __hash__(self):
         # Python 3 requires explicit overridden for __hash__ if __eq__ or
         # __cmp__ is overridden. See https://bugs.python.org/issue2235
         return super(Addr, self).__hash__()
@@ -47,7 +46,8 @@ class Addr(common.Addr):
             return 0
         elif self.get_addr() == "*":
             return 1
-        return 2
+        else:
+            return 2
 
     def conflicts(self, addr):
         r"""Returns if address could conflict with correct function of self.
@@ -98,7 +98,7 @@ class Addr(common.Addr):
         return self.get_addr_obj(port)
 
 
-class VirtualHost(object):
+class VirtualHost(object):  # pylint: disable=too-few-public-methods
     """Represents an Apache Virtualhost.
 
     :ivar str filep: file path of VH
@@ -126,6 +126,7 @@ class VirtualHost(object):
     def __init__(self, filep, path, addrs, ssl, enabled, name=None,
                  aliases=None, modmacro=False, ancestor=None):
 
+        # pylint: disable=too-many-arguments
         """Initialize a VH."""
         self.filep = filep
         self.path = path
@@ -139,7 +140,7 @@ class VirtualHost(object):
 
     def get_names(self):
         """Return a set of all names."""
-        all_names = set()  # type: Set[str]
+        all_names = set()
         all_names.update(self.aliases)
         # Strip out any scheme:// and <port> field from servername
         if self.name is not None:
@@ -166,19 +167,6 @@ class VirtualHost(object):
                 active="Yes" if self.enabled else "No",
                 modmacro="Yes" if self.modmacro else "No"))
 
-    def display_repr(self):
-        """Return a representation of VHost to be used in dialog"""
-        return (
-            "File: {filename}\n"
-            "Addresses: {addrs}\n"
-            "Names: {names}\n"
-            "HTTPS: {https}\n".format(
-                filename=self.filep,
-                addrs=", ".join(str(addr) for addr in self.addrs),
-                names=", ".join(self.get_names()),
-                https="Yes" if self.ssl else "No"))
-
-
     def __eq__(self, other):
         if isinstance(other, self.__class__):
             return (self.filep == other.filep and self.path == other.path and
@@ -250,7 +238,7 @@ class VirtualHost(object):
 
         # already_found acts to keep everything very conservative.
         # Don't allow multiple ip:ports in same set.
-        already_found = set()  # type: Set[str]
+        already_found = set()
 
         for addr in vhost.addrs:
             for local_addr in self.addrs:

certbot-apache/certbot_apache/options-ssl-apache.conf

@@ -8,7 +8,7 @@ SSLEngine on
 
 # Intermediate configuration, tweak to your needs
 SSLProtocol             all -SSLv2 -SSLv3
-SSLCipherSuite          ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS
+SSLCipherSuite          ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
 SSLHonorCipherOrder     on
 SSLCompression          off
 

certbot-apache/certbot_apache/override_arch.py

@@ -1,31 +0,0 @@
-""" Distribution specific override class for Arch Linux """
-import pkg_resources
-
-import zope.interface
-
-from certbot import interfaces
-
-from certbot_apache import configurator
-
-@zope.interface.provider(interfaces.IPluginFactory)
-class ArchConfigurator(configurator.ApacheConfigurator):
-    """Arch Linux specific ApacheConfigurator override class"""
-
-    OS_DEFAULTS = dict(
-        server_root="/etc/httpd",
-        vhost_root="/etc/httpd/conf",
-        vhost_files="*.conf",
-        logs_root="/var/log/httpd",
-        ctl="apachectl",
-        version_cmd=['apachectl', '-v'],
-        restart_cmd=['apachectl', 'graceful'],
-        conftest_cmd=['apachectl', 'configtest'],
-        enmod=None,
-        dismod=None,
-        le_vhost_ext="-le-ssl.conf",
-        handle_modules=False,
-        handle_sites=False,
-        challenge_location="/etc/httpd/conf",
-        MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
-    )

certbot-apache/certbot_apache/override_centos.py

@@ -1,217 +0,0 @@
-""" Distribution specific override class for CentOS family (RHEL, Fedora) """
-import logging
-
-import pkg_resources
-import zope.interface
-
-from certbot import errors
-from certbot import interfaces
-from certbot import util
-from certbot.errors import MisconfigurationError
-
-from acme.magic_typing import List  # pylint: disable=unused-import, no-name-in-module
-
-from certbot_apache import apache_util
-from certbot_apache import configurator
-from certbot_apache import parser
-
-
-logger = logging.getLogger(__name__)
-
-
-@zope.interface.provider(interfaces.IPluginFactory)
-class CentOSConfigurator(configurator.ApacheConfigurator):
-    """CentOS specific ApacheConfigurator override class"""
-
-    OS_DEFAULTS = dict(
-        server_root="/etc/httpd",
-        vhost_root="/etc/httpd/conf.d",
-        vhost_files="*.conf",
-        logs_root="/var/log/httpd",
-        ctl="apachectl",
-        version_cmd=['apachectl', '-v'],
-        restart_cmd=['apachectl', 'graceful'],
-        restart_cmd_alt=['apachectl', 'restart'],
-        conftest_cmd=['apachectl', 'configtest'],
-        enmod=None,
-        dismod=None,
-        le_vhost_ext="-le-ssl.conf",
-        handle_modules=False,
-        handle_sites=False,
-        challenge_location="/etc/httpd/conf.d",
-        MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "centos-options-ssl-apache.conf")
-    )
-
-    def config_test(self):
-        """
-        Override config_test to mitigate configtest error in vanilla installation
-        of mod_ssl in Fedora. The error is caused by non-existent self-signed
-        certificates referenced by the configuration, that would be autogenerated
-        during the first (re)start of httpd.
-        """
-
-        os_info = util.get_os_info()
-        fedora = os_info[0].lower() == "fedora"
-
-        try:
-            super(CentOSConfigurator, self).config_test()
-        except errors.MisconfigurationError:
-            if fedora:
-                self._try_restart_fedora()
-            else:
-                raise
-
-    def _try_restart_fedora(self):
-        """
-        Tries to restart httpd using systemctl to generate the self signed keypair.
-        """
-
-        try:
-            util.run_script(['systemctl', 'restart', 'httpd'])
-        except errors.SubprocessError as err:
-            raise errors.MisconfigurationError(str(err))
-
-        # Finish with actual config check to see if systemctl restart helped
-        super(CentOSConfigurator, self).config_test()
-
-    def _prepare_options(self):
-        """
-        Override the options dictionary initialization in order to support
-        alternative restart cmd used in CentOS.
-        """
-        super(CentOSConfigurator, self)._prepare_options()
-        self.options["restart_cmd_alt"][0] = self.option("ctl")
-
-    def get_parser(self):
-        """Initializes the ApacheParser"""
-        return CentOSParser(
-            self.option("server_root"), self.option("vhost_root"),
-            self.version, configurator=self)
-
-    def _deploy_cert(self, *args, **kwargs):  # pylint: disable=arguments-differ
-        """
-        Override _deploy_cert in order to ensure that the Apache configuration
-        has "LoadModule ssl_module..." before parsing the VirtualHost configuration
-        that was created by Certbot
-        """
-        super(CentOSConfigurator, self)._deploy_cert(*args, **kwargs)
-        if self.version < (2, 4, 0):
-            self._deploy_loadmodule_ssl_if_needed()
-
-    def _deploy_loadmodule_ssl_if_needed(self):
-        """
-        Add "LoadModule ssl_module <pre-existing path>" to main httpd.conf if
-        it doesn't exist there already.
-        """
-
-        loadmods = self.parser.find_dir("LoadModule", "ssl_module", exclude=False)
-
-        correct_ifmods = []  # type: List[str]
-        loadmod_args = []  # type: List[str]
-        loadmod_paths = []  # type: List[str]
-        for m in loadmods:
-            noarg_path = m.rpartition("/")[0]
-            path_args = self.parser.get_all_args(noarg_path)
-            if loadmod_args:
-                if loadmod_args != path_args:
-                    msg = ("Certbot encountered multiple LoadModule directives "
-                           "for LoadModule ssl_module with differing library paths. "
-                           "Please remove or comment out the one(s) that are not in "
-                           "use, and run Certbot again.")
-                    raise MisconfigurationError(msg)
-            else:
-                loadmod_args = path_args
-
-            if self.parser.not_modssl_ifmodule(noarg_path):  # pylint: disable=no-member
-                if self.parser.loc["default"] in noarg_path:
-                    # LoadModule already in the main configuration file
-                    if ("ifmodule/" in noarg_path.lower() or
-                        "ifmodule[1]" in noarg_path.lower()):
-                        # It's the first or only IfModule in the file
-                        return
-                # Populate the list of known !mod_ssl.c IfModules
-                nodir_path = noarg_path.rpartition("/directive")[0]
-                correct_ifmods.append(nodir_path)
-            else:
-                loadmod_paths.append(noarg_path)
-
-        if not loadmod_args:
-            # Do not try to enable mod_ssl
-            return
-
-        # Force creation as the directive wasn't found from the beginning of
-        # httpd.conf
-        rootconf_ifmod = self.parser.create_ifmod(
-            parser.get_aug_path(self.parser.loc["default"]),
-            "!mod_ssl.c", beginning=True)
-        # parser.get_ifmod returns a path postfixed with "/", remove that
-        self.parser.add_dir(rootconf_ifmod[:-1], "LoadModule", loadmod_args)
-        correct_ifmods.append(rootconf_ifmod[:-1])
-        self.save_notes += "Added LoadModule ssl_module to main configuration.\n"
-
-        # Wrap LoadModule mod_ssl inside of <IfModule !mod_ssl.c> if it's not
-        # configured like this already.
-        for loadmod_path in loadmod_paths:
-            nodir_path = loadmod_path.split("/directive")[0]
-            # Remove the old LoadModule directive
-            self.parser.aug.remove(loadmod_path)
-
-            # Create a new IfModule !mod_ssl.c if not already found on path
-            ssl_ifmod = self.parser.get_ifmod(nodir_path, "!mod_ssl.c",
-                                            beginning=True)[:-1]
-            if ssl_ifmod not in correct_ifmods:
-                self.parser.add_dir(ssl_ifmod, "LoadModule", loadmod_args)
-                correct_ifmods.append(ssl_ifmod)
-                self.save_notes += ("Wrapped pre-existing LoadModule ssl_module "
-                                    "inside of <IfModule !mod_ssl> block.\n")
-
-
-class CentOSParser(parser.ApacheParser):
-    """CentOS specific ApacheParser override class"""
-    def __init__(self, *args, **kwargs):
-        # CentOS specific configuration file for Apache
-        self.sysconfig_filep = "/etc/sysconfig/httpd"
-        super(CentOSParser, self).__init__(*args, **kwargs)
-
-    def update_runtime_variables(self):
-        """ Override for update_runtime_variables for custom parsing """
-        # Opportunistic, works if SELinux not enforced
-        super(CentOSParser, self).update_runtime_variables()
-        self.parse_sysconfig_var()
-
-    def parse_sysconfig_var(self):
-        """ Parses Apache CLI options from CentOS configuration file """
-        defines = apache_util.parse_define_file(self.sysconfig_filep, "OPTIONS")
-        for k in defines:
-            self.variables[k] = defines[k]
-
-    def not_modssl_ifmodule(self, path):
-        """Checks if the provided Augeas path has argument !mod_ssl"""
-
-        if "ifmodule" not in path.lower():
-            return False
-
-        # Trim the path to the last ifmodule
-        workpath = path.lower()
-        while workpath:
-            # Get path to the last IfModule (ignore the tail)
-            parts = workpath.rpartition("ifmodule")
-
-            if not parts[0]:
-                # IfModule not found
-                break
-            ifmod_path = parts[0] + parts[1]
-            # Check if ifmodule had an index
-            if parts[2].startswith("["):
-                # Append the index from tail
-                ifmod_path += parts[2].partition("/")[0]
-            # Get the original path trimmed to correct length
-            # This is required to preserve cases
-            ifmod_real_path = path[0:len(ifmod_path)]
-            if "!mod_ssl.c" in self.get_all_args(ifmod_real_path):
-                return True
-            # Set the workpath to the heading part
-            workpath = parts[0]
-
-        return False

certbot-apache/certbot_apache/override_darwin.py

@@ -1,31 +0,0 @@
-""" Distribution specific override class for macOS """
-import pkg_resources
-
-import zope.interface
-
-from certbot import interfaces
-
-from certbot_apache import configurator
-
-@zope.interface.provider(interfaces.IPluginFactory)
-class DarwinConfigurator(configurator.ApacheConfigurator):
-    """macOS specific ApacheConfigurator override class"""
-
-    OS_DEFAULTS = dict(
-        server_root="/etc/apache2",
-        vhost_root="/etc/apache2/other",
-        vhost_files="*.conf",
-        logs_root="/var/log/apache2",
-        ctl="apachectl",
-        version_cmd=['apachectl', '-v'],
-        restart_cmd=['apachectl', 'graceful'],
-        conftest_cmd=['apachectl', 'configtest'],
-        enmod=None,
-        dismod=None,
-        le_vhost_ext="-le-ssl.conf",
-        handle_modules=False,
-        handle_sites=False,
-        challenge_location="/etc/apache2/other",
-        MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
-    )

certbot-apache/certbot_apache/override_debian.py

@@ -1,146 +0,0 @@
-""" Distribution specific override class for Debian family (Ubuntu/Debian) """
-import logging
-
-import pkg_resources
-import zope.interface
-
-from certbot import errors
-from certbot import interfaces
-from certbot import util
-from certbot.compat import filesystem
-from certbot.compat import os
-
-from certbot_apache import apache_util
-from certbot_apache import configurator
-
-logger = logging.getLogger(__name__)
-
-
-@zope.interface.provider(interfaces.IPluginFactory)
-class DebianConfigurator(configurator.ApacheConfigurator):
-    """Debian specific ApacheConfigurator override class"""
-
-    OS_DEFAULTS = dict(
-        server_root="/etc/apache2",
-        vhost_root="/etc/apache2/sites-available",
-        vhost_files="*",
-        logs_root="/var/log/apache2",
-        ctl="apache2ctl",
-        version_cmd=['apache2ctl', '-v'],
-        restart_cmd=['apache2ctl', 'graceful'],
-        conftest_cmd=['apache2ctl', 'configtest'],
-        enmod="a2enmod",
-        dismod="a2dismod",
-        le_vhost_ext="-le-ssl.conf",
-        handle_modules=True,
-        handle_sites=True,
-        challenge_location="/etc/apache2",
-        MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
-    )
-
-    def enable_site(self, vhost):
-        """Enables an available site, Apache reload required.
-
-        .. note:: Does not make sure that the site correctly works or that all
-                  modules are enabled appropriately.
-
-        :param vhost: vhost to enable
-        :type vhost: :class:`~certbot_apache.obj.VirtualHost`
-
-        :raises .errors.NotSupportedError: If filesystem layout is not
-            supported.
-
-        """
-        if vhost.enabled:
-            return None
-
-        enabled_path = ("%s/sites-enabled/%s" %
-                        (self.parser.root,
-                         os.path.basename(vhost.filep)))
-        if not os.path.isdir(os.path.dirname(enabled_path)):
-            # For some reason, sites-enabled / sites-available do not exist
-            # Call the parent method
-            return super(DebianConfigurator, self).enable_site(vhost)
-        self.reverter.register_file_creation(False, enabled_path)
-        try:
-            os.symlink(vhost.filep, enabled_path)
-        except OSError as err:
-            if os.path.islink(enabled_path) and filesystem.realpath(
-               enabled_path) == vhost.filep:
-                # Already in shape
-                vhost.enabled = True
-                return None
-            else:
-                logger.warning(
-                    "Could not symlink %s to %s, got error: %s", enabled_path,
-                    vhost.filep, err.strerror)
-                errstring = ("Encountered error while trying to enable a " +
-                             "newly created VirtualHost located at {0} by " +
-                             "linking to it from {1}")
-                raise errors.NotSupportedError(errstring.format(vhost.filep,
-                                                                enabled_path))
-        vhost.enabled = True
-        logger.info("Enabling available site: %s", vhost.filep)
-        self.save_notes += "Enabled site %s\n" % vhost.filep
-        return None
-
-    def enable_mod(self, mod_name, temp=False):
-        """Enables module in Apache.
-
-        Both enables and reloads Apache so module is active.
-
-        :param str mod_name: Name of the module to enable. (e.g. 'ssl')
-        :param bool temp: Whether or not this is a temporary action.
-
-        :raises .errors.NotSupportedError: If the filesystem layout is not
-            supported.
-        :raises .errors.MisconfigurationError: If a2enmod or a2dismod cannot be
-            run.
-
-        """
-        avail_path = os.path.join(self.parser.root, "mods-available")
-        enabled_path = os.path.join(self.parser.root, "mods-enabled")
-        if not os.path.isdir(avail_path) or not os.path.isdir(enabled_path):
-            raise errors.NotSupportedError(
-                "Unsupported directory layout. You may try to enable mod %s "
-                "and try again." % mod_name)
-
-        deps = apache_util.get_mod_deps(mod_name)
-
-        # Enable all dependencies
-        for dep in deps:
-            if (dep + "_module") not in self.parser.modules:
-                self._enable_mod_debian(dep, temp)
-                self.parser.add_mod(dep)
-                note = "Enabled dependency of %s module - %s" % (mod_name, dep)
-                if not temp:
-                    self.save_notes += note + os.linesep
-                logger.debug(note)
-
-        # Enable actual module
-        self._enable_mod_debian(mod_name, temp)
-        self.parser.add_mod(mod_name)
-
-        if not temp:
-            self.save_notes += "Enabled %s module in Apache\n" % mod_name
-        logger.info("Enabled Apache %s module", mod_name)
-
-        # Modules can enable additional config files. Variables may be defined
-        # within these new configuration sections.
-        # Reload is not necessary as DUMP_RUN_CFG uses latest config.
-        self.parser.update_runtime_variables()
-
-    def _enable_mod_debian(self, mod_name, temp):
-        """Assumes mods-available, mods-enabled layout."""
-        # Generate reversal command.
-        # Try to be safe here... check that we can probably reverse before
-        # applying enmod command
-        if not util.exe_exists(self.option("dismod")):
-            raise errors.MisconfigurationError(
-                "Unable to find a2dismod, please make sure a2enmod and "
-                "a2dismod are configured correctly for certbot.")
-
-        self.reverter.register_undo_command(
-            temp, [self.option("dismod"), "-f", mod_name])
-        util.run_script([self.option("enmod"), mod_name])

certbot-apache/certbot_apache/override_fedora.py

@@ -1,98 +0,0 @@
-""" Distribution specific override class for Fedora 29+ """
-import pkg_resources
-import zope.interface
-
-from certbot import errors
-from certbot import interfaces
-from certbot import util
-
-from certbot_apache import apache_util
-from certbot_apache import configurator
-from certbot_apache import parser
-
-
-@zope.interface.provider(interfaces.IPluginFactory)
-class FedoraConfigurator(configurator.ApacheConfigurator):
-    """Fedora 29+ specific ApacheConfigurator override class"""
-
-    OS_DEFAULTS = dict(
-        server_root="/etc/httpd",
-        vhost_root="/etc/httpd/conf.d",
-        vhost_files="*.conf",
-        logs_root="/var/log/httpd",
-        ctl="httpd",
-        version_cmd=['httpd', '-v'],
-        restart_cmd=['apachectl', 'graceful'],
-        restart_cmd_alt=['apachectl', 'restart'],
-        conftest_cmd=['apachectl', 'configtest'],
-        enmod=None,
-        dismod=None,
-        le_vhost_ext="-le-ssl.conf",
-        handle_modules=False,
-        handle_sites=False,
-        challenge_location="/etc/httpd/conf.d",
-        MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            # TODO: eventually newest version of Fedora will need their own config
-            "certbot_apache", "centos-options-ssl-apache.conf")
-    )
-
-    def config_test(self):
-        """
-        Override config_test to mitigate configtest error in vanilla installation
-        of mod_ssl in Fedora. The error is caused by non-existent self-signed
-        certificates referenced by the configuration, that would be autogenerated
-        during the first (re)start of httpd.
-        """
-        try:
-            super(FedoraConfigurator, self).config_test()
-        except errors.MisconfigurationError:
-            self._try_restart_fedora()
-
-    def get_parser(self):
-        """Initializes the ApacheParser"""
-        return FedoraParser(
-            self.option("server_root"), self.option("vhost_root"),
-            self.version, configurator=self)
-
-    def _try_restart_fedora(self):
-        """
-        Tries to restart httpd using systemctl to generate the self signed keypair.
-        """
-        try:
-            util.run_script(['systemctl', 'restart', 'httpd'])
-        except errors.SubprocessError as err:
-            raise errors.MisconfigurationError(str(err))
-
-        # Finish with actual config check to see if systemctl restart helped
-        super(FedoraConfigurator, self).config_test()
-
-    def _prepare_options(self):
-        """
-        Override the options dictionary initialization to keep using apachectl
-        instead of httpd and so take advantages of this new bash script in newer versions
-        of Fedora to restart httpd.
-        """
-        super(FedoraConfigurator, self)._prepare_options()
-        self.options["restart_cmd"][0] = 'apachectl'
-        self.options["restart_cmd_alt"][0] = 'apachectl'
-        self.options["conftest_cmd"][0] = 'apachectl'
-
-
-class FedoraParser(parser.ApacheParser):
-    """Fedora 29+ specific ApacheParser override class"""
-    def __init__(self, *args, **kwargs):
-        # Fedora 29+ specific configuration file for Apache
-        self.sysconfig_filep = "/etc/sysconfig/httpd"
-        super(FedoraParser, self).__init__(*args, **kwargs)
-
-    def update_runtime_variables(self):
-        """ Override for update_runtime_variables for custom parsing """
-        # Opportunistic, works if SELinux not enforced
-        super(FedoraParser, self).update_runtime_variables()
-        self._parse_sysconfig_var()
-
-    def _parse_sysconfig_var(self):
-        """ Parses Apache CLI options from Fedora configuration file """
-        defines = apache_util.parse_define_file(self.sysconfig_filep, "OPTIONS")
-        for k in defines:
-            self.variables[k] = defines[k]

certbot-apache/certbot_apache/override_gentoo.py

@@ -1,75 +0,0 @@
-""" Distribution specific override class for Gentoo Linux """
-import pkg_resources
-
-import zope.interface
-
-from certbot import interfaces
-
-from certbot_apache import apache_util
-from certbot_apache import configurator
-from certbot_apache import parser
-
-@zope.interface.provider(interfaces.IPluginFactory)
-class GentooConfigurator(configurator.ApacheConfigurator):
-    """Gentoo specific ApacheConfigurator override class"""
-
-    OS_DEFAULTS = dict(
-        server_root="/etc/apache2",
-        vhost_root="/etc/apache2/vhosts.d",
-        vhost_files="*.conf",
-        logs_root="/var/log/apache2",
-        ctl="apache2ctl",
-        version_cmd=['apache2ctl', '-v'],
-        restart_cmd=['apache2ctl', 'graceful'],
-        restart_cmd_alt=['apache2ctl', 'restart'],
-        conftest_cmd=['apache2ctl', 'configtest'],
-        enmod=None,
-        dismod=None,
-        le_vhost_ext="-le-ssl.conf",
-        handle_modules=False,
-        handle_sites=False,
-        challenge_location="/etc/apache2/vhosts.d",
-        MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
-    )
-
-    def _prepare_options(self):
-        """
-        Override the options dictionary initialization in order to support
-        alternative restart cmd used in Gentoo.
-        """
-        super(GentooConfigurator, self)._prepare_options()
-        self.options["restart_cmd_alt"][0] = self.option("ctl")
-
-    def get_parser(self):
-        """Initializes the ApacheParser"""
-        return GentooParser(
-            self.option("server_root"), self.option("vhost_root"),
-            self.version, configurator=self)
-
-
-class GentooParser(parser.ApacheParser):
-    """Gentoo specific ApacheParser override class"""
-    def __init__(self, *args, **kwargs):
-        # Gentoo specific configuration file for Apache2
-        self.apacheconfig_filep = "/etc/conf.d/apache2"
-        super(GentooParser, self).__init__(*args, **kwargs)
-
-    def update_runtime_variables(self):
-        """ Override for update_runtime_variables for custom parsing """
-        self.parse_sysconfig_var()
-        self.update_modules()
-
-    def parse_sysconfig_var(self):
-        """ Parses Apache CLI options from Gentoo configuration file """
-        defines = apache_util.parse_define_file(self.apacheconfig_filep,
-                                                "APACHE2_OPTS")
-        for k in defines:
-            self.variables[k] = defines[k]
-
-    def update_modules(self):
-        """Get loaded modules from httpd process, and add them to DOM"""
-        mod_cmd = [self.configurator.option("ctl"), "modules"]
-        matches = self.parse_from_subprocess(mod_cmd, r"(.*)_module")
-        for mod in matches:
-            self.add_mod(mod.strip())

certbot-apache/certbot_apache/override_suse.py

@@ -1,31 +0,0 @@
-""" Distribution specific override class for OpenSUSE """
-import pkg_resources
-
-import zope.interface
-
-from certbot import interfaces
-
-from certbot_apache import configurator
-
-@zope.interface.provider(interfaces.IPluginFactory)
-class OpenSUSEConfigurator(configurator.ApacheConfigurator):
-    """OpenSUSE specific ApacheConfigurator override class"""
-
-    OS_DEFAULTS = dict(
-        server_root="/etc/apache2",
-        vhost_root="/etc/apache2/vhosts.d",
-        vhost_files="*.conf",
-        logs_root="/var/log/apache2",
-        ctl="apache2ctl",
-        version_cmd=['apache2ctl', '-v'],
-        restart_cmd=['apache2ctl', 'graceful'],
-        conftest_cmd=['apache2ctl', 'configtest'],
-        enmod="a2enmod",
-        dismod="a2dismod",
-        le_vhost_ext="-le-ssl.conf",
-        handle_modules=False,
-        handle_sites=False,
-        challenge_location="/etc/apache2/vhosts.d",
-        MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
-            "certbot_apache", "options-ssl-apache.conf")
-    )

certbot-apache/certbot_apache/parser.py

@@ -1,17 +1,14 @@
 """ApacheParser is a member object of the ApacheConfigurator class."""
-import copy
 import fnmatch
 import logging
+import os
 import re
 import subprocess
 import sys
 
 import six
 
-from acme.magic_typing import Dict, List, Set  # pylint: disable=unused-import, no-name-in-module
-
 from certbot import errors
-from certbot.compat import os
 
 from certbot_apache import constants
 
@@ -33,274 +30,86 @@ class ApacheParser(object):
     arg_var_interpreter = re.compile(r"\$\{[^ \}]*}")
     fnmatch_chars = set(["*", "?", "\\", "[", "]"])
 
-    def __init__(self, root, vhostroot=None, version=(2, 4),
-                 configurator=None):
+    def __init__(self, aug, root, vhostroot, version=(2, 4)):
         # Note: Order is important here.
 
-        # Needed for calling save() with reverter functionality that resides in
-        # AugeasConfigurator superclass of ApacheConfigurator. This resolves
-        # issues with aug.load() after adding new files / defines to parse tree
-        self.configurator = configurator
-
-        # Initialize augeas
-        self.aug = None
-        self.init_augeas()
-
-        if not self.check_aug_version():
-            raise errors.NotSupportedError(
-                "Apache plugin support requires libaugeas0 and augeas-lenses "
-                "version 1.2.0 or higher, please make sure you have you have "
-                "those installed.")
-
-        self.modules = set()  # type: Set[str]
-        self.parser_paths = {}  # type: Dict[str, List[str]]
-        self.variables = {}  # type: Dict[str, str]
+        # This uses the binary, so it can be done first.
+        # https://httpd.apache.org/docs/2.4/mod/core.html#define
+        # https://httpd.apache.org/docs/2.4/mod/core.html#ifdefine
+        # This only handles invocation parameters and Define directives!
+        self.parser_paths = {}
+        self.variables = {}
+        if version >= (2, 4):
+            self.update_runtime_variables()
 
+        self.aug = aug
         # Find configuration root and make sure augeas can parse it.
         self.root = os.path.abspath(root)
         self.loc = {"root": self._find_config_root()}
-        self.parse_file(self.loc["root"])
+        self._parse_file(self.loc["root"])
 
-        if version >= (2, 4):
-            # Look up variables from httpd and add to DOM if not already parsed
-            self.update_runtime_variables()
+        self.vhostroot = os.path.abspath(vhostroot)
 
         # This problem has been fixed in Augeas 1.0
         self.standardize_excl()
 
-        # Parse LoadModule directives from configuration files
-        self.parse_modules()
+        # Temporarily set modules to be empty, so that find_dirs can work
+        # https://httpd.apache.org/docs/2.4/mod/core.html#ifmodule
+        # This needs to come before locations are set.
+        self.modules = set()
+        self.init_modules()
 
         # Set up rest of locations
         self.loc.update(self._set_locations())
 
-        # list of the active include paths, before modifications
-        self.existing_paths = copy.deepcopy(self.parser_paths)
-
-        # Must also attempt to parse additional virtual host root
-        if vhostroot:
-            self.parse_file(os.path.abspath(vhostroot) + "/" +
-                            self.configurator.option("vhost_files"))
+        # Must also attempt to parse virtual host root
+        self._parse_file(self.vhostroot + "/" +
+                         constants.os_constant("vhost_files"))
 
         # check to see if there were unparsed define statements
         if version < (2, 4):
             if self.find_dir("Define", exclude=False):
                 raise errors.PluginError("Error parsing runtime variables")
 
-    def init_augeas(self):
-        """ Initialize the actual Augeas instance """
-
-        try:
-            import augeas
-        except ImportError:  # pragma: no cover
-            raise errors.NoInstallationError("Problem in Augeas installation")
-
-        self.aug = augeas.Augeas(
-            # specify a directory to load our preferred lens from
-            loadpath=constants.AUGEAS_LENS_DIR,
-            # Do not save backup (we do it ourselves), do not load
-            # anything by default
-            flags=(augeas.Augeas.NONE |
-                   augeas.Augeas.NO_MODL_AUTOLOAD |
-                   augeas.Augeas.ENABLE_SPAN))
-
-    def check_parsing_errors(self, lens):
-        """Verify Augeas can parse all of the lens files.
-
-        :param str lens: lens to check for errors
-
-        :raises .errors.PluginError: If there has been an error in parsing with
-            the specified lens.
-
-        """
-        error_files = self.aug.match("/augeas//error")
-
-        for path in error_files:
-            # Check to see if it was an error resulting from the use of
-            # the httpd lens
-            lens_path = self.aug.get(path + "/lens")
-            # As aug.get may return null
-            if lens_path and lens in lens_path:
-                msg = (
-                    "There has been an error in parsing the file {0} on line {1}: "
-                    "{2}".format(
-                    # Strip off /augeas/files and /error
-                    path[13:len(path) - 6],
-                    self.aug.get(path + "/line"),
-                    self.aug.get(path + "/message")))
-                raise errors.PluginError(msg)
-
-    def check_aug_version(self):
-        """ Checks that we have recent enough version of libaugeas.
-        If augeas version is recent enough, it will support case insensitive
-        regexp matching"""
-
-        self.aug.set("/test/path/testing/arg", "aRgUMeNT")
-        try:
-            matches = self.aug.match(
-                "/test//*[self::arg=~regexp('argument', 'i')]")
-        except RuntimeError:
-            self.aug.remove("/test/path")
-            return False
-        self.aug.remove("/test/path")
-        return matches
-
-    def unsaved_files(self):
-        """Lists files that have modified Augeas DOM but the changes have not
-        been written to the filesystem yet, used by `self.save()` and
-        ApacheConfigurator to check the file state.
-
-        :raises .errors.PluginError: If there was an error in Augeas, in
-            an attempt to save the configuration, or an error creating a
-            checkpoint
-
-        :returns: `set` of unsaved files
-        """
-        save_state = self.aug.get("/augeas/save")
-        self.aug.set("/augeas/save", "noop")
-        # Existing Errors
-        ex_errs = self.aug.match("/augeas//error")
-        try:
-            # This is a noop save
-            self.aug.save()
-        except (RuntimeError, IOError):
-            self._log_save_errors(ex_errs)
-            # Erase Save Notes
-            self.configurator.save_notes = ""
-            raise errors.PluginError(
-                "Error saving files, check logs for more info.")
-
-        # Return the original save method
-        self.aug.set("/augeas/save", save_state)
-
-        # Retrieve list of modified files
-        # Note: Noop saves can cause the file to be listed twice, I used a
-        # set to remove this possibility. This is a known augeas 0.10 error.
-        save_paths = self.aug.match("/augeas/events/saved")
-
-        save_files = set()
-        if save_paths:
-            for path in save_paths:
-                save_files.add(self.aug.get(path)[6:])
-        return save_files
-
-    def ensure_augeas_state(self):
-        """Makes sure that all Augeas dom changes are written to files to avoid
-        loss of configuration directives when doing additional augeas parsing,
-        causing a possible augeas.load() resulting dom reset
-        """
-
-        if self.unsaved_files():
-            self.configurator.save_notes += "(autosave)"
-            self.configurator.save()
-
-    def save(self, save_files):
-        """Saves all changes to the configuration files.
-
-        save() is called from ApacheConfigurator to handle the parser specific
-        tasks of saving.
-
-        :param list save_files: list of strings of file paths that we need to save.
-
-        """
-        self.configurator.save_notes = ""
-        self.aug.save()
-
-        # Force reload if files were modified
-        # This is needed to recalculate augeas directive span
-        if save_files:
-            for sf in save_files:
-                self.aug.remove("/files/"+sf)
-            self.aug.load()
-
-    def _log_save_errors(self, ex_errs):
-        """Log errors due to bad Augeas save.
-
-        :param list ex_errs: Existing errors before save
-
-        """
-        # Check for the root of save problems
-        new_errs = self.aug.match("/augeas//error")
-        # logger.error("During Save - %s", mod_conf)
-        logger.error("Unable to save files: %s. Attempted Save Notes: %s",
-                     ", ".join(err[13:len(err) - 6] for err in new_errs
-                               # Only new errors caused by recent save
-                               if err not in ex_errs), self.configurator.save_notes)
-
-    def add_include(self, main_config, inc_path):
-        """Add Include for a new configuration file if one does not exist
-
-        :param str main_config: file path to main Apache config file
-        :param str inc_path: path of file to include
-
-        """
-        if not self.find_dir(case_i("Include"), inc_path):
-            logger.debug("Adding Include %s to %s",
-                         inc_path, get_aug_path(main_config))
-            self.add_dir(
-                get_aug_path(main_config),
-                "Include", inc_path)
-
-            # Add new path to parser paths
-            new_dir = os.path.dirname(inc_path)
-            new_file = os.path.basename(inc_path)
-            self.existing_paths.setdefault(new_dir, []).append(new_file)
-
-    def add_mod(self, mod_name):
-        """Shortcut for updating parser modules."""
-        if mod_name + "_module" not in self.modules:
-            self.modules.add(mod_name + "_module")
-        if "mod_" + mod_name + ".c" not in self.modules:
-            self.modules.add("mod_" + mod_name + ".c")
-
-    def reset_modules(self):
-        """Reset the loaded modules list. This is called from cleanup to clear
-        temporarily loaded modules."""
-        self.modules = set()
-        self.update_modules()
-        self.parse_modules()
-
-    def parse_modules(self):
+    def init_modules(self):
         """Iterates on the configuration until no new modules are loaded.
 
         ..todo:: This should be attempted to be done with a binary to avoid
             the iteration issue.  Else... parse and enable mods at same time.
 
         """
-        mods = set()  # type: Set[str]
+        # Since modules are being initiated... clear existing set.
+        self.modules = set()
         matches = self.find_dir("LoadModule")
+
         iterator = iter(matches)
         # Make sure prev_size != cur_size for do: while: iteration
         prev_size = -1
 
-        while len(mods) != prev_size:
-            prev_size = len(mods)
+        while len(self.modules) != prev_size:
+            prev_size = len(self.modules)
 
             for match_name, match_filename in six.moves.zip(
                     iterator, iterator):
-                mod_name = self.get_arg(match_name)
-                mod_filename = self.get_arg(match_filename)
-                if mod_name and mod_filename:
-                    mods.add(mod_name)
-                    mods.add(os.path.basename(mod_filename)[:-2] + "c")
-                else:
-                    logger.debug("Could not read LoadModule directive from " +
-                                 "Augeas path: %s", match_name[6:])
-        self.modules.update(mods)
+                self.modules.add(self.get_arg(match_name))
+                self.modules.add(
+                    os.path.basename(self.get_arg(match_filename))[:-2] + "c")
 
     def update_runtime_variables(self):
-        """Update Includes, Defines and Includes from httpd config dump data"""
-        self.update_defines()
-        self.update_includes()
-        self.update_modules()
+        """"
 
-    def update_defines(self):
-        """Get Defines from httpd process"""
+        .. note:: Compile time variables (apache2ctl -V) are not used within
+            the dynamic configuration files.  These should not be parsed or
+            interpreted.
+
+        .. todo:: Create separate compile time variables...
+            simply for arg_get()
+
+        """
+        stdout = self._get_runtime_cfg()
 
         variables = dict()
-        define_cmd = [self.configurator.option("ctl"), "-t", "-D",
-                      "DUMP_RUN_CFG"]
-        matches = self.parse_from_subprocess(define_cmd, r"Define: ([^ \n]*)")
+        matches = re.compile(r"Define: ([^ \n]*)").findall(stdout)
         try:
             matches.remove("DUMP_RUN_CFG")
         except ValueError:
@@ -317,54 +126,15 @@ class ApacheParser(object):
 
         self.variables = variables
 
-    def update_includes(self):
-        """Get includes from httpd process, and add them to DOM if needed"""
-
-        # Find_dir iterates over configuration for Include and IncludeOptional
-        # directives to make sure we see the full include tree present in the
-        # configuration files
-        _ = self.find_dir("Include")
-
-        inc_cmd = [self.configurator.option("ctl"), "-t", "-D",
-                   "DUMP_INCLUDES"]
-        matches = self.parse_from_subprocess(inc_cmd, r"\(.*\) (.*)")
-        if matches:
-            for i in matches:
-                if not self.parsed_in_current(i):
-                    self.parse_file(i)
-
-    def update_modules(self):
-        """Get loaded modules from httpd process, and add them to DOM"""
-
-        mod_cmd = [self.configurator.option("ctl"), "-t", "-D",
-                       "DUMP_MODULES"]
-        matches = self.parse_from_subprocess(mod_cmd, r"(.*)_module")
-        for mod in matches:
-            self.add_mod(mod.strip())
-
-    def parse_from_subprocess(self, command, regexp):
-        """Get values from stdout of subprocess command
-
-        :param list command: Command to run
-        :param str regexp: Regexp for parsing
-
-        :returns: list parsed from command output
-        :rtype: list
-
-        """
-        stdout = self._get_runtime_cfg(command)
-        return re.compile(regexp).findall(stdout)
-
-    def _get_runtime_cfg(self, command):  # pylint: disable=no-self-use
+    def _get_runtime_cfg(self):  # pylint: disable=no-self-use
         """Get runtime configuration info.
-        :param command: Command to run
 
-        :returns: stdout from command
+        :returns: stdout from DUMP_RUN_CFG
 
         """
         try:
             proc = subprocess.Popen(
-                command,
+                constants.os_constant("define_cmd"),
                 stdout=subprocess.PIPE,
                 stderr=subprocess.PIPE,
                 universal_newlines=True)
@@ -373,10 +143,10 @@ class ApacheParser(object):
         except (OSError, ValueError):
             logger.error(
                 "Error running command %s for runtime parameters!%s",
-                command, os.linesep)
+                constants.os_constant("define_cmd"), os.linesep)
             raise errors.MisconfigurationError(
-                "Error accessing loaded Apache parameters: {0}".format(
-                command))
+                "Error accessing loaded Apache parameters: %s",
+                constants.os_constant("define_cmd"))
         # Small errors that do not impede
         if proc.returncode != 0:
             logger.warning("Error in checking parameter list: %s", stderr)
@@ -402,12 +172,12 @@ class ApacheParser(object):
         """
         filtered = []
         if args == 1:
-            for i, match in enumerate(matches):
-                if match.endswith("/arg"):
+            for i in range(len(matches)):
+                if matches[i].endswith("/arg"):
                     filtered.append(matches[i][:-4])
         else:
-            for i, match in enumerate(matches):
-                if match.endswith("/arg[%d]" % args):
+            for i in range(len(matches)):
+                if matches[i].endswith("/arg[%d]" % args):
                     # Make sure we don't cause an IndexError (end of list)
                     # Check to make sure arg + 1 doesn't exist
                     if (i == (len(matches) - 1) or
@@ -432,7 +202,7 @@ class ApacheParser(object):
         """
         # TODO: Add error checking code... does the path given even exist?
         #       Does it throw exceptions?
-        if_mod_path = self.get_ifmod(aug_conf_path, "mod_ssl.c")
+        if_mod_path = self._get_ifmod(aug_conf_path, "mod_ssl.c")
         # IfModule can have only one valid argument, so append after
         self.aug.insert(if_mod_path + "arg", "directive", False)
         nvh_path = if_mod_path + "directive[1]"
@@ -443,54 +213,22 @@ class ApacheParser(object):
             for i, arg in enumerate(args):
                 self.aug.set("%s/arg[%d]" % (nvh_path, i + 1), arg)
 
-    def get_ifmod(self, aug_conf_path, mod, beginning=False):
+    def _get_ifmod(self, aug_conf_path, mod):
         """Returns the path to <IfMod mod> and creates one if it doesn't exist.
 
         :param str aug_conf_path: Augeas configuration path
         :param str mod: module ie. mod_ssl.c
-        :param bool beginning: If the IfModule should be created to the beginning
-            of augeas path DOM tree.
-
-        :returns: Augeas path of the requested IfModule directive that pre-existed
-            or was created during the process. The path may be dynamic,
-            i.e. .../IfModule[last()]
-        :rtype: str
 
         """
         if_mods = self.aug.match(("%s/IfModule/*[self::arg='%s']" %
                                   (aug_conf_path, mod)))
-        if not if_mods:
-            return self.create_ifmod(aug_conf_path, mod, beginning)
-
+        if len(if_mods) == 0:
+            self.aug.set("%s/IfModule[last() + 1]" % aug_conf_path, "")
+            self.aug.set("%s/IfModule[last()]/arg" % aug_conf_path, mod)
+            if_mods = self.aug.match(("%s/IfModule/*[self::arg='%s']" %
+                                      (aug_conf_path, mod)))
         # Strip off "arg" at end of first ifmod path
-        return if_mods[0].rpartition("arg")[0]
-
-    def create_ifmod(self, aug_conf_path, mod, beginning=False):
-        """Creates a new <IfMod mod> and returns its path.
-
-        :param str aug_conf_path: Augeas configuration path
-        :param str mod: module ie. mod_ssl.c
-        :param bool beginning: If the IfModule should be created to the beginning
-            of augeas path DOM tree.
-
-        :returns: Augeas path of the newly created IfModule directive.
-            The path may be dynamic, i.e. .../IfModule[last()]
-        :rtype: str
-
-        """
-        if beginning:
-            c_path_arg = "{}/IfModule[1]/arg".format(aug_conf_path)
-            # Insert IfModule before the first directive
-            self.aug.insert("{}/directive[1]".format(aug_conf_path),
-                            "IfModule", True)
-            retpath = "{}/IfModule[1]/".format(aug_conf_path)
-        else:
-            c_path = "{}/IfModule[last() + 1]".format(aug_conf_path)
-            c_path_arg = "{}/IfModule[last()]/arg".format(aug_conf_path)
-            self.aug.set(c_path, "")
-            retpath = "{}/IfModule[last()]/".format(aug_conf_path)
-        self.aug.set(c_path_arg, mod)
-        return retpath
+        return if_mods[0][:len(if_mods[0]) - 3]
 
     def add_dir(self, aug_conf_path, directive, args):
         """Appends directive to the end fo the file given by aug_conf_path.
@@ -512,54 +250,6 @@ class ApacheParser(object):
         else:
             self.aug.set(aug_conf_path + "/directive[last()]/arg", args)
 
-    def add_dir_beginning(self, aug_conf_path, dirname, args):
-        """Adds the directive to the beginning of defined aug_conf_path.
-
-        :param str aug_conf_path: Augeas configuration path to add directive
-        :param str dirname: Directive to add
-        :param args: Value of the directive. ie. Listen 443, 443 is arg
-        :type args: list or str
-        """
-        first_dir = aug_conf_path + "/directive[1]"
-        self.aug.insert(first_dir, "directive", True)
-        self.aug.set(first_dir, dirname)
-        if isinstance(args, list):
-            for i, value in enumerate(args, 1):
-                self.aug.set(first_dir + "/arg[%d]" % (i), value)
-        else:
-            self.aug.set(first_dir + "/arg", args)
-
-    def add_comment(self, aug_conf_path, comment):
-        """Adds the comment to the augeas path
-
-        :param str aug_conf_path: Augeas configuration path to add directive
-        :param str comment: Comment content
-
-        """
-        self.aug.set(aug_conf_path + "/#comment[last() + 1]", comment)
-
-    def find_comments(self, arg, start=None):
-        """Finds a comment with specified content from the provided DOM path
-
-        :param str arg: Comment content to search
-        :param str start: Beginning Augeas path to begin looking
-
-        :returns: List of augeas paths containing the comment content
-        :rtype: list
-
-        """
-        if not start:
-            start = get_aug_path(self.root)
-
-        comments = self.aug.match("%s//*[label() = '#comment']" % start)
-
-        results = []
-        for comment in comments:
-            c_content = self.aug.get(comment)
-            if c_content and arg in c_content:
-                results.append(comment)
-        return results
-
     def find_dir(self, directive, arg=None, start=None, exclude=True):
         """Finds directive in the configuration.
 
@@ -619,7 +309,7 @@ class ApacheParser(object):
         else:
             arg_suffix = "/*[self::arg=~regexp('%s')]" % case_i(arg)
 
-        ordered_matches = []  # type: List[str]
+        ordered_matches = []
 
         # TODO: Wildcards should be included in alphabetical order
         # https://httpd.apache.org/docs/2.4/mod/core.html#include
@@ -636,20 +326,6 @@ class ApacheParser(object):
 
         return ordered_matches
 
-    def get_all_args(self, match):
-        """
-        Tries to fetch all arguments for a directive. See get_arg.
-
-        Note that if match is an ancestor node, it returns all names of
-        child directives as well as the list of arguments.
-
-        """
-
-        if match[-1] != "/":
-            match = match+"/"
-        allargs = self.aug.match(match + '*')
-        return [self.get_arg(arg) for arg in allargs]
-
     def get_arg(self, match):
         """Uses augeas.get to get argument value and interprets result.
 
@@ -663,10 +339,7 @@ class ApacheParser(object):
 
         # Note: normal argument may be a quoted variable
         # e.g. strip now, not later
-        if not value:
-            return None
-        else:
-            value = value.strip("'\"")
+        value = value.strip("'\"")
 
         variables = ApacheParser.arg_var_interpreter.findall(value)
 
@@ -755,9 +428,9 @@ class ApacheParser(object):
 
         # Attempts to add a transform to the file if one does not already exist
         if os.path.isdir(arg):
-            self.parse_file(os.path.join(arg, "*"))
+            self._parse_file(os.path.join(arg, "*"))
         else:
-            self.parse_file(arg)
+            self._parse_file(arg)
 
         # Argument represents an fnmatch regular expression, convert it
         # Split up the path and convert each into an Augeas accepted regex
@@ -793,10 +466,11 @@ class ApacheParser(object):
         if sys.version_info < (3, 6):
             # This strips off final /Z(?ms)
             return fnmatch.translate(clean_fn_match)[:-7]
-        # Since Python 3.6, it returns a different pattern like (?s:.*\.load)\Z
-        return fnmatch.translate(clean_fn_match)[4:-3]  # pragma: no cover
+        else:  # pragma: no cover
+            # Since Python 3.6, it returns a different pattern like (?s:.*\.load)\Z
+            return fnmatch.translate(clean_fn_match)[4:-3]
 
-    def parse_file(self, filepath):
+    def _parse_file(self, filepath):
         """Parse file with Augeas
 
         Checks to see if file_path is parsed by Augeas
@@ -806,9 +480,6 @@ class ApacheParser(object):
 
         """
         use_new, remove_old = self._check_path_actions(filepath)
-        # Ensure that we have the latest Augeas DOM state on disk before
-        # calling aug.load() which reloads the state from disk
-        self.ensure_augeas_state()
         # Test if augeas included file for Httpd.lens
         # Note: This works for augeas globs, ie. *.conf
         if use_new:
@@ -823,39 +494,6 @@ class ApacheParser(object):
                 self._add_httpd_transform(filepath)
                 self.aug.load()
 
-    def parsed_in_current(self, filep):
-        """Checks if the file path is parsed by current Augeas parser config
-        ie. returns True if the file is found on a path that's found in live
-        Augeas configuration.
-
-        :param str filep: Path to match
-
-        :returns: True if file is parsed in existing configuration tree
-        :rtype: bool
-        """
-        return self._parsed_by_parser_paths(filep, self.parser_paths)
-
-    def parsed_in_original(self, filep):
-        """Checks if the file path is parsed by existing Apache config.
-        ie. returns True if the file is found on a path that matches Include or
-        IncludeOptional statement in the Apache configuration.
-
-        :param str filep: Path to match
-
-        :returns: True if file is parsed in existing configuration tree
-        :rtype: bool
-        """
-        return self._parsed_by_parser_paths(filep, self.existing_paths)
-
-    def _parsed_by_parser_paths(self, filep, paths):
-        """Helper function that searches through provided paths and returns
-        True if file path is found in the set"""
-        for directory in paths.keys():
-            for filename in paths[directory]:
-                if fnmatch.fnmatch(filep, os.path.join(directory, filename)):
-                    return True
-        return False
-
     def _check_path_actions(self, filepath):
         """Determine actions to take with a new augeas path
 
@@ -875,7 +513,10 @@ class ApacheParser(object):
                 use_new = False
             else:
                 use_new = True
-            remove_old = new_file_match == "*"
+            if new_file_match == "*":
+                remove_old = True
+            else:
+                remove_old = False
         except KeyError:
             use_new = True
             remove_old = False
@@ -981,6 +622,7 @@ class ApacheParser(object):
         for name in location:
             if os.path.isfile(os.path.join(self.root, name)):
                 return os.path.join(self.root, name)
+
         raise errors.NoInstallationError("Could not find configuration root")
 
 

certbot-apache/certbot_apache/tests/apache-conf-files/apache-conf-test

@@ -3,11 +3,6 @@
 # A hackish script to see if the client is behaving as expected 
 # with each of the "passing" conf files.
 
-if [ -z "$SERVER" ]; then
-    echo "Please set SERVER to the ACME server's directory URL."
-    exit 1
-fi
-
 export EA=/etc/apache2/ 
 TESTDIR="`dirname $0`"
 cd $TESTDIR/passing
@@ -31,7 +26,6 @@ function Setup() {
     ErrorLog /tmp/error.log
     CustomLog /tmp/requests.log combined
 </VirtualHost>" | sudo tee $EA/sites-available/throwaway-example.conf >/dev/null
-        sudo ln -sf $EA/sites-available/throwaway-example.conf $EA/sites-enabled/throwaway-example.conf
     else
         TMP="/tmp/`basename \"$APPEND_APACHECONF\"`.$$"
         sudo cp -a "$APPEND_APACHECONF" "$TMP"
@@ -43,7 +37,6 @@ function Cleanup() {
     if [ "$APPEND_APACHECONF" = "" ] ; then
         sudo rm /etc/apache2/sites-{enabled,available}/"$f"
         sudo rm $EA/sites-available/throwaway-example.conf
-        sudo rm $EA/sites-enabled/throwaway-example.conf
     else
         sudo mv "$TMP" "$APPEND_APACHECONF"
     fi
@@ -51,7 +44,6 @@ function Cleanup() {
 
 # if our environment asks us to enable modules, do our best!
 if [ "$1" = --debian-modules ] ; then
-    sudo apt-get install -y apache2
     sudo apt-get install -y libapache2-mod-wsgi
     sudo apt-get install -y libapache2-mod-macro
 
@@ -61,16 +53,13 @@ if [ "$1" = --debian-modules ] ; then
     done
 fi
 
-CERTBOT_CMD="sudo $(command -v certbot) --server $SERVER -vvvv"
-CERTBOT_CMD="$CERTBOT_CMD --debug --apache --register-unsafely-without-email"
-CERTBOT_CMD="$CERTBOT_CMD --agree-tos certonly -t --no-verify-ssl"
 
 FAILS=0
 trap CleanupExit INT
 for f in *.conf ; do 
     echo -n  testing "$f"...
     Setup
-    RESULT=`echo c | $CERTBOT_CMD 2>&1`
+    RESULT=`echo c | sudo $(command -v certbot) -vvvv --debug --staging --apache --register-unsafely-without-email --agree-tos certonly -t 2>&1`
     if echo $RESULT | grep -Eq \("Which names would you like"\|"mod_macro is not yet"\) ; then
         echo passed
     else