test it

Fixes https://github.com/certbot/certbot/issues/8162.

I had to update the base of the Dockerfile to get a new enough version of Python 3. I also simplified things a lot and removed a lot of the comments that were essentially just describing how Dockerfiles work.

The most complicated changes here are in `testdata`. You can find a diff of the changes to `nginx.tar.gz` at https://gist.github.com/c7727db0cecf3f15f02439f085c73848.

The first problem was that there were some complaints from the new Apache/nginx/OpenSSL version about the 1024 bit RSA key so I updated `empty_cert.pem` both inside and outside of the tarball as well as the corresponding private key in the tarball to use a 2048 bit key.

The 2nd problem is trickier to understand. If you look at the output from nginx after loading the config from `lots/` you'll see it complaining about conflicting `server_name` directives for the directives I deleted. See https://dev.azure.com/certbot/certbot/_build/results?buildId=2578&view=logs&j=250aa146-b243-5f8f-bf86-17a529c9fb7e&t=9baa2014-9673-5e78-8f4f-7a463caf2bfa&l=1516.

After switching the tests to Python 3, tests on that domain started failing. What I believe to be happening is we were just lucky these tests were passing to begin with. In both the Apache and Nginx plugin, if there are conflicting virtual hosts like this, we just arbitrarily pick one. The relevant code here for nginx is 575092d603/certbot-nginx/certbot_nginx/_internal/configurator.py (L455)

I played around with a debugger and confirmed that before I removed the conflicting server names, there were two exact matches for the domain we were searching for here.

I think all that's going on is with the switch to Python 3, the vhost we happen to choose changes and "breaks" the test. I suspect this to be due to something like getting values out of a dict somewhere where the order of items in a dict while iterating over it is different between Python 2 and 3. I didn't track where this difference happens down, but I personally don't think it's a good use of time since I think the real problem here is that the nginx config being tested was invalid with conflicting `server` blocks.

I removed all references to the `server_name` causing conflicts in that nginx configuration because both server blocks had other domains that are being tested, but I could add either back if you prefer. You can see the `nginx_compat` test passing with these changes at https://dev.azure.com/certbot/certbot/_build/results?buildId=2587&view=logs&j=250aa146-b243-5f8f-bf86-17a529c9fb7e.

* update Dockerfile

* Fix apache_compat on py3.

* Update empty_cert.pem.

The command used here was `openssl req -key
certbot/certbot/tests/testdata/rsa2048_key.pem -new -subj '/CN=example.com'
-x509 >
certbot-compatibility-test/certbot_compatibility_test/testdata/empty_cert.pem`.

* update nginx.tar.gz

* Remove conflicting server_names

.azure-pipelines/templates/jobs/extended-tests-jobs.yml

@@ -68,8 +68,11 @@ jobs:
           TOXENV: le_auto_oraclelinux6
         docker-dev:
           TOXENV: docker_dev
-        farmtest-apache2:
-          PYTHON_VERSION: 3.7
+        macos-farmtest-apache2:
+          # We run one of these test farm tests on macOS to help ensure the
+          # tests continue to work on the platform.
+          IMAGE_NAME: macOS-10.15
+          PYTHON_VERSION: 3.8
           TOXENV: test-farm-apache2
         farmtest-leauto-upgrades:
           PYTHON_VERSION: 3.7

.azure-pipelines/templates/jobs/standard-tests-jobs.yml

@@ -2,71 +2,10 @@ jobs:
   - job: test
     strategy:
       matrix:
-        macos-py27:
-          IMAGE_NAME: macOS-10.14
-          PYTHON_VERSION: 2.7
-          TOXENV: py27
-        macos-py38:
-          IMAGE_NAME: macOS-10.14
-          PYTHON_VERSION: 3.8
-          TOXENV: py38
-        windows-py36:
-          IMAGE_NAME: vs2017-win2016
-          PYTHON_VERSION: 3.6
-          TOXENV: py36
-        windows-py37-cover:
-          IMAGE_NAME: vs2017-win2016
-          PYTHON_VERSION: 3.7
-          TOXENV: py37-cover
-        windows-integration-certbot:
-          IMAGE_NAME: vs2017-win2016
-          PYTHON_VERSION: 3.7
-          TOXENV: integration-certbot
-        linux-oldest-tests-1:
-          IMAGE_NAME: ubuntu-18.04
-          TOXENV: py27-{acme,apache,apache-v2,certbot}-oldest
-        linux-oldest-tests-2:
-          IMAGE_NAME: ubuntu-18.04
-          TOXENV: py27-{dns,nginx}-oldest
         linux-py27:
           IMAGE_NAME: ubuntu-18.04
           PYTHON_VERSION: 2.7
           TOXENV: py27
-        linux-py36:
-          IMAGE_NAME: ubuntu-18.04
-          PYTHON_VERSION: 3.6
-          TOXENV: py36
-        linux-py38-cover:
-          IMAGE_NAME: ubuntu-18.04
-          PYTHON_VERSION: 3.8
-          TOXENV: py38-cover
-        linux-py37-lint:
-          IMAGE_NAME: ubuntu-18.04
-          PYTHON_VERSION: 3.7
-          TOXENV: lint
-        linux-py36-mypy:
-          IMAGE_NAME: ubuntu-18.04
-          PYTHON_VERSION: 3.6
-          TOXENV: mypy
-        linux-integration:
-          IMAGE_NAME: ubuntu-18.04
-          PYTHON_VERSION: 2.7
-          TOXENV: integration
-          ACME_SERVER: pebble
-        apache-compat:
-          IMAGE_NAME: ubuntu-18.04
-          TOXENV: apache_compat
-        le-auto-xenial:
-          IMAGE_NAME: ubuntu-18.04
-          TOXENV: le_auto_xenial
-        apacheconftest:
-          IMAGE_NAME: ubuntu-18.04
-          PYTHON_VERSION: 2.7
-          TOXENV: apacheconftest-with-pebble
-        nginxroundtrip:
-          IMAGE_NAME: ubuntu-18.04
-          PYTHON_VERSION: 2.7
-          TOXENV: nginxroundtrip
     pool:
       vmImage: $(IMAGE_NAME)
     steps:

.azure-pipelines/templates/stages/test-and-package-stage.yml

@@ -2,5 +2,3 @@ stages:
   - stage: TestAndPackage
     jobs:
       - template: ../jobs/standard-tests-jobs.yml
-      - template: ../jobs/extended-tests-jobs.yml
-      - template: ../jobs/packaging-jobs.yml

.azure-pipelines/templates/steps/tox-steps.yml

@@ -14,6 +14,7 @@ steps:
         ca-certificates \
         nginx-light \
         openssl
+      geoiqgheoqigjeq
       sudo systemctl stop nginx
     condition: startswith(variables['IMAGE_NAME'], 'ubuntu')
     displayName: Install Linux dependencies

AUTHORS.md

@@ -61,6 +61,7 @@ Authors
 * [Daniel Albers](https://github.com/AID)
 * [Daniel Aleksandersen](https://github.com/da2x)
 * [Daniel Convissor](https://github.com/convissor)
+* [Daniel "Drex" Drexler](https://github.com/aeturnum)
 * [Daniel Huang](https://github.com/dhuang)
 * [Dave Guarino](https://github.com/daguar)
 * [David cz](https://github.com/dave-cz)

acme/acme/messages.py

@@ -315,6 +315,9 @@ class Registration(ResourceBody):
     # on new-reg key server ignores 'key' and populates it based on
     # JWS.signature.combined.jwk
     key = jose.Field('key', omitempty=True, decoder=jose.JWK.from_json)
+    # Contact field implements special behavior to allow messages that clear existing
+    # contacts while not expecting the `contact` field when loading from json.
+    # This is implemented in the constructor and *_json methods.
     contact = jose.Field('contact', omitempty=True, default=())
     agreement = jose.Field('agreement', omitempty=True)
     status = jose.Field('status', omitempty=True)
@@ -327,24 +330,73 @@ class Registration(ResourceBody):
 
     @classmethod
     def from_data(cls, phone=None, email=None, external_account_binding=None, **kwargs):
-        """Create registration resource from contact details."""
+        """
+        Create registration resource from contact details.
+
+        The `contact` keyword being passed to a Registration object is meaningful, so
+        this function represents empty iterables in its kwargs by passing on an empty
+        `tuple`.
+        """
+
+        # Note if `contact` was in kwargs.
+        contact_provided = 'contact' in kwargs
+
+        # Pop `contact` from kwargs and add formatted email or phone numbers
         details = list(kwargs.pop('contact', ()))
         if phone is not None:
             details.append(cls.phone_prefix + phone)
         if email is not None:
             details.extend([cls.email_prefix + mail for mail in email.split(',')])
-        kwargs['contact'] = tuple(details)
+
+        # Insert formatted contact information back into kwargs
+        # or insert an empty tuple if `contact` provided.
+        if details or contact_provided:
+            kwargs['contact'] = tuple(details)
 
         if external_account_binding:
             kwargs['external_account_binding'] = external_account_binding
 
         return cls(**kwargs)
 
+    def __init__(self, **kwargs):
+        """Note if the user provides a value for the `contact` member."""
+        if 'contact' in kwargs:
+            # Avoid the __setattr__ used by jose.TypedJSONObjectWithFields
+            object.__setattr__(self, '_add_contact', True)
+        super(Registration, self).__init__(**kwargs)
+
     def _filter_contact(self, prefix):
         return tuple(
             detail[len(prefix):] for detail in self.contact  # pylint: disable=not-an-iterable
             if detail.startswith(prefix))
 
+    def _add_contact_if_appropriate(self, jobj):
+        """
+        The `contact` member of Registration objects should not be required when
+        de-serializing (as it would be if the Fields' `omitempty` flag were `False`), but
+        it should be included in serializations if it was provided.
+
+        :param jobj: Dictionary containing this Registrations' data
+        :type jobj: dict
+
+        :returns: Dictionary containing Registrations data to transmit to the server
+        :rtype: dict
+        """
+        if getattr(self, '_add_contact', False):
+            jobj['contact'] = self.encode('contact')
+
+        return jobj
+
+    def to_partial_json(self):
+        """Modify josepy.JSONDeserializable.to_partial_json()"""
+        jobj = super(Registration, self).to_partial_json()
+        return self._add_contact_if_appropriate(jobj)
+
+    def fields_to_partial_json(self):
+        """Modify josepy.JSONObjectWithFields.fields_to_partial_json()"""
+        jobj = super(Registration, self).fields_to_partial_json()
+        return self._add_contact_if_appropriate(jobj)
+
     @property
     def phones(self):
         """All phones found in the ``contact`` field."""

acme/setup.py

@@ -6,7 +6,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [

acme/tests/messages_test.py

@@ -254,6 +254,19 @@ class RegistrationTest(unittest.TestCase):
         from acme.messages import Registration
         hash(Registration.from_json(self.jobj_from))
 
+    def test_default_not_transmitted(self):
+        from acme.messages import NewRegistration
+        empty_new_reg = NewRegistration()
+        new_reg_with_contact = NewRegistration(contact=())
+
+        self.assertEqual(empty_new_reg.contact, ())
+        self.assertEqual(new_reg_with_contact.contact, ())
+
+        self.assertTrue('contact' not in empty_new_reg.to_partial_json())
+        self.assertTrue('contact' not in empty_new_reg.fields_to_partial_json())
+        self.assertTrue('contact' in new_reg_with_contact.to_partial_json())
+        self.assertTrue('contact' in new_reg_with_contact.fields_to_partial_json())
+
 
 class UpdateRegistrationTest(unittest.TestCase):
     """Tests for acme.messages.UpdateRegistration."""

certbot-apache/setup.py

@@ -6,7 +6,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-auto

@@ -31,7 +31,7 @@ if [ -z "$VENV_PATH" ]; then
 fi
 VENV_BIN="$VENV_PATH/bin"
 BOOTSTRAP_VERSION_PATH="$VENV_PATH/certbot-auto-bootstrap-version.txt"
-LE_AUTO_VERSION="1.7.0"
+LE_AUTO_VERSION="1.8.0"
 BASENAME=$(basename $0)
 USAGE="Usage: $BASENAME [OPTIONS]
 A self-updating wrapper script for the Certbot ACME client. When run, updates
@@ -258,7 +258,7 @@ DeprecationBootstrap() {
 
 MIN_PYTHON_2_VERSION="2.7"
 MIN_PYVER2=$(echo "$MIN_PYTHON_2_VERSION" | sed 's/\.//')
-MIN_PYTHON_3_VERSION="3.5"
+MIN_PYTHON_3_VERSION="3.6"
 MIN_PYVER3=$(echo "$MIN_PYTHON_3_VERSION" | sed 's/\.//')
 # Sets LE_PYTHON to Python version string and PYVER to the first two
 # digits of the python version.
@@ -930,7 +930,7 @@ else
     error "Sorry, I don't know how to bootstrap Certbot on your operating system!"
     error
     error "You will need to install OS dependencies, configure virtualenv, and run pip install manually."
-    error "Please see https://letsencrypt.readthedocs.org/en/latest/contributing.html#prerequisites"
+    error "Please see https://certbot.eff.org/docs/contributing.html#prerequisites"
     error "for more info."
     exit 1
   }
@@ -1530,18 +1530,18 @@ letsencrypt==0.7.0 \
     --hash=sha256:105a5fb107e45bcd0722eb89696986dcf5f08a86a321d6aef25a0c7c63375ade \
     --hash=sha256:c36e532c486a7e92155ee09da54b436a3c420813ec1c590b98f635d924720de9
 
-certbot==1.7.0 \
-    --hash=sha256:84877127caf779c212d131d36399a45a8e13e06274e7a5e029845df5c84cd974 \
-    --hash=sha256:10b95bb86fb8f1dbbd27558bb42454d5995cbdb45d6c00d961ebba2a4bdc4355
-acme==1.7.0 \
-    --hash=sha256:ef0e84d670f59c096e9ed8c3bf9e6a7d22ee378fdb4175503c06cc485672c79a \
-    --hash=sha256:288d9bbb075278961d224e43f7f386c491d25366a98ed89a62771c5022978386
-certbot-apache==1.7.0 \
-    --hash=sha256:514c09a892964332c2485a38bd5720e4cf93e35998341af36eef5401ab165d89 \
-    --hash=sha256:99943b6406e0315f31c1f81e2ced6be38aee3ea24974ef4d7aeeda8202c1c3bc
-certbot-nginx==1.7.0 \
-    --hash=sha256:fea2387c92155635fbddb02758d5ba73f0d7af459959f91be0a1606fd2e43c55 \
-    --hash=sha256:d52ec3e711884100636c42b639d8959378562ea78183a273d120df808de2724f
+certbot==1.8.0 \
+    --hash=sha256:4bde86c53e30dc5bc0e78a0862045b053971703af727ac20c6a7da06596c7549 \
+    --hash=sha256:4837c516af6543ccd10d70f1498a2113bbdf9ef9a05d3a18b1558b291a2953e4
+acme==1.8.0 \
+    --hash=sha256:465033830a75f98042236f50f751f6e316735473ccb4edec0c718263f6c9ba8b \
+    --hash=sha256:ad8d067d14258d73ad2643439d9365913362308c04e66cc3010e39c868c5002d
+certbot-apache==1.8.0 \
+    --hash=sha256:8c9d981803e1156725fcfcf228afcb754b245c9d506e5b9f4fca948d6ae89aef \
+    --hash=sha256:a93c3a7ad929fe0ba5e0868e29ee2d0fe10aea2d4c638a902c4613a5c12c59b6
+certbot-nginx==1.8.0 \
+    --hash=sha256:e98e883b5ea7b29dd2e6a8ff286c7550a2d7af2fc859f47067303e510ad4fb52 \
+    --hash=sha256:fdb96c74fe42d90bbaf11a00314444ac5544ba87292a1b8b1d707f7561a3eacc
 
 UNLIKELY_EOF
     # -------------------------------------------------------------------------

certbot-compatibility-test/Dockerfile

@@ -1,47 +1,18 @@
-FROM debian:stretch
+FROM debian:buster
 MAINTAINER Brad Warren <bmw@eff.org>
 
-# no need to mkdir anything:
-# https://docs.docker.com/reference/builder/#copy
-# If <dest> doesn't exist, it is created along with all missing
-# directories in its path.
+RUN apt-get update && \
+    apt install python3-dev python3-venv gcc libaugeas0 libssl-dev \
+                 libffi-dev ca-certificates openssl -y
 
-# TODO: Install non-default Python versions for tox.
-# TODO: Install Apache/Nginx for plugin development.
-COPY letsencrypt-auto-source /opt/certbot/src/letsencrypt-auto-source
-RUN /opt/certbot/src/letsencrypt-auto-source/letsencrypt-auto --os-packages-only
+WORKDIR /opt/certbot/src
 
-# the above is not likely to change, so by putting it further up the
-# Dockerfile we make sure we cache as much as possible
+# We copy all contents of the build directory to allow us to easily use
+# things like tools/venv3.py which expects all of our packages to be available.
+COPY . .
 
-COPY certbot/setup.py certbot/README.rst certbot/CHANGELOG.md certbot/MANIFEST.in linter_plugin.py tox.cover.py tox.ini .pylintrc /opt/certbot/src/
-
-# all above files are necessary for setup.py, however, package source
-# code directory has to be copied separately to a subdirectory...
-# https://docs.docker.com/reference/builder/#copy: "If <src> is a
-# directory, the entire contents of the directory are copied,
-# including filesystem metadata. Note: The directory itself is not
-# copied, just its contents." Order again matters, three files are far
-# more likely to be cached than the whole project directory
-
-COPY certbot /opt/certbot/src/certbot/
-COPY acme /opt/certbot/src/acme/
-COPY certbot-apache /opt/certbot/src/certbot-apache/
-COPY certbot-nginx /opt/certbot/src/certbot-nginx/
-COPY certbot-compatibility-test /opt/certbot/src/certbot-compatibility-test/
-COPY tools /opt/certbot/src/tools
-
-RUN VIRTUALENV_NO_DOWNLOAD=1 virtualenv -p python2 /opt/certbot/venv && \
-    /opt/certbot/venv/bin/pip install -U setuptools && \
-    /opt/certbot/venv/bin/pip install -U pip
-ENV PATH /opt/certbot/venv/bin:$PATH
-RUN /opt/certbot/venv/bin/python \
-    /opt/certbot/src/tools/pip_install_editable.py \
-    /opt/certbot/src/acme \
-    /opt/certbot/src/certbot \
-    /opt/certbot/src/certbot-apache \
-    /opt/certbot/src/certbot-nginx \
-    /opt/certbot/src/certbot-compatibility-test
+RUN tools/venv3.py
+ENV PATH /opt/certbot/src/venv3/bin:$PATH
 
 # install in editable mode (-e) to save space: it's not possible to
 # "rm -rf /opt/certbot/src" (it's stays in the underlaying image);

certbot-compatibility-test/certbot_compatibility_test/test_driver.py

@@ -102,8 +102,10 @@ def _create_achalls(plugin):
         prefs = plugin.get_chall_pref(domain)
         for chall_type in prefs:
             if chall_type == challenges.HTTP01:
+                # challenges.HTTP01.TOKEN_SIZE is a float but os.urandom
+                # expects an integer.
                 chall = challenges.HTTP01(
-                    token=os.urandom(challenges.HTTP01.TOKEN_SIZE))
+                    token=os.urandom(int(challenges.HTTP01.TOKEN_SIZE)))
                 challb = acme_util.chall_to_challb(
                     chall, messages.STATUS_PENDING)
                 achall = achallenges.KeyAuthorizationAnnotatedChallenge(
@@ -137,7 +139,7 @@ def test_deploy_cert(plugin, temp_dir, domains):
     """Tests deploy_cert returning True if the tests are successful"""
     cert = crypto_util.gen_ss_cert(util.KEY, domains)
     cert_path = os.path.join(temp_dir, "cert.pem")
-    with open(cert_path, "w") as f:
+    with open(cert_path, "wb") as f:
         f.write(OpenSSL.crypto.dump_certificate(
             OpenSSL.crypto.FILETYPE_PEM, cert))
 
@@ -273,7 +275,7 @@ def _dirs_are_unequal(dir1, dir2):
             logger.error(str(dircmp.diff_files))
             return True
 
-        for subdir in dircmp.subdirs.itervalues():
+        for subdir in dircmp.subdirs.values():
             dircmps.append(subdir)
 
     return False

certbot-compatibility-test/certbot_compatibility_test/testdata/empty_cert.pem

@@ -1,13 +1,17 @@
 -----BEGIN CERTIFICATE-----
-MIICATCCAWoCCQCvMbKu4FHZ6zANBgkqhkiG9w0BAQsFADBFMQswCQYDVQQGEwJB
-VTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50ZXJuZXQgV2lkZ2l0
-cyBQdHkgTHRkMB4XDTE1MDcyMzIzMjc1MFoXDTE2MDcyMjIzMjc1MFowRTELMAkG
-A1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoMGEludGVybmV0
-IFdpZGdpdHMgUHR5IEx0ZDCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAws3o
-y46PMLM9Gr68pbex0MhdPr7Cq4rRe9BBpnOuHFdF35Ak0aPrzFwVzLlGOir94U11
-e5JYJDWJi+4FwLBRkOAfanjJ5GJ9BnEHSOdbtO+sv9uhbt+7iYOOUOngKSiJyUrM
-i1THAE+B1CenxZ1KHRQCke708zkK8jVuxLeIAOMCAwEAATANBgkqhkiG9w0BAQsF
-AAOBgQCC3LUP3MHk+IBmwHHZAZCX+6p4lop9SP6y6rDpWgnqEEeb9oFleHi2Rvzq
-7gxl6nS5AsaSzfAygJ3zWKTwVAZyU4GOQ8QTK+nHk3+LO1X4cDbUlQfm5+YuwKDa
-4LFKeovmrK6BiMLIc1J+MxUjLfCeVHYSdkZULTVXue0zif0BUA==
+MIICqDCCAZACCQCRC1UKg2WfRTANBgkqhkiG9w0BAQsFADAWMRQwEgYDVQQDDAtl
+eGFtcGxlLmNvbTAeFw0yMDA4MTkyMzM5MjdaFw0yMDA5MTgyMzM5MjdaMBYxFDAS
+BgNVBAMMC2V4YW1wbGUuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
+AQEA5tViHnJx4y+BbCb8Qz9uxsnqp1ynONR7ET/XL+M/jQ4xPeJg4L2uZ3YnogPc
+WdEoey17WXBg3KRqKfg+7PqIdGqVeonSCfXhD1HoGJRsThSUJ2fK3uoQ+zGgJTWR
+FYWa8Cb6xsuq0xaYtw2jaJBp+697Np60PWs4pY5FkadT50wZ0TYDnYt3NSAdn+Pt
+j3cpI4ocZZ2FLiOFn+UFOaRcetGtpnU1QwvmygD9tiL7kJ55B4CWGEv6DMRQk/UE
+eMUETzse1NkVlaxQ1TCd5iAfBTluiV30EpmmWa+OsXJWxCK+EEOkXD1r3CdXAldY
+nRYxJrn4udrFe69QX95wiRZNXwIDAQABMA0GCSqGSIb3DQEBCwUAA4IBAQCJvtDC
+875CK7SKNf006gSciXsNPNSVORGPjc/5OQ23baK4iPhxftI4LGZN8773N14jWp3E
+QnQLL1gZ9/G+98SlI5lm97a4m4XZyNaULbmQwRKgI22H0F1AWbvsG0SppjnhVlJ+
+93ZUqSQBXgbXelFHSsNfk1AB6Kvo6+UvS8s0vkz7SfkPOZGx0b+3RJSJZnZHvYih
+ggudN/jJggSgRrb+F6lpaelJE9pZsznJFb9R7mFI33AGBpQWV4r3p1ZbM1vGMqGc
+4PGBzDzi28BhLBplSOPZZxqRiINQzGiQ5T2SfN06usr7EafFr6+7YKNhgrCdlVjU
+thzJ5MgHZgALNXsh
 -----END CERTIFICATE-----

certbot-compatibility-test/setup.py

@@ -5,7 +5,7 @@ from setuptools import __version__ as setuptools_version
 from setuptools import find_packages
 from setuptools import setup
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 install_requires = [
     'certbot',

certbot-dns-cloudflare/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-cloudxns/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-digitalocean/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-dnsimple/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-dnsmadeeasy/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-gehirn/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [

certbot-dns-google/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-linode/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [

certbot-dns-luadns/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-nsone/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-ovh/certbot_dns_ovh/__init__.py

@@ -72,7 +72,7 @@ Examples
 
    certbot certonly \\
      --dns-ovh \\
-     --dns-ovh-credentials ~/.secrets/certbot/ohv.ini \\
+     --dns-ovh-credentials ~/.secrets/certbot/ovh.ini \\
      -d example.com
 
 .. code-block:: bash

certbot-dns-ovh/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-rfc2136/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-route53/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-dns-sakuracloud/setup.py

@@ -7,7 +7,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Please update tox.ini when modifying dependency version requirements
 install_requires = [

certbot-nginx/certbot_nginx/_internal/parser.py

@@ -496,7 +496,8 @@ def get_best_match(target_name, names):
 
 
 def _exact_match(target_name, name):
-    return name in (target_name, '.' + target_name)
+    target_lower = target_name.lower()
+    return name.lower() in (target_lower, '.' + target_lower)
 
 
 def _wildcard_match(target_name, name, start):
@@ -517,11 +518,11 @@ def _wildcard_match(target_name, name, start):
     if first not in ('*', ''):
         return False
 
-    target_name = '.'.join(parts)
-    name = '.'.join(match_parts)
+    target_name_lower = '.'.join(parts).lower()
+    name_lower = '.'.join(match_parts).lower()
 
     # Ex: www.eff.org matches *.eff.org, eff.org does not match *.eff.org
-    return target_name.endswith('.' + name)
+    return target_name_lower.endswith('.' + name_lower)
 
 
 def _regex_match(target_name, name):

certbot-nginx/setup.py

@@ -6,7 +6,7 @@ from setuptools import find_packages
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-version = '1.8.0.dev0'
+version = '1.9.0.dev0'
 
 # Remember to update local-oldest-requirements.txt when changing the minimum
 # acme/certbot version.

certbot-nginx/tests/parser_test.py

@@ -340,7 +340,9 @@ class NginxParserTest(util.NginxTest):
                  {'*.www.eff.org', 'www.*'},
                  {'*.org'},
                  set(),
-                 {'example.com'}]
+                 {'example.com'},
+                 {'www.Eff.org'},
+                 {'.efF.org'}]
         winners = [('exact', 'www.eff.org'),
                    (None, None),
                    ('exact', '.www.eff.org'),
@@ -353,7 +355,9 @@ class NginxParserTest(util.NginxTest):
                    ('wildcard_end', 'www.*'),
                    ('wildcard_start', '*.org'),
                    (None, None),
-                   (None, None)]
+                   (None, None),
+                   ('exact', 'www.Eff.org'),
+                   ('wildcard_start', '.efF.org')]
 
         for i, winner in enumerate(winners):
             self.assertEqual(winner,

certbot.wrapper

@@ -1,16 +1,38 @@
-#!/bin/bash
+#!/bin/sh
+#
+# TODO: We may want to consider rewriting this script in Python. See
+# https://github.com/certbot/certbot/issues/8251 for more info.
 set -e
-join() {
-    sep=$1
-    first=$2
-    if [ "$first" != "" ]; then
-        shift 2
-        echo -n "${first}"
-        for item in "$@"; do echo -n "${sep}${item}"; done
-        echo
-    fi
-}
 
-paths=$(for plugin_snap in $(snap connections certbot|sed -n '2,$p'|awk '$1=="content[certbot-1]"{print $3}'|cut -d: -f1); do echo /snap/$plugin_snap/current/lib/python3.8/site-packages; done)
-export CERTBOT_PLUGIN_PATH=$(join : $paths)
+# This code is based on snapcraft's own patch to work around this problem at
+# https://github.com/snapcore/snapcraft/blob/a97fb5c7ea553a1bd20f4887a7c3393e75761890/patches/ctypes_init.diff.
+# We may not build the Certbot snap for all of these architectures (and as of
+# writing this we do not), but we keep the code for them to avoid having to
+# solve this problem again in the future if we add support for new
+# architectures.
+case "${SNAP_ARCH}" in
+    'arm64')
+        ARCH_TRIPLET='aarch64-linux-gnu';;
+    'armhf')
+        ARCH_TRIPLET='arm-linux-gnueabihf';;
+    'i386')
+        ARCH_TRIPLET='i386-linux-gnu';;
+    'ppc64el')
+        ARCH_TRIPLET='powerpc64le-linux-gnu';;
+    'powerpc')
+        ARCH_TRIPLET='powerpc-linux-gnu';;
+    'amd64')
+        ARCH_TRIPLET='x86_64-linux-gnu';;
+    's390x')
+        ARCH_TRIPLET='s390x-linux-gnu';;
+    *)
+        echo "Unrecongized value of SNAP_ARCH: ${SNAP_ARCH}" >&2
+        exit 1
+esac
+
+export CERTBOT_AUGEAS_PATH="${SNAP}/usr/lib/${ARCH_TRIPLET}/libaugeas.so.0"
+
+CERTBOT_PLUGIN_PATH="$(curl -s --unix-socket /run/snapd.socket "http://localhost/v2/connections?snap=certbot&interface=content" | jq -r  '.result.established | map(select(.plug.plug == "plugin" and ."plug-attrs".content == "certbot-1") | "/snap/"+.slot.snap+"/current/lib/python3.8/site-packages/" ) | join(":")')"
+export CERTBOT_PLUGIN_PATH
+
 exec certbot "$@"

certbot/CHANGELOG.md

@@ -2,7 +2,7 @@
 
 Certbot adheres to [Semantic Versioning](https://semver.org/).
 
-## 1.8.0 - master
+## 1.9.0 - master
 
 ### Added
 
@@ -10,11 +10,32 @@ Certbot adheres to [Semantic Versioning](https://semver.org/).
 
 ### Changed
 
+*
+
+### Fixed
+
+* Fixed `server_name` case-sensitivity in the nginx plugin.
+
+More details about these changes can be found on our GitHub repo.
+
+## 1.8.0 - 2020-09-08
+
+### Added
+
+* Added the ability to remove email and phone contact information from an account 
+  using `update_account --register-unsafely-without-email`
+
+### Changed
+
 * Support for Python 3.5 has been removed.
 
 ### Fixed
 
-*
+* The problem causing the Apache plugin in the Certbot snap on ARM systems to
+  fail to load the Augeas library it depends on has been fixed.
+* The `acme` library can now tell the ACME server to clear contact information by passing an empty
+  `tuple` to the `contact` field of a `Registration` message. 
+* Fixed the `*** stack smashing detected ***` error in the Certbot snap on some systems.
 
 More details about these changes can be found on our GitHub repo.
 
@@ -36,7 +57,6 @@ More details about these changes can be found on our GitHub repo.
 
 ### Fixed
 
-*
 
 More details about these changes can be found on our GitHub repo.
 

certbot/README.rst

@@ -67,9 +67,9 @@ Let's Encrypt Website: https://letsencrypt.org
 
 Community: https://community.letsencrypt.org
 
-ACME spec: http://ietf-wg-acme.github.io/acme/
+ACME spec: `RFC 8555 <https://tools.ietf.org/html/rfc8555>`_
 
-ACME working area in github: https://github.com/ietf-wg-acme/acme
+ACME working area in github (archived): https://github.com/ietf-wg-acme/acme
 
 |build-status|
 

certbot/certbot/__init__.py

@@ -1,4 +1,4 @@
 """Certbot client."""
 
 # version number like 1.2.3a0, must have at least 2 parts, like 1.2
-__version__ = '1.8.0.dev0'
+__version__ = '1.9.0.dev0'

certbot/certbot/_internal/cli/__init__.py

@@ -171,13 +171,10 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False):
         ["register", "automation"], "--register-unsafely-without-email", action="store_true",
         default=flag_default("register_unsafely_without_email"),
         help="Specifying this flag enables registering an account with no "
-             "email address. This is strongly discouraged, because in the "
-             "event of key loss or account compromise you will irrevocably "
-             "lose access to your account. You will also be unable to receive "
-             "notice about impending expiration or revocation of your "
-             "certificates. Updates to the Subscriber Agreement will still "
-             "affect you, and will be effective 14 days after posting an "
-             "update to the web site.")
+             "email address. This is strongly discouraged, because you will be "
+             "unable to receive notice about impending expiration or "
+             "revocation of your certificates or problems with your Certbot "
+             "installation that will lead to failure to renew.")
     helpful.add(
         ["register", "update_account", "unregister", "automation"], "-m", "--email",
         default=flag_default("email"),

certbot/certbot/_internal/log.py

@@ -319,6 +319,9 @@ def post_arg_parse_except_hook(exc_type, exc_value, trace, debug, log_path):
     # logger.DEBUG should be used
     if debug or not issubclass(exc_type, Exception):
         assert constants.QUIET_LOGGING_LEVEL <= logging.ERROR
+        if exc_type is KeyboardInterrupt:
+            logger.error('Exiting due to user request.')
+            sys.exit(1)
         logger.error('Exiting abnormally:', exc_info=exc_info)
     else:
         logger.debug('Exiting abnormally:', exc_info=exc_info)

certbot/certbot/_internal/main.py

@@ -11,7 +11,7 @@ import josepy as jose
 import zope.component
 
 from acme import errors as acme_errors
-from acme.magic_typing import Union
+from acme.magic_typing import Union, Iterable, Optional  # pylint: disable=unused-import
 import certbot
 from certbot import crypto_util
 from certbot import errors
@@ -590,7 +590,7 @@ def _init_le_client(config, authenticator, installer):
     :type config: interfaces.IConfig
 
     :param authenticator: Acme authentication handler
-    :type authenticator: interfaces.IAuthenticator
+    :type authenticator: Optional[interfaces.IAuthenticator]
     :param installer: Installer object
     :type installer: interfaces.IInstaller
 
@@ -703,17 +703,17 @@ def update_account(config, unused_plugins):
 
     if not accounts:
         return "Could not find an existing account to update."
-    if config.email is None:
-        if config.register_unsafely_without_email:
-            return ("--register-unsafely-without-email provided, however, a "
-                    "new e-mail address must\ncurrently be provided when "
-                    "updating a registration.")
+    if config.email is None and not config.register_unsafely_without_email:
         config.email = display_ops.get_email(optional=False)
 
     acc, acme = _determine_account(config)
     cb_client = client.Client(config, acc, None, None, acme=acme)
+    # Empty list of contacts in case the user is removing all emails
+
+    acc_contacts = () # type: Iterable[str]
+    if config.email:
+        acc_contacts = ['mailto:' + email for email in config.email.split(',')]
     # We rely on an exception to interrupt this process if it didn't work.
-    acc_contacts = ['mailto:' + email for email in config.email.split(',')]
     prev_regr_uri = acc.regr.uri
     acc.regr = cb_client.acme.update_registration(acc.regr.update(
         body=acc.regr.body.update(contact=acc_contacts)))
@@ -722,8 +722,13 @@ def update_account(config, unused_plugins):
     # so that we can also continue to use the account object with acmev1.
     acc.regr = acc.regr.update(uri=prev_regr_uri)
     account_storage.update_regr(acc, cb_client.acme)
-    eff.prepare_subscription(config, acc)
-    add_msg("Your e-mail address was updated to {0}.".format(config.email))
+
+    if config.email is None:
+        add_msg("Any contact information associated with this account has been removed.")
+    else:
+        eff.prepare_subscription(config, acc)
+        add_msg("Your e-mail address was updated to {0}.".format(config.email))
+
     return None
 
 

certbot/certbot/display/ops.py

@@ -6,7 +6,6 @@ import zope.component
 from certbot import errors
 from certbot import interfaces
 from certbot import util
-from certbot.compat import misc
 from certbot.compat import os
 from certbot.display import util as display_util
 
@@ -33,9 +32,10 @@ def get_email(invalid=False, optional=True):
     msg = "Enter email address (used for urgent renewal and security notices)\n"
     unsafe_suggestion = ("\n\nIf you really want to skip this, you can run "
                          "the client with --register-unsafely-without-email "
-                         "but make sure you then backup your account key from "
-                         "{0}\n\n".format(os.path.join(
-                             misc.get_default_folder('config'), 'accounts')))
+                         "but you will then be unable to receive notice about "
+                         "impending expiration or revocation of your "
+                         "certificates or problems with your Certbot "
+                         "installation that will lead to failure to renew.\n\n")
     if optional:
         if invalid:
             msg += unsafe_suggestion

certbot/docs/cli-help.txt

@@ -118,7 +118,7 @@ optional arguments:
                         case, and to know when to deprecate support for past
                         Python versions and flags. If you wish to hide this
                         information from the Let's Encrypt server, set this to
-                        "". (default: CertbotACMEClient/1.7.0 (certbot(-auto);
+                        "". (default: CertbotACMEClient/1.8.0 (certbot(-auto);
                         OS_NAME OS_VERSION) Authenticator/XXX Installer/YYY
                         (SUBCOMMAND; flags: FLAGS) Py/major.minor.patchlevel).
                         The flags encoded in the user agent are: --duplicate,
@@ -373,13 +373,11 @@ register:
   --register-unsafely-without-email
                         Specifying this flag enables registering an account
                         with no email address. This is strongly discouraged,
-                        because in the event of key loss or account compromise
-                        you will irrevocably lose access to your account. You
-                        will also be unable to receive notice about impending
-                        expiration or revocation of your certificates. Updates
-                        to the Subscriber Agreement will still affect you, and
-                        will be effective 14 days after posting an update to
-                        the web site. (default: False)
+                        because you will be unable to receive notice about
+                        impending expiration or revocation of your
+                        certificates or problems with your Certbot
+                        installation that will lead to failure to renew.
+                        (default: False)
   -m EMAIL, --email EMAIL
                         Email used for registration and recovery contact. Use
                         comma to register multiple emails, ex:

certbot/docs/contributing.rst

@@ -18,6 +18,7 @@ Windows, you'll need to set up a (virtual) machine running an OS such as Linux
 and continue with these instructions on that UNIX-like OS.
 
 .. _local copy:
+.. _prerequisites:
 
 Running a local copy of the client
 ----------------------------------
@@ -578,33 +579,3 @@ effect. To do this, run::
 Now running the check for linting errors described above is as easy as::
 
   tox -e lint
-
-.. _prerequisites:
-
-Notes on OS dependencies
-========================
-
-OS-level dependencies can be installed like so:
-
-.. code-block:: shell
-
-   ./certbot-auto --debug --os-packages-only
-
-In general...
-
-* ``sudo`` is required as a suggested way of running privileged process
-* `Python`_ 2.7 or 3.6+ is required
-* `Augeas`_ is required for the Python bindings
-* ``virtualenv`` is used for managing other Python library dependencies
-
-.. _Python: https://wiki.python.org/moin/BeginnersGuide/Download
-.. _Augeas: http://augeas.net/
-.. _Virtualenv: https://virtualenv.pypa.io
-
-
-FreeBSD
--------
-
-FreeBSD by default uses ``tcsh``. In order to activate virtualenv (see
-above), you will need a compatible shell, e.g. ``pkg install bash &&
-bash``.

certbot/docs/install.rst

@@ -18,7 +18,7 @@ Certbot is packaged for many common operating systems and web servers. Check whe
 certbot.eff.org_, where you will also find the correct installation instructions for
 your system.
 
-.. Note:: Unless you have very specific requirements, we kindly suggest that you use the Certbot packages provided by your package manager (see certbot.eff.org_). If such packages are not available, we recommend using ``certbot-auto``, which automates the process of installing Certbot on your system.
+.. Note:: Unless you have very specific requirements, we kindly suggest that you use the installation instructions for your system found at certbot.eff.org_.
 
 .. _certbot.eff.org: https://certbot.eff.org
 
@@ -156,18 +156,17 @@ certificate. However, this mode of operation is unable to install
 certificates or configure your webserver, because our installer
 plugins cannot reach your webserver from inside the Docker container.
 
-Most users should use the operating system packages (see instructions at
-certbot.eff.org_) or, as a fallback, ``certbot-auto``. You should only
-use Docker if you are sure you know what you are doing and have a
-good reason to do so.
+Most users should use the instructions at certbot.eff.org_. You should only use
+Docker if you are sure you know what you are doing and have a good reason to do
+so.
 
 You should definitely read the :ref:`where-certs` section, in order to
 know how to manage the certs
 manually. `Our ciphersuites page <ciphers.html>`__
 provides some information about recommended ciphersuites. If none of
-these make much sense to you, you should definitely use the
-certbot-auto_ method, which enables you to use installer plugins
-that cover both of those hard topics.
+these make much sense to you, you should definitely use the installation method
+recommended for your system at certbot.eff.org_, which enables you to use
+installer plugins that cover both of those hard topics.
 
 If you're still not convinced and have decided to use this method, from
 the server that the domain you're requesting a certficate for resolves

certbot/docs/using.rst

@@ -392,7 +392,7 @@ abuse of the ACME protocol, as described
 .. _changing:
 
 Changing a Certificate's Domains
-================================
+--------------------------------
 
 The ``--cert-name`` flag can also be used to modify the domains a certificate contains,
 by specifying new domains using the ``-d`` or ``--domains`` flag. If certificate ``example.com``

certbot/tests/log_test.py

@@ -306,7 +306,7 @@ class PostArgParseExceptHookTest(unittest.TestCase):
         self.log_path = 'foo.log'
 
     def test_base_exception(self):
-        exc_type = KeyboardInterrupt
+        exc_type = BaseException
         mock_logger, output = self._test_common(exc_type, debug=False)
         self._assert_exception_logged(mock_logger.error, exc_type)
         self._assert_logfile_output(output)
@@ -342,6 +342,11 @@ class PostArgParseExceptHookTest(unittest.TestCase):
         self._assert_exception_logged(mock_logger.debug, exc_type)
         self._assert_quiet_output(mock_logger, output)
 
+    def test_keyboardinterrupt(self):
+        exc_type = KeyboardInterrupt
+        mock_logger, output = self._test_common(exc_type, debug=False)
+        mock_logger.error.assert_called_once_with('Exiting due to user request.')
+
     def _test_common(self, error_type, debug):
         """Returns the mocked logger and stderr output."""
         mock_err = six.StringIO()

certbot/tests/main_test.py

@@ -1404,6 +1404,43 @@ class MainTest(test_util.ConfigTestCase):
                  "user@example.org"])
             self.assertTrue("Could not find an existing account" in x[0])
 
+    @mock.patch('certbot._internal.main._determine_account')
+    @mock.patch('certbot._internal.eff.prepare_subscription')
+    @mock.patch('certbot._internal.main.account')
+    def test_update_account_remove_email(self, mocked_account_module, mock_prepare, mock_det_acc):
+        # Mock account storage and the account object returned
+        mocked_storage = mock.MagicMock()
+        mocked_account = mock.MagicMock()
+
+        mocked_account_module.AccountFileStorage.return_value = mocked_storage
+        mocked_storage.find_all.return_value = [mocked_account]
+        mock_det_acc.return_value = (mocked_account, "foo")
+
+        # Mock registration body to verify calls are made
+        mock_regr_body = mock.MagicMock()
+
+        # mocked_account.regr is overwritten in update, requiring an odd mock setup
+        mocked_account.regr.body = mock_regr_body
+
+        x = self._call(
+            ["update_account", "--register-unsafely-without-email"])
+
+
+        # When update succeeds, the return value of update_account() is None
+        self.assertTrue(x[0] is None)
+        # and we got supposedly did update the registration from
+        # the server
+        client_mock = x[3]
+        self.assertTrue(client_mock.Client().acme.update_registration.called)
+
+        self.assertTrue(mock_regr_body.update.called)
+        self.assertTrue('contact' in mock_regr_body.update.call_args[1])
+        self.assertEqual(mock_regr_body.update.call_args[1]['contact'], ())
+        # and we saved the updated registration on disk
+        self.assertTrue(mocked_storage.update_regr.called)
+        # ensure we didn't try to subscribe (no email to subscribe with)
+        self.assertFalse(mock_prepare.called)
+
     @mock.patch('certbot._internal.main.display_ops.get_email')
     @test_util.patch_get_utility()
     def test_update_account_with_email(self, mock_utility, mock_email):

letsencrypt-auto

@@ -31,7 +31,7 @@ if [ -z "$VENV_PATH" ]; then
 fi
 VENV_BIN="$VENV_PATH/bin"
 BOOTSTRAP_VERSION_PATH="$VENV_PATH/certbot-auto-bootstrap-version.txt"
-LE_AUTO_VERSION="1.7.0"
+LE_AUTO_VERSION="1.8.0"
 BASENAME=$(basename $0)
 USAGE="Usage: $BASENAME [OPTIONS]
 A self-updating wrapper script for the Certbot ACME client. When run, updates
@@ -258,7 +258,7 @@ DeprecationBootstrap() {
 
 MIN_PYTHON_2_VERSION="2.7"
 MIN_PYVER2=$(echo "$MIN_PYTHON_2_VERSION" | sed 's/\.//')
-MIN_PYTHON_3_VERSION="3.5"
+MIN_PYTHON_3_VERSION="3.6"
 MIN_PYVER3=$(echo "$MIN_PYTHON_3_VERSION" | sed 's/\.//')
 # Sets LE_PYTHON to Python version string and PYVER to the first two
 # digits of the python version.
@@ -930,7 +930,7 @@ else
     error "Sorry, I don't know how to bootstrap Certbot on your operating system!"
     error
     error "You will need to install OS dependencies, configure virtualenv, and run pip install manually."
-    error "Please see https://letsencrypt.readthedocs.org/en/latest/contributing.html#prerequisites"
+    error "Please see https://certbot.eff.org/docs/contributing.html#prerequisites"
     error "for more info."
     exit 1
   }
@@ -1530,18 +1530,18 @@ letsencrypt==0.7.0 \
     --hash=sha256:105a5fb107e45bcd0722eb89696986dcf5f08a86a321d6aef25a0c7c63375ade \
     --hash=sha256:c36e532c486a7e92155ee09da54b436a3c420813ec1c590b98f635d924720de9
 
-certbot==1.7.0 \
-    --hash=sha256:84877127caf779c212d131d36399a45a8e13e06274e7a5e029845df5c84cd974 \
-    --hash=sha256:10b95bb86fb8f1dbbd27558bb42454d5995cbdb45d6c00d961ebba2a4bdc4355
-acme==1.7.0 \
-    --hash=sha256:ef0e84d670f59c096e9ed8c3bf9e6a7d22ee378fdb4175503c06cc485672c79a \
-    --hash=sha256:288d9bbb075278961d224e43f7f386c491d25366a98ed89a62771c5022978386
-certbot-apache==1.7.0 \
-    --hash=sha256:514c09a892964332c2485a38bd5720e4cf93e35998341af36eef5401ab165d89 \
-    --hash=sha256:99943b6406e0315f31c1f81e2ced6be38aee3ea24974ef4d7aeeda8202c1c3bc
-certbot-nginx==1.7.0 \
-    --hash=sha256:fea2387c92155635fbddb02758d5ba73f0d7af459959f91be0a1606fd2e43c55 \
-    --hash=sha256:d52ec3e711884100636c42b639d8959378562ea78183a273d120df808de2724f
+certbot==1.8.0 \
+    --hash=sha256:4bde86c53e30dc5bc0e78a0862045b053971703af727ac20c6a7da06596c7549 \
+    --hash=sha256:4837c516af6543ccd10d70f1498a2113bbdf9ef9a05d3a18b1558b291a2953e4
+acme==1.8.0 \
+    --hash=sha256:465033830a75f98042236f50f751f6e316735473ccb4edec0c718263f6c9ba8b \
+    --hash=sha256:ad8d067d14258d73ad2643439d9365913362308c04e66cc3010e39c868c5002d
+certbot-apache==1.8.0 \
+    --hash=sha256:8c9d981803e1156725fcfcf228afcb754b245c9d506e5b9f4fca948d6ae89aef \
+    --hash=sha256:a93c3a7ad929fe0ba5e0868e29ee2d0fe10aea2d4c638a902c4613a5c12c59b6
+certbot-nginx==1.8.0 \
+    --hash=sha256:e98e883b5ea7b29dd2e6a8ff286c7550a2d7af2fc859f47067303e510ad4fb52 \
+    --hash=sha256:fdb96c74fe42d90bbaf11a00314444ac5544ba87292a1b8b1d707f7561a3eacc
 
 UNLIKELY_EOF
     # -------------------------------------------------------------------------

letsencrypt-auto-source/certbot-auto.asc

@@ -1,11 +1,11 @@
 -----BEGIN PGP SIGNATURE-----
 
-iQEzBAABCAAdFiEEos+1H6J1pyhiNOeyTRfJlc2XdfIFAl8pqd4ACgkQTRfJlc2X
-dfK7eAgAoYZrhhR4bEB+/mx8NjxT3gaS3rf6/r40OO04Dgb6gqnfunR1S9pvoGyE
-P3K1FsCftAYxeI5XH2gG8eQUS1rKXJT3sTr8aDqFOhZtiTiGyknL8iBJNpAG1n/I
-xRV/Y+HreBHU3hxNIS9I4LMxu0/4gJqtTvZONcO+NeFeOb80ED9IvvSEk6muTPYO
-YD46p1lf8omnx4wRvT8NblB7Tl0MNVvmk7i1fnYAKyItoh3TYBtarRk120lp48XZ
-5K9FPf2+RNyaU222Iv3Fy6q2d0nxysIWtlgObz/OVHiFUNPBKRPVHYQJlT2OwyIx
-gVHJkxuM5P/AxeD8rpwOJqhfHVUm2A==
-=FQqP
+iQEzBAABCAAdFiEEos+1H6J1pyhiNOeyTRfJlc2XdfIFAl9XuHIACgkQTRfJlc2X
+dfIU8wgAkwXao63sZxfiRfeQfzyM01oYEaqjp17gX/f0QhxvmrBUIdBKsF3TBZ9H
+7c3NYlBxJ31/a5PVfzElQJAzqMl4yEdlZK1mxKEepQycmW+vHOq8DOfpvOU957ro
+cRBpDcu5BK+/tKPqVTHpLRZX7SFjzpunwKmmdCz1JzxLuf0Wgrqmq678Yyh6rLdT
+96j7bDhHCDg0R2RC3hL1yk9HyMdh/nSKUYNnQdqAi/YSybclHXBU2NJURupMrei1
+6LLoE6I8wo4LXptCaM48kQEHBKGwdMWeimVkos0YbmIzcPbmmetmu+MvrL/T/Dz8
+6OEBdYbAkMdT2gzseq76CYEUeWhL0g==
+=1K2n
 -----END PGP SIGNATURE-----

letsencrypt-auto-source/letsencrypt-auto

@@ -31,7 +31,7 @@ if [ -z "$VENV_PATH" ]; then
 fi
 VENV_BIN="$VENV_PATH/bin"
 BOOTSTRAP_VERSION_PATH="$VENV_PATH/certbot-auto-bootstrap-version.txt"
-LE_AUTO_VERSION="1.8.0.dev0"
+LE_AUTO_VERSION="1.9.0.dev0"
 BASENAME=$(basename $0)
 USAGE="Usage: $BASENAME [OPTIONS]
 A self-updating wrapper script for the Certbot ACME client. When run, updates
@@ -930,7 +930,7 @@ else
     error "Sorry, I don't know how to bootstrap Certbot on your operating system!"
     error
     error "You will need to install OS dependencies, configure virtualenv, and run pip install manually."
-    error "Please see https://letsencrypt.readthedocs.org/en/latest/contributing.html#prerequisites"
+    error "Please see https://certbot.eff.org/docs/contributing.html#prerequisites"
     error "for more info."
     exit 1
   }
@@ -1530,18 +1530,18 @@ letsencrypt==0.7.0 \
     --hash=sha256:105a5fb107e45bcd0722eb89696986dcf5f08a86a321d6aef25a0c7c63375ade \
     --hash=sha256:c36e532c486a7e92155ee09da54b436a3c420813ec1c590b98f635d924720de9
 
-certbot==1.7.0 \
-    --hash=sha256:84877127caf779c212d131d36399a45a8e13e06274e7a5e029845df5c84cd974 \
-    --hash=sha256:10b95bb86fb8f1dbbd27558bb42454d5995cbdb45d6c00d961ebba2a4bdc4355
-acme==1.7.0 \
-    --hash=sha256:ef0e84d670f59c096e9ed8c3bf9e6a7d22ee378fdb4175503c06cc485672c79a \
-    --hash=sha256:288d9bbb075278961d224e43f7f386c491d25366a98ed89a62771c5022978386
-certbot-apache==1.7.0 \
-    --hash=sha256:514c09a892964332c2485a38bd5720e4cf93e35998341af36eef5401ab165d89 \
-    --hash=sha256:99943b6406e0315f31c1f81e2ced6be38aee3ea24974ef4d7aeeda8202c1c3bc
-certbot-nginx==1.7.0 \
-    --hash=sha256:fea2387c92155635fbddb02758d5ba73f0d7af459959f91be0a1606fd2e43c55 \
-    --hash=sha256:d52ec3e711884100636c42b639d8959378562ea78183a273d120df808de2724f
+certbot==1.8.0 \
+    --hash=sha256:4bde86c53e30dc5bc0e78a0862045b053971703af727ac20c6a7da06596c7549 \
+    --hash=sha256:4837c516af6543ccd10d70f1498a2113bbdf9ef9a05d3a18b1558b291a2953e4
+acme==1.8.0 \
+    --hash=sha256:465033830a75f98042236f50f751f6e316735473ccb4edec0c718263f6c9ba8b \
+    --hash=sha256:ad8d067d14258d73ad2643439d9365913362308c04e66cc3010e39c868c5002d
+certbot-apache==1.8.0 \
+    --hash=sha256:8c9d981803e1156725fcfcf228afcb754b245c9d506e5b9f4fca948d6ae89aef \
+    --hash=sha256:a93c3a7ad929fe0ba5e0868e29ee2d0fe10aea2d4c638a902c4613a5c12c59b6
+certbot-nginx==1.8.0 \
+    --hash=sha256:e98e883b5ea7b29dd2e6a8ff286c7550a2d7af2fc859f47067303e510ad4fb52 \
+    --hash=sha256:fdb96c74fe42d90bbaf11a00314444ac5544ba87292a1b8b1d707f7561a3eacc
 
 UNLIKELY_EOF
     # -------------------------------------------------------------------------

letsencrypt-auto-source/letsencrypt-auto.template

@@ -452,7 +452,7 @@ else
     error "Sorry, I don't know how to bootstrap Certbot on your operating system!"
     error
     error "You will need to install OS dependencies, configure virtualenv, and run pip install manually."
-    error "Please see https://letsencrypt.readthedocs.org/en/latest/contributing.html#prerequisites"
+    error "Please see https://certbot.eff.org/docs/contributing.html#prerequisites"
     error "for more info."
     exit 1
   }

letsencrypt-auto-source/pieces/certbot-requirements.txt

@@ -1,12 +1,12 @@
-certbot==1.7.0 \
-    --hash=sha256:84877127caf779c212d131d36399a45a8e13e06274e7a5e029845df5c84cd974 \
-    --hash=sha256:10b95bb86fb8f1dbbd27558bb42454d5995cbdb45d6c00d961ebba2a4bdc4355
-acme==1.7.0 \
-    --hash=sha256:ef0e84d670f59c096e9ed8c3bf9e6a7d22ee378fdb4175503c06cc485672c79a \
-    --hash=sha256:288d9bbb075278961d224e43f7f386c491d25366a98ed89a62771c5022978386
-certbot-apache==1.7.0 \
-    --hash=sha256:514c09a892964332c2485a38bd5720e4cf93e35998341af36eef5401ab165d89 \
-    --hash=sha256:99943b6406e0315f31c1f81e2ced6be38aee3ea24974ef4d7aeeda8202c1c3bc
-certbot-nginx==1.7.0 \
-    --hash=sha256:fea2387c92155635fbddb02758d5ba73f0d7af459959f91be0a1606fd2e43c55 \
-    --hash=sha256:d52ec3e711884100636c42b639d8959378562ea78183a273d120df808de2724f
+certbot==1.8.0 \
+    --hash=sha256:4bde86c53e30dc5bc0e78a0862045b053971703af727ac20c6a7da06596c7549 \
+    --hash=sha256:4837c516af6543ccd10d70f1498a2113bbdf9ef9a05d3a18b1558b291a2953e4
+acme==1.8.0 \
+    --hash=sha256:465033830a75f98042236f50f751f6e316735473ccb4edec0c718263f6c9ba8b \
+    --hash=sha256:ad8d067d14258d73ad2643439d9365913362308c04e66cc3010e39c868c5002d
+certbot-apache==1.8.0 \
+    --hash=sha256:8c9d981803e1156725fcfcf228afcb754b245c9d506e5b9f4fca948d6ae89aef \
+    --hash=sha256:a93c3a7ad929fe0ba5e0868e29ee2d0fe10aea2d4c638a902c4613a5c12c59b6
+certbot-nginx==1.8.0 \
+    --hash=sha256:e98e883b5ea7b29dd2e6a8ff286c7550a2d7af2fc859f47067303e510ad4fb52 \
+    --hash=sha256:fdb96c74fe42d90bbaf11a00314444ac5544ba87292a1b8b1d707f7561a3eacc

snap/hooks/configure

@@ -1,3 +1,3 @@
-#!/bin/bash -e
+#!/bin/sh -e
 
 exit 0

snap/hooks/prepare-plug-plugin

@@ -1,4 +1,4 @@
-#!/bin/bash -e
+#!/bin/sh -e
 
 if [ "$(snapctl get trust-plugin-with-root)" = "ok" ]; then
     # allow the connection, but reset config to allow for other slots to go through this auth flow

snap/snapcraft.yaml

@@ -24,7 +24,6 @@ apps:
     environment:
       PATH: "$SNAP/bin:$SNAP/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
       AUGEAS_LENS_LIB: "$SNAP/usr/share/augeas/lenses/dist"
-      LD_LIBRARY_PATH: "$SNAP/usr/lib/x86_64-linux-gnu/:$LD_LIBRARY_PATH"
       CERTBOT_SNAPPED: "True"
   renew:
     command: certbot.wrapper -q renew
@@ -32,7 +31,6 @@ apps:
     environment:
       PATH: "$SNAP/bin:$SNAP/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
       AUGEAS_LENS_LIB: $SNAP/usr/share/augeas/lenses/dist
-      LD_LIBRARY_PATH: "$SNAP/usr/lib/x86_64-linux-gnu/:$LD_LIBRARY_PATH"
       CERTBOT_SNAPPED: "True"
     # Run approximately twice a day with randomization
     timer: 00:00~24:00/2
@@ -44,7 +42,7 @@ parts:
     source: .
     constraints: [$SNAPCRAFT_PART_SRC/snap-constraints.txt]
     python-packages:
-      - git+https://github.com/basak/python-augeas.git@snap
+      - git+https://github.com/certbot/python-augeas.git@certbot-patched
       - ./acme
       - ./certbot
       - ./certbot-apache
@@ -54,7 +52,7 @@ parts:
       # Old versions of this file used to unstage
       # lib/python3.8/site-packages/augeas.py to avoid conflicts between
       # python-augeas 0.5.0 which was pinned in snap-constraints.txt and
-      # Robie's python-augeas fork which creates an auto-generated cffi file at
+      # our python-augeas fork which creates an auto-generated cffi file at
       # the same path. Since we've combined things in one part and removed the
       # python-augeas pinning, unstaging this file had a different, unintended
       # effect so we now stage the file to keep the auto-generated cffi file.
@@ -73,10 +71,14 @@ parts:
       - python3-distutils
       - python3-pkg-resources
       - python3.8-minimal
+      # added for certbot.wrapper script:
+      - curl
+      - jq
     # To build cryptography and cffi if needed
     build-packages: [gcc, libffi-dev, libssl-dev, git, libaugeas-dev, python3-dev]
     build-environment:
       - SNAPCRAFT_PYTHON_VENV_ARGS: --system-site-packages
+      - PIP_NO_BUILD_ISOLATION: "no"
     override-pull: |
         snapcraftctl pull
         cd $SNAPCRAFT_PART_SRC

tests/letstest/apache2_targets.yaml

@@ -8,12 +8,6 @@ targets:
     type: ubuntu
     virt: hvm
     user: ubuntu
-  - ami: ami-008680ee60f23c94b
-    name: ubuntu20.04_arm64
-    type: ubuntu
-    virt: hvm
-    user: ubuntu
-    machine_type: a1.medium
   - ami: ami-0545f7036167eb3aa
     name: ubuntu19.10
     type: ubuntu
@@ -36,6 +30,12 @@ targets:
     type: ubuntu
     virt: hvm
     user: admin
+  - ami: ami-0dcd54b7d2fff584f
+    name: debian10_arm64
+    type: ubuntu
+    virt: hvm
+    user: admin
+    machine_type: a1.medium
   - ami: ami-003f19e0e687de1cd
     name: debian9
     type: ubuntu

tests/letstest/multitester.py

@@ -99,7 +99,6 @@ PROFILE = None if cl_args.aws_profile == 'SET_BY_ENV' else cl_args.aws_profile
 # Globals
 #-------------------------------------------------------------------------------
 BOULDER_AMI = 'ami-072a9534772bec854' # premade shared boulder AMI 18.04LTS us-east-1
-LOGDIR = "letest-%d"%int(time.time()) #points to logging / working directory
 SECURITY_GROUP_NAME = 'certbot-security-group'
 SENTINEL = None #queue kill signal
 SUBNET_NAME = 'certbot-subnet'
@@ -233,43 +232,43 @@ def block_until_instance_ready(booting_instance, wait_time=5, extra_wait_time=20
 
 # Fabric Routines
 #-------------------------------------------------------------------------------
-def local_git_clone(local_cxn, repo_url):
+def local_git_clone(local_cxn, repo_url, log_dir):
     """clones master of repo_url"""
-    local_cxn.local('cd %s && if [ -d letsencrypt ]; then rm -rf letsencrypt; fi' % LOGDIR)
-    local_cxn.local('cd %s && git clone %s letsencrypt'% (LOGDIR, repo_url))
-    local_cxn.local('cd %s && tar czf le.tar.gz letsencrypt'% LOGDIR)
+    local_cxn.local('cd %s && if [ -d letsencrypt ]; then rm -rf letsencrypt; fi' % log_dir)
+    local_cxn.local('cd %s && git clone %s letsencrypt'% (log_dir, repo_url))
+    local_cxn.local('cd %s && tar czf le.tar.gz letsencrypt'% log_dir)
 
-def local_git_branch(local_cxn, repo_url, branch_name):
+def local_git_branch(local_cxn, repo_url, branch_name, log_dir):
     """clones branch <branch_name> of repo_url"""
-    local_cxn.local('cd %s && if [ -d letsencrypt ]; then rm -rf letsencrypt; fi' % LOGDIR)
+    local_cxn.local('cd %s && if [ -d letsencrypt ]; then rm -rf letsencrypt; fi' % log_dir)
     local_cxn.local('cd %s && git clone %s letsencrypt --branch %s --single-branch'%
-        (LOGDIR, repo_url, branch_name))
-    local_cxn.local('cd %s && tar czf le.tar.gz letsencrypt' % LOGDIR)
+        (log_dir, repo_url, branch_name))
+    local_cxn.local('cd %s && tar czf le.tar.gz letsencrypt' % log_dir)
 
-def local_git_PR(local_cxn, repo_url, PRnumstr, merge_master=True):
+def local_git_PR(local_cxn, repo_url, PRnumstr, log_dir, merge_master=True):
     """clones specified pull request from repo_url and optionally merges into master"""
-    local_cxn.local('cd %s && if [ -d letsencrypt ]; then rm -rf letsencrypt; fi' % LOGDIR)
-    local_cxn.local('cd %s && git clone %s letsencrypt' % (LOGDIR, repo_url))
+    local_cxn.local('cd %s && if [ -d letsencrypt ]; then rm -rf letsencrypt; fi' % log_dir)
+    local_cxn.local('cd %s && git clone %s letsencrypt' % (log_dir, repo_url))
     local_cxn.local('cd %s && cd letsencrypt && '
-        'git fetch origin pull/%s/head:lePRtest' % (LOGDIR, PRnumstr))
-    local_cxn.local('cd %s && cd letsencrypt && git checkout lePRtest' % LOGDIR)
+        'git fetch origin pull/%s/head:lePRtest' % (log_dir, PRnumstr))
+    local_cxn.local('cd %s && cd letsencrypt && git checkout lePRtest' % log_dir)
     if merge_master:
-        local_cxn.local('cd %s && cd letsencrypt && git remote update origin' % LOGDIR)
+        local_cxn.local('cd %s && cd letsencrypt && git remote update origin' % log_dir)
         local_cxn.local('cd %s && cd letsencrypt && '
-            'git merge origin/master -m "testmerge"' % LOGDIR)
-    local_cxn.local('cd %s && tar czf le.tar.gz letsencrypt' % LOGDIR)
+            'git merge origin/master -m "testmerge"' % log_dir)
+    local_cxn.local('cd %s && tar czf le.tar.gz letsencrypt' % log_dir)
 
-def local_repo_to_remote(cxn):
+def local_repo_to_remote(cxn, log_dir):
     """copies local tarball of repo to remote"""
     filename = 'le.tar.gz'
-    local_path = os.path.join(LOGDIR, filename)
+    local_path = os.path.join(log_dir, filename)
     cxn.put(local=local_path, remote='')
     cxn.run('tar xzf %s' % filename)
 
-def local_repo_clean(local_cxn):
+def local_repo_clean(local_cxn, log_dir):
     """delete tarball"""
     filename = 'le.tar.gz'
-    local_path = os.path.join(LOGDIR, filename)
+    local_path = os.path.join(log_dir, filename)
     local_cxn.local('rm %s' % local_path)
 
 def deploy_script(cxn, scriptpath, *args):
@@ -289,8 +288,8 @@ def config_and_launch_boulder(cxn, instance):
         deploy_script(cxn, 'scripts/boulder_config.sh')
         run_boulder(cxn)
 
-def install_and_launch_certbot(cxn, instance, boulder_url, target):
-    local_repo_to_remote(cxn)
+def install_and_launch_certbot(cxn, instance, boulder_url, target, log_dir):
+    local_repo_to_remote(cxn, log_dir)
     # This needs to be like this, I promise. 1) The env argument to run doesn't work.
     # See https://github.com/fabric/fabric/issues/1744. 2) prefix() sticks an && between
     # the commands, so it needs to be exports rather than no &&s in between for the script subshell.
@@ -339,7 +338,7 @@ def create_client_instance(ec2_client, target, security_group_id, subnet_id):
                          userdata=userdata)
 
 
-def test_client_process(fab_config, inqueue, outqueue, boulder_url):
+def test_client_process(fab_config, inqueue, outqueue, boulder_url, log_dir):
     cur_proc = mp.current_process()
     for inreq in iter(inqueue.get, SENTINEL):
         ii, instance_id, target = inreq
@@ -351,7 +350,7 @@ def test_client_process(fab_config, inqueue, outqueue, boulder_url):
         instance = ec2_client.Instance(id=instance_id)
 
         #save all stdout to log file
-        sys.stdout = open(LOGDIR+'/'+'%d_%s.log'%(ii,target['name']), 'w')
+        sys.stdout = open(log_dir+'/'+'%d_%s.log'%(ii,target['name']), 'w')
 
         print("[%s : client %d %s %s]" % (cur_proc.name, ii, target['ami'], target['name']))
         instance = block_until_instance_ready(instance)
@@ -361,7 +360,7 @@ def test_client_process(fab_config, inqueue, outqueue, boulder_url):
 
         with Connection(host_string, config=fab_config) as cxn:
             try:
-                install_and_launch_certbot(cxn, instance, boulder_url, target)
+                install_and_launch_certbot(cxn, instance, boulder_url, target, log_dir)
                 outqueue.put((ii, target, Status.PASS))
                 print("%s - %s SUCCESS"%(target['ami'], target['name']))
             except:
@@ -380,8 +379,8 @@ def test_client_process(fab_config, inqueue, outqueue, boulder_url):
                 pass
 
 
-def cleanup(cl_args, instances, targetlist, boulder_server):
-    print('Logs in ', LOGDIR)
+def cleanup(cl_args, instances, targetlist, boulder_server, log_dir):
+    print('Logs in ', log_dir)
     # If lengths of instances and targetlist aren't equal, instances failed to
     # start before running tests so leaving instances running for debugging
     # isn't very useful. Let's cleanup after ourselves instead.
@@ -418,22 +417,24 @@ def main():
 
     # Set up local copy of git repo
     #-------------------------------------------------------------------------------
-    print("Making local dir for test repo and logs: %s"%LOGDIR)
-    local_cxn.local('mkdir %s'%LOGDIR)
+    log_dir = "letest-%d"%int(time.time()) #points to logging / working directory
+    print("Making local dir for test repo and logs: %s"%log_dir)
+    local_cxn.local('mkdir %s'%log_dir)
 
     try:
-        # figure out what git object to test and locally create it in LOGDIR
+        # figure out what git object to test and locally create it in log_dir
         print("Making local git repo")
         if cl_args.pull_request != '~':
             print('Testing PR %s ' % cl_args.pull_request,
                   "MERGING into master" if cl_args.merge_master else "")
-            local_git_PR(local_cxn, cl_args.repo, cl_args.pull_request, cl_args.merge_master)
+            local_git_PR(local_cxn, cl_args.repo, cl_args.pull_request, log_dir,
+                         cl_args.merge_master)
         elif cl_args.branch != '~':
             print('Testing branch %s of %s' % (cl_args.branch, cl_args.repo))
-            local_git_branch(local_cxn, cl_args.repo, cl_args.branch)
+            local_git_branch(local_cxn, cl_args.repo, cl_args.branch, log_dir)
         else:
-            print('Testing current branch of %s' % cl_args.repo)
-            local_git_clone(local_cxn, cl_args.repo)
+            print('Testing current branch of %s' % cl_args.repo, log_dir)
+            local_git_clone(local_cxn, cl_args.repo, log_dir)
     except BaseException:
         print("FAIL: trouble with git repo")
         traceback.print_exc()
@@ -537,7 +538,7 @@ def main():
         # Install and launch client scripts in parallel
         #-------------------------------------------------------------------------------
         print("Uploading and running test script in parallel: %s"%cl_args.test_script)
-        print("Output routed to log files in %s"%LOGDIR)
+        print("Output routed to log files in %s"%log_dir)
         # (Advice: always use Manager.Queue, never regular multiprocessing.Queue
         # the latter has implementation flaws that deadlock it in some circumstances)
         manager = Manager()
@@ -550,8 +551,9 @@ def main():
 
 
         # initiate process execution
+        client_process_args=(fab_config, inqueue, outqueue, boulder_url, log_dir)
         for i in range(num_processes):
-            p = mp.Process(target=test_client_process, args=(fab_config, inqueue, outqueue, boulder_url))
+            p = mp.Process(target=test_client_process, args=client_process_args)
             jobs.append(p)
             p.daemon = True  # kills subprocesses if parent is killed
             p.start()
@@ -575,10 +577,10 @@ def main():
         outqueue.put(SENTINEL)
 
         # clean up
-        local_repo_clean(local_cxn)
+        local_repo_clean(local_cxn, log_dir)
 
         # print and save summary results
-        results_file = open(LOGDIR+'/results', 'w')
+        results_file = open(log_dir+'/results', 'w')
         outputs = [outq for outq in iter(outqueue.get, SENTINEL)]
         outputs.sort(key=lambda x: x[0])
         failed = False
@@ -600,7 +602,7 @@ def main():
             sys.exit(1)
 
     finally:
-        cleanup(cl_args, instances, targetlist, boulder_server)
+        cleanup(cl_args, instances, targetlist, boulder_server, log_dir)
 
 
 if __name__ == '__main__':

tests/letstest/scripts/test_apache2.sh

@@ -12,7 +12,7 @@ then
     # For apache 2.4, set up ServerName
     sudo sed -i '/ServerName/ s/#ServerName/ServerName/' $CONFFILE
     sudo sed -i '/ServerName/ s/www.example.com/'$PUBLIC_HOSTNAME'/' $CONFFILE
-    if [ $(python3 -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//') -ne 38 ]
+    if [ $(python3 -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//') -lt 36 ]
     then
         # Upgrade python version using pyenv because py3.5 is deprecated
         # Don't upgrade if it's already 3.8 because pyenv doesn't work great on arm, and

tests/letstest/targets.yaml

@@ -8,12 +8,6 @@ targets:
     type: ubuntu
     virt: hvm
     user: ubuntu
-  - ami: ami-008680ee60f23c94b
-    name: ubuntu20.04_arm64
-    type: ubuntu
-    virt: hvm
-    user: ubuntu
-    machine_type: a1.medium
   - ami: ami-0545f7036167eb3aa
     name: ubuntu19.10
     type: ubuntu
@@ -31,6 +25,12 @@ targets:
     type: ubuntu
     virt: hvm
     user: admin
+  - ami: ami-0dcd54b7d2fff584f
+    name: debian10_arm64
+    type: ubuntu
+    virt: hvm
+    user: admin
+    machine_type: a1.medium
   #-----------------------------------------------------------------------------
   # Other Redhat Distros
   - ami: ami-0916c408cb02e310b