Compare commits

...

27 Commits

Author SHA1 Message Date
Brad Warren
d0bb0c4cc8 quiet plz 2018-05-01 17:29:09 -07:00
Brad Warren
1cb821f567 Add set -e 2018-05-01 17:27:57 -07:00
Brad Warren
21267bae1f fail under 65 2018-05-01 17:27:57 -07:00
Brad Warren
bb220c87dd allow for running individual integration tests 2018-05-01 17:27:57 -07:00
Brad Warren
d33ab17e6e Remove nginx from certbot tests 2018-05-01 17:27:57 -07:00
Brad Warren
770766dd4a Rename Certbot integration tests 2018-05-01 17:27:57 -07:00
ohemorange
6d82061b42 Merge pull request #5880 from certbot/mypy-everything
Merge master into test-everything
2018-04-16 23:51:40 -07:00
Brad Warren
a4a8b6e994 Merge branch 'master' into mypy-everything 2018-04-16 08:44:40 -07:00
ohemorange
9defbc3aed Add before_install to all tests running on test-everything (#5806)
* add before_install to all tests running on test-everything

* actually always run before_install

* also don't override addons

* only install libaugeas0 for docker_dev
2018-03-29 17:18:15 -07:00
Brad Warren
f1461946b8 Merge pull request #5688 from certbot/update-test-everything
Update the test-everything branch
2018-03-11 15:45:14 -07:00
Brad Warren
b7472fb337 Merge branch 'master' into update-test-everything 2018-03-08 11:30:09 -08:00
Brad Warren
937e27db28 Merge branch 'master' into update-test-everything 2018-03-07 15:57:07 -08:00
Brad Warren
e64137fb53 Merge pull request #5668 from certbot/update-test-everything
Fix merge conflicts and add more ACMEv2 integration tests
2018-03-06 09:19:25 -08:00
Brad Warren
a0dc7dafda add python3 ACMEv2 integration tests 2018-03-05 12:02:05 -08:00
Brad Warren
02975c84d9 set up nginx-oldest integration tests 2018-03-05 12:01:26 -08:00
Brad Warren
5307600a03 use v1 for clarity 2018-03-05 12:00:44 -08:00
Brad Warren
b0ce7529aa Merge branch 'master' into update-test-everything 2018-03-05 12:00:07 -08:00
Joona Hoikkala
5abec9fa3c Python3 now installed as default let's upgrade instead of trying to install (#5665) 2018-03-05 10:01:46 -08:00
Brad Warren
8753959bb3 Merge pull request #5645 from certbot/update-test-everything
Update the test-everything branch
2018-03-02 11:17:01 -08:00
Brad Warren
184b384b58 Merge branch 'master' into update-test-everything 2018-03-01 10:21:55 -08:00
Brad Warren
13b6b3f1b3 Merge pull request #5401 from certbot/fix-everything
Fix test-everything
2018-01-09 18:33:13 -08:00
Brad Warren
f4a1547481 Merge branch 'master' into fix-everything 2018-01-09 17:33:17 -08:00
Brad Warren
164121fc15 Revert "Update test-everything (#5397)"
This reverts commit 349643c9b8.
2018-01-09 17:32:07 -08:00
Brad Warren
349643c9b8 Update test-everything (#5397)
* Use josepy instead of acme.jose. (#5203)

* Parse variables without whitespace separator correctly in CentOS family of distributions (#5318)

* Pin josepy in letsencrypt-auto (#5321)

* pin josepy in le-auto

* Put pinned versions in sorted order

* Pin dependencies in oldest tests (#5316)

* Add tools/merge_requirements.py

* Revert "Fix oldest tests by pinning Google DNS deps (#5000)"

This reverts commit f68fba2be2.

* Add tools/oldest_constraints.txt

* Remove oldest constraints from tox.ini

* Rename dev constraints file

* Update tools/pip_install.sh

* Update install_and_test.sh

* Fix pip_install.sh

* Don't cat when you can cp

* Add ng-httpsclient to dev constraints for oldest tests

* Bump tested setuptools version

* Update dev_constraints comment

* Better document oldest dependencies

* test against oldest versions we say we require

* Update dev constraints

* Properly handle empty lines

* Update constraints gen in pip_install

* Remove duplicated zope.component

* Reduce pyasn1-modules dependency

* Remove blank line

* pin back google-api-python-client

* pin back uritemplate

* pin josepy for oldest tests

* Undo changes to install_and_test.sh

* Update install_and_test.sh description

* use split instead of partition

* More pip dependency resolution workarounds (#5339)

* remove pyopenssl and six deps

* remove outdated tox.ini dep requirement

* Fix auto_tests on systems with new bootstrappers (#5348)

* Fix pytest on macOS in Travis (#5360)

* Add tools/pytest.sh

* pass TRAVIS through in tox.ini

* Use tools/pytest.sh to run pytest

* Add quiet to pytest.ini

* ignore pytest cache

* print as a string (#5359)

* Use apache2ctl modules for Gentoo systems. (#5349)

* Do not call Apache binary for module reset in cleanup()

* Use apache2ctl modules for Gentoo

* Broader git ignore for pytest cache files (#5361)

Make gitignore take pytest cache directories in to account, even if
they reside in subdirectories.

If pytest is run for a certain module, ie. `pytest certbot-apache` the
cache directory is created under `certbot-apache` directory.

* Fix letsencrypt-auto name and long forms of -n (#5375)

* Deprecate Python2.6 by using Python3 on CentOS/RHEL 6 (#5329)

* If there's no python or there's only python2.6 on red hat systems, install python3

* Always check for python2.6

* address style, documentation, nits

* factor out all initialization code

* fix up python version return value when no python installed

* add no python error and exit

* document DeterminePythonVersion parameters

* build letsencrypt-auto

* close brace

* build leauto

* fix syntax errors

* set USE_PYTHON_3 for all cases

* rip out NOCRASH

* replace NOCRASH, update LE_PYTHON set logic

* use built-in venv for py3

* switch to LE_PYTHON not affecting bootstrap selection and not overwriting LE_PYTHON

* python3ify fetch.py

* get fetch.py working with python2 and 3

* don't verify server certificates in fetch.py HttpsGetter

* Use SSLContext and an environment variable so that our tests continue to never verify server certificates.

* typo

* build

* remove commented out code

* address review comments

* add documentation for YES_FLAG and QUIET_FLAG

* Add tests to centos6 Dockerfile to make sure we install python3 if and only if appropriate to do so.

* Allow non-interactive revocation without deleting certificates (#5386)

* Add --delete-after-revoke flags

* Use delete_after_revoke value

* Add delete_after_revoke unit tests

* Add integration tests for delete-after-revoke.

* Have letsencrypt-auto do a real upgrade in leauto-upgrades option 2 (#5390)

* Make leauto_upgrades do a real upgrade

* Cleanup vars and output

* Sleep until the server is ready

* add simple_http_server.py

* Use a randomly assigned port

* s/realpath/readlink

* wait for server before getting port

* s/localhost/all interfaces

* update Apache ciphersuites (#5383)

* Fix macOS builds for Python2.7 in Travis (#5378)

* Add OSX Python2 tests

* Make sure python2 is originating from homebrew on macOS

* Upgrade the already installed python2 instead of trying to reinstall
2018-01-09 17:24:14 -08:00
Brad Warren
899b56514a Merge pull request #5315 from certbot/update-test-everything
Update the test everything branch
2017-12-08 17:48:21 -08:00
Brad Warren
fa3bc6d774 Merge branch 'master' into test-everything 2017-12-08 16:37:28 -08:00
Jacob Hoffman-Andrews
76f5f590ea Add back macOS and Boulder for test-everything.
This commit should not be merged to master (if that happens, just revert it).

In https://github.com/certbot/certbot/pull/5270, we removed the macOS and
most Boulder integration tests from .travis.yml because they made builds very
slow. We still want to run those tests nightly, though. The test-everything
branch will have the tests that are too expensive to run for every PR.
2017-12-01 16:12:44 -08:00
4 changed files with 540 additions and 487 deletions

View File

@@ -5,7 +5,14 @@ cache:
- $HOME/.cache/pip
before_install:
- '([ $TRAVIS_OS_NAME == linux ] && dpkg -s libaugeas0) || (brew update && brew install augeas python3 && brew upgrade python && brew link python)'
# In the test-everything branch, we merge master before running tests.
# This is because we want test-everything to test the code in master nightly
# in a Travis cron, but with a different set of tests than master has
# in .travis.yml.
- cp .travis.yml /tmp/travis.yml
- git pull origin master --strategy=recursive --strategy-option=theirs --no-edit
- if ! git diff .travis.yml /tmp/travis.yml ; then echo "Please merge master into test-everything"; exit 1; fi
- '([ $TRAVIS_OS_NAME == linux ] && dpkg -s libaugeas0) || (brew update && brew install augeas && brew upgrade python python3 && brew link python)'
before_script:
- 'if [ $TRAVIS_OS_NAME = osx ] ; then ulimit -n 1024 ; fi'
@@ -13,7 +20,15 @@ before_script:
matrix:
include:
- python: "2.7"
env: TOXENV=py27_install BOULDER_INTEGRATION=v1
env: TOXENV=cover FYI="this also tests py27"
- python: "2.7"
env: TOXENV=lint
- python: "2.7"
env: TOXENV=py27-nginx-oldest BOULDER_INTEGRATION=v1
sudo: required
services: docker
- python: "2.7"
env: TOXENV=py27-nginx-oldest BOULDER_INTEGRATION=v2
sudo: required
services: docker
- python: "2.7"
@@ -21,43 +36,73 @@ matrix:
sudo: required
services: docker
- python: "2.7"
env: TOXENV=cover FYI="this also tests py27"
- sudo: required
env: TOXENV=nginx_compat
env: TOXENV=py27_install BOULDER_INTEGRATION=v1
sudo: required
services: docker
before_install:
addons:
- python: "2.7"
env: TOXENV=lint
- python: "3.5"
env: TOXENV=mypy
- python: "2.7"
env: TOXENV='py27-{acme,apache,certbot,dns,nginx}-oldest'
sudo: required
services: docker
- python: "3.4"
env: TOXENV=py34
sudo: required
services: docker
- python: "3.6"
env: TOXENV=py36
sudo: required
services: docker
env: TOXENV='py27-{acme,apache,certbot,dns}-oldest'
- sudo: required
env: TOXENV=apache_compat
services: docker
before_install:
addons:
- sudo: required
env: TOXENV=nginx_compat
services: docker
- sudo: required
env: TOXENV=le_auto_precise
services: docker
- sudo: required
env: TOXENV=le_auto_trusty
services: docker
before_install:
- sudo: required
env: TOXENV=le_auto_wheezy
services: docker
- sudo: required
env: TOXENV=le_auto_centos6
services: docker
- sudo: required
env: TOXENV=docker_dev
services: docker
addons:
apt:
packages: # don't install nginx and apache
- libaugeas0
- python: "2.7"
env: TOXENV=apacheconftest
sudo: required
- python: "3.4"
env: TOXENV=py34 BOULDER_INTEGRATION=v1
sudo: required
services: docker
- python: "3.5"
env: TOXENV=mypy
- python: "3.4"
env: TOXENV=py34 BOULDER_INTEGRATION=v2
sudo: required
services: docker
- python: "3.5"
env: TOXENV=py35 BOULDER_INTEGRATION=v1
sudo: required
services: docker
- python: "3.5"
env: TOXENV=py35 BOULDER_INTEGRATION=v2
sudo: required
services: docker
- python: "3.6"
env: TOXENV=py36 BOULDER_INTEGRATION=v1
sudo: required
services: docker
- python: "3.6"
env: TOXENV=py36 BOULDER_INTEGRATION=v2
sudo: required
services: docker
- python: "2.7"
env: TOXENV=nginxroundtrip
- language: generic
env: TOXENV=py27
os: osx
- language: generic
env: TOXENV=py36
os: osx
# Only build pushes to the master branch, PRs, and branches beginning with
@@ -102,9 +147,9 @@ after_success: '[ "$TOXENV" == "cover" ] && coveralls'
notifications:
email: false
irc:
channels:
- secure: "SGWZl3ownKx9xKVV2VnGt7DqkTmutJ89oJV9tjKhSs84kLijU6EYdPnllqISpfHMTxXflNZuxtGo0wTDYHXBuZL47w1O32W6nzuXdra5zC+i4sYQwYULUsyfOv9gJX8zWAULiK0Z3r0oho45U+FR5ZN6TPCidi8/eGU+EEPwaAw="
on_success: never
on_failure: always
use_notice: true
# irc:
# channels:
# - secure: "SGWZl3ownKx9xKVV2VnGt7DqkTmutJ89oJV9tjKhSs84kLijU6EYdPnllqISpfHMTxXflNZuxtGo0wTDYHXBuZL47w1O32W6nzuXdra5zC+i4sYQwYULUsyfOv9gJX8zWAULiK0Z3r0oho45U+FR5ZN6TPCidi8/eGU+EEPwaAw="
# on_success: never
# on_failure: always
# use_notice: true

View File

@@ -1,461 +1,14 @@
#!/bin/bash
# Simple integration test. Make sure to activate virtualenv beforehand
# (source venv/bin/activate) and that you are running Boulder test
# instance (see ./boulder-fetch.sh).
#
# Environment variables:
# SERVER: Passed as "certbot --server" argument.
#
# Note: this script is called by Boulder integration test suite!
set -eux
set -e
. ./tests/integration/_common.sh
export PATH="$PATH:/usr/sbin" # /usr/sbin/nginx
cleanup_and_exit() {
EXIT_STATUS=$?
if SERVER_STILL_RUNNING=`ps -p $python_server_pid -o pid=`
then
echo Kill server subprocess, left running by abnormal exit
kill $SERVER_STILL_RUNNING
fi
if [ -f "$HOOK_DIRS_TEST" ]; then
rm -f "$HOOK_DIRS_TEST"
fi
exit $EXIT_STATUS
}
trap cleanup_and_exit EXIT
export HOOK_DIRS_TEST="$(mktemp)"
renewal_hooks_root="$config_dir/renewal-hooks"
renewal_hooks_dirs=$(echo "$renewal_hooks_root/"{pre,deploy,post})
renewal_dir_pre_hook="$(echo $renewal_hooks_dirs | cut -f 1 -d " ")/hook.sh"
renewal_dir_deploy_hook="$(echo $renewal_hooks_dirs | cut -f 2 -d " ")/hook.sh"
renewal_dir_post_hook="$(echo $renewal_hooks_dirs | cut -f 3 -d " ")/hook.sh"
# Creates hooks in Certbot's renewal hook directory that write to a file
CreateDirHooks() {
for hook_dir in $renewal_hooks_dirs; do
mkdir -p $hook_dir
hook_path="$hook_dir/hook.sh"
cat << EOF > "$hook_path"
#!/bin/bash -xe
if [ "\$0" = "$renewal_dir_deploy_hook" ]; then
if [ -z "\$RENEWED_DOMAINS" -o -z "\$RENEWED_LINEAGE" ]; then
echo "Environment variables not properly set!" >&2
exit 1
if [ "$TOXENV" != "py27-nginx-oldest" ]; then
tests/certbot-boulder-integration.sh
fi
if [ "$TOXENV" != "py27-certbot-oldest" ]; then
# Most CI systems set this variable to true.
# If the tests are running as part of CI, Nginx should be available.
if ${CI:-false} || type nginx; then
certbot-nginx/tests/boulder-integration.sh
fi
fi
echo \$(basename \$(dirname "\$0")) >> "\$HOOK_DIRS_TEST"
EOF
chmod +x "$hook_path"
done
}
# Asserts that the hooks created by CreateDirHooks have been run once and
# resets the file.
#
# Arguments:
# The number of times the deploy hook should have been run. (It should run
# once for each certificate that was issued in that run of Certbot.)
CheckDirHooks() {
expected="pre\n"
for ((i=0; i<$1; i++)); do
expected=$expected"deploy\n"
done
expected=$expected"post"
if ! diff "$HOOK_DIRS_TEST" <(echo -e "$expected"); then
echo "Unexpected directory hook output!" >&2
echo "Expected:" >&2
echo -e "$expected" >&2
echo "Got:" >&2
cat "$HOOK_DIRS_TEST" >&2
exit 1
fi
rm -f "$HOOK_DIRS_TEST"
export HOOK_DIRS_TEST="$(mktemp)"
}
common_no_force_renew() {
certbot_test_no_force_renew \
--authenticator standalone \
--installer null \
"$@"
}
common() {
common_no_force_renew \
--renew-by-default \
"$@"
}
export HOOK_TEST="/tmp/hook$$"
CheckHooks() {
if [ $(head -n1 "$HOOK_TEST") = "wtf.pre" ]; then
expected="wtf.pre\ndeploy\n"
if [ $(sed '3q;d' "$HOOK_TEST") = "deploy" ]; then
expected=$expected"deploy\nwtf2.pre\n"
else
expected=$expected"wtf2.pre\ndeploy\n"
fi
expected=$expected"deploy\ndeploy\nwtf.post\nwtf2.post"
else
expected="wtf2.pre\ndeploy\n"
if [ $(sed '3q;d' "$HOOK_TEST") = "deploy" ]; then
expected=$expected"deploy\nwtf.pre\n"
else
expected=$expected"wtf.pre\ndeploy\n"
fi
expected=$expected"deploy\ndeploy\nwtf2.post\nwtf.post"
fi
if ! cmp --quiet <(echo -e "$expected") "$HOOK_TEST" ; then
echo Hooks did not run as expected\; got >&2
cat "$HOOK_TEST" >&2
echo -e "Expected\n$expected" >&2
rm "$HOOK_TEST"
exit 1
fi
rm "$HOOK_TEST"
}
# Checks if deploy is in the hook output and deletes the file
DeployInHookOutput() {
CONTENTS=$(cat "$HOOK_TEST")
rm "$HOOK_TEST"
grep deploy <(echo "$CONTENTS")
}
# Asserts that there is a saved renew_hook for a lineage.
#
# Arguments:
# Name of lineage to check
CheckSavedRenewHook() {
if ! grep renew_hook "$config_dir/renewal/$1.conf"; then
echo "Hook wasn't saved as renew_hook" >&2
exit 1
fi
}
# Asserts the deploy hook was properly run and saved and deletes the hook file
#
# Arguments:
# Lineage name of the issued cert
CheckDeployHook() {
if ! DeployInHookOutput; then
echo "The deploy hook wasn't run" >&2
exit 1
fi
CheckSavedRenewHook $1
}
# Asserts the renew hook wasn't run but was saved and deletes the hook file
#
# Arguments:
# Lineage name of the issued cert
# Asserts the deploy hook wasn't run and deletes the hook file
CheckRenewHook() {
if DeployInHookOutput; then
echo "The renew hook was incorrectly run" >&2
exit 1
fi
CheckSavedRenewHook $1
}
# Cleanup coverage data
coverage erase
# test for regressions of #4719
get_num_tmp_files() {
ls -1 /tmp | wc -l
}
num_tmp_files=$(get_num_tmp_files)
common --csr / && echo expected error && exit 1 || true
common --help
common --help all
common --version
if [ $(get_num_tmp_files) -ne $num_tmp_files ]; then
echo "New files or directories created in /tmp!"
exit 1
fi
CreateDirHooks
common register
for dir in $renewal_hooks_dirs; do
if [ ! -d "$dir" ]; then
echo "Hook directory not created by Certbot!" >&2
exit 1
fi
done
common register --update-registration --email example@example.org
common plugins --init --prepare | grep webroot
# We start a server listening on the port for the
# unrequested challenge to prevent regressions in #3601.
python ./tests/run_http_server.py $http_01_port &
python_server_pid=$!
certname="le1.wtf"
common --domains le1.wtf --preferred-challenges tls-sni-01 auth \
--cert-name $certname \
--pre-hook 'echo wtf.pre >> "$HOOK_TEST"' \
--post-hook 'echo wtf.post >> "$HOOK_TEST"'\
--deploy-hook 'echo deploy >> "$HOOK_TEST"'
kill $python_server_pid
CheckDeployHook $certname
python ./tests/run_http_server.py $tls_sni_01_port &
python_server_pid=$!
certname="le2.wtf"
common --domains le2.wtf --preferred-challenges http-01 run \
--cert-name $certname \
--pre-hook 'echo wtf.pre >> "$HOOK_TEST"' \
--post-hook 'echo wtf.post >> "$HOOK_TEST"'\
--deploy-hook 'echo deploy >> "$HOOK_TEST"'
kill $python_server_pid
CheckDeployHook $certname
certname="le.wtf"
common certonly -a manual -d le.wtf --rsa-key-size 4096 --cert-name $certname \
--manual-auth-hook ./tests/manual-http-auth.sh \
--manual-cleanup-hook ./tests/manual-http-cleanup.sh \
--pre-hook 'echo wtf2.pre >> "$HOOK_TEST"' \
--post-hook 'echo wtf2.post >> "$HOOK_TEST"' \
--renew-hook 'echo deploy >> "$HOOK_TEST"'
CheckRenewHook $certname
certname="dns.le.wtf"
common -a manual -d dns.le.wtf --preferred-challenges dns,tls-sni run \
--cert-name $certname \
--manual-auth-hook ./tests/manual-dns-auth.sh \
--manual-cleanup-hook ./tests/manual-dns-cleanup.sh \
--pre-hook 'echo wtf2.pre >> "$HOOK_TEST"' \
--post-hook 'echo wtf2.post >> "$HOOK_TEST"' \
--renew-hook 'echo deploy >> "$HOOK_TEST"'
CheckRenewHook $certname
common certonly --cert-name newname -d newname.le.wtf
export CSR_PATH="${root}/csr.der" KEY_PATH="${root}/key.pem" \
OPENSSL_CNF=examples/openssl.cnf
./examples/generate-csr.sh le3.wtf
common auth --csr "$CSR_PATH" \
--cert-path "${root}/csr/cert.pem" \
--chain-path "${root}/csr/chain.pem"
openssl x509 -in "${root}/csr/cert.pem" -text
openssl x509 -in "${root}/csr/chain.pem" -text
common --domains le3.wtf install \
--cert-path "${root}/csr/cert.pem" \
--key-path "${root}/key.pem"
CheckCertCount() {
CERTCOUNT=`ls "${root}/conf/archive/$1/cert"* | wc -l`
if [ "$CERTCOUNT" -ne "$2" ] ; then
echo Wrong cert count, not "$2" `ls "${root}/conf/archive/$1/"*`
exit 1
fi
}
CheckCertCount "le.wtf" 1
# This won't renew (because it's not time yet)
common_no_force_renew renew
CheckCertCount "le.wtf" 1
if [ -s "$HOOK_DIRS_TEST" ]; then
echo "Directory hooks were executed for non-renewal!" >&2;
exit 1
fi
rm -rf "$renewal_hooks_root"
# renew using HTTP manual auth hooks
common renew --cert-name le.wtf --authenticator manual
CheckCertCount "le.wtf" 2
# test renewal with no executables in hook directories
for hook_dir in $renewal_hooks_dirs; do
touch "$hook_dir/file"
mkdir "$hook_dir/dir"
done
# renew using DNS manual auth hooks
common renew --cert-name dns.le.wtf --authenticator manual
CheckCertCount "dns.le.wtf" 2
# test with disabled directory hooks
rm -rf "$renewal_hooks_root"
CreateDirHooks
# This will renew because the expiry is less than 10 years from now
sed -i "4arenew_before_expiry = 4 years" "$root/conf/renewal/le.wtf.conf"
common_no_force_renew renew --rsa-key-size 2048 --no-directory-hooks
CheckCertCount "le.wtf" 3
if [ -s "$HOOK_DIRS_TEST" ]; then
echo "Directory hooks were executed with --no-directory-hooks!" >&2
exit 1
fi
# The 4096 bit setting should persist to the first renewal, but be overridden in the second
size1=`wc -c ${root}/conf/archive/le.wtf/privkey1.pem | cut -d" " -f1`
size2=`wc -c ${root}/conf/archive/le.wtf/privkey2.pem | cut -d" " -f1`
size3=`wc -c ${root}/conf/archive/le.wtf/privkey3.pem | cut -d" " -f1`
# 4096 bit PEM keys are about ~3270 bytes, 2048 ones are about 1700 bytes
if [ "$size1" -lt 3000 ] || [ "$size2" -lt 3000 ] || [ "$size3" -gt 1800 ] ; then
echo key sizes violate assumptions:
ls -l "${root}/conf/archive/le.wtf/privkey"*
exit 1
fi
# --renew-by-default is used, so renewal should occur
[ -f "$HOOK_TEST" ] && rm -f "$HOOK_TEST"
common renew
CheckCertCount "le.wtf" 4
CheckHooks
CheckDirHooks 5
# test with overlapping directory hooks on the command line
common renew --cert-name le2.wtf \
--pre-hook "$renewal_dir_pre_hook" \
--deploy-hook "$renewal_dir_deploy_hook" \
--post-hook "$renewal_dir_post_hook"
CheckDirHooks 1
# test with overlapping directory hooks in the renewal conf files
common renew --cert-name le2.wtf
CheckDirHooks 1
# manual-dns-auth.sh will skip completing the challenge for domains that begin
# with fail.
common -a manual -d dns1.le.wtf,fail.dns1.le.wtf \
--allow-subset-of-names \
--preferred-challenges dns,tls-sni \
--manual-auth-hook ./tests/manual-dns-auth.sh \
--manual-cleanup-hook ./tests/manual-dns-cleanup.sh
if common certificates | grep "fail\.dns1\.le\.wtf"; then
echo "certificate should not have been issued for domain!" >&2
exit 1
fi
# ECDSA
openssl ecparam -genkey -name secp384r1 -out "${root}/privkey-p384.pem"
SAN="DNS:ecdsa.le.wtf" openssl req -new -sha256 \
-config "${OPENSSL_CNF:-openssl.cnf}" \
-key "${root}/privkey-p384.pem" \
-subj "/" \
-reqexts san \
-outform der \
-out "${root}/csr-p384.der"
common auth --csr "${root}/csr-p384.der" \
--cert-path "${root}/csr/cert-p384.pem" \
--chain-path "${root}/csr/chain-p384.pem"
openssl x509 -in "${root}/csr/cert-p384.pem" -text | grep 'ASN1 OID: secp384r1'
# OCSP Must Staple
common auth --must-staple --domains "must-staple.le.wtf"
openssl x509 -in "${root}/conf/live/must-staple.le.wtf/cert.pem" -text | grep '1.3.6.1.5.5.7.1.24'
# revoke by account key
common revoke --cert-path "$root/conf/live/le.wtf/cert.pem" --delete-after-revoke
# revoke renewed
common revoke --cert-path "$root/conf/live/le1.wtf/cert.pem" --no-delete-after-revoke
if [ ! -d "$root/conf/live/le1.wtf" ]; then
echo "cert deleted when --no-delete-after-revoke was used!"
exit 1
fi
common delete --cert-name le1.wtf
# revoke by cert key
common revoke --cert-path "$root/conf/live/le2.wtf/cert.pem" \
--key-path "$root/conf/live/le2.wtf/privkey.pem"
# Get new certs to test revoke with a reason, by account and by cert key
common --domains le1.wtf
common revoke --cert-path "$root/conf/live/le1.wtf/cert.pem" \
--reason cessationOfOperation
common --domains le2.wtf
common revoke --cert-path "$root/conf/live/le2.wtf/cert.pem" \
--key-path "$root/conf/live/le2.wtf/privkey.pem" \
--reason keyCompromise
common unregister
out=$(common certificates)
subdomains="le dns.le newname.le must-staple.le"
for subdomain in $subdomains; do
domain="$subdomain.wtf"
if ! echo $out | grep "$domain"; then
echo "$domain not in certificates output!"
exit 1;
fi
done
# Testing that revocation also deletes by default
subdomains="le1 le2"
for subdomain in $subdomains; do
domain="$subdomain.wtf"
if echo $out | grep "$domain"; then
echo "Revoked $domain in certificates output! Should not be!"
exit 1;
fi
done
# Test that revocation raises correct error if --cert-name and --cert-path don't match
common --domains le1.wtf
common --domains le2.wtf
out=$(common revoke --cert-path "$root/conf/live/le1.wtf/fullchain.pem" --cert-name "le2.wtf" 2>&1) || true
if ! echo $out | grep "or both must point to the same certificate lineages."; then
echo "Non-interactive revoking with mismatched --cert-name and --cert-path "
echo "did not raise the correct error!"
exit 1
fi
# Revoking by matching --cert-name and --cert-path deletes
common --domains le1.wtf
common revoke --cert-path "$root/conf/live/le1.wtf/fullchain.pem" --cert-name "le1.wtf"
out=$(common certificates)
if echo $out | grep "le1.wtf"; then
echo "Cert le1.wtf should've been deleted! Was revoked via matching --cert-path & --cert-name"
exit 1
fi
# Test that revocation doesn't delete if multiple lineages share an archive dir
common --domains le1.wtf
common --domains le2.wtf
sed -i "s|^archive_dir = .*$|archive_dir = $root/conf/archive/le1.wtf|" "$root/conf/renewal/le2.wtf.conf"
#common update_symlinks # not needed, but a bit more context for what this test is about
out=$(common revoke --cert-path "$root/conf/live/le1.wtf/cert.pem")
if ! echo $out | grep "Not deleting revoked certs due to overlapping archive dirs"; then
echo "Deleted a cert that had an overlapping archive dir with another lineage!"
exit 1
fi
cert_name="must-staple.le.wtf"
common delete --cert-name $cert_name
archive="$root/conf/archive/$cert_name"
conf="$root/conf/renewal/$cert_name.conf"
live="$root/conf/live/$cert_name"
for path in $archive $conf $live; do
if [ -e $path ]; then
echo "Lineage not properly deleted!"
exit 1
fi
done
# Test ACMEv2-only features
if [ "${BOULDER_INTEGRATION:-v1}" = "v2" ]; then
common -a manual -d '*.le4.wtf,le4.wtf' --preferred-challenges dns \
--manual-auth-hook ./tests/manual-dns-auth.sh \
--manual-cleanup-hook ./tests/manual-dns-cleanup.sh
fi
# Most CI systems set this variable to true.
# If the tests are running as part of CI, Nginx should be available.
if ${CI:-false} || type nginx;
then
. ./certbot-nginx/tests/boulder-integration.sh
fi
coverage report --fail-under 67 -m

View File

@@ -0,0 +1,454 @@
#!/bin/bash
# Simple integration test. Make sure to activate virtualenv beforehand
# (source venv/bin/activate) and that you are running Boulder test
# instance (see ./boulder-fetch.sh).
#
# Environment variables:
# SERVER: Passed as "certbot --server" argument.
#
# Note: this script is called by Boulder integration test suite!
set -eux
. ./tests/integration/_common.sh
export PATH="$PATH:/usr/sbin" # /usr/sbin/nginx
cleanup_and_exit() {
EXIT_STATUS=$?
if SERVER_STILL_RUNNING=`ps -p $python_server_pid -o pid=`
then
echo Kill server subprocess, left running by abnormal exit
kill $SERVER_STILL_RUNNING
fi
if [ -f "$HOOK_DIRS_TEST" ]; then
rm -f "$HOOK_DIRS_TEST"
fi
exit $EXIT_STATUS
}
trap cleanup_and_exit EXIT
export HOOK_DIRS_TEST="$(mktemp)"
renewal_hooks_root="$config_dir/renewal-hooks"
renewal_hooks_dirs=$(echo "$renewal_hooks_root/"{pre,deploy,post})
renewal_dir_pre_hook="$(echo $renewal_hooks_dirs | cut -f 1 -d " ")/hook.sh"
renewal_dir_deploy_hook="$(echo $renewal_hooks_dirs | cut -f 2 -d " ")/hook.sh"
renewal_dir_post_hook="$(echo $renewal_hooks_dirs | cut -f 3 -d " ")/hook.sh"
# Creates hooks in Certbot's renewal hook directory that write to a file
CreateDirHooks() {
for hook_dir in $renewal_hooks_dirs; do
mkdir -p $hook_dir
hook_path="$hook_dir/hook.sh"
cat << EOF > "$hook_path"
#!/bin/bash -xe
if [ "\$0" = "$renewal_dir_deploy_hook" ]; then
if [ -z "\$RENEWED_DOMAINS" -o -z "\$RENEWED_LINEAGE" ]; then
echo "Environment variables not properly set!" >&2
exit 1
fi
fi
echo \$(basename \$(dirname "\$0")) >> "\$HOOK_DIRS_TEST"
EOF
chmod +x "$hook_path"
done
}
# Asserts that the hooks created by CreateDirHooks have been run once and
# resets the file.
#
# Arguments:
# The number of times the deploy hook should have been run. (It should run
# once for each certificate that was issued in that run of Certbot.)
CheckDirHooks() {
expected="pre\n"
for ((i=0; i<$1; i++)); do
expected=$expected"deploy\n"
done
expected=$expected"post"
if ! diff "$HOOK_DIRS_TEST" <(echo -e "$expected"); then
echo "Unexpected directory hook output!" >&2
echo "Expected:" >&2
echo -e "$expected" >&2
echo "Got:" >&2
cat "$HOOK_DIRS_TEST" >&2
exit 1
fi
rm -f "$HOOK_DIRS_TEST"
export HOOK_DIRS_TEST="$(mktemp)"
}
common_no_force_renew() {
certbot_test_no_force_renew \
--authenticator standalone \
--installer null \
"$@"
}
common() {
common_no_force_renew \
--renew-by-default \
"$@"
}
export HOOK_TEST="/tmp/hook$$"
CheckHooks() {
if [ $(head -n1 "$HOOK_TEST") = "wtf.pre" ]; then
expected="wtf.pre\ndeploy\n"
if [ $(sed '3q;d' "$HOOK_TEST") = "deploy" ]; then
expected=$expected"deploy\nwtf2.pre\n"
else
expected=$expected"wtf2.pre\ndeploy\n"
fi
expected=$expected"deploy\ndeploy\nwtf.post\nwtf2.post"
else
expected="wtf2.pre\ndeploy\n"
if [ $(sed '3q;d' "$HOOK_TEST") = "deploy" ]; then
expected=$expected"deploy\nwtf.pre\n"
else
expected=$expected"wtf.pre\ndeploy\n"
fi
expected=$expected"deploy\ndeploy\nwtf2.post\nwtf.post"
fi
if ! cmp --quiet <(echo -e "$expected") "$HOOK_TEST" ; then
echo Hooks did not run as expected\; got >&2
cat "$HOOK_TEST" >&2
echo -e "Expected\n$expected" >&2
rm "$HOOK_TEST"
exit 1
fi
rm "$HOOK_TEST"
}
# Checks if deploy is in the hook output and deletes the file
DeployInHookOutput() {
CONTENTS=$(cat "$HOOK_TEST")
rm "$HOOK_TEST"
grep deploy <(echo "$CONTENTS")
}
# Asserts that there is a saved renew_hook for a lineage.
#
# Arguments:
# Name of lineage to check
CheckSavedRenewHook() {
if ! grep renew_hook "$config_dir/renewal/$1.conf"; then
echo "Hook wasn't saved as renew_hook" >&2
exit 1
fi
}
# Asserts the deploy hook was properly run and saved and deletes the hook file
#
# Arguments:
# Lineage name of the issued cert
CheckDeployHook() {
if ! DeployInHookOutput; then
echo "The deploy hook wasn't run" >&2
exit 1
fi
CheckSavedRenewHook $1
}
# Asserts the renew hook wasn't run but was saved and deletes the hook file
#
# Arguments:
# Lineage name of the issued cert
# Asserts the deploy hook wasn't run and deletes the hook file
CheckRenewHook() {
if DeployInHookOutput; then
echo "The renew hook was incorrectly run" >&2
exit 1
fi
CheckSavedRenewHook $1
}
# Cleanup coverage data
coverage erase
# test for regressions of #4719
get_num_tmp_files() {
ls -1 /tmp | wc -l
}
num_tmp_files=$(get_num_tmp_files)
common --csr / && echo expected error && exit 1 || true
common --help
common --help all
common --version
if [ $(get_num_tmp_files) -ne $num_tmp_files ]; then
echo "New files or directories created in /tmp!"
exit 1
fi
CreateDirHooks
common register
for dir in $renewal_hooks_dirs; do
if [ ! -d "$dir" ]; then
echo "Hook directory not created by Certbot!" >&2
exit 1
fi
done
common register --update-registration --email example@example.org
common plugins --init --prepare | grep webroot
# We start a server listening on the port for the
# unrequested challenge to prevent regressions in #3601.
python ./tests/run_http_server.py $http_01_port &
python_server_pid=$!
certname="le1.wtf"
common --domains le1.wtf --preferred-challenges tls-sni-01 auth \
--cert-name $certname \
--pre-hook 'echo wtf.pre >> "$HOOK_TEST"' \
--post-hook 'echo wtf.post >> "$HOOK_TEST"'\
--deploy-hook 'echo deploy >> "$HOOK_TEST"'
kill $python_server_pid
CheckDeployHook $certname
python ./tests/run_http_server.py $tls_sni_01_port &
python_server_pid=$!
certname="le2.wtf"
common --domains le2.wtf --preferred-challenges http-01 run \
--cert-name $certname \
--pre-hook 'echo wtf.pre >> "$HOOK_TEST"' \
--post-hook 'echo wtf.post >> "$HOOK_TEST"'\
--deploy-hook 'echo deploy >> "$HOOK_TEST"'
kill $python_server_pid
CheckDeployHook $certname
certname="le.wtf"
common certonly -a manual -d le.wtf --rsa-key-size 4096 --cert-name $certname \
--manual-auth-hook ./tests/manual-http-auth.sh \
--manual-cleanup-hook ./tests/manual-http-cleanup.sh \
--pre-hook 'echo wtf2.pre >> "$HOOK_TEST"' \
--post-hook 'echo wtf2.post >> "$HOOK_TEST"' \
--renew-hook 'echo deploy >> "$HOOK_TEST"'
CheckRenewHook $certname
certname="dns.le.wtf"
common -a manual -d dns.le.wtf --preferred-challenges dns,tls-sni run \
--cert-name $certname \
--manual-auth-hook ./tests/manual-dns-auth.sh \
--manual-cleanup-hook ./tests/manual-dns-cleanup.sh \
--pre-hook 'echo wtf2.pre >> "$HOOK_TEST"' \
--post-hook 'echo wtf2.post >> "$HOOK_TEST"' \
--renew-hook 'echo deploy >> "$HOOK_TEST"'
CheckRenewHook $certname
common certonly --cert-name newname -d newname.le.wtf
export CSR_PATH="${root}/csr.der" KEY_PATH="${root}/key.pem" \
OPENSSL_CNF=examples/openssl.cnf
./examples/generate-csr.sh le3.wtf
common auth --csr "$CSR_PATH" \
--cert-path "${root}/csr/cert.pem" \
--chain-path "${root}/csr/chain.pem"
openssl x509 -in "${root}/csr/cert.pem" -text
openssl x509 -in "${root}/csr/chain.pem" -text
common --domains le3.wtf install \
--cert-path "${root}/csr/cert.pem" \
--key-path "${root}/key.pem"
CheckCertCount() {
CERTCOUNT=`ls "${root}/conf/archive/$1/cert"* | wc -l`
if [ "$CERTCOUNT" -ne "$2" ] ; then
echo Wrong cert count, not "$2" `ls "${root}/conf/archive/$1/"*`
exit 1
fi
}
CheckCertCount "le.wtf" 1
# This won't renew (because it's not time yet)
common_no_force_renew renew
CheckCertCount "le.wtf" 1
if [ -s "$HOOK_DIRS_TEST" ]; then
echo "Directory hooks were executed for non-renewal!" >&2;
exit 1
fi
rm -rf "$renewal_hooks_root"
# renew using HTTP manual auth hooks
common renew --cert-name le.wtf --authenticator manual
CheckCertCount "le.wtf" 2
# test renewal with no executables in hook directories
for hook_dir in $renewal_hooks_dirs; do
touch "$hook_dir/file"
mkdir "$hook_dir/dir"
done
# renew using DNS manual auth hooks
common renew --cert-name dns.le.wtf --authenticator manual
CheckCertCount "dns.le.wtf" 2
# test with disabled directory hooks
rm -rf "$renewal_hooks_root"
CreateDirHooks
# This will renew because the expiry is less than 10 years from now
sed -i "4arenew_before_expiry = 4 years" "$root/conf/renewal/le.wtf.conf"
common_no_force_renew renew --rsa-key-size 2048 --no-directory-hooks
CheckCertCount "le.wtf" 3
if [ -s "$HOOK_DIRS_TEST" ]; then
echo "Directory hooks were executed with --no-directory-hooks!" >&2
exit 1
fi
# The 4096 bit setting should persist to the first renewal, but be overridden in the second
size1=`wc -c ${root}/conf/archive/le.wtf/privkey1.pem | cut -d" " -f1`
size2=`wc -c ${root}/conf/archive/le.wtf/privkey2.pem | cut -d" " -f1`
size3=`wc -c ${root}/conf/archive/le.wtf/privkey3.pem | cut -d" " -f1`
# 4096 bit PEM keys are about ~3270 bytes, 2048 ones are about 1700 bytes
if [ "$size1" -lt 3000 ] || [ "$size2" -lt 3000 ] || [ "$size3" -gt 1800 ] ; then
echo key sizes violate assumptions:
ls -l "${root}/conf/archive/le.wtf/privkey"*
exit 1
fi
# --renew-by-default is used, so renewal should occur
[ -f "$HOOK_TEST" ] && rm -f "$HOOK_TEST"
common renew
CheckCertCount "le.wtf" 4
CheckHooks
CheckDirHooks 5
# test with overlapping directory hooks on the command line
common renew --cert-name le2.wtf \
--pre-hook "$renewal_dir_pre_hook" \
--deploy-hook "$renewal_dir_deploy_hook" \
--post-hook "$renewal_dir_post_hook"
CheckDirHooks 1
# test with overlapping directory hooks in the renewal conf files
common renew --cert-name le2.wtf
CheckDirHooks 1
# manual-dns-auth.sh will skip completing the challenge for domains that begin
# with fail.
common -a manual -d dns1.le.wtf,fail.dns1.le.wtf \
--allow-subset-of-names \
--preferred-challenges dns,tls-sni \
--manual-auth-hook ./tests/manual-dns-auth.sh \
--manual-cleanup-hook ./tests/manual-dns-cleanup.sh
if common certificates | grep "fail\.dns1\.le\.wtf"; then
echo "certificate should not have been issued for domain!" >&2
exit 1
fi
# ECDSA
openssl ecparam -genkey -name secp384r1 -out "${root}/privkey-p384.pem"
SAN="DNS:ecdsa.le.wtf" openssl req -new -sha256 \
-config "${OPENSSL_CNF:-openssl.cnf}" \
-key "${root}/privkey-p384.pem" \
-subj "/" \
-reqexts san \
-outform der \
-out "${root}/csr-p384.der"
common auth --csr "${root}/csr-p384.der" \
--cert-path "${root}/csr/cert-p384.pem" \
--chain-path "${root}/csr/chain-p384.pem"
openssl x509 -in "${root}/csr/cert-p384.pem" -text | grep 'ASN1 OID: secp384r1'
# OCSP Must Staple
common auth --must-staple --domains "must-staple.le.wtf"
openssl x509 -in "${root}/conf/live/must-staple.le.wtf/cert.pem" -text | grep '1.3.6.1.5.5.7.1.24'
# revoke by account key
common revoke --cert-path "$root/conf/live/le.wtf/cert.pem" --delete-after-revoke
# revoke renewed
common revoke --cert-path "$root/conf/live/le1.wtf/cert.pem" --no-delete-after-revoke
if [ ! -d "$root/conf/live/le1.wtf" ]; then
echo "cert deleted when --no-delete-after-revoke was used!"
exit 1
fi
common delete --cert-name le1.wtf
# revoke by cert key
common revoke --cert-path "$root/conf/live/le2.wtf/cert.pem" \
--key-path "$root/conf/live/le2.wtf/privkey.pem"
# Get new certs to test revoke with a reason, by account and by cert key
common --domains le1.wtf
common revoke --cert-path "$root/conf/live/le1.wtf/cert.pem" \
--reason cessationOfOperation
common --domains le2.wtf
common revoke --cert-path "$root/conf/live/le2.wtf/cert.pem" \
--key-path "$root/conf/live/le2.wtf/privkey.pem" \
--reason keyCompromise
common unregister
out=$(common certificates)
subdomains="le dns.le newname.le must-staple.le"
for subdomain in $subdomains; do
domain="$subdomain.wtf"
if ! echo $out | grep "$domain"; then
echo "$domain not in certificates output!"
exit 1;
fi
done
# Testing that revocation also deletes by default
subdomains="le1 le2"
for subdomain in $subdomains; do
domain="$subdomain.wtf"
if echo $out | grep "$domain"; then
echo "Revoked $domain in certificates output! Should not be!"
exit 1;
fi
done
# Test that revocation raises correct error if --cert-name and --cert-path don't match
common --domains le1.wtf
common --domains le2.wtf
out=$(common revoke --cert-path "$root/conf/live/le1.wtf/fullchain.pem" --cert-name "le2.wtf" 2>&1) || true
if ! echo $out | grep "or both must point to the same certificate lineages."; then
echo "Non-interactive revoking with mismatched --cert-name and --cert-path "
echo "did not raise the correct error!"
exit 1
fi
# Revoking by matching --cert-name and --cert-path deletes
common --domains le1.wtf
common revoke --cert-path "$root/conf/live/le1.wtf/fullchain.pem" --cert-name "le1.wtf"
out=$(common certificates)
if echo $out | grep "le1.wtf"; then
echo "Cert le1.wtf should've been deleted! Was revoked via matching --cert-path & --cert-name"
exit 1
fi
# Test that revocation doesn't delete if multiple lineages share an archive dir
common --domains le1.wtf
common --domains le2.wtf
sed -i "s|^archive_dir = .*$|archive_dir = $root/conf/archive/le1.wtf|" "$root/conf/renewal/le2.wtf.conf"
#common update_symlinks # not needed, but a bit more context for what this test is about
out=$(common revoke --cert-path "$root/conf/live/le1.wtf/cert.pem")
if ! echo $out | grep "Not deleting revoked certs due to overlapping archive dirs"; then
echo "Deleted a cert that had an overlapping archive dir with another lineage!"
exit 1
fi
cert_name="must-staple.le.wtf"
common delete --cert-name $cert_name
archive="$root/conf/archive/$cert_name"
conf="$root/conf/renewal/$cert_name.conf"
live="$root/conf/live/$cert_name"
for path in $archive $conf $live; do
if [ -e $path ]; then
echo "Lineage not properly deleted!"
exit 1
fi
done
# Test ACMEv2-only features
if [ "${BOULDER_INTEGRATION:-v1}" = "v2" ]; then
common -a manual -d '*.le4.wtf,le4.wtf' --preferred-challenges dns \
--manual-auth-hook ./tests/manual-dns-auth.sh \
--manual-cleanup-hook ./tests/manual-dns-cleanup.sh
fi
coverage report --fail-under 65 -m

View File

@@ -25,6 +25,7 @@ certbot_test_no_force_renew () {
omit_patterns="*/*.egg-info/*,*/dns_common*,*/setup.py,*/test_*,*/tests/*"
omit_patterns="$omit_patterns,*_test.py,*_test_*,certbot-apache/*"
omit_patterns="$omit_patterns,certbot-compatibility-test/*,certbot-dns*/"
omit_patterns="$omit_patterns,certbot-nginx/*"
coverage run \
--append \
--source $sources \