Went through log and fixed as much as I could

Signed-off-by: Peter Conrad <pconrad@pconrad-ltm6.internal.salesforce.com>

README.rst

@@ -4,7 +4,10 @@ Certbot is part of EFF’s effort to encrypt the entire Internet. Secure communi
 
 Anyone who has gone through the trouble of setting up a secure website knows what a hassle getting and maintaining a certificate is. Certbot and Let’s Encrypt can automate away the pain and let you turn on and manage HTTPS with simple commands. Using Certbot and Let's Encrypt is free, so there’s no need to arrange payment.
 
-How you use Certbot depends on the configuration of your web server. The best way to get started is to use our `interactive guide <https://certbot.eff.org>`_. It generates instructions based on your configuration settings. In most cases, you’ll need `root or administrator access <https://certbot.eff.org/faq/#does-certbot-require-root-administrator-privileges>`_ to your web server to run Certbot.
+.. Note:: The best way to get started is to use our `interactive Guide <https://certbot.eff.org>`_.
+
+The Interactive Guide generates instructions based on your web server and operating system. 
+In most cases, you’ll need `root or administrator access <https://certbot.eff.org/faq/#does-certbot-require-root-administrator-privileges>`_ to your web server to run Certbot.
 
 If you’re using a hosted service and don’t have direct access to your web server, you might not be able to use Certbot. Check with your hosting provider for documentation about uploading certificates or using certificates issued by Let’s Encrypt.
 
@@ -22,6 +25,13 @@ Until May 2016, Certbot was named simply ``letsencrypt`` or ``letsencrypt-auto``
 depending on install method. Instructions on the Internet, and some pieces of the
 software, may still refer to this older name.
 
+Certbot is packaged for many common operating systems and web servers. Check whether
+``certbot`` (or ``letsencrypt``) is packaged for your web server's OS by visiting
+certbot.eff.org_, where you will also find the correct installation instructions for
+your system.
+
+
+
 Contributing
 ------------
 
@@ -46,16 +56,18 @@ In many cases, you can just run ``certbot-auto`` or ``certbot``, and the
 client will guide you through the process of obtaining and installing certs
 interactively.
 
-For full command line help, you can type::
+For full command line help, you can type
 
-  ./certbot-auto --help all
+::
+  ./certbot-auto --help all
 
 
 You can also tell it exactly what you want it to do from the command line.
 For instance, if you want to obtain a cert for ``example.com``,
 ``www.example.com``, and ``other.example.net``, using the Apache plugin to both
-obtain and install the certs, you could do this::
+obtain and install the certs, you could do this
 
+::
   ./certbot-auto --apache -d example.com -d www.example.com -d other.example.net
 
 (The first time you run the command, it will make an account, and ask for an
@@ -63,16 +75,17 @@ email and agreement to the Let's Encrypt Subscriber Agreement; you can
 automate those with ``--email`` and ``--agree-tos``)
 
 If you want to use a webserver that doesn't have full plugin support yet, you
-can still use "standalone" or "webroot" plugins to obtain a certificate::
+can still use "standalone" or "webroot" plugins to obtain a certificate
 
-  ./certbot-auto certonly --standalone --email admin@example.com -d example.com -d www.example.com -d other.example.net
+::
+  ./certbot-auto certonly --standalone --email admin@example.com -d example.com -d www.example.com -d other.example.net
 
 
 Understanding the client in more depth
 --------------------------------------
 
 To understand what the client is doing in detail, it's important to
-understand the way it uses plugins.  Please see the `explanation of
+understand the way it uses plugins.  Please see the `explanation of
 plugins <https://certbot.eff.org/docs/using.html#plugins>`_ in
 the User Guide.
 
@@ -104,20 +117,24 @@ ACME working area in github: https://github.com/ietf-wg-acme/acme
 .. _Freenode: https://webchat.freenode.net?channels=%23letsencrypt
 
 .. |build-status| image:: https://travis-ci.org/certbot/certbot.svg?branch=master
-   :target: https://travis-ci.org/certbot/certbot
+   :target: https://travis-ci.org/certbot/certbot
-   :alt: Travis CI status
+   :alt: Travis CI status
+
 
 .. |coverage| image:: https://coveralls.io/repos/certbot/certbot/badge.svg?branch=master
-   :target: https://coveralls.io/r/certbot/certbot
+   :target: https://coveralls.io/r/certbot/certbot
-   :alt: Coverage status
+   :alt: Coverage status
+
 
 .. |docs| image:: https://readthedocs.org/projects/letsencrypt/badge/
-   :target: https://readthedocs.org/projects/letsencrypt/
+   :target: https://readthedocs.org/projects/letsencrypt/
-   :alt: Documentation status
+   :alt: Documentation status
+
 
 .. |container| image:: https://quay.io/repository/letsencrypt/letsencrypt/status
-   :target: https://quay.io/repository/letsencrypt/letsencrypt
+   :target: https://quay.io/repository/letsencrypt/letsencrypt
-   :alt: Docker Repository on Quay.io
+   :alt: Docker Repository on Quay.io
+
 
 .. Do not modify this comment unless you know what you're doing. tag:links-end
 
@@ -143,13 +160,12 @@ Current Features
   - other server software via `third party plugins <https://certbot.eff.org/docs/using.html#third-party-plugins>`_
 
 * The private key is generated locally on your system.
-* Can talk to the Let's Encrypt CA or optionally to other ACME
+* Can talk to the Let's Encrypt CA or optionally to other ACME-compliant services.
-  compliant services.
 * Can get domain-validated (DV) certificates.
 * Can revoke certificates.
 * Adjustable RSA key bit-length (2048 (default), 4096, ...).
 * Can optionally install a http -> https redirect, so your site effectively
-  runs https only (Apache only)
+  runs HTTPS only (Apache only)
 * Fully automated.
 * Configuration changes are logged and can be reverted.
 * Supports an interactive text UI, or can be driven entirely from the
@@ -159,3 +175,5 @@ Current Features
 .. Do not modify this comment unless you know what you're doing. tag:features-end
 
 For extensive documentation on using and contributing to Certbot, go to https://certbot.eff.org/docs. If you would like to contribute to the project or run the latest code from git, you should read our `developer guide <https://certbot.eff.org/docs/contributing.html>`_.
+
+

docs/challenges.rst

@@ -1,56 +1,53 @@
+
+.. _challenges:
+
+
 Challenges
 ==========
 
-To receive a certificate from Let's Encrypt certificate authority (CA), you must pass a *challenge* to 
+To receive a certificate from Let's Encrypt certificate authority (CA), you must pass a *challenge_* to 
-prove you control each of the domain names that will be listed in the certificate. A challenge is one of 
+prove you control each of the domain names that will be listed in the certificate. A challenge is  one of 
 three tasks that only someone who controls the domain should be able to accomplish:
 
-* Posting a specified file in a specified location on a web site (the HTTP-01 challenge)
+* Posting a specified file in a specified location on a web site (See the IETF draft of the `HTTP-01`_ challenge)
-* Offering a specified temporary certificate on a web site (the TLS-SNI-01 challenge) 
+* Offering a specified temporary certificate on a web site (See the IETF draft of the `TLS-SNI-01`_ challenge) 
-* Posting a specified DNS record in the domain name system (the DNS-01 challenge)
+* Posting a specified DNS record in the domain name system (See the IETF draft of the `DNS-01`_ challenge)
+
+The DNS-01 challenge, in particular, requires configuration of a DNS server on
+port 53, though that's often not the same machine as your webserver. 
 
 It’s possible to complete each type of challenge *automatically* (Certbot directly makes the necessary 
 changes itself, or runs another program that does so), or *manually* (Certbot tells you to make a 
 certain change, and you edit a configuration file of some kind in order to accomplish it). Certbot's 
 design favors performing challenges automatically, and this is the normal case for most users of Certbot.
 
-Some plugins offer an *authenticator*, meaning that they can satisfy challenges:
+.. _TLS-SNI-01: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7.3
+.. _HTTP-01: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7.2
+.. _DNS-01: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7.4
 
-* Apache plugin: (TLS-SNI-01) Tries to edit your Apache configuration files to temporarily serve 
-  a Certbot-generated certificate for a specified name. Use the Apache plugin when you're running 
-  Certbot on a web server with Apache listening on port 443.
-* NGINX plugin: (TLS-SNI-01) Tries to edit your NGINX configuration files to temporarily serve a 
-  Certbot-generated certificate for a specified name. Use the NGINX plugin when you're running 
-  Certbot on a web server with NGINX listening on port 443.
-* Webroot plugin: (HTTP-01) Tries to place a file where it can be served over HTTP on port 80 by a
-  web server running on your system. Use the Webroot plugin when you're running Certbot on 
-  a web server with any server application listening on port 80 serving files from a folder on disk in response.
-* Standalone plugin: (TLS-SNI-01 or HTTP-01) Tries to run a temporary web server listening on either HTTP on 
-  port 80 (for HTTP-01) or HTTPS on port 443 (for TLS-SNI-01). Use the Standalone plugin if no existing program 
-  is listening to these ports. Choose TLS-SNI-01 or HTTP-01 using the `--preferred-challenges` option.
-* Manual plugin: (DNS-01 or HTTP-01) Either tells you what changes to make to your configuration or updates 
-  your DNS records using an external script (for DNS-01) or your webroot (for HTTP-01). Use the Manual 
-  plugin if you have the technical knowledge to make configuration changes yourself when asked to do so. 
 
 Tips for Challenges
 -------------------
 General tips:
 
 * Run Certbot on your web server, not on your laptop or another server. It’s usually the easiest way to get a certificate.
-* Use a tool like the DNSchecker at dnsstuff.com to check your DNS records to make sure 
+* Use a tool like the DNSchecker at dnsstuff.com to check your DNS records to make sure
   there are no serious errors. A DNS error can prevent a certificate authority from 
   issuing a certificate, even if it does not prevent your site from loading in a browser.
 * If you are using Apache or NGINX plugins, make sure the configuration of your Apache or NGINX server is correct.
 
+
+.. _http_01_challenge:
+
 HTTP-01 Challenge
-~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^
 
 * Make sure the domain name exists and is already pointed to the public IP address of the server where 
   you’re requesting the certificate.
 * Make sure port 80 is open, publicly reachable from the Internet, and not blocked by a router or firewall.
 * When using the Webroot plugin or the manual plugin, make sure the the webroot directory exists and that you
   specify it properly. If you set the webroot directory for example.com to `/var/www/example.com`
-  then a file placed in `/var/www/example.com/.well-known/acme-challenge/testfile` should appear on
+  then a file placed in `/var/www/example.com/.well-known/acme-challenge/testfile` should appear on 
   your web site at `http://example.com/.well-known/acme-challenge/testfile` (A redirection to HTTPS
   is OK here and should not stop the challenge from working.)
 * In some web server configurations, all pages are dynamically generated by some kind of framework, 
@@ -63,8 +60,11 @@ HTTP-01 Challenge
 * When using the Standalone plugin, make sure another program is not already listening to port 80 on the server.
 * When using the Webroot plugin, make sure there is a web server listening on port 80.
 
+
+.. _tls_sni_01_challenge:
+
 TLS-SNI-01 Challenge
-~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^
 
 * The TLS-SNI-01 challenge doesn’t work with content delivery networks (CDNs) 
   like CloudFlare and Akamai because the domain name is pointed at the CDN, not directly at your server.
@@ -77,9 +77,14 @@ TLS-SNI-01 Challenge
   In that case, please file a bug to help us improve certbot!
 * When using the Standalone plugin, make sure another program is not already listening to port 443 on the server.
 
+
+.. _dns_01_challenge:
+
 DNS-01 Challenge
-~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^
 
 * When using the manual plugin, make sure your DNS records are correctly updated; 
   you must be able to make appropriate changes to your DNS zone in order to pass the challenge.
 
+
+

docs/configure.rst

@@ -0,0 +1,92 @@
+Configuration Files
+=====================
+
+.. contents:: Table of Contents
+   :local:
+
+This is another new section that will talk about what can be configured and will explain 
+the interaction between the renewal configuration file and the global configuration file. 
+The text about the global configuration file has been moved here from using.rst
+
+.. _renewal-config-file:
+
+
+Renewal Configuration File
+
+Modifying the Renewal Configuration File
+----------------------------------------
+
+When a certificate is issued, by default Certbot creates a renewal configuration file that
+tracks the options that were selected when Certbot was run. This allows Certbot
+to use those same options again when it comes time for renewal. These renewal
+configuration files are located at ``/etc/letsencrypt/renewal/CERTNAME``.
+
+For advanced certificate management tasks, it is possible to manually modify the certificate's
+renewal configuration file, but this is discouraged since it can easily break Certbot's
+ability to renew your certificates. If you choose to modify the renewal configuration file
+we advise you to test its validity with the ``certbot renew --dry-run`` command.
+
+.. warning:: Modifying any files in ``/etc/letsencrypt`` can damage them so Certbot can no longer properly manage its certificates, and we do not recommend doing so.
+
+For most tasks, it is safest to limit yourself to pointing symlinks at the files there, or using
+``--deploy-hook`` to copy / make new files based upon those files, if your operational situation requires it
+(for instance, combining certificates and keys in different way, or having copies of things with different
+specific permissions that are demanded by other programs).
+
+If the contents of ``/etc/letsencrypt/archive/CERTNAME`` are moved to a new folder, first specify
+the new folder's name in the renewal configuration file, then run ``certbot update_symlinks`` to
+point the symlinks in ``/etc/letsencrypt/live/CERTNAME`` to the new folder.
+
+If you would like the live certificate files whose symlink location Certbot updates on each run to
+reside in a different location, first move them to that location, then specify the full path of
+each of the four files in the renewal configuration file. Since the symlinks are relative links,
+you must follow this with an invocation of ``certbot update_symlinks``.
+
+For example, say that a certificate's renewal configuration file previously contained the following
+directives::
+
+  archive_dir = /etc/letsencrypt/archive/example.com
+  cert = /etc/letsencrypt/live/example.com/cert.pem
+  privkey = /etc/letsencrypt/live/example.com/privkey.pem
+  chain = /etc/letsencrypt/live/example.com/chain.pem
+  fullchain = /etc/letsencrypt/live/example.com/fullchain.pem
+
+The following commands could be used to specify where these files are located::
+
+  mv /etc/letsencrypt/archive/example.com /home/user/me/certbot/example_archive
+  sed -i 's,/etc/letsencrypt/archive/example.com,/home/user/me/certbot/example_archive,' /etc/letsencrypt/renewal/example.com.conf
+  mv /etc/letsencrypt/live/example.com/*.pem /home/user/me/certbot/
+  sed -i 's,/etc/letsencrypt/live/example.com,/home/user/me/certbot,g' /etc/letsencrypt/renewal/example.com.conf
+  certbot update_symlinks
+
+.. _config-file:
+
+Global Configuration File
+=========================
+
+Certbot accepts a global configuration file that applies its options to all invocations
+of Certbot. Certificate specific configuration choices should be set in the ``.conf``
+files that can be found in ``/etc/letsencrypt/renewal``.
+
+By default no cli.ini file is created, after creating one 
+it is possible to specify the location of this configuration file with
+``certbot-auto --config cli.ini`` (or shorter ``-c cli.ini``). An
+example configuration file is shown below:
+
+.. include:: ../examples/cli.ini
+   :code: ini
+
+By default, the following locations are searched:
+
+- ``/etc/letsencrypt/cli.ini``
+- ``$XDG_CONFIG_HOME/letsencrypt/cli.ini`` (or ``~/.config/letsencrypt/cli.ini`` if ``$XDG_CONFIG_HOME`` is not set).
+
+Since this configuration file applies to all invocations of certbot it is incorrect
+to list domains in it. Listing domains in cli.ini may prevent renewal from working.
+Additionally due to how arguments in cli.ini are parsed, options which wish to
+not be set should not be listed. Options set to false will instead be read
+as being set to true by older versions of Certbot, since they have been listed
+in the config file.
+
+.. keep it up to date with constants.py
+

docs/contributing.rst

@@ -20,7 +20,7 @@ running:
 
 .. code-block:: shell
 
-   git clone https://github.com/certbot/certbot
+  git clone https://github.com/certbot/certbot
 
 If you're on macOS, we recommend you skip the rest of this section and instead
 run Certbot in Docker. You can find instructions for how to do this :ref:`here
@@ -31,17 +31,17 @@ a new plugin is introduced.
 
 .. code-block:: shell
 
-   cd certbot
+  cd certbot
-   ./certbot-auto --os-packages-only
+  ./certbot-auto --os-packages-only
-   ./tools/venv.sh
+  ./tools/venv.sh
 
 Then in each shell where you're working on the client, do:
 
 .. code-block:: shell
 
-   source ./venv/bin/activate
+  source ./venv/bin/activate
-   export SERVER=https://acme-staging.api.letsencrypt.org/directory
+  export SERVER=https://acme-staging.api.letsencrypt.org/directory
-   source tests/integration/_common.sh
+  source tests/integration/_common.sh
 
 After that, your shell will be using the virtual environment, and you run the
 client by typing `certbot` or `certbot_test`. The latter is an alias that
@@ -123,8 +123,8 @@ Boulder:
 
 .. code-block:: shell
 
-   export SERVER=http://localhost:4000/directory
+  export SERVER=http://localhost:4000/directory
-   source tests/integration/_common.sh
+  source tests/integration/_common.sh
 
 Run the integration tests using:
 
@@ -264,7 +264,7 @@ the virtualenv used by `certbot-auto`, but they will be wiped away when
    it with any necessary API changes.
 
 .. _`setuptools entry points`:
-    http://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
+   http://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
 
 .. _coding-style:
 
@@ -277,27 +277,26 @@ Please:
 
 2. Read `PEP 8 - Style Guide for Python Code`_.
 
-3. Follow the `Google Python Style Guide`_, with the exception that we
+3. Follow the `Google Python Style Guide`_, with the exception that we use `Sphinx-style`_ documentation::
-   use `Sphinx-style`_ documentation::
 
-        def foo(arg):
+  def foo(arg):
-            """Short description.
+      """Short description.
 
-            :param int arg: Some number.
+      :param int arg: Some number.
 
-            :returns: Argument
+      :returns: Argument
-            :rtype: int
+      :rtype: int
 
-            """
+      """
-            return arg
+      return arg
 
 4. Remember to use ``pylint``.
 
-.. _Google Python Style Guide:
+.. _Google Python Style Guide: https://google.github.io/styleguide/pyguide.html
-  https://google.github.io/styleguide/pyguide.html
+
 .. _Sphinx-style: http://sphinx-doc.org/
-.. _PEP 8 - Style Guide for Python Code:
+
-  https://www.python.org/dev/peps/pep-0008
+.. _PEP 8 - Style Guide for Python Code: https://www.python.org/dev/peps/pep-0008
 
 Submitting a pull request
 =========================
@@ -337,7 +336,7 @@ manually updating this script, run the build script, which lives at
 
 .. code-block:: shell
 
-   python letsencrypt-auto-source/build.py
+  python letsencrypt-auto-source/build.py
 
 Running ``build.py`` will update the ``letsencrypt-auto-source/letsencrypt-auto``
 script.  Note that the ``certbot-auto`` and ``letsencrypt-auto`` scripts in the root
@@ -383,8 +382,8 @@ testing Certbot. This is especially useful for macOS users. To install Docker
 Compose, follow the instructions at https://docs.docker.com/compose/install/.
 
 .. note:: Linux users can simply run ``pip install docker-compose`` to get
-  Docker Compose after installing Docker Engine and activating your shell as
+   Docker Compose after installing Docker Engine and activating your shell as
-  described in the :ref:`Getting Started <getting_started>` section.
+   described in the :ref:`Getting Started <getting_started>` section.
 
 Now you can develop on your host machine, but run Certbot and test your changes
 in Docker. When using ``docker-compose`` make sure you are inside your clone of
@@ -448,3 +447,5 @@ for the package.
 FreeBSD by default uses ``tcsh``. In order to activate virtualenv (see
 above), you will need a compatible shell, e.g. ``pkg install bash &&
 bash``.
+
+

docs/how.rst

@@ -0,0 +1,248 @@
+=====================
+How it Works
+=====================
+
+This new section will cover general info about how the following things work. 
+This is not a how-to, just a way of connecting the dots so that when people 
+are doing these tasks later, it all makes sense.
+
+XXX Description of the order of events. That is... you pick your webserver and OS on the interactive installation tool. Behind the scenes, certbot knows how to modify the config file and handle the challenge. You can optionally do hooks. You get authenticated and the cert gets installed. A config file gets created so that automatic renewal can happen. After that, you can manage, modify, or delete certs.
+
+
+.. include:: challenges.rst
+
+
+.. _plugins:
+
+
+Plugins
+=======
+
+The Certbot client supports two types of plugins for
+obtaining and installing certificates: authenticators and installers.
+
+Authenticators are plugins used with the ``certonly`` command to obtain a certificate.
+The authenticator satisfies a challenge to validate that you
+control the domain(s) you are requesting a certificate for, obtains a certificate for the specified
+domain(s), and places the certificate in the ``/etc/letsencrypt`` directory on your
+machine. The authenticator does not install the certificate (it does not edit any of your server's configuration files to serve the
+obtained certificate). If you specify multiple domains to authenticate, they will
+all be listed in a single certificate. To obtain multiple separate certificates
+you will need to run Certbot multiple times.
+
+
+Installers are Plugins used with the ``install`` command to install a certificate.
+These plugins can modify your webserver's configuration to
+serve your website over HTTPS using certificates obtained by certbot.
+
+Plugins that do both can be used with the ``certbot run`` command, which is the default
+when no command is specified. The ``run`` subcommand can also be used to specify
+a combination_ of distinct authenticator and installer plugins.
+
+=========== ==== ==== =============================================================== =============================
+Plugin      Auth Inst Notes                                                           Challenge types (and port)
+=========== ==== ==== =============================================================== =============================
+apache     Y    Y    | Automates obtaining and installing a certificate with Apache  :ref:`TLS-SNI-01 <tls_sni_01_challenge>` (443)
+                      | 2.4 on Debian-based distributions with ``libaugeas0`` 1.0+.
+webroot    Y    N    | Obtains a certificate by writing to the webroot directory of  :ref:`HTTP-01 <http_01_challenge>` (80)
+                      | an already running webserver.
+nginx      Y    Y    | Automates obtaining and installing a certificate with Nginx.  :ref:`TLS-SNI-01 <tls_sni_01_challenge>` (443)
+                      | Shipped with Certbot 0.9.0.
+standalone Y    N    | Uses a "standalone" webserver to obtain a certificate.        :ref:`HTTP-01 <http_01_challenge>` (80) or
+                      | Requires port 80 or 443 to be available. This is useful on    :ref:`TLS-SNI-01 <tls_sni_01_challenge>` (443)
+                      | systems with no webserver, or when direct integration with
+                      | the local webserver is not supported or not desired.
+manual     Y    N    | Helps you obtain a certificate by giving you instructions to  :ref:`HTTP-01 <http_01_challenge>` (80),
+                      | perform domain validation yourself. Additionally allows you   :ref:`DNS-01 <dns_01_challenge>` (53) or
+                      | to specify scripts to automate the validation task in a       :ref:`TLS-SNI-01 <tls_sni_01_challenge>` (443)
+                      | customized way.
+=========== ==== ==== =============================================================== =============================
+
+A few
+plugins support more than one challenge type, in which case you can choose one
+with ``--preferred-challenges``.
+
+.. _third-party-plugins:
+
+Third-party plugins
+-------------------
+
+There are also a number of third-party plugins for the client, provided by
+other developers. Many are beta/experimental, but some are already in
+widespread use:
+
+=========== ==== ==== ===============================================================
+Plugin      Auth Inst Notes
+=========== ==== ==== ===============================================================
+plesk_      Y    Y    Integration with the Plesk web hosting tool
+haproxy_    Y    Y    Integration with the HAProxy load balancer
+s3front_    Y    Y    Integration with Amazon CloudFront distribution of S3 buckets
+gandi_      Y    Y    Integration with Gandi's hosting products and API
+varnish_    Y    N    Obtain certificates via a Varnish server
+external_   Y    N    A plugin for convenient scripting (See also ticket 2782_)
+icecast_    N    Y    Deploy certificates to Icecast 2 streaming media servers
+pritunl_    N    Y    Install certificates in pritunl distributed OpenVPN servers
+proxmox_    N    Y    Install certificates in Proxmox Virtualization servers
+postfix_    N    Y    STARTTLS Everywhere is becoming a Certbot Postfix/Exim plugin
+heroku_     Y    Y    Integration with Heroku SSL
+=========== ==== ==== ===============================================================
+
+.. _plesk: https://github.com/plesk/letsencrypt-plesk
+.. _haproxy: https://github.com/greenhost/certbot-haproxy
+.. _s3front: https://github.com/dlapiduz/letsencrypt-s3front
+.. _gandi: https://github.com/Gandi/letsencrypt-gandi
+.. _icecast: https://github.com/e00E/lets-encrypt-icecast
+.. _varnish: http://git.sesse.net/?p=letsencrypt-varnish-plugin
+.. _2782: https://github.com/certbot/certbot/issues/2782
+.. _pritunl: https://github.com/kharkevich/letsencrypt-pritunl
+.. _proxmox: https://github.com/kharkevich/letsencrypt-proxmox
+.. _external: https://github.com/marcan/letsencrypt-external
+.. _postfix: https://github.com/EFForg/starttls-everywhere
+.. _heroku: https://github.com/gboudreau/certbot-heroku
+
+If you're interested, you can also :ref:`write your own plugin <dev-plugin>`.
+
+
+.. _lock-files:
+
+Lock Files
+==========
+
+When processing a validation Certbot writes a number of lock files on your system
+to prevent multiple instances from overwriting each other's changes. This means
+that be default two instances of Certbot will not be able to run in parallel.
+
+Since the directories used by Certbot are configurable, Certbot
+will write a lock file for all of the directories it uses. This include Certbot's
+``--work-dir``, ``--logs-dir``, and ``--config-dir``. By default these are
+``/var/lib/letsencrypt``, ``/var/logs/letsencrypt``, and ``/etc/letsencrypt``
+respectively. Additionally if you are using Certbot with Apache or nginx it will
+lock the configuration folder for that program, which are typically also in the
+``/etc`` directory.
+
+Note that these lock files will only prevent other instances of Certbot from
+using those directories, not other processes. If you'd like to run multiple
+instances of Certbot simultaneously you should specify different directories
+as the ``--work-dir``, ``--logs-dir``, and ``--config-dir`` for each instance
+of Certbot that you would like to run.
+
+
+
+.. _hooks:
+
+Pre and Post Validation Hooks
+=============================
+
+Certbot allows for the specification of pre and post validation hooks when run
+in manual mode. The flags to specify these scripts are ``--manual-auth-hook``
+and ``--manual-cleanup-hook`` respectively and can be used as follows::
+
+  certbot certonly --manual --manual-auth-hook /path/to/http/authenticator.sh --manual-cleanup-hook /path/to/http/cleanup.sh -d secure.example.com
+
+This will run the ``authenticator.sh`` script, attempt the validation, and then run
+the ``cleanup.sh`` script. Additionally certbot will pass relevant environment
+variables to these scripts:
+
+- ``CERTBOT_DOMAIN``: The domain being authenticated
+- ``CERTBOT_VALIDATION``: The validation string (HTTP-01 and DNS-01 only)
+- ``CERTBOT_TOKEN``: Resource name part of the HTTP-01 challenge (HTTP-01 only)
+- ``CERTBOT_CERT_PATH``: The challenge SSL certificate (TLS-SNI-01 only)
+- ``CERTBOT_KEY_PATH``: The private key associated with the aforementioned SSL certificate (TLS-SNI-01 only)
+- ``CERTBOT_SNI_DOMAIN``: The SNI name for which the ACME server expects to be presented the self-signed certificate located at ``$CERTBOT_CERT_PATH`` (TLS-SNI-01 only)
+
+Additionally for cleanup:
+
+- ``CERTBOT_AUTH_OUTPUT``: Whatever the auth script wrote to stdout
+
+Example usage for HTTP-01::
+
+  certbot certonly --manual --preferred-challenges=http --manual-auth-hook /path/to/http/authenticator.sh --manual-cleanup-hook /path/to/http/cleanup.sh -d secure.example.com
+
+/path/to/http/authenticator.sh
+
+.. code-block:: none
+
+   #!/bin/bash
+   echo $CERTBOT_VALIDATION > /var/www/htdocs/.well-known/acme-challenge/$CERTBOT_TOKEN
+
+/path/to/http/cleanup.sh
+
+.. code-block:: none
+
+   #!/bin/bash
+   rm -f /var/www/htdocs/.well-known/acme-challenge/$CERTBOT_TOKEN
+
+Example usage for DNS-01 (Cloudflare API v4) (for example purposes only, do not use as-is)::
+
+  certbot certonly --manual --preferred-challenges=dns --manual-auth-hook /path/to/dns/authenticator.sh --manual-cleanup-hook /path/to/dns/cleanup.sh -d secure.example.com
+
+/path/to/dns/authenticator.sh
+
+.. code-block:: none
+
+   #!/bin/bash
+
+   # Get your API key from https://www.cloudflare.com/a/account/my-account
+   API_KEY="your-api-key"
+   EMAIL="your.email@example.com"
+
+   # Strip only the top domain to get the zone id
+   DOMAIN=$(expr match "$CERTBOT_DOMAIN" '.*\.\(.*\..*\)')
+
+   # Get the Cloudflare zone id
+   ZONE_EXTRA_PARAMS="status=active&page=1&per_page=20&order=status&direction=desc&match=all"
+   ZONE_ID=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones?name=$DOMAIN&$ZONE_EXTRA_PARAMS" \
+        -H     "X-Auth-Email: $EMAIL" \
+        -H     "X-Auth-Key: $API_KEY" \
+        -H     "Content-Type: application/json" | python -c "import sys,json;print(json.load(sys.stdin)['result'][0]['id'])")
+
+   # Create TXT record
+   CREATE_DOMAIN="_acme-challenge.$CERTBOT_DOMAIN"
+   RECORD_ID=$(curl -s -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records" \
+        -H     "X-Auth-Email: $EMAIL" \
+        -H     "X-Auth-Key: $API_KEY" \
+        -H     "Content-Type: application/json" \
+        --data '{"type":"TXT","name":"'"$CREATE_DOMAIN"'","content":"'"$CERTBOT_VALIDATION"'","ttl":120}' \
+                | python -c "import sys,json;print(json.load(sys.stdin)['result']['id'])")
+   # Save info for cleanup
+   if [ ! -d /tmp/CERTBOT_$CERTBOT_DOMAIN ];then
+           mkdir -m 0700 /tmp/CERTBOT_$CERTBOT_DOMAIN
+   fi
+   echo $ZONE_ID > /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID
+   echo $RECORD_ID > /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID
+
+   # Sleep to make sure the change has time to propagate over to DNS
+   sleep 25
+
+/path/to/dns/cleanup.sh
+
+.. code-block:: none
+
+   #!/bin/bash
+
+   # Get your API key from https://www.cloudflare.com/a/account/my-account
+   API_KEY="your-api-key"
+   EMAIL="your.email@example.com"
+
+   if [ -f /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID ]; then
+           ZONE_ID=$(cat /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID)
+           rm -f /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID
+   fi
+
+   if [ -f /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID ]; then
+           RECORD_ID=$(cat /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID)
+           rm -f /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID
+   fi
+
+   # Remove the challenge TXT record from the zone
+   if [ -n "${ZONE_ID}" ]; then
+       if [ -n "${RECORD_ID}" ]; then
+           curl -s -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/$RECORD_ID" \
+                   -H "X-Auth-Email: $EMAIL" \
+                   -H "X-Auth-Key: $API_KEY" \
+                   -H "Content-Type: application/json"
+       fi
+   fi
+
+
+

docs/index.rst

@@ -4,13 +4,18 @@ Welcome to the Certbot documentation!
 .. toctree::
    :maxdepth: 2
 
+   start
    intro
-   what
+      what
+      how
+      challenges
    install
    using
    contributing
    packaging
    resources
+   reference
+      configure
 
 .. toctree::
    :maxdepth: 1
@@ -24,3 +29,4 @@ Indices and tables
 * :ref:`genindex`
 * :ref:`modindex`
 * :ref:`search`
+

docs/install.rst

@@ -1,22 +1,16 @@
 =====================
-Get Certbot
+Manual Installation
 =====================
 
 .. contents:: Table of Contents
    :local:
 
+.. Note:: The easiest way to install Certbot is by visiting `certbot.eff.org`_, 
+where you can find the correct installation instructions for many web server 
+and OS combinations.
 
-About Certbot
-=============
 
-Certbot is packaged for many common operating systems and web servers. Check whether
+.. _certbot.eff.org: https://certbot.eff.org/
-``certbot`` (or ``letsencrypt``) is packaged for your web server's OS by visiting
-certbot.eff.org_, where you will also find the correct installation instructions for
-your system.
-
-.. Note:: Unless you have very specific requirements, we kindly suggest that you use the Certbot packages provided by your package manager (see certbot.eff.org_). If such packages are not available, we recommend using ``certbot-auto``, which automates the process of installing Certbot on your system.
-
-.. _certbot.eff.org: https://certbot.eff.org
 
 
 System Requirements
@@ -77,23 +71,6 @@ For full command line help, you can type::
 
   ./certbot-auto --help all
 
-Problems with Python virtual environment
-----------------------------------------
-
-On a low memory system such as VPS with less than 512MB of RAM, the required dependencies of Certbot will fail to build.
-This can be identified if the pip outputs contains something like ``internal compiler error: Killed (program cc1)``.
-You can workaround this restriction by creating a temporary swapfile::
-
-  user@webserver:~$ sudo fallocate -l 1G /tmp/swapfile
-  user@webserver:~$ sudo chmod 600 /tmp/swapfile
-  user@webserver:~$ sudo mkswap /tmp/swapfile
-  user@webserver:~$ sudo swapon /tmp/swapfile
-
-Disable and remove the swapfile once the virtual environment is constructed::
-
-  user@webserver:~$ sudo swapoff /tmp/swapfile
-  user@webserver:~$ sudo rm /tmp/swapfile
-
 Running with Docker
 -------------------
 
@@ -241,3 +218,207 @@ whole process is described in the :doc:`contributing`.
    ./venv/bin/...``. These modes of operation might corrupt your operating
    system and are **not supported** by the Certbot team!
 
+Problems with Python virtual environment
+----------------------------------------
+
+On a low memory system such as VPS with less than 512MB of RAM, the required dependencies of Certbot will fail to build.
+This can be identified if the pip outputs contains something like ``internal compiler error: Killed (program cc1)``.
+You can workaround this restriction by creating a temporary swapfile::
+
+  user@webserver:~$ sudo fallocate -l 1G /tmp/swapfile
+  user@webserver:~$ sudo chmod 600 /tmp/swapfile
+  user@webserver:~$ sudo mkswap /tmp/swapfile
+  user@webserver:~$ sudo swapon /tmp/swapfile
+
+Disable and remove the swapfile once the virtual environment is constructed::
+
+  user@webserver:~$ sudo swapoff /tmp/swapfile
+  user@webserver:~$ sudo rm /tmp/swapfile
+
+
+.. _getting_certs:
+
+
+Getting Certificates
+==================== 
+
+
+
+
+XXX This section needs to have command-line examples for each plug-in.
+
+.. _apache:
+
+Apache
+------
+
+The Apache plugin currently requires an OS with augeas version 1.0; currently `it
+supports
+<https://github.com/certbot/certbot/blob/master/certbot-apache/certbot_apache/constants.py>`_
+modern OSes based on Debian, Fedora, SUSE, Gentoo and Darwin.
+This automates both obtaining *and* installing certificates on an Apache
+webserver. To specify this plugin on the command line, simply include
+``--apache``.
+
+* Apache plugin: (TLS-SNI-01) Tries to edit your Apache configuration files to temporarily serve 
+  a Certbot-generated certificate for a specified name. Use the Apache plugin when you're running 
+  Certbot on a web server with Apache listening on port 443.
+
+.. _manual:
+
+Manual
+------
+
+If you'd like to obtain a certificate running ``certbot`` on a machine
+other than your target webserver or perform the steps for domain
+validation yourself, you can use the manual plugin. While hidden from
+the UI, you can use the plugin to obtain a certificate by specifying
+``certonly`` and ``--manual`` on the command line. This requires you
+to copy and paste commands into another terminal session, which may
+be on a different computer.
+
+The manual plugin can use either the ``http``, ``dns`` or the
+``tls-sni`` challenge. You can use the ``--preferred-challenges`` option
+to choose the challenge of your preference.
+
+The ``http`` challenge will ask you to place a file with a specific name and
+specific content in the ``/.well-known/acme-challenge/`` directory directly
+in the top-level directory (“web root”) containing the files served by your
+webserver. In essence it's the same as the webroot_ plugin, but not automated.
+
+When using the ``dns`` challenge, ``certbot`` will ask you to place a TXT DNS
+record with specific contents under the domain name consisting of the hostname
+for which you want a certificate issued, prepended by ``_acme-challenge``.
+
+For example, for the domain ``example.com``, a zone file entry would look like::
+        _acme-challenge.example.com. 300 IN TXT "gfj9Xq...Rg85nM"
+
+When using the ``tls-sni`` challenge, ``certbot`` will prepare a self-signed
+SSL certificate for you with the challenge validation appropriately
+encoded into a subjectAlternatNames entry. You will need to configure
+your SSL server to present this challenge SSL certificate to the ACME
+server using SNI.
+
+Additionally you can specify scripts to prepare for validation and
+perform the authentication procedure and/or clean up after it by using
+the ``--manual-auth-hook`` and ``--manual-cleanup-hook`` flags. This is
+described in more depth in the hooks section.
+
+* Manual plugin: (DNS-01 or HTTP-01) Either tells you what changes to make to your configuration or updates 
+  your DNS records using an external script (for DNS-01) or your webroot (for HTTP-01). Use the Manual 
+  plugin if you have the technical knowledge to make configuration changes yourself when asked to do so. 
+
+.. _nginx:
+
+Nginx
+-----
+
+The Nginx plugin has been distributed with Certbot since version 0.9.0 and should
+work for most configurations. We recommend backing up Nginx
+configurations before using it (though you can also revert changes to
+configurations with ``certbot --nginx rollback``). You can use it by providing
+the ``--nginx`` flag on the commandline::
+
+   certbot –nginx
+
+* NGINX plugin: (TLS-SNI-01) Tries to edit your NGINX configuration files to temporarily serve a
+  Certbot-generated certificate for a specified name. Use the NGINX plugin when you're running 
+  Certbot on a web server with NGINX listening on port 443.
+
+.. _standalone:
+
+
+Standalone
+----------
+
+Use standalone mode to obtain a certificate if you don't want to use (or don't currently have)
+existing server software. The standalone plugin does not rely on any other server
+software running on the machine where you obtain the certificate.
+
+To obtain a certificate using a "standalone" webserver, you can use the
+standalone plugin by including ``certonly`` and ``--standalone``
+on the command line. This plugin needs to bind to port 80 or 443 in
+order to perform domain validation, so you may need to stop your
+existing webserver. To control which port the plugin uses, include
+one of the options shown below on the command line.
+
+    * ``--preferred-challenges http`` to use port 80
+    * ``--preferred-challenges tls-sni`` to use port 443
+
+It must still be possible for your machine to accept inbound connections from
+the Internet on the specified port using each requested domain name.
+
+.. note:: The ``--standalone-supported-challenges`` option has been
+   deprecated since ``certbot`` version 0.9.0.
+
+* Standalone plugin: (TLS-SNI-01 or HTTP-01) Tries to run a temporary web server listening on either HTTP on 
+  port 80 (for HTTP-01) or HTTPS on port 443 (for TLS-SNI-01). Use the Standalone plugin if no existing program 
+  is listening to these ports. Choose TLS-SNI-01 or HTTP-01 using the `--preferred-challenges` option.
+
+
+.. _webroot:
+
+Webroot
+-------
+
+If you're running a local webserver for which you have the ability
+to modify the content being served, and you'd prefer not to stop the
+webserver during the certificate issuance process, you can use the webroot
+plugin to obtain a certificate by including ``certonly`` and ``--webroot`` on
+the command line. In addition, you'll need to specify ``--webroot-path``
+or ``-w`` with the top-level directory ("web root") containing the files
+served by your webserver. For example, ``--webroot-path /var/www/html``
+or ``--webroot-path /usr/share/nginx/html`` are two common webroot paths.
+
+If you're getting a certificate for many domains at once, the plugin
+needs to know where each domain's files are served from, which could
+potentially be a separate directory for each domain. When requesting a
+certificate for multiple domains, each domain will use the most recently
+specified ``--webroot-path``. So, for instance::
+
+  certbot certonly --webroot -w /var/www/example/ -d www.example.com -d example.com -w /var/www/other -d other.example.net -d another.other.example.net
+
+would obtain a single certificate for all of those names, using the 
+``/var/www/example`` webroot directory for the first two, and ``/var/www/other`` for the second two.
+
+The webroot plugin works by creating a temporary file for each of your requested
+domains in ``${webroot-path}/.well-known/acme-challenge``. Then the Let's Encrypt
+validation server makes HTTP requests to validate that the DNS for each
+requested domain resolves to the server running certbot. An example request
+made to your web server would look like::
+
+  66.133.109.36 - - [05/Jan/2016:20:11:24 -0500] "GET /.well-known/acme-challenge/HGr8U1IeTW4kY_Z6UIyaakzOkyQgPr_7ArlLgtZE8SX HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
+
+Note that to use the webroot plugin, your server must be configured to serve
+files from hidden directories. If ``/.well-known`` is treated specially by
+your webserver configuration, you might need to modify the configuration
+to ensure that files inside ``/.well-known/acme-challenge`` are served by
+the webserver.
+
+* Webroot plugin: (HTTP-01) Tries to place a file where it can be served over HTTP on port 80 by a
+  web server running on your system. Use the Webroot plugin when you're running Certbot on 
+  a web server with any server application listening on port 80 serving files from a folder on disk in response.
+
+
+.. _combination:
+
+
+Combining plugins
+-----------------
+
+Sometimes you may want to specify a combination of distinct authenticator and
+installer plugins. To do so, specify the authenticator plugin with
+``--authenticator`` or ``-a`` and the installer plugin with ``--installer`` or
+``-i``.
+
+For instance, you may want to create a certificate using the webroot_ plugin
+for authentication and the apache_ plugin for installation, perhaps because you
+use a proxy or CDN for SSL and only want to secure the connection between them
+and your origin server, which cannot use the tls_sni_01_challege_ due to the
+intermediate proxy.
+
+::
+    certbot run -a webroot -i apache -w /var/www/html -d example.com
+
+
+

docs/intro.rst

@@ -1,5 +1,5 @@
 =====================
-Introduction
+Overview
 =====================
 
 .. note::
@@ -8,3 +8,9 @@ Introduction
 .. include:: ../README.rst
     :start-after: tag:intro-begin
     :end-before: tag:intro-end
+
+.. include:: what.rst
+
+.. include:: how.rst
+
+

docs/reference.rst

@@ -0,0 +1,35 @@
+=====================
+Reference
+=====================
+
+.. include:: configure.rst
+
+
+Certbot Logs
+============
+
+
+.. _log-rotation:
+
+Log Rotation
+------------
+
+By default certbot stores status logs in ``/var/log/letsencrypt``. By default
+certbot will begin rotating logs once there are 1000 logs in the log directory.
+Meaning that once 1000 files are in ``/var/log/letsencrypt`` Certbot will delete
+the oldest one to make room for new logs. The number of subsequent logs can be
+changed by passing the desired number to the command line flag
+``--max-log-backups``.
+
+.. _command-line:
+
+Certbot Command-line Reference
+==============================
+
+Certbot supports a lot of command line options. Here's the full list, from
+``certbot --help all``:
+
+.. literalinclude:: cli-help.txt
+
+
+

docs/start.rst

@@ -0,0 +1,44 @@
+=====================
+Quick Start
+=====================
+
+Start Here
+============
+
+Use the the `interactive installation guide <https://certbot.eff.org>`_.
+
+What you’ll find there:
+
+* Installation instructions
+* Instructions for getting a certificate
+* Automated renewal instructions
+
+You only need to go to the Manual Installation section if you have special needs: 
+an unusual server or configuration, if you need to modify an existing certificate, 
+if you need to run special scripts automatically before or after installation.
+
+
+Getting Help
+============
+
+If you're having problems, we recommend posting on the Let's Encrypt
+`Community Forum <https://community.letsencrypt.org>`_.
+
+You can also chat with us on IRC: `(#letsencrypt @
+freenode) <https://webchat.freenode.net?channels=%23letsencrypt>`_
+
+If you find a bug in the software, please do report it in our `issue
+tracker <https://github.com/certbot/certbot/issues>`_. Remember to
+give us as much information as possible:
+
+- copy and paste exact command line used and the output (though mind
+  that the latter might include some personally identifiable
+  information, including your email and domains)
+- copy and paste logs from ``/var/log/letsencrypt`` (though mind they
+  also might contain personally identifiable information)
+- copy and paste ``certbot --version`` output
+- your operating system, including specific version
+- specify which installation method you've chosen
+
+
+

docs/using.rst

@@ -1,269 +1,19 @@
-==========
+Working with Certificates
-User Guide
+=========================
-==========
 
 .. contents:: Table of Contents
    :local:
 
-Certbot Commands
+Before You Begin
 ================
 
-Certbot uses a number of different commands (also referred
-to as "subcommands") to request specific actions such as
-obtaining, renewing, or revoking certificates. The most important
-and commonly-used commands will be discussed throughout this
-document; an exhaustive list also appears near the end of the document.
-
 The ``certbot`` script on your web server might be named ``letsencrypt`` if your system uses an older package, or ``certbot-auto`` if you used an alternate installation method. Throughout the docs, whenever you see ``certbot``, swap in the correct name as needed.
 
-.. _plugins:
 
-Getting certificates (and choosing plugins)
+* Figure out which plugin to use
-===========================================
+* Use the plugin to get the certificate
 
-The Certbot client supports two types of plugins for
-obtaining and installing certificates: authenticators and installers.
 
-Authenticators are plugins used with the ``certonly`` command to obtain a certificate.
-The authenticator validates that you
-control the domain(s) you are requesting a certificate for, obtains a certificate for the specified
-domain(s), and places the certificate in the ``/etc/letsencrypt`` directory on your
-machine. The authenticator does not install the certificate (it does not edit any of your server's configuration files to serve the
-obtained certificate). If you specify multiple domains to authenticate, they will
-all be listed in a single certificate. To obtain multiple separate certificates
-you will need to run Certbot multiple times.
-
-Installers are Plugins used with the ``install`` command to install a certificate.
-These plugins can modify your webserver's configuration to
-serve your website over HTTPS using certificates obtained by certbot.
-
-Plugins that do both can be used with the ``certbot run`` command, which is the default
-when no command is specified. The ``run`` subcommand can also be used to specify
-a combination_ of distinct authenticator and installer plugins.
-
-=========== ==== ==== =============================================================== =============================
-Plugin      Auth Inst Notes                                                           Challenge types (and port)
-=========== ==== ==== =============================================================== =============================
-apache_     Y    Y    | Automates obtaining and installing a certificate with Apache  tls-sni-01_ (443)
-                      | 2.4 on Debian-based distributions with ``libaugeas0`` 1.0+.
-webroot_    Y    N    | Obtains a certificate by writing to the webroot directory of  http-01_ (80)
-                      | an already running webserver.
-nginx_      Y    Y    | Automates obtaining and installing a certificate with Nginx.  tls-sni-01_ (443)
-                      | Shipped with Certbot 0.9.0.
-standalone_ Y    N    | Uses a "standalone" webserver to obtain a certificate.        http-01_ (80) or
-                      | Requires port 80 or 443 to be available. This is useful on    tls-sni-01_ (443)
-                      | systems with no webserver, or when direct integration with
-                      | the local webserver is not supported or not desired.
-manual_     Y    N    | Helps you obtain a certificate by giving you instructions to  http-01_ (80),
-                      | perform domain validation yourself. Additionally allows you   dns-01_ (53) or
-                      | to specify scripts to automate the validation task in a       tls-sni-01_ (443)
-                      | customized way.
-=========== ==== ==== =============================================================== =============================
-
-Under the hood, plugins use one of several ACME protocol challenges_ to
-prove you control a domain. The options are http-01_ (which uses port 80),
-tls-sni-01_ (port 443) and dns-01_ (requiring configuration of a DNS server on
-port 53, though that's often not the same machine as your webserver). A few
-plugins support more than one challenge type, in which case you can choose one
-with ``--preferred-challenges``.
-
-There are also many third-party-plugins_ available. Below we describe in more detail
-the circumstances in which each plugin can be used, and how to use it.
-
-.. _challenges: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7
-.. _tls-sni-01: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7.3
-.. _http-01: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7.2
-.. _dns-01: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7.4
-
-Apache
-------
-
-The Apache plugin currently requires an OS with augeas version 1.0; currently `it
-supports
-<https://github.com/certbot/certbot/blob/master/certbot-apache/certbot_apache/constants.py>`_
-modern OSes based on Debian, Fedora, SUSE, Gentoo and Darwin.
-This automates both obtaining *and* installing certificates on an Apache
-webserver. To specify this plugin on the command line, simply include
-``--apache``.
-
-Webroot
--------
-
-If you're running a local webserver for which you have the ability
-to modify the content being served, and you'd prefer not to stop the
-webserver during the certificate issuance process, you can use the webroot
-plugin to obtain a certificate by including ``certonly`` and ``--webroot`` on
-the command line. In addition, you'll need to specify ``--webroot-path``
-or ``-w`` with the top-level directory ("web root") containing the files
-served by your webserver. For example, ``--webroot-path /var/www/html``
-or ``--webroot-path /usr/share/nginx/html`` are two common webroot paths.
-
-If you're getting a certificate for many domains at once, the plugin
-needs to know where each domain's files are served from, which could
-potentially be a separate directory for each domain. When requesting a
-certificate for multiple domains, each domain will use the most recently
-specified ``--webroot-path``. So, for instance,
-
-::
-
-    certbot certonly --webroot -w /var/www/example/ -d www.example.com -d example.com -w /var/www/other -d other.example.net -d another.other.example.net
-
-would obtain a single certificate for all of those names, using the
-``/var/www/example`` webroot directory for the first two, and
-``/var/www/other`` for the second two.
-
-The webroot plugin works by creating a temporary file for each of your requested
-domains in ``${webroot-path}/.well-known/acme-challenge``. Then the Let's Encrypt
-validation server makes HTTP requests to validate that the DNS for each
-requested domain resolves to the server running certbot. An example request
-made to your web server would look like:
-
-::
-
-    66.133.109.36 - - [05/Jan/2016:20:11:24 -0500] "GET /.well-known/acme-challenge/HGr8U1IeTW4kY_Z6UIyaakzOkyQgPr_7ArlLgtZE8SX HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
-
-Note that to use the webroot plugin, your server must be configured to serve
-files from hidden directories. If ``/.well-known`` is treated specially by
-your webserver configuration, you might need to modify the configuration
-to ensure that files inside ``/.well-known/acme-challenge`` are served by
-the webserver.
-
-Nginx
------
-
-The Nginx plugin has been distributed with Certbot since version 0.9.0 and should
-work for most configurations. We recommend backing up Nginx
-configurations before using it (though you can also revert changes to
-configurations with ``certbot --nginx rollback``). You can use it by providing
-the ``--nginx`` flag on the commandline.
-
-::
-
-   certbot --nginx
-
-Standalone
-----------
-
-Use standalone mode to obtain a certificate if you don't want to use (or don't currently have)
-existing server software. The standalone plugin does not rely on any other server
-software running on the machine where you obtain the certificate.
-
-To obtain a certificate using a "standalone" webserver, you can use the
-standalone plugin by including ``certonly`` and ``--standalone``
-on the command line. This plugin needs to bind to port 80 or 443 in
-order to perform domain validation, so you may need to stop your
-existing webserver. To control which port the plugin uses, include
-one of the options shown below on the command line.
-
-    * ``--preferred-challenges http`` to use port 80
-    * ``--preferred-challenges tls-sni`` to use port 443
-
-It must still be possible for your machine to accept inbound connections from
-the Internet on the specified port using each requested domain name.
-
-.. note:: The ``--standalone-supported-challenges`` option has been
-   deprecated since ``certbot`` version 0.9.0.
-
-Manual
-------
-
-If you'd like to obtain a certificate running ``certbot`` on a machine
-other than your target webserver or perform the steps for domain
-validation yourself, you can use the manual plugin. While hidden from
-the UI, you can use the plugin to obtain a certificate by specifying
-``certonly`` and ``--manual`` on the command line. This requires you
-to copy and paste commands into another terminal session, which may
-be on a different computer.
-
-The manual plugin can use either the ``http``, ``dns`` or the
-``tls-sni`` challenge. You can use the ``--preferred-challenges`` option
-to choose the challenge of your preference.
-
-The ``http`` challenge will ask you to place a file with a specific name and
-specific content in the ``/.well-known/acme-challenge/`` directory directly
-in the top-level directory (“web root”) containing the files served by your
-webserver. In essence it's the same as the webroot_ plugin, but not automated.
-
-When using the ``dns`` challenge, ``certbot`` will ask you to place a TXT DNS
-record with specific contents under the domain name consisting of the hostname
-for which you want a certificate issued, prepended by ``_acme-challenge``.
-
-For example, for the domain ``example.com``, a zone file entry would look like:
-
-::
-
-        _acme-challenge.example.com. 300 IN TXT "gfj9Xq...Rg85nM"
-
-When using the ``tls-sni`` challenge, ``certbot`` will prepare a self-signed
-SSL certificate for you with the challenge validation appropriately
-encoded into a subjectAlternatNames entry. You will need to configure
-your SSL server to present this challenge SSL certificate to the ACME
-server using SNI.
-
-Additionally you can specify scripts to prepare for validation and
-perform the authentication procedure and/or clean up after it by using
-the ``--manual-auth-hook`` and ``--manual-cleanup-hook`` flags. This is
-described in more depth in the hooks_ section.
-
-.. _combination:
-
-Combining plugins
------------------
-
-Sometimes you may want to specify a combination of distinct authenticator and
-installer plugins. To do so, specify the authenticator plugin with
-``--authenticator`` or ``-a`` and the installer plugin with ``--installer`` or
-``-i``.
-
-For instance, you may want to create a certificate using the webroot_ plugin
-for authentication and the apache_ plugin for installation, perhaps because you
-use a proxy or CDN for SSL and only want to secure the connection between them
-and your origin server, which cannot use the tls-sni-01_ challenge due to the
-intermediate proxy.
-
-::
-
-    certbot run -a webroot -i apache -w /var/www/html -d example.com
-
-.. _third-party-plugins:
-
-Third-party plugins
--------------------
-
-There are also a number of third-party plugins for the client, provided by
-other developers. Many are beta/experimental, but some are already in
-widespread use:
-
-=========== ==== ==== ===============================================================
-Plugin      Auth Inst Notes
-=========== ==== ==== ===============================================================
-plesk_      Y    Y    Integration with the Plesk web hosting tool
-haproxy_    Y    Y    Integration with the HAProxy load balancer
-s3front_    Y    Y    Integration with Amazon CloudFront distribution of S3 buckets
-gandi_      Y    Y    Integration with Gandi's hosting products and API
-varnish_    Y    N    Obtain certificates via a Varnish server
-external_   Y    N    A plugin for convenient scripting (See also ticket 2782_)
-icecast_    N    Y    Deploy certificates to Icecast 2 streaming media servers
-pritunl_    N    Y    Install certificates in pritunl distributed OpenVPN servers
-proxmox_    N    Y    Install certificates in Proxmox Virtualization servers
-postfix_    N    Y    STARTTLS Everywhere is becoming a Certbot Postfix/Exim plugin
-heroku_     Y    Y    Integration with Heroku SSL
-=========== ==== ==== ===============================================================
-
-.. _plesk: https://github.com/plesk/letsencrypt-plesk
-.. _haproxy: https://github.com/greenhost/certbot-haproxy
-.. _s3front: https://github.com/dlapiduz/letsencrypt-s3front
-.. _gandi: https://github.com/Gandi/letsencrypt-gandi
-.. _icecast: https://github.com/e00E/lets-encrypt-icecast
-.. _varnish: http://git.sesse.net/?p=letsencrypt-varnish-plugin
-.. _2782: https://github.com/certbot/certbot/issues/2782
-.. _pritunl: https://github.com/kharkevich/letsencrypt-pritunl
-.. _proxmox: https://github.com/kharkevich/letsencrypt-proxmox
-.. _external: https://github.com/marcan/letsencrypt-external
-.. _postfix: https://github.com/EFForg/starttls-everywhere
-.. _heroku: https://github.com/gboudreau/certbot-heroku
-
-If you're interested, you can also :ref:`write your own plugin <dev-plugin>`.
 
 .. _managing-certs:
 
@@ -359,7 +109,7 @@ abuse of the ACME protocol, as described
 .. _changing:
 
 Changing a Certificate's Domains
-================================
+--------------------------------
 
 The ``--cert-name`` flag can also be used to modify the domains a certificate contains,
 by specifying new domains using the ``-d`` or ``--domains`` flag. If certificate ``example.com``
@@ -562,58 +312,11 @@ commands into your individual environment.
   you will need to use the ``--post-hook`` since the exit status will be 0 both on successful renewal
   and when renewal is not necessary.
 
-.. _renewal-config-file:
-
-
-Modifying the Renewal Configuration File
-----------------------------------------
-
-When a certificate is issued, by default Certbot creates a renewal configuration file that
-tracks the options that were selected when Certbot was run. This allows Certbot
-to use those same options again when it comes time for renewal. These renewal
-configuration files are located at ``/etc/letsencrypt/renewal/CERTNAME``.
-
-For advanced certificate management tasks, it is possible to manually modify the certificate's
-renewal configuration file, but this is discouraged since it can easily break Certbot's
-ability to renew your certificates. If you choose to modify the renewal configuration file
-we advise you to test its validity with the ``certbot renew --dry-run`` command.
-
-.. warning:: Modifying any files in ``/etc/letsencrypt`` can damage them so Certbot can no longer properly manage its certificates, and we do not recommend doing so.
-
-For most tasks, it is safest to limit yourself to pointing symlinks at the files there, or using
-``--deploy-hook`` to copy / make new files based upon those files, if your operational situation requires it
-(for instance, combining certificates and keys in different way, or having copies of things with different
-specific permissions that are demanded by other programs).
-
-If the contents of ``/etc/letsencrypt/archive/CERTNAME`` are moved to a new folder, first specify
-the new folder's name in the renewal configuration file, then run ``certbot update_symlinks`` to
-point the symlinks in ``/etc/letsencrypt/live/CERTNAME`` to the new folder.
-
-If you would like the live certificate files whose symlink location Certbot updates on each run to
-reside in a different location, first move them to that location, then specify the full path of
-each of the four files in the renewal configuration file. Since the symlinks are relative links,
-you must follow this with an invocation of ``certbot update_symlinks``.
-
-For example, say that a certificate's renewal configuration file previously contained the following
-directives::
-
-  archive_dir = /etc/letsencrypt/archive/example.com
-  cert = /etc/letsencrypt/live/example.com/cert.pem
-  privkey = /etc/letsencrypt/live/example.com/privkey.pem
-  chain = /etc/letsencrypt/live/example.com/chain.pem
-  fullchain = /etc/letsencrypt/live/example.com/fullchain.pem
-
-The following commands could be used to specify where these files are located::
-
-  mv /etc/letsencrypt/archive/example.com /home/user/me/certbot/example_archive
-  sed -i 's,/etc/letsencrypt/archive/example.com,/home/user/me/certbot/example_archive,' /etc/letsencrypt/renewal/example.com.conf
-  mv /etc/letsencrypt/live/example.com/*.pem /home/user/me/certbot/
-  sed -i 's,/etc/letsencrypt/live/example.com,/home/user/me/certbot,g' /etc/letsencrypt/renewal/example.com.conf
-  certbot update_symlinks
 
 
 .. _where-certs:
 
+
 Where are my certificates?
 ==========================
 
@@ -677,224 +380,4 @@ The following files are available:
    could convert using ``openssl``. You can automate that with
    ``--deploy-hook`` if you're using automatic renewal_.
 
-.. _hooks:
 
-Pre and Post Validation Hooks
-=============================
-
-Certbot allows for the specification of pre and post validation hooks when run
-in manual mode. The flags to specify these scripts are ``--manual-auth-hook``
-and ``--manual-cleanup-hook`` respectively and can be used as follows:
-
-::
-
- certbot certonly --manual --manual-auth-hook /path/to/http/authenticator.sh --manual-cleanup-hook /path/to/http/cleanup.sh -d secure.example.com
-
-This will run the ``authenticator.sh`` script, attempt the validation, and then run
-the ``cleanup.sh`` script. Additionally certbot will pass relevant environment
-variables to these scripts:
-
-- ``CERTBOT_DOMAIN``: The domain being authenticated
-- ``CERTBOT_VALIDATION``: The validation string (HTTP-01 and DNS-01 only)
-- ``CERTBOT_TOKEN``: Resource name part of the HTTP-01 challenge (HTTP-01 only)
-- ``CERTBOT_CERT_PATH``: The challenge SSL certificate (TLS-SNI-01 only)
-- ``CERTBOT_KEY_PATH``: The private key associated with the aforementioned SSL certificate (TLS-SNI-01 only)
-- ``CERTBOT_SNI_DOMAIN``: The SNI name for which the ACME server expects to be presented the self-signed certificate located at ``$CERTBOT_CERT_PATH`` (TLS-SNI-01 only)
-
-Additionally for cleanup:
-
-- ``CERTBOT_AUTH_OUTPUT``: Whatever the auth script wrote to stdout
-
-Example usage for HTTP-01:
-
-::
-
- certbot certonly --manual --preferred-challenges=http --manual-auth-hook /path/to/http/authenticator.sh --manual-cleanup-hook /path/to/http/cleanup.sh -d secure.example.com
-
-/path/to/http/authenticator.sh
-
-.. code-block:: none
-
-   #!/bin/bash
-   echo $CERTBOT_VALIDATION > /var/www/htdocs/.well-known/acme-challenge/$CERTBOT_TOKEN
-
-/path/to/http/cleanup.sh
-
-.. code-block:: none
-
-   #!/bin/bash
-   rm -f /var/www/htdocs/.well-known/acme-challenge/$CERTBOT_TOKEN
-
-Example usage for DNS-01 (Cloudflare API v4) (for example purposes only, do not use as-is)
-
-::
-
- certbot certonly --manual --preferred-challenges=dns --manual-auth-hook /path/to/dns/authenticator.sh --manual-cleanup-hook /path/to/dns/cleanup.sh -d secure.example.com
-
-/path/to/dns/authenticator.sh
-
-.. code-block:: none
-
-   #!/bin/bash
-
-   # Get your API key from https://www.cloudflare.com/a/account/my-account
-   API_KEY="your-api-key"
-   EMAIL="your.email@example.com"
-
-   # Strip only the top domain to get the zone id
-   DOMAIN=$(expr match "$CERTBOT_DOMAIN" '.*\.\(.*\..*\)')
-
-   # Get the Cloudflare zone id
-   ZONE_EXTRA_PARAMS="status=active&page=1&per_page=20&order=status&direction=desc&match=all"
-   ZONE_ID=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones?name=$DOMAIN&$ZONE_EXTRA_PARAMS" \
-        -H     "X-Auth-Email: $EMAIL" \
-        -H     "X-Auth-Key: $API_KEY" \
-        -H     "Content-Type: application/json" | python -c "import sys,json;print(json.load(sys.stdin)['result'][0]['id'])")
-
-   # Create TXT record
-   CREATE_DOMAIN="_acme-challenge.$CERTBOT_DOMAIN"
-   RECORD_ID=$(curl -s -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records" \
-        -H     "X-Auth-Email: $EMAIL" \
-        -H     "X-Auth-Key: $API_KEY" \
-        -H     "Content-Type: application/json" \
-        --data '{"type":"TXT","name":"'"$CREATE_DOMAIN"'","content":"'"$CERTBOT_VALIDATION"'","ttl":120}' \
-                | python -c "import sys,json;print(json.load(sys.stdin)['result']['id'])")
-   # Save info for cleanup
-   if [ ! -d /tmp/CERTBOT_$CERTBOT_DOMAIN ];then
-           mkdir -m 0700 /tmp/CERTBOT_$CERTBOT_DOMAIN
-   fi
-   echo $ZONE_ID > /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID
-   echo $RECORD_ID > /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID
-
-   # Sleep to make sure the change has time to propagate over to DNS
-   sleep 25
-
-/path/to/dns/cleanup.sh
-
-.. code-block:: none
-
-   #!/bin/bash
-
-   # Get your API key from https://www.cloudflare.com/a/account/my-account
-   API_KEY="your-api-key"
-   EMAIL="your.email@example.com"
-
-   if [ -f /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID ]; then
-           ZONE_ID=$(cat /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID)
-           rm -f /tmp/CERTBOT_$CERTBOT_DOMAIN/ZONE_ID
-   fi
-
-   if [ -f /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID ]; then
-           RECORD_ID=$(cat /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID)
-           rm -f /tmp/CERTBOT_$CERTBOT_DOMAIN/RECORD_ID
-   fi
-
-   # Remove the challenge TXT record from the zone
-   if [ -n "${ZONE_ID}" ]; then
-       if [ -n "${RECORD_ID}" ]; then
-           curl -s -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/$RECORD_ID" \
-                   -H "X-Auth-Email: $EMAIL" \
-                   -H "X-Auth-Key: $API_KEY" \
-                   -H "Content-Type: application/json"
-       fi
-   fi
-
-.. _lock-files:
-
-Lock Files
-==========
-
-When processing a validation Certbot writes a number of lock files on your system
-to prevent multiple instances from overwriting each other's changes. This means
-that be default two instances of Certbot will not be able to run in parallel.
-
-Since the directories used by Certbot are configurable, Certbot
-will write a lock file for all of the directories it uses. This include Certbot's
-``--work-dir``, ``--logs-dir``, and ``--config-dir``. By default these are
-``/var/lib/letsencrypt``, ``/var/logs/letsencrypt``, and ``/etc/letsencrypt``
-respectively. Additionally if you are using Certbot with Apache or nginx it will
-lock the configuration folder for that program, which are typically also in the
-``/etc`` directory.
-
-Note that these lock files will only prevent other instances of Certbot from
-using those directories, not other processes. If you'd like to run multiple
-instances of Certbot simultaneously you should specify different directories
-as the ``--work-dir``, ``--logs-dir``, and ``--config-dir`` for each instance
-of Certbot that you would like to run.
-
-.. _config-file:
-
-Configuration file
-==================
-
-Certbot accepts a global configuration file that applies its options to all invocations
-of Certbot. Certificate specific configuration choices should be set in the ``.conf``
-files that can be found in ``/etc/letsencrypt/renewal``.
-
-By default no cli.ini file is created, after creating one 
-it is possible to specify the location of this configuration file with
-``certbot-auto --config cli.ini`` (or shorter ``-c cli.ini``). An
-example configuration file is shown below:
-
-.. include:: ../examples/cli.ini
-   :code: ini
-
-By default, the following locations are searched:
-
-- ``/etc/letsencrypt/cli.ini``
-- ``$XDG_CONFIG_HOME/letsencrypt/cli.ini`` (or
-  ``~/.config/letsencrypt/cli.ini`` if ``$XDG_CONFIG_HOME`` is not
-  set).
-
-Since this configuration file applies to all invocations of certbot it is incorrect
-to list domains in it. Listing domains in cli.ini may prevent renewal from working.
-Additionally due to how arguments in cli.ini are parsed, options which wish to
-not be set should not be listed. Options set to false will instead be read
-as being set to true by older versions of Certbot, since they have been listed
-in the config file.
-
-.. keep it up to date with constants.py
-
-.. _log-rotation:
-
-Log Rotation
-============
-
-By default certbot stores status logs in ``/var/log/letsencrypt``. By default
-certbot will begin rotating logs once there are 1000 logs in the log directory.
-Meaning that once 1000 files are in ``/var/log/letsencrypt`` Certbot will delete
-the oldest one to make room for new logs. The number of subsequent logs can be
-changed by passing the desired number to the command line flag
-``--max-log-backups``.
-
-.. _command-line:
-
-Certbot command-line options
-============================
-
-Certbot supports a lot of command line options. Here's the full list, from
-``certbot --help all``:
-
-.. literalinclude:: cli-help.txt
-
-Getting help
-============
-
-If you're having problems, we recommend posting on the Let's Encrypt
-`Community Forum <https://community.letsencrypt.org>`_.
-
-You can also chat with us on IRC: `(#letsencrypt @
-freenode) <https://webchat.freenode.net?channels=%23letsencrypt>`_
-
-If you find a bug in the software, please do report it in our `issue
-tracker <https://github.com/certbot/certbot/issues>`_. Remember to
-give us as much information as possible:
-
-- copy and paste exact command line used and the output (though mind
-  that the latter might include some personally identifiable
-  information, including your email and domains)
-- copy and paste logs from ``/var/log/letsencrypt`` (though mind they
-  also might contain personally identifiable information)
-- copy and paste ``certbot --version`` output
-- your operating system, including specific version
-- specify which installation method you've chosen

docs/what.rst

@@ -18,7 +18,7 @@ You can use Certbot to easily obtain and configure a free certificate from Let's
 joint project of EFF, Mozilla, and many other sponsors.
 
 Certificates and Lineages
-=========================
+-------------------------
 
 Certbot introduces the concept of a *lineage,* which is a collection of all the versions of a certificate
 plus Certbot configuration information maintained for that certificate from
@@ -29,3 +29,4 @@ a new one.
 
 See also:
 :ref:`updating_certs`
+