Secure repository

Posted By Barry van Acker on 24 Sep 2018 in CICD in docker |


In the previous post, we introduced a Nexus repository and prepared it for use with docker. The individual repositories are present, and outbound communication has been established. However, we still can’t use the Nexus repository from docker. Docker is quite strict in its communication and requires a secure repository with encrypted connections. This means setting up an SSL-secured reverse-proxy to facilitate the communication.

This post is part of a series about creating a continues integration platform for home use.

 

  Create an artifact repository

  Configure the artifact repository

  Secure the artifact repository

 Create the Jenkins master

 Add a Jenkins slave

 Creating a sample project.

Setup the secure repository proxy

We will start by creating a folder for the reverse-proxy. This folder will hold the information needed to build a docker image specific for our need. It will hold the configuration for the proxy, which will be Nginx, and it will hold the certificates. This is the quickest and easiest way to build an image, but lacks some re-use potential. For now we will proceed with this simple setup, and we will use self-signed certificates.

In the demo folder, run the following commands.

You will be asked to fill in some details like your organisation name etc. These can be entered as you like. The only important question is the FQDN. This is the name by which the user will access the docker repository. This can be an official domain name you own, like docker.mycompany.com, a domain name setup on your local netwerk, a well known ip number (not user friendly), or (like I am using for local development) you can choose a name like mydocker, and add a mapping from mydocker to the correct ip number in your host file on every computer that is using the repository. (Requires root permissions on the clients).

You will see something like this:

You will now see two files in the certs folder: a domain.crt file containing your public certificate, and a domain.key file containing the private key. Make sure to keep the last one secret, and only use it on the reverse proxy.

 

Bonus: Generate the certificate using docker

If you are on windows, or just don’t wish to install openssl in order to generate one certificate, try using a docker image to create the certificate:

While the container is running, open a new commandline. You can find the id with docker ps and copy the certificate out of it using docker cp <containerid>:/certs .

 

Configure Nginx

We now have finished the preparations and are ready to start configuring Nginx.

Create the file https.conf in the folder reverse, and start adding the following upstreams:

Each upstream refers to a docker repository we configured in Nexus in the previous posts. An upstream is a destination where Nginx can forward it’s requests to. The reference is by hostname and portnumber. The hostname matches the name of the Nexus container in the docker-compose.yml, while the port number matches the http port we defined for each repository individually during the configuration of Nexus.

Next, we add a header field mapping that is required for the docker repository system.

Finally, we start adding the listeners for the inbound requests. The first listener will be on port 443, which is the default https port as well as the default docker registry port. This will allow us to use just mydocker as a destination, without specifying a port number.

Let’s analyse the above configuration:

  • A server definition creates a listener for inbound requests on a given port
  • We tell Nginx to listen on port 443for ssl-encrypted http2 requests. The interfaces we use are the ip4 and ip6 interfaces of this host.
  • Nginx should expect mydocker or mydocker.local as hostname. This is the hostname you’d type in a browser, before it is resolved to the ip-number. It must match the FQDN of the certificate we created earlier.
  • The public certificate and private key are provided. We will have to add the files to the docker image later on.
  • A required docker header is added.
  • Not all ssl protocols are secure. Some are outdated. Some are not supported by docker. We list the protocols and ciphers we wish to use.
  • Docker transfers can be huge. You might want to transfer a 16G image. We remove the max-size limit on the request, so that the client is allowed to send this much information. This also means that we can’t encode the entire request in memory, but we have to use a chunked approach.
  • The docker repository we use is version 2, so we expect the path to start with /v2/ which allows us to add a v1 or v3 with different settings if we ever need to.
  • Exclude old docker versions that don’t play nice.
  • We add the header mapping we defined at the very beginning of the http2.conf file, right after the upstreams. The mapping is required because add_header by itself only allows fixed data.
  • Finally, we tell Nginx what to do with the incoming request. The request should be forwared (proxied) to the upstream docker-public, which we defined at the very start. Nexus will need some extra headers again, this time they are related to the way a proxy server talks to the proxied server. It is used to forward information, such as the protocol used between the client and the proxy, the ip number of the client etc. We also set a large timeout, because storing large binaries might take some time.

What did we do?

  • We have forwarded the default docker port towards the Nexus docker-public repository. This is the group repository for our docker images, which means that when we pull an image from this default endpoint, the image will be retrieved from one of the following locations: docker-releases, docker-snapshots or docker-hub.

This endpoint allows us to find any docker image we created ourselves, or from the public docker-hub repository on the internet. We don’t need to know in which repository it is stored, all magic is handled by Nexus. Great.

The next step is storing docker images. We don’t want to use the generic port for this, but rather, we would like to specify what kind of image we are storing: is it a snapshot build created during development, or is it a candidate release build that might end up on production?

For this, we introduce two endpoints in Nginx, in a way similar to the default endpoint.

and

Note that the only differences are:

  • The listen port has changed for both ip4 and ip6
  • The proxy_pass destination has changed to the corresponding upstream.

This concludes our Nginx configuration. You can close the editor on https.conf. All we need to do now is to bundle the software, configuration and certificates in a docker image.

Bundling the package

In the reverse folder, we create a file Dockerfile and add the following content:

This specifies that we use the official Nginx distribution from docker-hub. We select a specific version to avoid update problems in the future. Our certificates are copied to the location we specified in the configuration file. Finally we copy the configuration file itself to the default location where Nginx expects it to be.

We have ended up with the following file-structure for the reverse proxy image:

Validate your results by typing docker build . on the command-line inside the reverse folder. It should download the base image and add the required files.

Bring it together

We now have a reverse proxy configured to forward all traffic towards the Nexus repository. All we need is to put them together in a single docker-compose environment, so that they can communicate. Go to the root folder of your project and edit the docker-compose.yml file. We will add some lines, so that the result will be:

Everything we added is in the reverse-proxy service:

  • The docker image will be identified by the name reverse-proxy
  • It is not a downloaded image like nexus, but instead it’s a locally build image that can be found in the folder reverse
  • It exposes a number of ports to the outside world, most specifically port 443, 8082 and 8083. The others are there for future use.

 

Running and testing

Now that we have both Nexus and Nginx in the docker-compose, it is time to start using it. Make sure your previous compose is stopped by typing docker-compose stop

Go to the main directory and build the composition by running docker-compose build (without the . that docker build . uses). You should see output like this:

Now we are ready to run the composition for the first time. Run it with docker-compose up so that it creates missing volumes if needed. To run it afterwards, use docker-compose start instead.

Before we can login, we need to make sure we can find the host mydocker. As discussed before, it needs to be registered. The simplest way is to register the name on the local machine:

On linux, edit the file /etc/hosts

On windows, edit the file C:\Windows\System32\drivers\etc\hosts

Add the following line:

This tells your computer that any traffic for mydocker will be routed towards the loopback ip number.

 

Now we can start testing. Try to log in to your repository by entering the following command

docker login mydocker

You will be asked for credentials. Provide the username and password for the user you created.

The command should end with the message “Login succeeded”

You are now logged on to the group repository that also contains the reference to docker-hub. Confirm this by doing a docker pull nginx

It should show:

Now try a docker push nginx. This should give you a denied message: you have no permissions to push to the nginx image on docker-hub.

Lets store this image in our docker repository. Begin by logging in to our snapshot repo: enter docker login mydocker:8083 and provide the usercredentials.

Tag and push the image:

You should see the layers being uploaded.

Verify the data in Nexus. 

It should show the nginx image you just uploaded:

You can find more details if you drill-down deeper.

 

Securing the admin interface

Now that we have secured the Docker interface, we can add the admin interface as well. Edit the https.conf file and add the admin port as an upstream at the start of the file.

Scroll down to the server component for port 443. This server contains one location for /v2/. What we want is to lroute trafic on /v2/ towards the Docker repository, and to route other data towards the admin pages. This works because no admin pages exists that use /v2/ as prefix.

Below the location /v2/ we add a new location. Make sure it is still inside the server section for port 443.

Test the new endpoint by rebuilding and starting your docker-compose. Make sure you can log on to the admin page. You’ll most likely need to accept the untrusted certificate before you can continue.

Finally, go to the docker-compose.yml and remove the following lines from the nexus configuration. The port will no longer be public.

Instead we will open a private port. Inside the expose section of nexus add:

Now our http admin port is no longer visible from the outside world, and we can only access it using the public https endpoint of the proxy.

 

Recap

We have introduced a reverse-proxy in order to create a secure repository. The proxy provides a https-secured Docker endpoint, which ensures that the transferred data is not intercepted. The data remains private and unmodified during transport.

In our case, we used a self-generated certificate for encryption. One important thing to note is that we didn’t have to install the certificate in docker. Docker accepts our certificate, even if it isn’t signed by a trusted certificate authority. Should future versions of Docker enforce the trust of te certificate, you’ll need to add the public certificate in the certificate folder on every client, which can be found at

Upcoming post

In the next post, we will deploy the Jenkins master in the docker-compose network and set it up as a work orchestration server.

Share This

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close