Setup Private Docker Registry

We use cookies and related technologies to remember user preferences, for security, to analyse our traffic, and to enable website functionality. Azure Container Registry now supports Azure Private Link, enabling private endpoints from a virtual network to be placed on a registry. Private endpoints are accessible from within the virtual network, using private IP addresses. We recommend using private endpoints instead of service endpoints in most network scenarios. I ran into the same issue when trying to do a pull from a private registry. I tried to install the certificate on the client and didn’t work, so I deleted it, then I realized that if I stop the docker service that is running as a systemd service, and start the docker daemon by hand with dockerd, I’m able to download the images. ECR is configured as a credsHelper and our CI machines used docker login to the private registry. Docker pull works for both registries, but attempting to build an image that references the private registry in the Dockerfile would fail with access forbidden. This is all on Linux machines, no OSX involved.

-->

Azure Virtual Network provides secure, private networking for your Azure and on-premises resources. A service endpoint allows you to secure your container registry's public IP address to only your virtual network. This endpoint gives traffic an optimal route to the resource over the Azure backbone network. The identities of the virtual network and the subnet are also transmitted with each request.

This article shows how to configure a container registry service endpoint (preview) in a virtual network.

Important

To start an instance of the registry, you’ll set up a docker-compose.yml file to define the location where your registry will be storing its data. On the server you have created to host your private Docker Registry, you can create a docker-registry directory, move into it, and then create a data subfolder with the following commands. As of v0.1.10, you have to configure your private registry credentials, but you can specify this registry as a default registry so that all system images are pulled from the designated private registry. You can use the command rke config -system-images to get the list of default system images to populate your private registry.

Azure Container Registry now supports Azure Private Link, enabling private endpoints from a virtual network to be placed on a registry. Private endpoints are accessible from within the virtual network, using private IP addresses. We recommend using private endpoints instead of service endpoints in most network scenarios.

Configuring a registry service endpoint is available in the Premium container registry service tier. For information about registry service tiers and limits, see Azure Container Registry service tiers.

Preview limitations

  • Future development of service endpoints for Azure Container Registry isn't currently planned. We recommend using private endpoints instead.
  • You can't use the Azure portal to configure service endpoints on a registry.
  • Only an Azure Kubernetes Service cluster or Azure virtual machine can be used as a host to access a container registry using a service endpoint. Other Azure services including Azure Container Instances aren't supported.
  • Service endpoints for Azure Container Registry aren't supported in the Azure US Government cloud or Azure China cloud.

Important

  • Azure Security Center can't currently perform image vulnerability scanning in a registry that restricts access to private endpoints, selected subnets, or IP addresses.
  • Instances of Azure services including Azure DevOps Services, Web Apps, and Azure Container Instances are also unable to access a network-restricted container registry.
  • Certain other Azure service instances can securely access a network-restricted container registry. For more information, see Allow trusted services to securely access a network-restricted container registry.

Prerequisites

  • To use the Azure CLI steps in this article, Azure CLI version 2.0.58 or later is required. If you need to install or upgrade, see Install Azure CLI.

  • If you don't already have a container registry, create one (Premium tier required) and push a sample image such as hello-world from Docker Hub. For example, use the Azure portal or the Azure CLI to create a registry.

  • If you want to restrict registry access using a service endpoint in a different Azure subscription, register the resource provider for Azure Container Registry in that subscription. For example:

Create a Docker-enabled virtual machine

For test purposes, use a Docker-enabled Ubuntu VM to access an Azure container registry. To use Azure Active Directory authentication to the registry, also install the Azure CLI on the VM. If you already have an Azure virtual machine, skip this creation step.

You may use the same resource group for your virtual machine and your container registry. This setup simplifies clean-up at the end but isn't required. If you choose to create a separate resource group for the virtual machine and virtual network, run az group create. The following example assumes you've set environment variables for the resource group name and registry location:

Now deploy a default Ubuntu Azure virtual machine with az vm create. The following example creates a VM named myDockerVM.

It takes a few minutes for the VM to be created. When the command completes, take note of the publicIpAddress displayed by the Azure CLI. Use this address to make SSH connections to the VM.

Install Docker on the VM

After the VM is running, make an SSH connection to the VM. Replace publicIpAddress with the public IP address of your VM.

Run the following commands to install Docker on the Ubuntu VM:

After installation, run the following command to verify that Docker is running properly on the VM:

Output:

Install the Azure CLI

Follow the steps in Install Azure CLI with apt to install the Azure CLI on your Ubuntu virtual machine. For example:

Exit the SSH connection.

Configure network access for registry

In this section, configure your container registry to allow access from a subnet in an Azure virtual network. Steps are provided using the Azure CLI.

Add a service endpoint to a subnet

Setup Private Docker Registry Windows 10

When you create a VM, Azure by default creates a virtual network in the same resource group. The name of the virtual network is based on the name of the virtual machine. For example, if you name your virtual machine myDockerVM, the default virtual network name is myDockerVMVNET, with a subnet named myDockerVMSubnet. Verify this by using the az network vnet list command:

Output:

Setup

Use the az network vnet subnet update command to add a Microsoft.ContainerRegistry service endpoint to your subnet. Substitute the names of your virtual network and subnet in the following command:

Use the az network vnet subnet show command to retrieve the resource ID of the subnet. You need this in a later step to configure a network access rule.

Output:

Change default network access to registry

By default, an Azure container registry allows connections from hosts on any network. To limit access to a selected network, change the default action to deny access. Substitute the name of your registry in the following az acr update command:

Add network rule to registry

Use the az acr network-rule add command to add a network rule to your registry that allows access from the VM's subnet. Substitute the container registry's name and the resource ID of the subnet in the following command:

Verify access to the registry

After waiting a few minutes for the configuration to update, verify that the VM can access the container registry. Make an SSH connection to your VM, and run the az acr login command to login to your registry.

You can perform registry operations such as run docker pull to pull a sample image from the registry. Substitute an image and tag value appropriate for your registry, prefixed with the registry login server name (all lowercase):

Docker successfully pulls the image to the VM.

This example demonstrates that you can access the private container registry through the network access rule. However, the registry can't be accessed from a login host that doesn't have a network access rule configured. If you attempt to login from another host using the az acr login command or docker login command, output is similar to the following:

Restore default registry access

To restore the registry to allow access by default, remove any network rules that are configured. Then set the default action to allow access.

Remove network rules

To see a list of network rules configured for your registry, run the following az acr network-rule list command:

Setup Private Docker Registry

For each rule that is configured, run the az acr network-rule remove command to remove it. For example:

Allow access

Substitute the name of your registry in the following az acr update command:

Setup Private Docker Registry Centos

Clean up resources

Setup Private Docker Registry

If you created all the Azure resources in the same resource group and no longer need them, you can optionally delete the resources by using a single az group delete command:

Set Up Private Docker Registry Centos

Next steps

Setup Private Docker Registry Kubernetes

  • To restrict access to a registry using a private endpoint in a virtual network, see Configure Azure Private Link for an Azure container registry.
  • If you need to set up registry access rules from behind a client firewall, see Configure rules to access an Azure container registry behind a firewall.

A private Docker registry allows you to share your custom base images within your organization,keeping a consistent, private, and centralized source of truth for the building blocks of your architecture.A private Docker registry gives you better performances for big clusters and high-frequency roll-outs,plus added features like access authentication.

In an earlier post, we had a look at how one could store Docker images in Exoscale’s S3-compatible object storage.

We described how to configure a Docker registry storing images on Exoscale’s Object Storage, yet this keeps the local Docker instance responsible for the processing itself.

To improve availability, a registry would be better hosted on an external server.
Let’s see how to setup a private registry, and then later how to secure the whole thing.We will split the process in two, setting up first and securing the registry later.

What is a private Docker registry anyway?

After building a Docker image on your machine, it’s possible to run it on the spot.But if you’re a software provider, what if you want to share the image with the whole world?Or, what if you want to privately share the image with your team?

A registry can be considered private if pulling requires authentication

A Docker registry is a place where you can store your images i.e.docker push, and let third-parties get them i.e.docker pull.Docker Hub is the default registry.For example, let’s run:

In a very simplified way, the process goes like this:

  1. Check if the hello-world image is found locally
  2. If it isn’t, pull it from Docker Hub
  3. Register it in the local Docker.The image is now available locally
  4. Run it via the local Docker daemon

Note that while you can pull freely, pushing still requires some kind of authentication.
A registry can be considered private if pulling requires authentication too.

For example, GitLab, a popular Continuous Integration platform,provides a Docker registry per project among more traditional “build” capabilities,and it can be configured to be freely accessible or private.

However, GitLab’s registry is a solution that is still a bit rough around the edges.
It’s quite hard to remove images (while it’s possible to untag them though), andmore importantly, using the SaaS version of Gitlab’s registry is an all-or-nothing option:there’s no way to customize it e.g. integrate it with one’s identity store for authenticated access.

Let’s start from scratch instead, and publish our private registry on Exoscale’s cloud servers.

How to set up a private Docker registry

Good news, a Docker registry is just a Docker image!So, in order to set up a Docker registry, you first need to…setup Docker itself.

How to securely install the latest Docker release

There are two ways to install Docker:

  • From a package:this requires downloading a specific package and manually installing it e.g.dpkg my-package.deb
  • From a repository e.g.apt-get install my-package.

Installing from a repository makes updates to installed packages applied in an automated way.
The biggest advantage of this approach is that the system will always benefit from the latest security patch.In the light of some recent security scandals related to an outdated library/package, it seems there’s no other way.

Let’s choose option 2 using a fresh Ubuntu instance. To install Docker is just as easy as:

However, getting the latest and greatest version requires a bit more effort,as the official Ubuntu repository lags behind the Docker release cycle.

To do so, the official Docker package repository should first be added to the list of available repositories.This requires some preliminary setup, as it is a security-sensitive operation.

  1. As key system components, packages should be signed and verified.The provider will sign a package using a private key, and provide a public key so that a third-party can check it’s genuine.Since the check process is handled by the system, we need to provide it with the Docker GPG public key first.

  2. If somebody hacked the previous site to set its own key, it could impersonate the Docker organization and sign malicious packages.We need to assert this is the correct key:

    0EBFCD88 is part of Docker’s public key.This should output something akin to the following:

    Notice that the uid contains the string 0EBF CD88 that we searched for, as well as Docker Release (CE deb) <[email protected]>.At this point, we have allowed the installation of packages signed by Docker.

  3. Add the proper repository:

  4. Install Docker (finally):

To test the correct installation, execute this command:

This should yield the following:

Great, let’s pat ourselves on the back because at this point, Docker is (finally) ready to use.Remember that we had to process all those steps because the out-of-the-box Ubuntu repository doesn’t contain the latest version of Docker.

How to install the Docker registry on a virtual machine

Now is time for the proper Docker registry installation.Interestingly enough, this might be the easiest part of the whole setup process.As stated above, a Docker registry is just a specific running container, registry.

That command:

  • downloads the registry image which is tagged 2.This tag references the latest version of the registry at the time of this writing.
  • exposes port 5000 to the host, under the same port
  • gives the container the name registry instead of assigning it a random name

To make sure that the registry is running, a simple docker ps should display the following (abridged for readability):

How to push a custom Docker image to a remote private registry

Now, to test that the registry behaves as attended, let’s push a basic image to our brand-new shiny registry.We will use the hello-world image.

The goal is now to push the local image to the registry available remotely.

If you followed the above procedure, you might have noticed the hello-world image is already available on the remote Docker.Now, you might think if you push, nothing will change because the image is already present remotely, but this is actually wrong.

There’s a clear difference between an image available to Docker, and an image stored in a Docker registry.

  • In the first case, it can be listed and run by the Docker daemon to which it belongs.
  • In the second case, it cannot.

For an image stored in a registry to be run requires it to be first pulled to a Docker instance.

There’s one constraint though, the image’s name needs to be prefixed with the registry’s URL (whether domain-based or IP), port included e.g.159.100.243.157:5000/hello-world.
This is the responsibility of the docker tag command:

The new label should now appear:

Although listed with two different labels, the very same hello-world image is referenced, so that the disk space is used only once.You can make sure of that by looking at the image ID (4ab4c602aa5e in our case).

Labels are in fact pretty similar to links created by the ln command.If you remove one of the labels, the image will still be available with the other one.In order to remove the image altogether, it has to be referenced without a label.

Now is time to push the image to the registry, with the docker push command:

Chances are high to get the following error output:

Docker expects a secured channel by default, and that’s naturally a very good thing.But TLS adds another layer of complexity, and possible issues, so let’s skip that for now and we’ll come back on the subject a bit later.

Configuring Docker to accept connections to unsecure registries depends on your OS,but it’s quite straightforward. In all cases you will need to update a daemon.json file.

On Linux the .json file is located /etc/docker/daemon.json and assuming no other setting is present in the file, it should look like this:

You can create the file if does not exist, and you will need to restart Docker afterwards for the changes to take effect.On macOS you do it using the user interface, and the changes will automatically restart the daemon:

  1. Click on the Docker icon
  2. Select Preferences… in the menu
  3. Select the Daemon tab
  4. Check the checkbox named Experimental features
  5. In the first list box, enter the address (URL or IP) of the unsecure registry e.g. 159.100.243.157:5000

Wait a bit for the Docker daemon to restart, then push again to the registry with the same command-line as above.This time, it should be a success:

The image should now be safely stored on the Docker registry we set up.In order to make sure of that, we can ask this remote registry what images it contains.Fortunately, the registry also offers a web API to query stored images.

From any machine, type:

This should return:

Security considerations

Setup Private Docker Registry Windows

Congratulations, you managed to install your own private Docker registry and push your image to it!Remember that at this point, the registry is not secured, and in more than one way.

We have seen while pushing to the registry that Docker expects a secured channel by default, but we have skipped it to keep things simple.Moreover, anybody can push to, or pull from the registry…

Setup Private Docker Registry Windows

You should never leave it in such a state for a real-world setup!If you do, bad things will happen sooner or later, as Aeroflot can attest.

Setup Private Docker Registry Download

We will show you how to secure your registry in another blog post, follow us on Twitter to stay tuned!

Setup Private Docker Registry Centos 7

References