Containerization of applications gives a certain peace of mind to everyone. It’s great - the way applications can be wrapped into platform-agnostic containers which run on any system. Containers provide a generic way to run your applications on any host which makes them very unique. In my previous posts I talked about some basics of containers and we even went through an example where we containerized a simple NodeJS application. Once the application image is ready, it can be shared with anyone using Docker Hub or any locally hosted image repository and multiple instances can be spun anywhere.
In this post, I shall touch base on the networking between container instances. This is just a primer to get you started with Docker networking. The question is - how do containers communicate with each other? We shall try to get an understanding around the questions below.
- How do container instances hosted on the same host communicate?
- How do container instances hosted on different hosts communicate?
- Which ports are used by containers?
- What about network security?
- Do the containers have their own mac address?
- What are the ways in which we can configure network settings on containers?
Enabling networking on containers makes a lot of sense because it opens up multiple possibilities. Containers can actually communicate with external services like web servers, database applications, make API calls and much more - just like a normal system connected to a network - and the other system need not be docker aware. Without networking it wouldn’t make much sense to have containers as all they would do is run locally.
Below are some ways Docker manages network settings for running containers. Docker achieves network isolation by manipulating
iptables policies. It can be configured to use IPv6 addresses. Whenever a container is connected to a Docker network, it is assigned an IP address. Thus, Docker also acts as a DHCP server for each container.
Docker containers make use of bridge networks to communicate with other containers hosted on the same host. Bridge facilitates network isolation for the containers from the host network. Bridge network is created as soon as Docker is started on the host machine and containers connect to this bridge by default. Bridge networks could be hardware connected to host machine or software components. It is recommended to use user defined bridges as against default one, as the latter is considered legacy. User defined bridges have more advantages as below:
- Automatic DNS resolution: Containers connected on user defined bridges can identify other containers by the name they are assigned. In case of a default bridge, containers need to be identified using IP addresses.
- Better network isolation: Containers which are spun without using explicit –network flag, are connected to default bridge - which could be a risk.
- Containers can be attached detached from user defined network os on the fly. In case of a default bridge, you need to stop and start the containers with modified network settings.
- User defined networks are configurable, thus containers with different networking requirements can have their say.
- All containers connected to default bridge share environment variables - which can be of risk.
In the figure above, container A and D communicate using Default bridge - because that’s the default setting if you do not mention the
--network flag along with user defined network names. Container B and C communicate using
User Defined Bridge 2.
User Defined Bridge 1 is just defined and never used.
For situations where there is a network of Docker containers spread across different Docker hosts, overlay networks are used to enable communication. By default, 2 networks are created on the Docker host whenever they initialize or join Docker swarm. They are:
- ingress - used for control and data communication between various swarm nodes.
- docker_gwbridge - used for communication between Docker hosts. They work on the physical network of the host.
Referring to the diagram, we have 2 docker hosts hosting multiple containers within them (A1, B1, C1 on Host 1 and A2, B2, C2 on Host 2). Every docker host uses docker_gwbridge which acts as a virtual bridge between docker hosts and uses the host’s physical network for communication. Ingress can be understood as a network at container/services level which handles the data exchange between services/containers.
Overlay networks can be created in the same way Bridge networks are created, by specifying ‘overlay’ key in the command as below: docker network create -d overlay myoverlaynetwork
Primarily overlay networks are used in swarm mode, but that doesn’t mean they cannot be used by individual Docker containers. To enable this, overlay networks are created by passing the ‘–attachable’ flag. docker network create -d overlay –attachable myoverlaynetwork
Management service traffic is encrypted by default using AES Algorithm in GCM mode and application data can also be encrypted by passing –opt encrypted while creating an overlay network. This enables IPSEC encryption using IPSEC tunnels. Encrypting the overlay network has some performance impact, so it is advised to test the performance before implementing it in production.
Of course, overlay networks have default settings but can be customized to have different names, specify subnet ranges, specify maximum transmission unit value or specify gateway IP address. In order to customize ingress networks they need to be recreated. Only one ingress network can be created at a given time. All the services connected to this ingress network should be removed/stopped. Remove the current overlay ingress network and create it again by specifying customization options. Lastly, reconnect/restart all the services to this ingress network.
Similarly, docker_gwbridge can also be customized with these [Link here] options. This customization needs to be done before joining the swarm as this is part of the host machine’s kernel.
This is the most simple and straightforward way to enable networking on your Docker containers. Using this technique, the host machine’s network is directly used and network isolation is not used. This can be useful in cases where performance is important. To make use of the host network, –network host flag is set while running a docker container. By doing this, if you happen to publish a container service on a particular port X, this service will be available on host machine’s port X.
Host networks can also be used for services in swarm mode. In this case, communication between different hosts still happens using an overlay network, but the data which is sent to the container is sent on the host’s port. Since this method uses the host’s port directly, you cannot run multiple containers on the same port. Whereas in overlay and bridge networks, since the network interface of the container and host system is isolated, multiple services can be spun due to subnetting.
Macvlan network mode is used when it is required to assign mac addresses to your containers. As far as possible, use of this should be avoided by making use of other network modes due to some concerns as below:
- It is easy to make use of multiple IP addresses for the same mac address - this can be inappropriate.
- NIC of the host machine should have “promiscuous mode” [LINK HERE] turned on. Cloud providers usually don’t allow this. So this is only possible on your own devices. Macvlan networks are mostly used when dealing with legacy systems where it is required to supply MAC addresses of integration/communication. This can be done in 2 modes - bridge and trunk bridge. In bridge mode, containers use the host machine’s physical network interface while in trunk bridge mode, a sub-interface is created.
Despite all the network interfaces we have talked about here - if your container doesn’t need to be on the network, then the network can be completely disabled by passing the
--network none flag.
As mentioned at the start of the post, this is just a primer and should give you a decent idea of how networking in Docker containers work. If you have any questions, do leave a comment below or get in touch with me to share a practical aspect of any of these in detail.