Assigning Static IPs from a Specific Physical Subnet to Docker Containers

The official documentation regarding bridge and IP configuration can be found at: https://docs.docker.com/articles/networking/

Recently, several clients have prepared to deploy Docker on some production lines for testing purposes. Here, I will share the network configuration process with you:

1. The host machine (running ubuntu-14.04.1-server-amd64) uses bridged networking mode (the default is DHCP mode, but static IP is another option). The subnet of the bridge should match the physical subnet. The current physical subnet is 192.168.5.1/24, the gateway is 192.168.1.1, and the physical network interface is eth0.

Edit the configuration file /etc/network/interfaces to customize the bridge br0:

root@ubuntu-docker:~# cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
#auto eth0
#iface eth0 inet dhcp
#
auto br0
iface br0 inet static
address 192.168.5.116
netmask 255.255.255.0
gateway 192.168.5.1
bridge_ports eth0
bridge_stp off
dns-nameservers 192.168.1.12 192.168.1.13
root@ubuntu-docker:~#

After saving and exiting the configuration, restart the network:

root@ubuntu-docker:~# ifdown -a && ifup -a

After configuration, use ifconfig to view the status of br0 and eth0 as shown below (the host’s IP address is 192.168.5.116):

br0       Link encap:Ethernet  HWaddr 00:0c:29:dc:47:11
inet addr:192.168.5.116  Bcast:192.168.5.255  Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fedc:4711/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:737 errors:0 dropped:0 overruns:0 frame:0
TX packets:226 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:70000 (70.0 KB)  TX bytes:26394 (26.3 KB)

eth0      Link encap:Ethernet  HWaddr 00:0c:29:dc:47:11
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:70065 errors:0 dropped:151 overruns:0 frame:0
TX packets:19621 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:23933126 (23.9 MB)  TX bytes:2321651 (2.3 MB)

2. After the bridge is configured, the remaining task is Docker-related network configuration.

a) The network mode for container startup must be set to none, specified using –net=none, for example:

docker run -it --rm --net=none eeed74b237f9 /bin/bash

b) Obtain the target container ID. Mine is dfe83012cda2.

root@ubuntu-docker:~# docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
dfe83012cda2        centos:centos6.6    /bin/bash           About an hour ago   Up 23 minutes                           test
root@ubuntu-docker:~#

c) The steps to configure the IP are somewhat tedious, so I wrote a script. You can pass the relevant parameters to the script to set the IP. For example, to set a static IP address of 192.168.5.123, subnet mask 255.255.255.0, and gateway 192.168.5.1 for container dfe83012cda2:

root@ubuntu-docker:~# sh manual_con_static_ip.sh dfe83012cda2 192.168.5.123 24 192.168.5.1

Note that when the container restarts, the set IP will be lost, requiring you to re-execute this script once.

The content of the script is as follows (written with reference to the official documentation):

#!/bin/bash
if [ -z $1 ] || [ -z $2 ] || [ -z $3 ] || [ -z $4 ] || [ -z $5 ];
then
echo "*****Input the necessary parameters: CONTAINERID IP MASK GATEWAY ETHNAME"
echo "*****Call the script like: sh manual_con_static_ip.sh  b0e18b6a4432 192.168.5.123 24 192.168.5.1 deth0"
exit
fi

CONTAINERID=$1
SETIP=$2
SETMASK=$3
GATEWAY=$4
ETHNAME=$5

# Check if the host network interface exists
ifconfig $ETHNAME > /dev/null 2>&1
if [ $? -eq 0 ]; then
read -p "$ETHNAME exist, do you want delete it? y/n " del
if [[ $del == 'y' ]]; then
ip link del $ETHNAME
else
exit
fi
fi
#
pid=`docker inspect -f '{{.State.Pid}}' $CONTAINERID`
mkdir -p /var/run/netns
find -L /var/run/netns -type l -delete

if [ -f /var/run/netns/$pid ]; then
rm -f /var/run/netns/$pid
fi
ln -s /proc/$pid/ns/net /var/run/netns/$pid
#
ip link add $ETHNAME type veth peer name B
brctl addif br3 $ETHNAME
ip link set $ETHNAME up
ip link set B netns $pid
# Delete existing eth0 inside the container first
ip netns exec $pid ip link del eth0 > /dev/null 2>&1
# Set new network card eth0 for the container
ip netns exec $pid ip link set dev B name eth0
ip netns exec $pid ip link set eth0 up
ip netns exec $pid ip addr add $SETIP/$SETMASK dev eth0
ip netns exec $pid ip route add default via $GATEWAY

Below is an explanation of the script’s commands line by line:

pid=`docker inspect -f '{{.State.Pid}}' $CONTAINERID`

To operate on the container, we need to obtain the container’s Process ID (PID). docker inspect can view the underlying information of the container. To view all underlying related information for container dfe83012cda2, use docker inspect dfe83012cda2. The -f parameter allows formatted output of given information, such as checking the container’s status:

root@ubuntu-docker:~# docker inspect -f '{{.State.Running}}' dfe83012cda2
true
root@ubuntu-docker:~#

Create a net namespace for the container and establish a point-to-point connection (between the container’s namespace network card and the network card generated on the host). Ensure the directory /var/run/netns where net namespaces are stored exists, then delete invalid links in that directory, and finally create a symbolic link of the container’s net namespace file to /var/run/netns, allowing ip netns to read it.

mkdir -p /var/run/netns
find -L /var/run/netns -type l -delete
ln -s /proc/$pid/ns/net /var/run/netns/$pid

Create two directly connected network cards (A and B connected directly) on the host. Use B as the network card inside the container and A as the network card on the host.

ip link add A type veth peer name B

Bridge network card A to br0 and start network card A:

brctl addif br0 A
ip link set A up

Add network card B to the corresponding container’s net namespace. Once network card B is added to the container’s net namespace, the host will no longer be able to see this network card information (executing ip netns reads from /var/run/netns by default).

ip link set B netns $pid

ip netns exec can enter the container’s net namespace and is used to configure network parameters for the container’s net namespace, specifically configuring network card B inside the container:

ip netns exec $pid ip link set dev B name eth0
ip netns exec $pid ip link set eth0 up
ip netns exec $pid ip addr add $SETIP/$SETMASK dev eth0
ip netns exec $pid ip route add default via $GATEWAY

Next, let’s understand the process of Docker creating networks for containers (http://dockerpool.com/static/books/docker_practice/underly/network.html). This will clarify why we configure it this way.

1) Create a pair of virtual interfaces and place one in the local host and the other in the new container;

2) Bridge the local host end to the default docker0 or a specified bridge, giving it a unique name, such as veth65f9;

3) Place the container end into the new container and rename it as eth0. This interface is only visible within the container’s namespace;

4) Obtain an available address from the bridge’s address range and assign it to the container’s eth0, configuring the default route to the bridged network card veth65f9.

When we start a container using the –net=none parameter, Docker does not configure the container’s network. If container network availability is required, we must configure the container’s network following the same steps.