Detailed introduction to Docker storage driver
I recently worked on a project and did not know how to use the Docker storage driver during this period, so I looked for information online and solved it. I will record it here.
Purpose
Docker is an open source application container engine that mainly uses the Linux kernel namespace to achieve sandbox isolation and uses Cgroup to achieve resource limitation. Docker is a lightweight Linux container used for unified development and deployment, trying to solve the "dependence hell" problem, combining dependent services and components, similar to containers used by ships, and achieving rapid installation and deployment.
1. The basic architecture of Docker - Client and Daemon
Let us first understand the basic architecture and startup process of docker. In fact, Docker adopts a C/S architecture, including the client and server. Docker daemon accepts requests from customers as a server and processes these requests (create, run, submit containers). The client and the server communicate on the same machine through the RESTful API. In the specific process of use, after executing service docker start, the docker deamon daemon process is generated on the host, run in the background and wait for receiving messages from the client (i.e., input docker commands, such as docker pull xxx, docker run…, docker commit xxx) to achieve interaction with docker deamon. After starting the docker service, you can see the docker process.
Default
[root@localhost ~]# ps -aux | grep dockerroot 11701 0.0 0.4 359208 16624 ? Ssl 21:05 0:00 /usr/bin/docker -d -H fd:// --selinux-enabled --insecure-registry 186.100.8.216:5000root 11861 0.0 0.0 113004 2256 pts/0 S+ 23:01 0:00 grep --color=auto docker
This is mainly because when specifying the file system later, you need to configure the specific storage driver in /etc/sysconfig/docker (this will write a special blog), and then start Docker Daemon, and cannot operate through the parameters of the run command. You can also set it directly from the host command line through docker d.
2. Docker storage method - Storage Driver
The core part of the Docker model is to effectively utilize the hierarchical mirroring mechanism. The mirror can be inherited through hierarchy. Based on the basic image (without the parent image), various specific application images can be made. Different Docker containers can share some basic file system layers, and at the same time, coupled with their own unique change layers, greatly improving storage efficiency. The main mechanism is to hierarchical model and mount different directories to the same virtual filesystem (unite several directories into a single virtual filesystem, from this article). Several different storage drivers are used for mirror storage docker, including: aufs, devicemapper, btrfs and overlay (from the official website). The following is a brief introduction to different storage drivers.
AUFS
AUFS (AnotherUnionFS) is a joint file system. AUFS supports setting readonly, readwrite and whiteout-able permissions for each member directory (similar to Git). At the same time, there is a similar concept in AUFS, where branches with read-only permissions can be modified logically incrementally (not affecting the read-only part). AUFS's only storage driver can realize the sharing of executable and shareable runtime libraries between containers, so when you run hundreds of runtimes with the same program code or runtime libraries, AUFS is a very good choice.
Device mapper
Device mapper is a mapping framework mechanism from logical devices to physical devices provided in the Linux 2.6 kernel. Under this mechanism, users can easily formulate management strategies for implementing storage resources according to their needs (see details). Device mapper driver will create a 100G simple file containing your images and containers, each container is limited to a volume of 10G size (note: this is a sparse file automatically created using loopback, specifically data and metadata under /var/lib/docker/devicemapper/devicemapper, which can be expanded dynamically). You can adjust the size of the Docker container, for specific reference) You can use the parameter -s to specify the driver when starting the docker daemon, that is, you can set the docker storage driver by docker -d -s devicemapper. First close the docker service and execute the command:
Default
[root@localhost ~]# docker -d -s devicemapperINFO[0000] +job serveapi(unix:///var/run/docker.sock) INFO[0000] Listening for HTTP on unix (/var/run/docker.sock) INFO[0000] +job init_networkdriver() INFO[0000] -job init_networkdriver() = OK (0) INFO[0000] Loading containers: start. ....INFO[0000] Loading containers: done. INFO[0000] docker daemon: 1.4.0 4595d4f/1.4.0; execdriver: native-0.2; graphdriver: devicemapper INFO[0000] +job acceptconnections() INFO[0000] -job acceptconnections() = OK (0)
In addition, docker can specify the storage-opt parameter when starting the container, but now only devicemapper can accept parameter settings. There will be targeted blog displays later.
BTRFS
Btrfs Driver can be very efficient in docker build. However, like devicemapper, it does not support shared storage between devices (participate in the official website). Btrfs supports snapshots and clones, and can also easily manage multiple physical devices. (For details, please refer to IBM's introduction to Btrfs)
overlay
Overlay is very similar to AUFS, but its performance is better than AUFS and has good memory utilization. It has now been merged into Linux kernel 3.18. Specific use command: docker ds overlay
Note on the official website: It is currently unsupported on btrfs or any Copy on Write filesystem and should only be used over ext4 partitions.
3 Docker directory structure
The two most important concepts of Docker are mirrors and containers. So where is the image we pull down stored? After the mirror run container is started, where is the content of our operation modified? Because the specific drivers are different, the final implementation effect is different. Let’s analyze the storage structure of docker using Device Mapper storage driver as an example.
1. Enter the /var/lib/docker directory and list the contents:
Default
[root@localhost ~]# cd /var/lib/docker/[root@localhost docker]# lscontainers devicemapper execdriver graph init linkgraph.db repositories-devicemapper tmp trust volumes
According to the content of the directory, it is obvious that the devicemapper driver is used.
Note: The folders shown below are all under /var/lib/docker.
2. Which folder does pull image file exist? ( refer to)
The image information of pull is saved in the graph folder, and the content of the image exists under devicemapper/ devicemapper/data.
3. Where does the started container run?
The started container configuration information is saved in containers, and the execdriver/native/ is also viewed.
The contents of the operation in the container are saved under devicemapper/devicemapper/data.
4. The role of graph
In the Docker architecture, it plays the custodian of the downloaded container image and the recorder of the relationship between the downloaded container image. In the local directory of graph, the specific information stored for each container image is: the metadata (json) of the container image, the layer size (layersize) information of the container image, and the specific rootfs represented by the container image.
5. Experimental test:
- Initially no container is enabled:
Default
[root@localhost docker]# ll containers/total 0
- Start a container:
Default
[root@localhost docker]# docker run -i -t --rm centos:7 /bin/bash[root@187a8f9d2865 /]#
The UUID of the started container=187a8f9d2865
- Before starting the container, check the actual size of the file under /var/lib/docker/devicemapper/devicemapper/
Default
[root@bhDocker216 docker]# du -h devicemapper/devicemapper/*2.1G devicemapper/devicemapper/data3.5M devicemapper/devicemapper/metadata
- View on host
Default
[root@bhDocker216 docker]# ls containers/187a8f9d2865c2ac***91b981
Check the contents of the started container below the UUID folder:
Default
[root@bhDocker216 containers]# ll 187a8f9d2865c2ac***91b981total 24-rw-----. 1 root root 273 Mar 5 23:59 187a8f9d2865***-json.log-rw-r--. 1 root root 1683 Mar 5 23:58 config.json-rw-r--. 1 root root 334 Mar 5 23:58 hostconfig.json-rw-r--. 1 root root 13 Mar 5 23:58 hostname-rw-r--. 1 root root 174 Mar 5 23:58 hosts-rw-r--. 1 root root 69 Mar 5 23:58 resolv.conf- Add file in the startup container and view it.
First create a file in the running container:
Default
[root@8a1e3ad05d9e /]# dd if=/dev/zero of=floppy.img bs=512 count=57605760+0 records in5760+0 records out2949120 bytes (2.9 MB) copied, 0.0126794 s, 233 MB/s
Then view the file under /var/lib/docker/devicemapper/devicemapper/:
Default
[root@bhDocker216 docker]# du -h devicemapper/devicemapper/*5.5G devicemapper/devicemapper/data4.6M devicemapper/devicemapper/metadata
The size of this place is a bit different because I first executed #dd if=/dev/zero of=test.txt bs=1M count=8000 to create an 8G-sized file. I terminated it because it was too slow, but I can clearly see that both folders have changed (added) when operating in the running container.
- Check the graph. When only one image (Ubuntu14.10) is pulled, 7 long UUID named directories appear. How did this come about?
Use docker images tree to list the mirror tree structure, and we can see the hierarchical storage structure of the mirror. The final Ubuntu (Layer 7) is based on layer 6 changes, that is, the n-th layer in this logical tree is based on the n-1 changes, and the n-layer depends on the n-1 image. The 0th layer, size is 0, is called base image.
- What is the content in the graph/UUID directory?
Default
[root@localhost graph]# ll 01bf15a18638145eb*** -htotal 8.0K-rw-----. 1 root root 1.6K Mar 5 18:02 json-rw-----. 1 root root 9 Mar 5 18:02 layersize
View the content of layersize: the size of the digital representation layer (unit: B). josn: Save the metadata of this image (such as: size, architecture, config, container, **parent's UUID**, etc.).
- View devicemapper/devicemapper folder
There are two folders, data and metadata. In fact, device mapper driver stores both the mirror and container files in the **data** file. You can view the size of data and metadata through docker info. In addition, you can use du h (used above) to view the actual sizes of these two sparse files.
- execdriver
Default
[root@bhDocker216 docker]# ls execdriver/native/8a1e3ad05d9e66a455e683a2c***2437bdcccdfdfa//View the content inside: [root@bhDocker216 8a1e3ad05d9e66a455e***]# lscontainer.json state.json
- volumes
Volumes without the -v parameter are empty. After testing, if the container is started, add the -v parameter, a UUID will be displayed in the volumes folder. A global search will be performed on the host and found only under volumes. It has nothing to do with the image and the UUID of the container.
Default
[root@bhDocker216 docker]# find / -name 86eb77f9f5e25676f100***d5a/var/lib/docker/volumes/86eb77f9f5e25676f100***d5a//View the content inside: [root@bhDocker216 volumes]# ls 86eb77f9f5e25676f100***d5aconfig.json[root@bhDocker216 volumes]# cat 86eb77f9f5e25676f100***d5a /config.json {"ID":"86eb77f9f5e25676f100a89ba727bc15185303236aae0dcf4c17223e37651d5a","Path":"/home/data","IsBindMount":true,"Writable":true} Table description of folder function
Make a summary, organize a table, and explain the functions of different folders under /var/lib/docker:
Thank you for reading, I hope it can help you. Thank you for your support for this site!