Linux Control Groups

Three components

How they work

cgroup

create mount point for hierarchy

$ mkdir cgroup-test

Mount hierarchy

$ sudo mount -t cgroup -o none,name=cgroup-test cgroup-test ./cgroup-test

Below files will be found on cgroup-test

Extend two cgroups

/cgroup-test $ sudo mkdir cgroup-1
/cgroup-test $ sudo mkdir cgroup-2

Check tree structure

/cgroup-test $ tree
.
├── cgroup-1
│   ├── cgroup.clone_children
│   ├── cgroup.procs
│   ├── notify_on_release
│   └── tasks
├── cgroup-2
│   ├── cgroup.clone_children
│   ├── cgroup.procs
│   ├── notify_on_release
│   └── tasks
├── cgroup.clone_children
├── cgroup.procs
├── cgroup.sane_behavior
├── notify_on_release
├── release_agent
└── tasks

Add or move process in cgroup A process in hierarchy of Cgrups can only exist in one cgroup node. All processes are defaultly exist on root node, but can be moved to other cgroup node. The only thing to move is write pid to tasks of cgroup node.

/cgroup-test $ echo $$
8099

/cgroup-test $ cat tasks
...
8099
...

/cgroup-test$ cd cgroup-1
/cgroup-test/cgroup-1$ cat tasks
/cgroup-test/cgroup-1$ sudo sh -c "echo $$ >> tasks"
/cgroup-test/cgroup-1$ cat tasks
8099
17858
/cgroup-test/cgroup-1$ cat /proc/8099/cgroup
12:name=cgroup-test:/cgroup-1
11:freezer:/
10:hugetlb:/
9:pids:/user.slice/user-1000.slice
8:cpu,cpuacct:/user.slice
7:blkio:/user.slice
6:cpuset:/
5:net_cls,net_prio:/
4:memory:/user.slice
3:perf_event:/
2:devices:/user.slice
1:name=systemd:/user.slice/user-1000.slice/session-2.scope

Limit cgroup resource When mount hierarchy, there is no link between hierarchy and subsystem, so can’t limit resources by using cgroup of hierarchy. But there is default hierarchy for every subsystem.

$ mount | grep memory
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)

As you can see, /sys/fs/cgroup/memory is mount at memory subsystem’s hierarchy.

Now, we can create cgroup on this hierarchy to limit resources like memory

Before this, install stress by sudo apt-get install stress.

# try stress process with 20M memory without any limitation
/sys/fs/cgroup/memory$ stress --vm-bytes 20m --vm-keep -m 1

# create a cgroup and limit max size memory 10m
/sys/fs/cgroup/memory$ sudo mkdir test-limit-memory && cd test-limit-memory

/sys/fs/cgroup/memory/test-limit-memory$ sudo sh -c "echo "10m" > memory.limit_in_bytes"

/sys/fs/cgroup/memory/test-limit-memory$ sudo sh -c "echo $$ > tasks"
/sys/fs/cgroup/memory/test-limit-memory$ stress --vm-bytes 200m --vm-keep -m 1

How docker working to limit resource

Docker create cgroup for every container. And using cgroup to limit resources.

$ docker run -itd -m 128m --rm --name="tools" dockercraft/mysql-client
$ cd /sys/fs/cgroup/memory/docker/3a4680003809860500984dfcafa9c690bb944314864e6da28f5a947864da6784/
$ cat memory.limit_in_bytes
134217728

Linux subsystem

cgroup-bin

apt-get install cgroup-bin

lssubsys List all subsystems support by Kernel.

Hierarchy

link cgroup like tree structure.