How to Configuring Linux ulimit with Docker and AWS ECS

What is ulimit?

The purpose of ulimits is to limit a program's resource utilization to prevent a run-away bug or security breach from bringing the whole system down. It is easy for modern applications to exceed default open files and other limits very quickly.

How to check the Hard limit and soft limit?

Check hard limit

ulimit -a -H

Check soft limit :

ulimit -a -S

Configuring limits in Docker containers

Controlling the limits becomes a bit trickier when Docker is involved. The Docker daemon runs as root user. By default, the same limits apply to the application running within a container as they would to the Docker daemon.

Adjusting limits within a container, however, requires privileges not available to the application inside the Docker container and must be done as parameters to the docker run command.

For example:

docker run -it --ulimit nofile=2048:2048 ubuntu bash
root@01da5d7fa50b:/# ulimit -n
2048

Configuring Docker container limits in AWS ECS

Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service.

AWS ECS is organized around task definitions, clusters, tasks, and services. A task definition describes the docker images and various parameters for the container.

Within the ECS cluster when you ‘Create new Task Definition’, There are two option

1. Fargate Based
2. EC2 Based

Once you select any of the above methods, you have an option to provide detailed container definition

Inside container definition, you have an option to tweak various ulimits

Another way to change this value If you have created EC2 based cluster you can ssh to your ec2 instance and check the below file

check /etc/sysconfig/docker in your Amazon ECS-optimized AMI

$ cat /etc/sysconfig/docker
# The max number of open files for the daemon itself, and all
# running containers. The default value of 1048576 mirrors the value
# used by the systemd service unit.
DAEMON_MAXFILES=1048576# Additional startup options for the Docker daemon, for example:
# OPTIONS=” — ip-forward=true — iptables=true”
# By default we limit the number of open files per container
OPTIONS=” — default-ulimit nofile=1024:4096"

You can change the desired value as below.

OPTIONS=” — default-ulimit nofile=1024000:1024000"

Conclusion

Limits are a critical application tuning parameter. Cloud Docker services have their own mechanism for configuring ulimits. In this article, I talked about AWS ECS.