Skip to main content

ResourceQuota

When several users or teams share a cluster with a fixed number of nodes, there is a concern that one team could use more than its fair share of resources Resource quotas are a tool for administrators to address this concern.

A resource quota, defined by a ResourceQuota object, provides constraints that limit aggregate resource consumption per namespace. It can limit the quantity of objects that can be created in a namespace by type, as well as the total amount of compute resources that may be consumed by resources in that namespace.

Compute Resource Quotas​

These limits control how much CPU and memory can be allocated to pods in a namespace.

Let's create file resource_quota.yaml and put this following configuration.

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: limited
spec:
  hard:
    requests.cpu: "1"
    requests.memory: "2Gi"
    limits.cpu: "2"
    limits.memory: "4Gi"
  • request.cpu: Maximum total CPU requested across all pods in this namespace cannot exceed more than 1 core.
  • request.memory: Maximum total memory requested across all pods in this namespace cannot exceed more than 2GB.
  • limits.cpu: Maximum total CPU usage of all pods combined in this namespace cannot exceed more than 2 cores.
  • limits.memory: Maximum total memory usage of all pods combined in this namespace cannot exceed more than 4GB.

Apply and Validate​

Let's apply the configuration file using kubectl apply command.

➜ kubectl apply -f resource_quota.yaml 
resourcequota/compute-quota created

We can validate it using kubectl describe quota in the specific namespace like this.

➜ kubectl -n limited describe quota
Name:            compute-quota
Namespace:       limited
Resource         Used  Hard
--------         ----  ----
limits.cpu       0     2
limits.memory    0     4Gi
requests.cpu     0     1
requests.memory  0     2Gi

Storage Resource Quotas​

These quotas limit how much storage (Persistent Volumes) can be created in a namespace.

Add few more line to our configuration file before in the spec.hard section to define the storage limit.

requests.storage: "64Gi"
persistentvolumeclaims: "10"
  • requests.storage: The total storage requested by PVC in this namespace cannot exceed 64GB.
  • persistentvolumeclaims: The total number of PVC in this namespace cannot exceed 10.

Apply and Validate​

Let's apply our configuration file and validate.

➜ kubectl apply -f resource_quota.yaml
resourcequota/compute-quota configured
➜ kubectl -n limited describe quota   
Name:                   compute-quota
Namespace:              limited
Resource                Used  Hard
--------                ----  ----
limits.cpu              0     2
limits.memory           0     4Gi
persistentvolumeclaims  0     10
requests.cpu            0     1
requests.memory         0     2Gi
requests.storage        0     64Gi

Object Count Quotas​

These quotas limit the number of objects (Pods, Services, ConfigMaps, etc.) a namespace can have.

Add few more line to our configuration file before in the spec.hard section to define pods, services, secrets, and configmaps limit.

pods: "5"
services: "2"
secrets: "2"
configmaps: "2"
  • pods: The total number of Pods in a non-terminal state in this namespace cannot exceed 5. A pod is in a terminal state if .status.phase in (Failed, Succeeded) is true.
  • services: The total number of services in this namespace cannot exceed 2.
  • secrets: The total number of secrets in this namespace cannot exceed 2.
  • configmaps: The total number of configmaps in this namespace cannot exceed 2.

Apply and Validate​

Again, lets apply our configuration and validate.

➜ kubectl apply -f resource_quota.yaml
➜ kubectl -n limited describe quota   
Name:                   compute-quota
Namespace:              limited
Resource                Used  Hard
--------                ----  ----
configmaps              1     2
limits.cpu              0     2
limits.memory           0     4Gi
persistentvolumeclaims  0     10
pods                    0     5
requests.cpu            0     1
requests.memory         0     2Gi
requests.storage        0     64Gi
secrets                 0     2
services                0     2

We can see all of the resource limit that we just created. This will ensure this namespace resource will not exceed our defined limit.

What Happened if Limit Exceeded?​

  • If creating or updating a resource violates a quota constraint, the request will fail with HTTP status code 403 FORBIDDEN with a message explaining the constraint that would have been violated.

  • If quotas are enabled in a namespace for compute resources like cpu and memory, users must specify requests or limits for those values; otherwise, the quota system may reject pod creation.

  • We can use LimitRange to enforce default request and limit for compute resource.

References​