Reserving CPU and memory

Modal jobs are reserved 128 MiB of memory and 0.1 CPU cores by default. However, if there is free memory or CPU capacity on a worker, containers are free to spike above these limits.

Keep in mind that these reservations should be set to the minimum value required to have your application function correctly.

CPU cores

If you have code that must run on a larger number of cores, you can request that using the cpu argument. This allows you to specify a floating-point number of CPU cores:

import modal

app = modal.App()  # Note: prior to April 2024, "app" was called "stub"

@app.function(cpu=8.0)
def my_function():
    # code here will have access to at least 8.0 cores
    ...

Memory

If you have code that needs more guaranteed memory, you can request it using the memory argument. This expects an integer number of megabytes:

import modal

app = modal.App()  # Note: prior to April 2024, "app" was called "stub"

@app.function(memory=32768)
def my_function():
    # code here will have access to at least 32 GiB of RAM
    ...

How much can I request?

For both CPU and memory, a maximum is enforced at function creation time to ensure your application can be scheduled for execution. Requests exceeding the maximum will be rejected with an InvalidError.

As the platform grows, we plan to support larger CPU and memory reservations.

Resource limits

CPU limits

Modal containers have a soft CPU limit that is set at 4 physical cores above the CPU request. Given that the default CPU request is 0.1 cores the soft CPU limit is 4.1 cores. Above this limit the host will begin to throttle the CPU usage of the container.

Memory limits

Modal containers can have a hard memory limit which will ‘Out of Memory’ (OOM) kill containers which attempt to exceed the limit. This functionality is useful when a container has a serious memory leak. You can set the limit and have the container killed to avoid paying for the leaked GBs of memory.

mem_request = 1024
mem_limit = 2048
@app.function(
    memory=(mem_request, mem_limit),
)

Specify this limit using the memory parameter on Modal Functions.

Disk limits

Running Modal containers have access to many GBs of SSD disk, but the amount of writes is limited by:

1. The size of the underlying worker’s SSD disk capacity
2. A per-container disk quota that is set in the 100s of GBs.

Hitting either limit will cause the container’s disk writes to be rejected, which typically manifests as an OSError.