Skip to main content
GPUs accelerate computational workloads through parallel processing. Originally designed for graphics rendering, modern GPUs are essential for AI models, large-scale data processing, and other compute-intensive applications. Cerebrium provides GPU access through configuration in the cerebrium.toml file, without requiring infrastructure management.

​
Specifying GPUs

Configure GPUs in the [cerebrium.hardware] section of cerebrium.toml, specifying the type (compute parameter) and quantity (gpu_count). Additional deployment and scaling considerations are covered in the sections below.

​
Available GPUs

The platform offers GPUs ranging from cost-effective development options to high-end enterprise hardware.
The identifier is used in the cerebrium.toml file. It consists of the GPU model generation and model name to avoid ambiguity.
GPU selection is also possible using the --compute and --gpu-count flags during application initialization.

​
Multi-GPU Configuration

Multiple GPUs are configured in the cerebrium.toml file:

​
GPU Preference Lists

The compute parameter also accepts a list of acceptable GPU types in preference order. The platform allocates the most preferred type with available capacity and falls back to the next entry when needed:
  • Up to 5 entries, ordered from most to least preferred
  • All entries must belong to the same hardware family. NVIDIA GPU types cannot be mixed with CPU or AWS accelerators (INF2, TRN1)
  • cerebrium run uses only the first entry
Accepting more GPU types widens the pool of capacity an app can run on and reduces the likelihood of request queuing.

​
Availability

GPU availability varies by region and provider. Narrowing the provider and region constraints increases the likelihood of request queuing. See GPU availability by region. For guaranteed burst capacity, contact the enterprise plan team.