We keep popular open-source models always-on: there's no boot time, they're just ready to go. Always-on models are charged per-token.

Here's the list of our always-on models:

LoRAs of the following models are always-on, and are charged per-token.

LoRA sizes are measured in "ranks," starting at rank-8; we support up to rank-64 LoRAs kept always-on, and we run them in FP8 precision. The rank is set during the finetuning process: if you create your own LoRA, you'll be able to set exactly what rank you want using standard configuration for your training framework.

For LoRAs of base models not listed in the table above, we support running them on-demand as long as vLLM does; however, since the base models aren't kept always-on, you'll be charged our standard on-demand pricing for the base model (and no additional charge for the LoRA).

We support launching all other LLMs on-demand on cloud GPUs. There's no configuration necessary: just enter the Hugging Face  link for any model, and we'll automatically run it for you in our friendly chat UI or API.

We'll automatically detect the number of GPUs you need to run the model. Here's our current GPU pricing:

Our on-demand GPU rates are very competitive: for example, an 80GB GPU is ~2x cheaper on Synthetic than on competing services like Replicate or Modal Labs.

We automatically calculate the type and number of GPUs required for a model repository for you. We don't quantize on-demand models: they're launched in whatever precision the underlying repo uses; typically, BF16, with the exception of Jamba-based models which are launched in FP8. Quantizing past FP8 can significantly harm model performance.

On-demand model context length is capped to a maximum of 32k tokens.