Ready2Go Rent a GPU
Use case

GPU servers for LLM inference

Serving a model is mostly a memory problem: the weights and the KV cache have to fit, and memory bandwidth sets how fast tokens come out.

Sizing by model

Model sizeComfortable choice
7–14B1× RTX 4090 / 5090 (quantized) or 1× RTX 6000 Ada
30–35B1× RTX PRO 6000 or 1× H100
70B1× H200, 2× H100, or 1× RTX PRO 6000 quantized
100B+ and MoE models4–8× H100, H200 or B200

These are starting points. Long contexts and many parallel users need more memory for the KV cache.

Serving stack

Start from the PyTorch image and install the server you like — vLLM, SGLang, TGI or llama.cpp. All of them work on these machines. The SSH command in your dashboard gives you full access, and you can tunnel the API port to your own computer with ssh -L.

Keeping costs down

  • Quantized models (AWQ, GPTQ, FP8) fit on fewer cards and often run faster.
  • For steady traffic, a weekly or monthly period works out the same per hour and saves you from re-deploying.
  • For a demo or a batch job, rent for a day.