Practice AI Projects for Free with Ollama Cloud Models

Every AI project needs an AI API. That part is unavoidable — you need a model to generate text, classify intent, summarize documents, or write code. The question is which API you point at while you are still figuring things out.

For a long time I just reached for a paid provider. It worked, but it came with two problems I did not love.

The first was cost. I could not tell in advance what a week of experimenting would cost. A retry loop here, a longer prompt there, and suddenly the bill is a number I did not plan for. There is no natural ceiling when you are the one writing the calls.

The second was key handling. A paid API key is a secret that has to live somewhere. It lives in an environment variable, then in a .env file, then maybe in a shell profile, and one careless git add away from being public. If you are experienced you have rituals for this. If you are new to AI development, this is exactly the kind of sharp edge that ends a project early.

Local models solve both problems. No bill, no key in your repository. But local inference has its own catch: it needs hardware. Running a 30B-class model on a machine without a serious GPU means the model is loaded into system RAM and partially onto the CPU, and your laptop fans spin up, the machine gets hot, and every other program slows down. On a modest development box, “run it locally” quietly becomes “run it locally and lose your afternoon”.

The middle ground

What I actually wanted was something with the ergonomics of a local model and the capability of a big one, without the bill and without handling a paid credential.

Ollama cloud models give exactly that. A model tagged -cloud does not download any weights to your machine. Instead, your local Ollama server keeps a small manifest and proxies the request to Ollama’s hosted inference. You keep talking to http://localhost:11434, the API surface stays identical, and the inference happens on someone else’s GPU.

The setup is three steps.

1. Install Ollama

Download the installer for your platform from ollama.com/download, or on Linux and macOS:

curl -fsSL https://ollama.com/install.sh | sh

On Windows the installer puts ollama on your PATH. Verify it:

ollama --version

2. Sign in and load a free cloud model

Cloud models require an Ollama account. Sign in once:

ollama signin

This prints a URL, you confirm in the browser, and your machine stores a key pair locally. Note what this key is not: it is not a paid API credential for a third-party provider, and it never gets embedded in your application code.

Now load a cloud model:

ollama pull gemma4:31b-cloud

Watch the size column afterwards:

$ ollama list
NAME                ID              SIZE    MODIFIED
gemma4:31b-cloud    ef09f235533c    -       Less than a second ago

That - in the SIZE column is the point. A normal model shows something like 20 GB because the weights were downloaded to your disk. A cloud model downloads a manifest of a few hundred bytes and nothing else. You get a 31B-parameter model with a 256K context window without giving up 20 GB of disk space or a single minute of GPU time.

You can also chat with it directly to sanity-check it:

ollama run gemma4:31b-cloud

3. Call it through the local OpenAI-compatible API

This is the part that makes it useful for real projects. Ollama exposes an OpenAI-compatible endpoint at http://localhost:11434/v1, so the official OpenAI Python SDK works against it unchanged. You only change the base URL and the model name.

pip install openai
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama",  # required by the SDK, but not checked by Ollama
)

response = client.chat.completions.create(
    model="gemma4:31b-cloud",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Summarize the CAP theorem in three sentences."},
    ],
)

print(response.choices[0].message.content)

That api_key="ollama" placeholder deserves a note. The OpenAI SDK refuses to send a request without an API key, so something has to be there. Ollama’s local server does not validate it. Nothing secret is being smuggled into your project, and if you commit this file to Git, you have committed nothing sensitive.

One detail that catches people out: use the -cloud suffix in the model name. Through the local server, gemma4:31b-cloud resolves to the hosted model, while gemma4:31b returns model not found because that tag refers to local weights you never downloaded. The hosted API at ollama.com uses the shorter name; the local proxy uses the one shown by ollama list.

You can verify the raw endpoint with curl, which is handy when debugging:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:31b-cloud",
    "messages": [{"role": "user", "content": "Say hello in one sentence."}]
  }'

Things worth knowing

Free is not unlimited. The free tier includes a monthly allowance of starter usage credits and access to starter models, with a limited number of concurrent requests. When you outgrow it, you either add usage credits or move to a paid plan. Ollama meters cloud usage per token at each model’s published rate, and gemma4 sits at $0.14 per million input tokens and $0.40 per million output tokens, so the credits drain slowly for normal experimentation. Check your usage and the pricing page for current numbers — the tiers change fairly often.

“Cloud” means the prompt leaves your machine. This is the honest tradeoff. Ollama states that cloud prompts and responses are never logged and never used for training, and that it requires no-logging, no-training, zero-retention terms from the hosting providers it works with. That is a policy promise, not something you can verify from your desk. If a specific prompt is sensitive enough that you need to prove it never left, pull a local model instead — ollama pull gemma4:e2b runs fully offline. The -cloud suffix is the only signal that tells you where the inference happens, so keep it visible in your model names.

Cloud models are not local models. ollama ps shows the difference clearly: a local model lists a size and a processor (42 GB, 100% GPU), a cloud model shows neither, because your GPU never woke up.

The same setup works for coding agents. Ollama can configure a coding agent to point at a cloud model in one command, which is useful if you want to drive an agent against a large model without a subscription:

ollama launch claude --model gemma4:31b-cloud
ollama launch codex --model gemma4:31b-cloud
ollama launch opencode --model gemma4:31b-cloud

Where I landed

I now develop every AI feature against a cloud model first. The economics are right for practice: no subscription, no per-token meter I have to watch, no provider key in my repository. When a feature graduates to something real — with real users, real traffic, and a latency budget — switching to a paid endpoint is a two-line change, because the code already speaks the OpenAI protocol. The base_url and model are the only things that differ.

That is the whole trick. http://localhost:11434/v1 behaves exactly like a hosted API, so you can treat “no paid provider yet” as a configuration detail rather than a blocker. Pick a model name, point the SDK at localhost, and start building.

Share :