A few lines of code. Every LLM.

1. Deploy TokenHub

   Grab the Docker image or an installer from the download page and run it on your own server. It listens on port 8080 by default; create the admin account the first time you open http://localhost:8080.

   Download & install System requirements

   # Docker example (image name announced at release)
   docker run -d --name tokenhub \
     -p 8080:8080 \
     -v $(pwd)/tokenhub-data:/data \
     <IMAGE_TBD>:2.3.0

2. Create an API key

   In your own TokenHub admin console, add models (cloud provider keys or local model endpoints), then create a key under "API Keys". Each key can be scoped to a project, a set of models and a token quota. Keys start with th- and are shown in full only once.

   # Keep keys in env vars, never in your repo
   export TOKENHUB_API_KEY="th-your-api-key"
   # Replace with your TokenHub address
   export TOKENHUB_BASE_URL="http://localhost:8080/v1"

3. Install the SDK

   Use the official OpenAI SDK as is; no extra dependencies. Any HTTP client works too.

   # Python
   pip install openai
   
   # Node.js
   npm install openai

4. Make your first call

   Set model to a specific model (for example deepseek-chat, qwen-plus or ollama/qwen2.5:7b), or to auto and let your routing policy choose.

   cURL Python Node.js Copy

   # Replace with your TokenHub address
   curl http://localhost:8080/v1/chat/completions \
     -H "Authorization: Bearer $TOKENHUB_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "deepseek-chat",
       "messages": [
         {"role": "system", "content": "You are a helpful business assistant."},
         {"role": "user", "content": "Describe TokenHub in three sentences."}
       ],
       "temperature": 0.7
     }'

   import os
   from openai import OpenAI
   
   client = OpenAI(
       base_url="http://localhost:8080/v1",  # replace with your TokenHub address
       api_key=os.environ["TOKENHUB_API_KEY"],
   )
   
   resp = client.chat.completions.create(
       model="deepseek-chat",
       messages=[
           {"role": "system", "content": "You are a helpful business assistant."},
           {"role": "user", "content": "Describe TokenHub in three sentences."},
       ],
       temperature=0.7,
   )
   print(resp.choices[0].message.content)
   print(resp.usage)  # prompt / completion / total tokens

   import OpenAI from "openai";
   
   const client = new OpenAI({
     baseURL: "http://localhost:8080/v1", // replace with your TokenHub address
     apiKey: process.env.TOKENHUB_API_KEY,
   });
   
   const resp = await client.chat.completions.create({
     model: "deepseek-chat",
     messages: [
       { role: "system", content: "You are a helpful business assistant." },
       { role: "user", content: "Describe TokenHub in three sentences." },
     ],
     temperature: 0.7,
   });
   
   console.log(resp.choices[0].message.content);

5. Turn on streaming

   Set stream: true to receive output as Server-Sent Events (SSE). The first tokens arrive sooner, which suits chat interfaces. The final message is data: [DONE].

   cURL Python Node.js Copy

   curl -N http://localhost:8080/v1/chat/completions \
     -H "Authorization: Bearer $TOKENHUB_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "qwen-plus",
       "stream": true,
       "messages": [
         {"role": "user", "content": "Write an opening line for a product launch"}
       ]
     }'
   
   # Response (excerpt):
   # data: {"choices":[{"index":0,"delta":{"content":"Welcome"}}]}
   # data: {"choices":[{"index":0,"delta":{"content":" everyone"}}]}
   # data: [DONE]

   stream = client.chat.completions.create(
       model="qwen-plus",
       messages=[{"role": "user", "content": "Write an opening line for a product launch"}],
       stream=True,
   )
   
   for chunk in stream:
       if chunk.choices and chunk.choices[0].delta.content:
           print(chunk.choices[0].delta.content, end="", flush=True)

   const stream = await client.chat.completions.create({
     model: "qwen-plus",
     messages: [{ role: "user", content: "Write an opening line for a product launch" }],
     stream: true,
   });
   
   for await (const chunk of stream) {
     process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
   }

6. Check usage and logs

   Every call's model, latency, tokens and cost are logged in real time. Search them under "Call Logs" in your admin console, or see totals by department and project on the Live Dashboard.

   Explore the Live Dashboard See call logging

# ① Pull models
ollama pull qwen2.5:7b
ollama pull bge-m3

# ② Listen on the LAN and start
OLLAMA_HOST=0.0.0.0:11434 ollama serve

# ③ TokenHub local model config (equivalent YAML)
provider: ollama
name: local-qwen
base_url: http://10.0.0.21:11434
models:
  - qwen2.5:7b
  - bge-m3          # also used as the knowledge base embedding model

# ④ Call with model="local-qwen/qwen2.5:7b"

# ① Start the server (example: one machine, 2 GPUs)
vllm serve Qwen/Qwen2.5-14B-Instruct \
  --port 8000 \
  --tensor-parallel-size 2 \
  --api-key local-secret

# ② Connectivity check
curl http://10.0.0.22:8000/v1/models \
  -H "Authorization: Bearer local-secret"

# ③ TokenHub local model config (equivalent YAML)
provider: openai-compatible
name: local-vllm
base_url: http://10.0.0.22:8000/v1
api_key: ${VLLM_API_KEY}
models:
  - id: Qwen/Qwen2.5-14B-Instruct
    alias: qwen2.5-14b
fallback: deepseek-chat   # switch automatically if local is down

* Commands and config are illustrative; adjust IPs, models and field names to your environment and the released version. Xinference, LM Studio, SGLang and similar servers connect the same way as an "OpenAI-compatible service".