From first prompt to production,
all in one platform
Connecting models is just the start. TokenHub ships a complete toolkit: Playground, prompt templates, request logs, monitoring and alerts, a cost center, semantic caching and an open API. It all deploys with TokenHub on your own servers, so developers, ops and managers work from the same place.
Everything your company needs to get real value from LLMs
Building and debugging, observability, cost control and integrations, all ready out of the box.
Model Playground
Run one prompt across several models side by side and compare quality, latency and cost at a glance.
Prompt templates
Shared team templates with variables and versioning, referenced straight from your API calls.
Chat apps / agent flows Planned
Visually wire up conversation flows, tool calls and knowledge bases, then ship internal apps fast.
Request logs
Inputs, outputs, routing steps, latency and token details for every request, searchable and replayable.
Monitoring & alerts
Rules on latency, error rate, spend and rate limits, with alerts via WeCom, DingTalk, Feishu (Lark) or email.
Cost center
Break spend down by department, project, member and model, and watch budgets in real time.
Quotas
Token and spend quotas plus RPM / TPM limits per team, project, employee or key. Overages are blocked or downgraded automatically. See Employee Quotas →
Caching
Exact-match plus semantic caching: similar questions hit the cache, cutting latency and duplicate spend.
Multi-tenancy
Isolate data across subsidiaries and business units, each with its own models, quotas and permissions.
Content safety
Sensitive-term and PII redaction plus two-way moderation. Violations are blocked and logged.
Batch jobs
Upload a file and run it through a model in bulk, ideal for labeling, classification and offline summaries. Schedule it for off-peak hours.
Webhooks
Push alerts, quota events and job completions to your own systems or chat bots in real time.
OpenAPI
Automate members, projects, keys, usage and cost data through the management API.
Smart routing & failover
Pick models automatically by cost, latency and capability, and fail over seamlessly when one goes down.
Live Dashboard & reports
Real-time views of calls, tokens, cost and performance, plus scheduled reports.
More on the way
Check the changelog for the latest releases and what's coming next.
The four features teams use most
The screens below are illustrations with sample data.
One question, several models, side by side
Stop picking models on gut feel. Enter a prompt once and compare answer quality, response time and per-call cost across cloud and local models. When you like a setup, save it as a template or generate the code to call it.
- Compare up to 4 models at once, each with its own parameters
- Attach a knowledge base to see the before-and-after of RAG
- Export ready-to-run Python, Node.js or cURL code
Explain our travel expense process to a new hire in three sentences.
Submit a travel request in the portal and get it approved before you go. Keep every receipt and itinerary. Within 15 days of returning, upload them to the expense system for finance to review and reimburse.
① Get your trip approved first. ② Stay within the city lodging limits and keep receipts. ③ File online within 15 days of returning (see Travel Policy, section 3).
Get approval first, travel within policy and keep your receipts, then submit an expense report when you're back and wait for finance to review it.
Know exactly where every dollar goes
Every call's cost is attributed to a department, project and member automatically. Set monthly budgets per team, get warned as they approach the limit, and choose what happens when they're exceeded: fall back to a cheaper model or pause calls.
- Slice by department, project, member or model
- Convert local-model GPU usage into cost and compare it with cloud spend
- Budget alerts and overage policies: notify, downgrade or block
Catch problems before your users do
Set alert rules on latency, error rates, provider rate limits, spend and key expiry. When one fires, you hear about it right away via WeCom, DingTalk, Feishu (Lark), email or webhook, and TokenHub can act on its own, such as switching to a backup model.
- Thresholds and durations per model, project or provider
- Severity levels, quiet hours and on-call recipients
- Tied into routing: fail over automatically when an upstream breaks
-
P1
Error spike
5xx rate > 2% for 3 minPhoneFeishu -
P2
High latency
P95 latency > 3s for 5 minWeComEmail -
P2
Upstream rate limit
Provider 429s > 50 / minFail overWebhook -
P3
Budget warning
Dept. monthly spend > 90% of budgetEmail -
P3
Key expiring
Expires in < 7 daysDingTalk
Trace every step of every request
Auth, caching, knowledge base retrieval, routing, time to first token and streaming: every stage is timed and recorded. Figuring out why a response was slow or wrong is no longer guesswork.
- Search by request ID, member, project, model or status code
- See full inputs and outputs, plus the knowledge base chunks that matched
- Choose what gets stored, and keep only summaries of sensitive content
- Auth & rate limit4 ms
- Cache miss8 ms
- KB retrieval + rerank168 ms
- Routing decision3 ms
- First token427 ms
- Streaming1,232 ms
Works with the tools you already use
TokenHub speaks the OpenAI API. Any tool or framework that lets you set a custom base URL usually just needs a new address and key.
SDKs & frameworks
Use the official OpenAI SDKs or your favorite app framework as is.
IDE plugins
Roll out coding models to every engineer, with usage tracked by team and member.
Low-code platforms
Add TokenHub as a model provider in your visual builder and you're set.
* Common OpenAI-compatible tools shown as examples. Actual compatibility depends on your own testing.
Download TokenHub and try every feature yourself
Free for personal and non-commercial use, and up and running on your own server in minutes. For commercial deployments, contact us about a commercial license.