Does Tokoscope store my prompts?
+
We store prompt previews (first 100 characters) for display in the dashboard. Full prompts are only stored temporarily when compression is enabled, and are never used to train AI models or shared with third parties.
Does the SDK add latency to my API calls?
+
Tracking events are sent in background threads — they never block your main application. The only latency added is the cache check before each API call, which typically takes 50-100ms. On a cache hit, you save the full LLM round-trip time.
What happens if Tokoscope goes down?
+
The SDK is designed to fail silently. If our servers are unreachable, your LLM calls continue working normally — no errors, no disruption. Tokoscope never sits in the critical path of your application.
Can I cancel anytime?
+
Yes. Cancel anytime from your account settings. You keep access until the end of your billing period. No cancellation fees, no questions asked.
How does semantic caching work?
+
Semantic caching uses OpenAI embeddings to compare the meaning of incoming prompts against cached ones. If a new prompt is 85% or more similar to a cached prompt, we return the cached response instead of calling the LLM. This catches variations like "What's the capital of France?" and "Name the French capital city." — saving tokens on both.
Which LLM providers are supported?
+
OpenAI, Anthropic, and Gemini are fully supported in both the JavaScript and Python SDKs. Mistral and Ollama support is coming soon.