The original title is "Token Counting Done Right: Stop Using tiktoken for Claude"

Short summary

OpenAI's tiktoken systematically undercounts Claude tokens by 15-20%, causing budget projections to be wildly wrong. Use Anthropic's countTokens() SDK endpoint instead—it's a free 3-line API call that returns the exact count your model will charge for. Key gotchas: token counts differ between Claude versions (never cache across versions), and in practice, output tokens—not input tokens—dominate your real generation costs.

• •tiktoken undercounts Claude tokens by 15-20% because different models use different tokenizers
• •Use Anthropic SDK's countTokens() method for exact counts matching your model
• •Never cache token counts across model versions; re-run countTokens for each version

Generated with AI, which can make mistakes.