OneTaskBy Tim King · Updated
AI Token Counter
Browser-Based o200k_base Encoding, Each Token Shown, Plus Characters and Words Per Token
OneTask is free, with no sign-up, and built by one person.
If this tool saved you time, a coffee helps keep new ones coming.
How the AI Token Counter works
This AI token counter counts the tokens in any text exactly for OpenAI's GPT-4o and later models, using the o200k_base tokenizer. Claude and Gemini use other tokenizers, so their counts are estimates. Paste a prompt, a document or code to see the token count, characters and words per token, and every token split out or as IDs.
How the AI token counter works
Language models do not read letters or words: they read tokens, chunks of text from a fixed vocabulary learned by byte-pair encoding. The o200k_base vocabulary has about 200,000 entries. Common English words are usually one token with their leading space, so " the" is one token, while rare words, names, numbers and non-Latin scripts split into several. The counter runs the real tokenizer, an open-source MIT-licensed port, in your browser, loading it only when you first count.
A rule-of-thumb formula for English prose is tokens ≈ words ÷ 0.75, or about four characters per token. Code, numbers and other languages break that rule, which is why an exact count is worth having.
First example. "The quick brown fox jumps over the lazy dog." is 44 characters, 9 words and 10 tokens: 4.4 characters per token, close to the rule of thumb.
Second example. The UUID 3f2504e0-4f89-11d3-9a0c-0305e82c3301 is 36 characters but 27 tokens, only 1.33 characters per token, because random hexadecimal rarely matches a vocabulary entry.
How many tokens is a word? On average about 1.3 tokens per English word, but a long technical word can be four or five.
Token counts for sample text under the o200k_base encoding
| Sample | Text | Characters | Tokens | Characters per token |
|---|---|---|---|---|
| A single English word | hello |
5 | 1 | 5.00 |
| A common long word | understanding |
13 | 2 | 6.50 |
| A rare long word | floccinaucinihilipilification |
29 | 9 | 3.22 |
| A short sentence | The quick brown fox jumps over the lazy dog. |
44 | 10 | 4.40 |
| A number | 1234567890 |
10 | 4 | 2.50 |
| A date | 2026-10-01 |
10 | 6 | 1.67 |
| A price | $1,299.99 |
9 | 6 | 1.50 |
| An email address | someone@example.com |
19 | 3 | 6.33 |
| A URL | https://www.example.com/path?q=1 |
32 | 9 | 3.56 |
| A UUID | 3f2504e0-4f89-11d3-9a0c-0305e82c3301 |
36 | 27 | 1.33 |
| A line of JavaScript | const total = items.reduce((a, b) => a + b, 0); |
47 | 18 | 2.61 |
| A line of Python | def add(a, b): return a + b |
27 | 10 | 2.70 |
| JSON object | {"name": "Ada", "age": 36} |
26 | 12 | 2.17 |
| French sentence | Le renard brun saute par-dessus le chien. |
41 | 10 | 4.10 |
| German compound | Donaudampfschifffahrtsgesellschaft |
34 | 10 | 3.40 |
| Spanish sentence | El zorro marrón salta sobre el perro. |
37 | 11 | 3.36 |
| Chinese sentence | 敏捷的棕色狐狸跳过了懒狗。 |
13 | 14 | 0.93 |
| Japanese sentence | 素早い茶色の狐が怠け者の犬を飛び越える。 |
20 | 20 | 1.00 |
| Russian sentence | Быстрая коричневая лиса прыгает через собаку. |
45 | 17 | 2.65 |
| Arabic sentence | الثعلب البني السريع يقفز فوق الكلب. |
35 | 14 | 2.50 |
| Hindi sentence | तेज़ भूरी लोमड़ी आलसी कुत्ते पर कूदती है। |
41 | 19 | 2.16 |
| Emoji | 👍 |
1 | 1 | 1.00 |
| Family emoji | 👨👩👧👦 |
7 | 11 | 0.64 |
| Twenty spaces | ···················· |
20 | 1 | 20.00 |
| Repeated letters | aaaaaaaaaaaaaaaaaaaa |
20 | 3 | 6.67 |
Tips and pitfalls
Other model families use their own tokenizers. Claude, Gemini and Llama counts for the same text are usually in the same range as o200k_base, though they can differ by 10 to 20 percent or more, so treat this count as a close estimate for them and exact for GPT-4o, GPT-4.1, GPT-5 and the o-series. Chat APIs also add a few tokens per message for roles and formatting.
FAQ
Which models does this token count match?
It is exact for OpenAI models that use o200k_base, including GPT-4o, GPT-4.1, GPT-5 and the o1, o3 and o4 reasoning models. Older GPT-4 and GPT-3.5 used cl100k_base, which gives slightly higher counts for most text.
Does it count Claude or Gemini tokens?
Not exactly. Anthropic and Google use their own tokenizers and do not publish them all for offline use. Counts usually land in the same range as this one, close enough for budgeting and checking context limits.
Why do numbers and other languages use more tokens?
The vocabulary was learned mostly from English text and code, so common English words get their own tokens. Long numbers, accented words and scripts such as Chinese or Hindi are split into smaller pieces, often one token per character or more.
What does "Tokens split with |" show?
It decodes each token back to text and joins them with a vertical bar, so you can see exactly where the tokenizer cuts. Leading spaces belong to the token after them, which is why most English tokens start with a space.
Is my text sent to OpenAI?
No. The tokenizer file is downloaded once from this site and runs in your browser. The text you paste is never uploaded, so you can safely count private prompts, code and documents.
See also: all AI tools, Dev Utilities, Formatters & Validators, Data & Encoding, Security & Identifiers, SEO, Accessibility.
Related tools
Context Window Calculator
Check whether a prompt, documents and reply fit a model's context window.
LLM Cost Calculator
Price LLM API calls per request and per month, with caching.
Word Counter
Words, six readability scores, keyword density and platform limits.
Character Counter
Characters against SMS, meta and social post limits.
Prompt Formatter
Turn loose notes into a structured prompt with XML tags or headings.