Skip to content
Vietnamese text use 2-3x more tokens than English for same meaning, how you handle cost?
question1 min read76 viewsSep 16, 2026

Vietnamese text use 2-3x more tokens than English for same meaning, how you handle cost?

Nguyen Minh Anh@minhanh_ml

September 16, 2026 · 1 min read

#1 in Stochastic Parrots

76 views3 comments

Hi all, first post here. I build support chatbot for an e-commerce company, customers write in Vietnamese and English. I notice our token cost per conversation is much higher for Vietnamese. I test same FAQ answer in both language, the Vietnamese version is around 2.3x tokens of the English version with the tokenizer of our model.

It make two problem:

  • cost per conversation is higher
  • our RAG chunk size is set by characters (1000 chars), so Vietnamese chunk become much bigger in tokens, sometime we hit context limit when we retrieve top 8

What I try: reduce top k to 5 (quality drop a bit), shorter system prompt. How you handle this for non-English language? Should I chunk by token? Or translate everything to English first and translate back? That feel risky for product name and address.

Accepted answer

Lin Wei#2 in Stochastic Parrots2 likes

yes chunk by tokens, with the same tokenizer as the model, not characters. we do this for chinese product description, same issue. other thing that help us: 1. measure cost per language separately in the dashboard so you know the real number 2. keep the long static system prompt at the start and use prompt caching if your provider support it, the static part become much cheaper 3. translate-then-answer we tried. product names and promo terms got mangled. not worth 4. some newer model have better tokenizer for accented text, worth to benchmark 2-3 model on your real chats, difference can be big

View in the thread

#llm#rag#nlp#cost

Written by

Nguyen Minh Anh

ML engineer in Ho Chi Minh City. Building LLM support bots for e-commerce in Vietnamese and English. Evals, RAG, and explaining to boss why the bot said that.

View profile

Get the best of #stochastic-parrots weekly

One email each Monday with the most useful posts. No spam; unsubscribe in one click.

Comments

3

Sign in to join the conversation.

Sign in to comment

Accepted answer

Lin Wei@linwei#2 in Stochastic Parrots
yes chunk by tokens, with the same tokenizer as the model, not characters. we do this for chinese product description, same issue. other thing that help us: 1. measure cost per language separately in the dashboard so you know the real number 2. keep the long static system prompt at the start and use prompt caching if your provider support it, the static part become much cheaper 3. translate-then-answer we tried. product names and promo terms got mangled. not worth 4. some newer model have better tokenizer for accented text, worth to benchmark 2-3 model on your real chats, difference can be big
2 net votes
Nguyen Minh Anh@minhanh_ml#1 in Stochastic Parrots
Thank you so much! Chunk by token fix the context limit problem, and the caching cut maybe 30% cost on long chat. Will benchmark other model next sprint
0 net votes
Olivia Martin@liv_codes#3 in Stochastic Parrots
this is so interesting, never thought about tokenizers being unfair to some languages. adding to my "things to mention in interviews to sound smart" list
1 net votes