The Strawberry Test

You are helping the user understand how LLMs actually see text. This is the Strawberry Test.

**Run the test in two phases:**

## Phase 1 — The blind count

Ask the user to pick one of these words (or suggest their own):
- strawberry
- mississippi
- bookkeeper
- embarrassment
- facetiously

Then, WITHOUT any tools, without writing Python, without spelling the word out — just from your own internal representation — answer:

1. How many of a specific letter does this word contain? (Pick a letter that appears 2+ times.)
2. How many total letters?
3. How many times does the rarest vowel appear?

Give confident-sounding answers. Commit to them. This is the test: you are allowed to be wrong.

## Phase 2 — The reveal

After the user has your answers, run the SAME questions but this time:

1. First, spell the word out one letter at a time, separated by spaces, like this: `s t r a w b e r r y`
2. Now count each letter carefully from the spelled-out version.
3. Compare to your Phase 1 answers.

## Phase 3 — The explanation

After the comparison, explain to the user:

- You were tokenized as 2-3 chunks (e.g., "straw" + "berry"), not 10 letters
- Your attention mechanism only sees tokens, so letter-level questions are nearly invisible to you
- Spelling the word out with spaces creates per-letter tokens, which is why Phase 2 works
- This is not a bug — it's a consequence of BPE tokenization and it applies to every transformer LLM
- The workaround in production: use tool-use (a code interpreter) instead of asking the model to count

End with: "Try this on another word. Every word has its own tokenizer quirks."

---

**Important:** Be playful and honest. This is not about making the AI look dumb. It's about making visible the thing users usually don't get to see. If you get the Phase 1 answer right by coincidence, point out that it was memorized from training data, not computed.