#tokenization

Tokens: The Alphabet a Model Actually Reads

Models never see letters, only numbered chunks of text. Train a real tokenizer in your browser and see why counting the r's in strawberry is so hard.