A language model such as ChatGPT, Gemini or Claude does not read words the way we do. Before processing text, it breaks it into small units called tokens. A token can be a whole word, part of a word or a punctuation mark. In English, a word is on average a little more than one token. Hebrew usually needs more tokens for the same amount of text, because models were trained mainly on English and split Hebrew words into smaller pieces.
Tokens are also the unit of price. Companies that use models through a software interface (API) pay by the number of tokens going into the model (the input, meaning the question and any documents attached) and coming out of it (the answer). Usually an output token costs more than an input token, because generating text takes more computation. Prices are published per million tokens and vary widely between a small, cheap model and a large, advanced one.
Why does it cost money? Because every token is computed on expensive chips, mostly Nvidia graphics processors, in data centers that consume enormous amounts of electricity. When you get an answer from ChatGPT for free, someone pays for the computation: the company, out of money from paying subscribers and investors.
This is where AI agents come in. A regular chatbot answers one question and stops. An agent, for example one that writes code, works in a loop: it reads files, writes, runs, reads the result, fixes and repeats, and on each round it sends the model the whole context again. A single task can consume millions of tokens. That is what makes AI coding tools expensive, and why Apple, according to Bloomberg, limits engineers to a cap of $200 a day, and some teams to even less.
What does it mean for a regular user? Mainly that the price of 'free' will not last forever. Companies already limit the number of messages to advanced models on free tiers and keep adding pricier plans. And for those who use AI at work: short, focused questions, without pasting entire documents when not needed, are not only a way to get a better answer but also a way to save money for whoever is paying.
