synthetic

    How many tokens are in a prompt?

    Large language models are trained on "tokens": word fragments, rather than individual words or characters. Different LLMs can have a slightly different tokenizers: that is, they break up the same set of words into slightly different sets of tokens.

    A good rule of thumb is that there's usually around three tokens for every two words, plus a few tokens of chat metadata — but that can vary based on the model, and based on your prompt's complexity. We built this calculator to help you get a sense for how many tokens a prompt uses for different open-source models.

    This model's tokenizer uses...

    (Input token price: $0.55/million tokens. See pricing for all models →)
    • A Midsummer Night's Dream, by William Shakespeare
       
      26,331 total tokens
      Price: 1.448 cents
    • We Shall Fight on the Beaches, by Winston Churchill
       
      4,419 total tokens
      Price: 0.243 cents
    • The Tell-Tale Heart, by Edgar Allen Poe
       
      2,702 total tokens
      Price: 0.149 cents
    • I Have a Dream, by Martin Luther King, Jr.
       
      1,954 total tokens
      Price: 0.107 cents
    • Not Like Us, by Kendrick Lamar
       
      1,393 total tokens
      Price: 0.077 cents
    • Hotline Bling, by Drake
       
      610 total tokens
      Price: 0.034 cents
    • Yellow Submarine, by The Beatles
       
      319 total tokens
      Price: 0.018 cents
    Want to run these (and other) open-source models? Sign up for Synthetic.
    Sign upLog in
    synthetic