synthetic

    How many tokens are in a prompt?

    Large language models are trained on "tokens": word fragments, rather than individual words or characters. Different LLMs can have a slightly different tokenizers: that is, they break up the same set of words into slightly different sets of tokens.

    A good rule of thumb is that there's usually around three tokens for every two words, plus a few tokens of chat metadata — but that can vary based on the model, and based on your prompt's complexity. We built this calculator to help you get a sense for how many tokens a prompt uses for different open-source models.

    This model's tokenizer uses...

    (Input token price: $2.00/million tokens. See pricing for all models →)
    • A Midsummer Night's Dream, by William Shakespeare
       
      26,123 total tokens
      Price: 5.225 cents
    • We Shall Fight on the Beaches, by Winston Churchill
       
      4,473 total tokens
      Price: 0.895 cents
    • The Tell-Tale Heart, by Edgar Allen Poe
       
      2,710 total tokens
      Price: 0.542 cents
    • I Have a Dream, by Martin Luther King, Jr.
       
      1,971 total tokens
      Price: 0.394 cents
    • Not Like Us, by Kendrick Lamar
       
      1,428 total tokens
      Price: 0.286 cents
    • Hotline Bling, by Drake
       
      602 total tokens
      Price: 0.12 cents
    • Yellow Submarine, by The Beatles
       
      322 total tokens
      Price: 0.064 cents
    Want to run these (and other) open-source models? Sign up for Synthetic.
    Sign upLog in
    synthetic