JSON
{
"prompt": "OpenAI's large language models process text using tokens, which are common sequences of characters found in a set of text.",
"metadata": {
"createdBy": "perdixlabs",
"intent": "token-visualizer",
"version": "1.0.0"
},
"instructions": [
{
"role": "system",
"content": "Explain how the model uses tokens to predict the next chunk of text."
},
{
"role": "user",
"content": "Use concise wording and capture the spirit of the OpenAI tokenizer playground UI."
}
]
}
TOON
Provide JSON to materialize TOON output and tokens.