¶ What is an LLM?
An LLM is a Large Language Model. They are the AI “brain” behind most popular AI applications and our mod.
There are a large variety of LLMs to choose from online, such as ChatGPT, LLama, Sonnet, DeepSeek, etc.
Our mod is designed to be compatible with as many LLMs as possible.
Different LLMs can have different “quirks” in regards to roleplay.
For example, DeepSeek can act rather mean when role-playing as a character. While ChatGPT can be very tasked focused.
We want to provide you the option to plug different LLM's in and out to find the one that best suites your roleplay experience!
There is a community ran LLM tier-list here: https://docs.google.com/spreadsheets/d/1UtAR_r18wskmTMMsg8IlhVvr1Fn9tHvRJT8drH6RuzY/edit?gid=1257158105#gid=1257158105
¶ What are Tokens?
LLM tokens are the basic units that large language models process text with. Think of them as the "vocabulary" of AI models like GPT, Claude, and LLaMA.
Whenever you generate an LLM response, you will use tokens for your input/output. Most online providers will only charge you for your Token usage.
- Tokens can be words, parts of words, punctuation, or other meaningful units
- Languages typically use 1-2 tokens per word in English, more for complex words
- Models have token limits (context windows) that constrain input/output length
- Token count affects processing speed and cost of AI operations
Tokens determine both how much text an LLM can process at once and how it understands language structure.
¶
JSON LLMs
CHIM primarily takes advantage of JSON LLM’s for most of the AI interactions.
This is a very important point as a lot of features of this mod are only possible thanks to this technology.
To keep the technical details short, JSON based LLMs allows us to force the AI output to always be in a predictable format.
This means running scripts, animations, who speaks next, etc. Frankly, the sky's the limit for coding AI interactions with JSON LLMs. It's really cool tech!
Here are some examples of JSON LLMs:
OpenAI: GPT-3.5, GPT-4, and GPT-4o (including mini).
OpenRouter: Meta-llama/llama-3-70b-instruct and command-r. CHIM LLM Support List
KoboldCPP: We did tests using openhermes-2.5-mistral-7b.Q8_0.gguf
Example of a JSON output.
[RAW] => {"character":"The Narrator","listener":"RANGROO","mood":"amused","action":"Talk","target":"","message":"You want me to attack that monster? I'm afraid I'm just a voice in your head, RANGROO. The fighting is all up to you. But I can offer you some advice: that beast looks like a tough one. Are you sure you're ready for it?"}
See how it is in an array (a formatted data structure) that we can pick apart and use to play scripts ingame.
This data package tells us the mood, the person who is listening, the action to take, and the message for the NPC (in this case the Narrator) to say.
This format can be considered to hinder AI roleplaying capabilities with their dialogue responses.
However the interaction which is now possible is what we find much more important in regards to a good roleplay.
The goal of this mod is meaningful AI interactions, not just conversations.
With good prompting you can make some rather colourful characters using this system.
We still do take advantage of non-JSON LLMs to summarize memory, dynamic profiles and writing diary entries.
¶ LLM Parameters and what do they do?
- Temperature: Controls randomness; higher = more creative, lower = more focused. [0-2]
- Presence Penalty: Reduces repetition by discouraging repeated topics. [(-2)-2]
- Frequency Penalty: Reduces repeated words or phrases. [0-2]
- Repetition Penalty: Stops the model from repeating itself. [0-2]
- Top_p: Chooses tokens with a combined probability up to p. [0-1]
- Top_k: Picks from the top k most likely words. [0-100]
- Min_p: Ignore words with very low probability. [0-1]
- Top_a: Adjusts word probabilities for better balance. [0-1]