Soulgaze allows AI NPC’s to visually see the game. This works by sending an image of the game to an appropriate AI service whenever you press the “Soulgaze” MCM hotkey. They will then describe the visuals in the image and talk about it.
You can alter how it works under Soulgaze (ITT) in the configuration wizard. You can use an OpenRouter model by changing the OpenAI connector URL.
Soulgaze can potentially use up a lot of tokens as image processing is expensive for LLMs.