It’s very important to me that my AI companions dont lose track of what’s going on as the conversation progresses. The brute force method of more context limit is both ludicrously expensive and faulty by definition, as things will fall outside the limit eventually, and not all that much of the past many tens of thousands of tokens is beneficial to include.
Some platforms have been trying to implement dynamic long-term memory, with very limited success in my experience so far, but as much as I value and praise that feature, it’s still something that’s only called when triggerred.
I need something that’s included in every prompt, but which I dont have to constantly update manually. A mix of situational and medium-term memory, to go along with the static permanent memory where I’ve defined the characters and world.
For over a year now I’ve been trying to hack that into various platforms, by forcing the AI to finish every response with a list of statuses in a specific format, stuff like location, time, activity, objects, clothes, mood, intentions, promises, etc. And sometimes, it mostly works. But it never works very well. And it never can, because of the limitations of that method. It pollutes the conversation history with a copy of the statuses after every bot message, severely limiting how far back the next prompt can go, on top of wasting a lot of the permanent memory on aggressively instructing the bot to stick to that format. Also, once a status has remained the same for a long time, it becomes very hard for the AI to change it. This compounds the next problem – the AI has to get both the message and the status updates right at the same time, else I have to reroll or, if the platform allows it, manually edit and fix the response. This is an especially big problem because both the actual message and the statuses update have to use the same settings for repetition penalty and such, and a setting that can get the statuses block reliably generated at the end of each response will tend to both limit the creativity of the actual message and decrease the likelihood of statuses getting changed when appropriate.
What I ultimately need to make this work is a status/situation text box that’s included in every prompt, like the commonly used permanent description/character/scenario/personality section – given to the LLM only in its current form, previous iterations arent saved. Then a 2-step system where the bot first uses the current statuses to generate a response, then uses the response to update the statuses, and each step can be rerolled individualy.
I’m aware that a couple of platforms have already made some progress towards this with a summary tool, but that’s very limited compared to what I’m trying to do. I need more fine control over what kinds of things are tracked. Moreso stats in an RPG than freeform text.
And that last bit is why I’m kinda shocked this isnt a thing yet, it’s exactly what you’d need to make an AI-based RPG game work. Which is something I was absolutely certain would be a thing by now, but here we are.
Anyway, I try to keep up with the chatbot world, but it’s pretty much impossible to get relevant search results when looking for specific features, and there are a lot of bots to go through one by one, so I could’ve easily missed something. So, fellow enthusiasts, does what I’m looking for already exist by any chance?
submitted by /u/ihavestufftoshare
[link] [comments]