Hi all, this is less of a guide, more of a ‘where to start’.
It’s easy to run local models on your computer, hence free, private and forever yours.
The place to get models: https://huggingface.co/
You run the AI models with popular front ends like:
https://github.com/LostRuins/koboldcpp
https://github.com/oobabooga/text-generation-webui
On top of them, you can run Silly Tavern:
https://github.com/SillyTavern/SillyTavern
I just gave example to 2 front ends, but there are many more, these two are just my own recommendations.
1-Click installers exists for both. For more advanced functionality, you install Silly Tavern, which you could think about as an extension for power users. You can find guides on YouTube if you want a more detailed visual guide, it’s easy.
You don’t need insane hardware to run AI models, smaller models (1B to 4B in size) could easily be run on CPU alone.
Better, smarter models = you’ll need better hardware.
Stay free.
submitted by /u/Sicarius_The_First
[link] [comments]