App Discovery: LM Studio -- explore your personal ChatGPT / Gemini / Llama / Claude type chatbot

If you have access to tech news, you've no doubt heard of the wonder world of "AI", that seems to be smart about answering questions about... Anything, from general knowledge to math, and can act as anything from career coach to fitness coach. Some examples are OpenAI's ChatGPT, Google's Gemini, Anthropic's Claude, or Facebook's Llama, but these are actually "large language models", i.e. a pool of knowledge, verified and validated by humans and "learning models" that curates the pool of knowledge, where it both retained the knowledge as well as a way to compose sentences that are human-like. It had NOT yet learned how to think... It just fools you into thinking that it does. 

And one way these LLMs learn is by taking the input you gave it as a part of their dataset. That's why you are able to give it thumbs up / down ratings... You are helping the maker validate its output, and continually tune the variables. And this brings up some privacy concerns, as not everything you ask these LLMs will remain "private". In fact, one of the most given advice when it comes to "build resume with AI" is you need to "sanitize" the resume so your personal details do not appear, lest it got scraped and incorporated into a future training dataset. 

What if I tell you that there are "smaller" LLMs you can run locally on your own PC, as long as you have enough RAM and VRAM on your GPU? So you don't have to worry about privacy (because you didn't upload anything)? Obviously these "smaller" LLMs are less capable than their cloud-based brothers, but they may be enough for your needs, and you can try a couple different ones. 

Photo by Hitesh Choudhary on Unsplash

Go download LMStudio from https://lmstudio.ai/  as it's available for Windows (requires CPU that supports AVX2), the new Mx CPU Macs, and there's an experimental build running on Linux. GPU is optional. 

Once you got it downloaded and installed, then use it to download one of the smaller LLMs, such as Phi3 from Google, but there are dozens and dozens to choose from. Then let LMStudio load it into memory (if it fits) and you can keyboard chat with it just like their cloud-based cousins (sorry, nothing like the multi-modal input like ChatGPT-4o yet). Got tired of it? Go download another LLM and play with that. They are a few GB each. 

If you are interested in this field, and want to learn prompt engineering, yet don't want to stay online indefinitely or pay for "plus" or "premium", doing it locally is a viable alternative. 

So go download it today and have fun! 

P.S. You will be able to TRAIN your own agent soon... so it sounds like you, and haves like you, and maybe even perform actions for you like an avatar!