Running LLM models at the crib

[Macallik86]

I've had a passing interesting in AI for the last few years, but only realized in the last month or so that I don't need an expensive laptop to run models locally.

Share what you are running, what front-end you are using, any tips/tricks you know of, etc.

 

[bnew]

i've used LM Studio when i tested some quantized 6B & 7B models.

LM Studio - Discover, download, and run local LLMs

Run Llama, Gemma 3, DeepSeek locally on your computer.

lmstudio.ai

About LM Studio | LM Studio Docs

Learn how to run Llama, DeepSeek, Phi, and other LLMs locally with LM Studio.

lmstudio.ai

GitHub - nomic-ai/gpt4all: GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use. - nomic-ai/gpt4all

github.com

 

[Ty Daniels]

For simplicity sake there is also https://pinokio.computer/

Also depends on what you are trying to do, general LLM, or AI Art.
Pinokio is simple to use and to get started

 

[greenvale]

Just use jupyter and hugging face

 

[Macallik86]

Just use jupyter and hugging face

I already have jupyterlab in Docker for data analysis, but that UX is meh. I might as well just use the terminal instead lol.

I have been using Jan.ai and it does allow you to grab new models directly from huggingface.


What models are yall using and what are your use cases @bnew @greenvale @Ty Daniels

 

[Ty Daniels]

I already have jupyterlab in Docker for data analysis, but that UX is meh. I might as well just use the terminal instead lol.

I have been using Jan.ai and it does allow you to grab new models directly from huggingface.


What models are yall using and what are your use cases @bnew @greenvale @Ty Daniels

I'm mainly using it for AI Art/Editing, more so Stable Diffusion 1.5 and XL, along With Flux

Using Forge UI, Krita AI Diffusion, and sometimes Fooocus

Tools I use
- Krita (with Krita Ai Difussion) (Like Adobe's Generative Fill, but Free)
- Pinokio
- Stablity Matrix
- Google Colab
- ControlNet (SD 1.5 and XL)

I've also played with LLama installed locally, but mainly use ChatGPT(Claude etc...) for any non-art related tasks

 

[Macallik86]

Damn I'm actually running KDE but never tried Krita. I'll have to spin the block for that one.