Run a local AI model on Windows and turn notes into actions

Llama Windows, announced on 13 October, puts a local AI runtime in the Windows 11 notification area. Its model catalogue and chat window let you download an open model and use it without opening a terminal. The app is free and open source.[1]

Here is a first task with a concrete finish: turn a short set of notes into an action list, then check every assignment against the original. This is a walkthrough of current official documentation; the Windows application was not run for this article.

Get one model ready

Start on a Windows 11 PC with an internet connection for the application and model downloads, and room for the selected model. Go to llama.app and choose Download for Windows 11. The project also provides a .msixbundle installer for x64 and ARM64 through its official releases.[3][4]

1. Launch Llama Windows and click its llama icon beside the Windows clock. If the icon is hidden, open the notification-area overflow.
2. Open Recommended Models. Compare the displayed download sizes and choose a modest model for your first attempt. Let the download finish.
3. Find it in Available, load it, and open the web chat. The application starts or connects to the llama.cpp runtime for you.

Before asking it to read a long document, open the model details. The current app offers context-length choices with memory estimates. Context is the amount of text the model can work with at once; increasing it can require more working memory. A model’s file size is therefore not a complete memory budget.[2]

For this small exercise, keep the suggested settings. There is little benefit in starting with the largest download or maximum context when the first question is whether the selected model can follow a few precise instructions.

Give it a job with a checkable answer

Paste the following invented notes into the chat. They contain two assigned actions and one item that remains unassigned.

Then ask: “Extract the agreed actions. For each, give the task, owner and deadline. Put unassigned ideas in a separate section. Use only information in these notes. If something is missing, say it is not specified.”

The expected assignments are Lena with the brochure on Thursday, and Omar with the venue by Friday. The price-list idea should remain unassigned, with no deadline. The launch date should remain undecided. A neat-looking list that assigns the price list to someone has changed the record.

If the answer mixes discussion with decisions, ask it to explain which sentence supports each assignment and try a fresh conversation. Keep the original notes alongside the output. Once the result is right, copy the list into your usual document or task app; that completes the exercise.

Try the same task without a connection

Llama’s documented design runs downloaded models locally and supports offline use. After the downloads have finished, you can disconnect the PC from the internet and repeat the same short exercise in a new chat. This checks your installed setup’s ability to complete that local task.[3]

The Windows repository says the app sends no telemetry and keeps prompts and inference local; downloads and release checks still make network requests. Those statements describe Llama itself. A separate application connected to its local API can have its own data handling.[2]

If a model will not load, try a shorter context setting or a smaller model before repeating the exercise. If the chat opens but the task is handled poorly, compare another suitable model using exactly the same notes and instructions. Keep the questions separate: can this PC run the model, and does that model produce the result you need?[2]

The useful milestone is a small piece of work you can keep: a correctly assigned action list, produced by a model available on your own machine. From there, increase the length or complexity one step at a time.

Report an error or suggest a correction