local.ai

? local.ai

A desktop app for local, private, secured AI experimentation. Included out-of-the box are:

• A known-good model API and a model downloader, with descriptions such as recommended hardware specs, model license, blake3/sha256 hashes etc...
• A simple note-taking app, with inference config PER note. The note and its config are output into plain text .mdx format
• A model inference streaming server (/completion endpoint, similar to OpenAI)

It's made to be used alongside https://github.com/alexanderatallah/window.ai/ as a simple way to have a local inference server up and running in no time. window.ai + local.ai enable every web app to utilize AI without incurring any cost from either the developer or the user!

Right now, local.ai uses the https://github.com/rustformers/llm rust crate at its core. Check them out, they are super cool!

Install

Go to the site at https://www.localai.app/ and click the button for your machine's architecture. You can also find the build manually in the GitHub release page.

Windows and MacOS binaries are signed under Plasmo Corp. - a company owned by the author of this project (@louisgv).

You may also build from source!

Demo

local.ai.demo.v0.2.x.mp4

? Development

Here's how to run the project locally:

Prerequisites

1. node >= 18
2. rust >= 1.69
3. pnpm >= 8

Workflow

git submodule update --init --recursive
pnpm i
pnpm dev

? License

• Desktop/Web App: GNU GPLv3
• Client SDK: MIT

? Trivia

Why the backpack?

Ties into the bring your own model concept -- Alex from window.ai

Why GPLv3?

Anything AI-related including their derivatives should be open-source for all to inspect. GPLv3 enforces this chain of open-source.

Is there a community?

• GitHub discussion
• Checkout this poll

Where should I ask question?

• Here

I made something with local.ai, where should I post it?

• Here

I have some nice things to say about local.ai, where should I post it?

• Here
• Also, consider giving the repo a Star ️

The naming seems close to LocalAI?

• When I first started the project and got the domain localai.app, I had no idea LocalAI was a thing. A friend of mine forwarded me a link to that project mid May, and I was like dang it, let's just add a dot and call it a day (for now...) - local "dot" ai vs LocalAI lol
• We might rename the project.

Do you accept contribution/PR?

Absolutely - Please note that any contribution toward this repo shall be relicensed under GPLv3. There are many ways to contribute, such as:

• Documentation via wiki edit submission and known model documentation
• Testing and filing BUG issue
• For beginner, check out Good First Issues
• For seasoned developers, check out Unassigned Help Wanted Issues