Skip to content
Dev Tools Library

llama.cpp

LLM inference in C/C++, efficient even on ordinary CPUs.

LibraryOpen source
Open site(opens in a new tab)Repository(opens in a new tab)

C/C++ inference engine for language models, with quantization and support for CPU, GPU and Apple Silicon; the base of several local tools.

When to use

For fine control and maximum performance when running local models on varied hardware.

When to avoid

If you want something simple to install and use; Ollama and LM Studio package it better.

Alternatives

They do a similar job. Compare before choosing.

  • Ollama

    Runs language models on your machine with one command.

    CLIOpen source★ 185.4K

Related

From the same area, with a close purpose.

  • LM Studio

    Graphical app to download and chat with local models.

    AppFree

  • LangChain

    Framework to chain LLMs, tools and data.

    FrameworkOpen source★ 148.7K

  • Bruno

    Open source API client that keeps collections as files, in Git.

    AppOpen source★ 48K

  • AI SDK

    TypeScript toolkit for AI apps, with streaming and multiple providers.

    LibraryOpen source★ 27.6K