awesome-local-llm

Awesome local LLM

A curated list of awesome platforms, tools, practices and resources that helps run LLMs locally

Inference platforms
Inference engines
User Interfaces
Large Language Models
Tools
Hardware
Tutorials
Communities

Inference platforms

LM Studio - discover, download and run local LLMs
unsloth - unified web UI for training and running open models like Qwen, DeepSeek, and Gemma locally
LocalAI - the free, open-source alternative to OpenAI, Claude and others
jan - an open source alternative to ChatGPT that runs 100% offline on your computer
ChatBox - user-friendly desktop client app for AI models/LLMs
lemonade - a local LLM server with GPU and NPU Acceleration

Back to Table of Contents

Inference engines

ollama - get up and running with LLMs
llama.cpp - LLM inference in C/C++
vllm - a high-throughput and memory-efficient inference and serving engine for LLMs
exo - run your own AI cluster at home with everyday devices
BitNet - official inference framework for 1-bit LLMs
sglang - a fast serving framework for large language models and vision language models
TensorRT-LLM - provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs
Nano-vLLM - a lightweight vLLM implementation built from scratch
omlx - LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
koboldcpp - run GGUF models easily with a KoboldAI UI
mistral.rs - fast, flexible LLM inference
dynamo - a datacenter scale distributed inference serving framework
flashinfer - kernel library for LLM serving
mlx-lm - generate text and fine-tune large language models on Apple silicon with MLX
gpustack - simple, scalable AI model deployment on GPU clusters
LiteRT-LM - Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices
mlx-vlm - a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX
executorch - on-device AI across mobile, embedded and edge for PyTorch
mini-sglang - a lightweight yet high-performance inference framework for Large Language Models
distributed-llama - connect home devices into a powerful cluster to accelerate LLM inference
LiteRT - Google's on-device framework for high-performance ML & GenAI deployment on edge platforms, via efficient conversion, runtime, and optimization
ik_llama.cpp - llama.cpp fork with additional SOTA quants and improved performance
sonar - large-scale LLM inference engine based on vLLM
FastFlowLM - run LLMs on AMD Ryzen™ AI NPUs
tokenspeed - a speed-of-light LLM inference engine
krasis - a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
vllm-gfx906 - vLLM for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60
llm-scaler - run LLMs on Intel Arc™ Pro B60 GPUs

Back to Table of Contents

User Interfaces

Open WebUI - User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
Lobe Chat - an open-source, modern design AI chat framework
Text generation web UI - LLM UI with advanced features, easy setup, and multiple backend support
SillyTavern - LLM Frontend for Power Users
Page Assist - Use your locally running AI models to assist you in your web browsing

Back to Table of Contents

Large Language Models

Explorers, Benchmarks, Leaderboards

Arena - benchmark & compare the best AI models
AI Models & API Providers Analysis - understand the AI landscape to choose the best model and provider for your use case
SWE-rebench - a continuously evolving and decontaminated benchmark for software engineering LLMs
BullshitBench - measure whether AI models challenge nonsensical prompts instead of confidently answering them
LLM Explorer - explore list of the open-source LLM models
Dubesor LLM Benchmark table - small-scale manual performance comparison benchmark
oobabooga benchmark - a list sorted by size (on disk) for each score
CyberGym - evaluating AI agents' real-world cybersecurity capabilities at scale
vakra - a benchmark for evaluating multi-hop, multi-source tool-calling in AI agents

Back to Table of Contents

Model providers

Qwen - powered by Alibaba Cloud
Mistral AI - a pioneering French artificial intelligence startup
Tencent - a profile of a Chinese multinational technology conglomerate and holding company
Unsloth AI - focusing on making AI more accessible to everyone (GGUFs etc.)
bartowski - providing GGUF versions of popular LLMs
Beijing Academy of Artificial Intelligence - a private non-profit organization engaged in AI research and development
Open Thoughts - a team of researchers and engineers curating the best open reasoning datasets

Back to Table of Contents

Specific models

General purpose

DeepSeek-V4 - a collection of the DeepSeek V4 LLMs
Qwen3.6 - a collection of the latest generation Qwen LLMs
NVIDIA Nemotron v3 - a family of open models from NVIDIA with open weights, training data and recipes, delivering leading efficiency and accuracy for building specialized AI agents
Gemma 4 - a family of open models built by Google DeepMind, that are multimodal, handling text and image input (with audio supported on small models) and generating text output
Mistral Medium 3.5 - The first flaship models from Mistral AI handling instruction-following, reasoning, and coding in a single set of opened-weights
gpt-oss - a collection of open-weight models from OpenAI, designed for powerful reasoning, agentic tasks, and versatile developer use cases
gpt-oss-puzzle-88B - a deployment-optimized large language model developed by NVIDIA, derived from OpenAI's gpt-oss-120b
Hunyuan - a collection of Tencent's open-source efficient LLMs designed for versatile deployment across diverse computational environments
Phi-4 - a family of small language, multi-modal and reasoning models from Microsoft
OpenReasoning-Nemotron - a collection of models from NVIDIA, trained on 5M reasoning traces for math, code and science
Kimi K2.5 - a collection of open-source, native multimodal agentic models from Moonshot AI that advances practical capabilities in long-horizon coding, coding-driven design, proactive autonomous execution, and swarm-based task orchestration
GLM-5.2 - a Z.ai's flagship model for long-horizon tasks
Ling-3.0-flash - a native hybrid reasoning model from inclusionAI, operating with 124B total and 5.1B active parameters
Granite 4.1 - efficient language models from IBM for multilingual generation, coding, RAG, and AI assistant workflows
EXAONE-4.5 - LG's First Open-Weight Vision-Language Model for Industrial Intelligence
Step-3.5-Flash - most capable open-source foundation model, engineered to deliver frontier reasoning and agentic capabilities with exceptional efficiency
Nex-N2 - a collection of agent models built for real-world productivity scenarios

Back to Table of Contents

Coding

Qwen3-Coder-Next - a collection of Qwen's open-weight language models designed specifically for coding agents and local development
Devstral 2 - a couple of agentic LLMs for software engineering tasks, excelling at using tools to explore codebases, edit multiple files, and power SWE Agents
Mellum 2 - an assistant model trained by JetBrain
MiniMax-M3 - a native multimodal model with 1M context
MiniMax-M2 - a collection of SOTA models for real-world dev & agents
Laguna-S-2.1 - a 118B total parameter Mixture-of-Experts model with 8B activated parameters per token, designed for agentic coding and long-horizon work
SWE-FastContext - a family of code-search models from Microsoft powering the Explore subagent for coding agents
OmniCoder-9B - a 9-billion parameter coding agent model built by Tesslate, fine-tuned on top of Qwen3.5-9B's hybrid architecture
NousCoder-14B - a competitive programming model post-trained on Qwen3-14B via reinforcement learning
MusaCoder-27B - a code model developed by Moore Threads for PyTorch-to-CUDA/MUSA native kernel generation

Back to Table of Contents

Multimodal

Qwen3-Omni - a collection of the natively end-to-end multilingual omni-modal foundation models from Qwen
GLM-4.6V - a collection of open source multimodal models with native tool use from Zhipu AI

Back to Table of Contents

Image

Qwen-Image - a collection of models for image generation, edit and decomposition from Qwen
Qwen3-VL - a collection of the most powerful vision-language models in the Qwen series to date
GLM-Image - an image generation model
Granite Vision - multimodal models from IBM built for visual document analysis and image understanding
HunyuanImage - a collection of image generation models from Tencent
HunyuanVideo - a collection of video generation models from Tencent
Vidi - a collection of models for multimodal video understanding and creation
FastVLM - a collection of VLMs with efficient vision encoding from Apple
MiniCPM-o & MiniCPM-V - multimodal models with leading performance
LFM2-VL - a colection of vision-language models, designed for on-device deployment
ClipTagger-12b - a vision-language model (VLM) designed for video understanding at massive scale

Back to Table of Contents

Audio

whisper-large-v3 - a state-of-the-art model for automatic speech recognition (ASR) and speech translation from OpenAI
Nemotron Speech - a collection of open, state-of-the-art, production‑ready enterprise speech models from NVIDIA for ASR, TTS, Speaker Diarization and S2SOpenAI
NVIDIA NemotronLabs VoiceChat 11B - a 11B end-to-end, real-time speech full duplex (FD) model from NVIDIA for conversational AI that jointly performs streaming speech understanding and speech generation
Qwen3-ASR - a collection of models that support language identification and ASR for 52 languages and dialects
Qwen3-TTS - a collection of TTS models that cover 10 major languages as well as multiple dialectal voice profiles to meet global application needs
Granite Speech - a collection of compact and efficient speech-language models from IBM, specifically designed for multilingual automatic speech recognition (ASR) and bidirectional automatic speech translation (AST)
Voxtral-Small-24B-2507 - an enhancement of Mistral Small 3, incorporating state-of-the-art audio input capabilities while retaining best-in-class text performance
Voxtral-Mini-4B-Realtime-2602 - a multilingual, realtime speech-transcription model and among the first open-source solutions to achieve accuracy comparable to offline systems with a delay of <500ms
Voxtral-4B-TTS-2603 - frontier, open-weights text-to-speech model that’s fast, instantly adaptable, and produces lifelike speech for voice agents
chatterbox - first production-grade open-source TTS model
VibeVoice - a collection of frontier text-to-speech models from Microsoft
Kitten TTS - a collection of open-source realistic text-to-speech models designed for lightweight deployment and high-quality voice synthesis
Streaming Sortformer Diarizer 4spk v2.1 - a streaming version of a novel end-to-end neural model for speaker diarization from NVIDIA

Back to Table of Contents

Retrieval-Augmented Generation

Nemotron RAG - a set of tools to build retrieval-augmented generation (RAG) systems, improve search and ranking accuracy, and extract structured data from complex docs
Qwen3-Embedding - a collection of the latest proprietary Qwen models, specifically designed for text embedding and ranking tasks
Qwen3-VL-Embedding - an addition to the Qwen embedding models, specifically designed for multimodal information retrieval and cross-modal understanding
Qwen3-Reranker - a collection of the latest proprietary Qwen models, engineered to refine embedding results
Qwen3-VL-Reranker - an addition to the Qwen embedding models, specifically designed for multimodal information retrieval and cross-modal understanding

Back to Table of Contents

Safeguards

Shieldstral 1.0 3B - a compact 3B-parameter, policy-adaptive multimodal safety classifier
Granite Guardian - a collection of safety models from IBM for detecting risks, toxicity, and hallucinations in LLM workflows
Qwen3Guard - a collection of safety moderation models built upon Qwen3
NemoGuard - a collection of models from NVIDIA for content safety, topic-following and security guardrails
Nemotron-3.5-Content-Safety - a small language model (SLM) that uses Google's Gemma-3-4B-it as the base and is fine-tuned by NVIDIA on multimodal, multilingual, and reasoning-oriented content-safety datasets
Privasis - a collection of lightweight text-sanitization models from NVIDIA designed to remove or abstract sensitive information from text according to a user-provided sanitization instruction
SingGuard - a collection of policy-adaptive multimodal LLM Guardrails with dynamic reasoning
HARC - a family of safety-aligned instruction models from Microsoft trained with HARC
gpt-oss-safeguard - a collection of safety reasoning models built-upon gpt-oss from OpenAI
privacy-filter - a bidirectional token-classification model from OpenAI for personally identifiable information (PII) detection and masking in text
AprielGuard - a safeguard model designed to detect and mitigate both safety risks and security threats in LLM interactions

Back to Table of Contents

Miscellaneous

Intern-S2 - a collection of multimodal foundation models for scientific intelligence and long-horizon agents
Fara1.5 - a collection of multimodal computer use agents (CUA) for web browsers from Microsoft
Marco-MoE - a suit of multilingual MoE models with highly-sparse architectures
Jan-v3 - a 4B baseline model for fine-tuning, designed for downstream work: improved instruction following out of the box, strong starting point for fine-tuning and effective lightweight coding assistance
Jan-v2-VL - a family of VLM focused on reliable, many-step task execution
Nemotron-Orchestrator-8B - a state-of-the-art 8B orchestration model designed to solve complex, multi-turn agentic tasks by coordinating a diverse set of expert models and tools
Arch-Router-1.5B - the fastest LLM router model that aligns to subjective usage preferences
Waypoint - a collection of real-time interactive video world models
Hunyuan3D - a collection of everything related (models, datasets etc.) to 3D assets generation from Tencent
Hunyuan-GameCraft-1.0 - a novel framework for high-dynamic interactive video generation in game environments
void-model - a model from Netflix that removes objects from videos along with all interactions they induce on the scene — not just secondary effects like shadows and reflections, but physical interactions like objects falling when a person is removed

Back to Table of Contents

Tools

Models

llmfit - hundreds of models & providers, one command to find what runs on your hardware
outlines - structured outputs for LLMs
llama-swap - reliable model swapping for any local OpenAI compatible server - llama.cpp, vllm, etc.
llguidance - super-fast structured outputs

Back to Table of Contents

Agent Frameworks

AutoGPT - a powerful platform that allows you to create, deploy, and manage continuous AI agents that automate complex workflows
langflow - a powerful tool for building and deploying AI-powered agents and workflows
langchain - build context-aware reasoning applications
anything-llm - the all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more
autogen - a programming framework for agentic AI
Flowise - build AI agents, visually
pi - AI agent toolkit: coding agent CLI, unified LLM API, TUI & web UI libraries, Slack bot, vLLM pods
llama_index - the leading framework for building LLM-powered agents over your data
crewAI - a framework for orchestrating role-playing, autonomous AI agents
agno - a full-stack framework for building Multi-Agent Systems with memory, knowledge and reasoning
sim - open-source platform to build and deploy AI agent workflows
openai-agents-python - a lightweight, powerful framework for multi-agent workflows
NemoClaw - run OpenClaw more securely inside NVIDIA OpenShell with managed inference
SuperAGI - an open-source framework to build, manage and run useful Autonomous AI Agents
camel - the first and the best multi-agent framework
pydantic-ai - a Python agent framework designed to help you quickly, confidently, and painlessly build production grade applications and workflows with Generative AI
txtai - all-in-one open-source AI framework for semantic search, LLM orchestration and language model workflows
agent-framework - a framework for building, orchestrating and deploying AI agents and multi-agent workflows with support for Python and .NET
archgw - a high-performance proxy server that handles the low-level work in building agents: like applying guardrails, routing prompts to the right agent, and unifying access to LLMs, etc.
genkit - open-source framework for building AI-powered apps in JavaScript, Go, and Python, built and used in production by Google
ClaraVerse - privacy-first, fully local AI workspace with Ollama LLM chat, tool calling, agent builder, Stable Diffusion, and embedded n8n-style automation
NeMo-Agent-Toolkit - an open-source library for efficiently connecting and optimizing teams of AI agents
ragbits - building blocks for rapid development of GenAI applications

Back to Table of Contents

Model Context Protocol

mindsdb - federated query engine for AI - the only MCP Server you'll ever need
github-mcp-server - GitHub's official MCP Server
playwright-mcp - Playwright MCP server
chrome-devtools-mcp - Chrome DevTools for coding agents
n8n-mcp - a MCP for Claude Desktop / Claude Code / Windsurf / Cursor to build n8n workflows for you
awslabs/mcp - AWS MCP Servers — helping you get the most out of AWS, wherever you use MCP
mcp-atlassian - MCP server for Atlassian tools (Confluence, Jira)
dbhub - zero-dependency, token-efficient database MCP server for Postgres, MySQL, SQL Server, MariaDB, SQLite

Back to Table of Contents

Retrieval-Augmented Generation

pathway - Python ETL framework for stream processing, real-time analytics, LLM pipelines and RAG
graphrag - a modular graph-based RAG system
LightRAG - simple and fast RAG
haystack - AI orchestration framework to build customizable, production-ready LLM applications, best suited for building RAG, question answering, semantic search or conversational agent chatbots
vanna - an open-source Python RAG framework for SQL generation and related functionality
graphiti - build real-time knowledge graphs for AI Agents
onyx - the AI platform connected to your company's docs, apps, and people
claude-context - make entire codebase the context for any coding agent
pipeshub-ai - a fully extensible and explainable workplace AI platform for enterprise search and workflow automation

Back to Table of Contents

Coding Agents

opencode - a AI coding agent built for the terminal
zed - a next-generation code editor designed for high-performance collaboration with humans and AI
OpenHands - a platform for software development agents powered by AI
cline - autonomous coding agent right in your IDE, capable of creating/editing files, executing commands, using the browser, and more with your permission every step of the way
aider - AI pair programming in your terminal
tabby - an open-source GitHub Copilot alternative, set up your own LLM-powered code completion server
continue - create, share, and use custom AI code assistants with our open-source IDE extensions and hub of models, rules, prompts, docs, and other building blocks
void - an open-source Cursor alternative, use AI agents on your codebase, checkpoint and visualize changes, and bring any model or host locally
goose - an open-source, extensible AI agent that goes beyond code suggestions
Roo-Code - a whole dev team of AI agents in your code editor
crush - the glamourous AI coding agent for your favourite terminal
kilocode - open source AI coding assistant for planning, building, and fixing code
humanlayer - the best way to get AI coding agents to solve hard problems in complex codebases
99 - neovim AI agent done right
ProxyAI - the leading open-source AI copilot for JetBrains

Back to Table of Contents

Computer Use

open-interpreter - a natural language interface for computers
OmniParser - a simple screen parsing tool towards pure vision based GUI agent
openwork - an open-source alternative to Claude Cowork, powered by OpenCode
cua - the Docker Container for Computer-Use AI Agents
Agent-S - an open agentic framework that uses computers like a human
self-operating-computer - a framework to enable multimodal models to operate a computer
OpenRoom - a browser-based desktop where AI Agent operates every app through natural language, from MiniMaxAI

Back to Table of Contents

Browser Automation

puppeteer - a JavaScript API for Chrome and Firefox
playwright - a framework for Web Testing and Automation
browser-use - make websites accessible for AI agents
firecrawl - turn entire websites into LLM-ready markdown or structured data
stagehand - the AI Browser Automation Framework
nanobrowser - open-source Chrome extension for AI-powered web automation

Back to Table of Contents

Memory Management

mem0 - universal memory layer for AI Agents
mempalace - the highest-scoring AI memory system ever benchmarked
letta - the stateful agents framework with memory, reasoning, and context management
supermemory - memory engine and app that is extremely fast, scalable
cognee - memory for AI Agents in 5 lines of code
LMCache - supercharge your LLM with the fastest KV Cache Layer
memU - an open-source memory framework for AI companions
reasoning-bank - a memory mechanism for agents that learns from both successful and failed trajectories, with reasoning stored as memory content

Back to Table of Contents

Testing, Evaluation and Observability

langfuse - an open-source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more
opik - debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards
openllmetry - an open-source observability for your LLM application, based on OpenTelemetry
giskard - an open-source evaluation & testing for AI & LLM systems
agenta - an open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place
Evaluator - open-source library for scalable, reproducible evaluation of AI models and benchmarks

Back to Table of Contents

Research

Perplexica - an open-source alternative to Perplexity AI, the AI-powered search engine
gpt-researcher - an LLM based autonomous agent that conducts deep local and web research on any topic and generates a long report with citations
SurfSense - an open-source alternative to NotebookLM / Perplexity / Glean
open-notebook - an open-source implementation of Notebook LM with more flexibility and features
RD-Agent - automate the most critical and valuable aspects of the industrial R&D process
local-deep-researcher - fully local web research and report writing assistant
local-deep-research - an AI-powered research assistant for deep, iterative research
maestro - an AI-powered research application designed to streamline complex research tasks

Back to Table of Contents

Training and Fine-tuning

heretic - fully automatic censorship removal for language models
sentence-transformers - a Python library for using and training embedding and reranker models for applications like retrieval augmented generation, semantic search, and more
trl - train transformer language models with reinforcement learning
OpenRLHF - an easy-to-use, high-performance open-source RLHF framework built on Ray, vLLM, ZeRO-3 and HuggingFace Transformers, designed to make RLHF training simple and accessible
slime - an LLM post-training framework for RL Scaling
Kiln - the easiest tool for fine-tuning LLM models, synthetic data generation, and collaborating on datasets
OpenEnv - an interface library for RL post training with environments
augmentoolkit - train an open-source LLM on new facts
RL - scalable toolkit for efficient model reinforcement
miles - an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime
Gym - evaluate and improve models and agents using environments
SpecForge - train speculative decoding models effortlessly and port them smoothly to SGLang serving

Back to Table of Contents

Security and Sandboxing

garak - the LLM vulnerability scanner from NVIDIA
Guardrails - an open-source toolkit from NVIDIA for easily adding programmable guardrails to LLM-based conversational systems
OpenShell - the safe, private runtime for autonomous AI agents from NVIDIA
CubeSandbox - instant, concurrent, secure & lightweight sandbox for AI agents

Back to Table of Contents

Miscellaneous

context7 - up-to-date code documentation for LLMs and AI code editors
deepwiki-open - open source DeepWiki: AI-powered wiki generator for GitHub/Gitlab/Bitbucket repositories
cai - Cybersecurity AI (CAI), the framework for AI Security
speakr - a personal, self-hosted web application designed for transcribing audio recordings
presenton - an open-source AI presentation generator and API
OmniGen2 - exploration to advanced multimodal generation
4o-ghibli-at-home - a powerful, self-hosted AI photo stylizer built for performance and privacy
Observer - local open-source micro-agents that observe, log and react, all while keeping your data private and secure
mobile-use - a powerful, open-source AI agent that controls your Android or IOS device using natural language
gabber - build AI applications that can see, hear, and speak using your screens, microphones, and cameras as inputs
promptcat - a zero-dependency prompt manager/catalog/library in a single HTML file

Back to Table of Contents

Hardware

Alex Ziskind - tests of pcs, laptops, gpus etc. capable of running LLMs
Digital Spaceport - reviews of various builds designed for LLM inference
Donato Capitella - practical and insightful tutorials on running LLMs locally
JetsonHacks - information about developing on NVIDIA Jetson Development Kits
Miyconst - tests of various types of hardware capable of running LLMs
Kolosal - LLM Memory calculator - estimate the RAM requirements of any GGUF model instantly
LLM Inference VRAM & GPU Requirement Calculator - calculate how many GPUs you need to deploy LLMs
ZLUDA - CUDA on non-NVIDIA GPUs
Strix Halo AI Toolboxes - toolboxes for GenAI on AMD Ryzen AI MAX+: containerized environments for LLMs, Image Generation, and Fine-tuning
Strix Halo Wiki - a website to gather important information and practical guides for systems powered by AMD Ryzen AI MAX and MAX+ processors
ai-notes - random AI notes for working with local models or playing around with random machine learning bits

Back to Table of Contents

Tutorials

Models

Let's reproduce GPT-2 (124M)
nanochat - a full-stack implementation of an LLM like ChatGPT in a single, clean, minimal, hackable, dependency-lite codebase, designed to run on a single 8XH100 node via scripts like speedrun.sh, that run the entire pipeline start to end
Knowledge Distillation: How LLMs train each other
gguf-docs - Docs for GGUF quantization (unofficial)
Embarrassingly Simple Self-Distillation Improves Code Generation

Back to Table of Contents

Prompt Engineering

Prompt Engineering Guide - guides, papers, lecture, notebooks and resources for prompt engineering
Prompt Engineering by NirDiamant - a comprehensive collection of tutorials and implementations for Prompt Engineering techniques, ranging from fundamental concepts to advanced strategies
Prompting guide 101 - a quick-start handbook for effective prompts by Google
Prompt Engineering by Google - prompt engineering by Google
Prompt Engineering by Anthropic - prompt engineering by Anthropic
Prompt Engineering Interactive Tutorial - Prompt Engineering Interactive Tutorial by Anthropic
Real world prompting - real world prompting tutorial by Anthropic
Prompt evaluations - prompt evaluations course by Anthropic
system-prompts-and-models-of-ai-tools - a collection of system prompts extracted from AI tools
system_prompts_leaks - a collection of extracted System Prompts from popular chatbots like ChatGPT, Claude & Gemini
Prompt from Codex - Prompt used to steer behavior of OpenAI's Codex

Back to Table of Contents

Context Engineering

Context-Engineering - a frontier, first-principles handbook inspired by Karpathy and 3Blue1Brown for moving beyond prompt engineering to the wider discipline of context design, orchestration, and optimization
Awesome-Context-Engineering - a comprehensive survey on Context Engineering: from prompt engineering to production-grade AI systems

Back to Table of Contents

Inference

vLLM Production Stack - vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Back to Table of Contents

Agents

superpowers - an agentic skills framework & software development methodology that works
GenAI Agents - tutorials and implementations for various Generative AI Agent techniques
500+ AI Agent Projects - a curated collection of AI agent use cases across various industries
12-Factor Agents - principles for building reliable LLM applications
Agents towards production - end-to-end, code-first tutorials covering every layer of production-grade GenAI agents, guiding you from spark to scale with proven patterns and reusable blueprints for real-world launches
agents.md - a simple, open format for guiding coding agents
Agent Skills - a simple, open format for giving agents new capabilities and expertise
skills - Hugging Face Skills are definitions for AI/ML tasks like dataset creation, model training and evaluation
LLM Agents & Ecosystem Handbook - one-stop handbook for building, deploying, and understanding LLM agents with 60+ skeletons, tutorials, ecosystem guides, and evaluation tools
601 real-world gen AI use cases - 601 real-world gen AI use cases from the world's leading organizations by Google
A practical guide to building agents - a practical guide to building agents by OpenAI

Back to Table of Contents

Retrieval-Augmented Generation

Pathway AI Pipelines - ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data
RAG Techniques - various advanced techniques for Retrieval-Augmented Generation (RAG) systems
Controllable RAG Agent - an advanced Retrieval-Augmented Generation (RAG) solution for complex question answering that uses sophisticated graph based algorithm to handle the tasks
LangChain RAG Cookbook - a collection of modular RAG techniques, implemented in LangChain + Python