PowerInfer

About this project

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU TL;DR PowerInfer is a CPU/GPU LLM inference engine leveraging activation locality for your device. Project Kanban Latest News 🔥 — [2026/1/5] We released Tiiny AI Pocket Lab, the world's first pocket-size supercomputer. It runs GPT-OSS-120B (int4) locally at 20 tokens/s. Featured at CES 2026. — [2025/7/27] We released SmallThinker-21BA3B-Instruct and SmallThinker-4BA0.6B-Instruct. We also released a corresponding framework for efficient on-device inference. — [2024/6/11] We are thrilled to introduce PowerInfer-2, our highly optimized inference framework designed specifically for smartphones. With…

GitHub

Reviews

Built by

Tiiny-AI

Imported from GitHub · not yet claimed on GitPalace

View developer pageSign in with GitHub to claim

Maintain Tiiny-AI/PowerInfer? Claiming verifies admin access through your GitHub account and gives you control of this listing.

You might also like

openclaw

AI

The AI that really does things. Any OS. Any Platform. The lobster way. 🦞

★ New389K stars

by openclaw

superpowers

AI

An agentic skills framework & software development methodology that works.

★ New282.4K stars

by obra

hermes-agent

AI

The agent that grows with you

★ New242.5K stars

by NousResearch

n8n

AI

Fair-code workflow automation platform with native AI capabilities. Combine visual building with custom code, self-host

★ New203.6K stars

by n8n-io