gemma.cpp
lightweight, standalone C++ inference engine for Google's Gemma models.
About this project
gemma.cpp gemma.cpp is a lightweight, standalone C++ inference engine for the Gemma foundation models from Google. For additional information about Gemma, see ai.google.dev/gemma. Model weights, including gemma.cpp specific artifacts, are available on kaggle. Who is this project for? Modern LLM inference engines are sophisticated systems, often with bespoke capabilities extending beyond traditional neural network runtimes. With this comes opportunities for research and innovation through co-design of high level algorithms and low-level computation. However, there is a gap between deployment-oriented C++ inference runtimes, which are not designed for experimentation, and Python-centric…
Technologies
Project health
GitHub
Reviews
Built by
Maintain google/gemma.cpp? Claiming verifies admin access through your GitHub account and gives you control of this listing.
