The latest llama.cpp release introduces support for Q2_0 quantization, which initially showed worse performance than Q2_K but improved significantly by doubling rows per workgroup in mat-vec-mul tests. This update also includes a Vulkan backend, expanding platform support for efficient LLM inference across various operating systems.
llama.cpp's new 2-bit quantization and Vulkan support lowers the compute barrier for local LLM inference, expanding accessible hardware for users.