The new release introduces ragged-tile MoE prefill FP16 GEMM optimizations, allowing the skipping of padded expert tiles to improve efficiency, particularly at low tokens-per-expert. It also adds quarter-granularity ragged MoE tile-skip with 8-column skip-groups, further optimizing performance by independently skipping empty trailing groups.
llama.cpp's OpenCL optimizations reduce wasted compute cycles for MoE models, making inference more efficient on diverse hardware platforms.