» Tag
model-porting
2 postsPorting Gemma-4 (2B/4B/12B) to AWS Inferentia2: A Field Report
How Gemma-4's mixed attention heads and KV-sharing broke AWS's vendor stack, and the tracing/sharding fix that made Inferentia2 inference work.
Porting Gemma-4 (2B/4B/12B) to AWS Inferentia2: A Field Report
How Gemma-4 was ported to AWS Inferentia2 by bypassing vLLM/NxD, solving mixed attention heads and KV-sharing via direct model tracing.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.com