» Tag
llm
536 postsllama.garden: A Nostr-Powered Decentralized LLM Torrent Network
llama.garden distributes LLM weights via BitTorrent, using a Nostr-based catalog and Hugging Face servers as fallback seeds so downloads work even without seeders.
Why price per 1M tokens is a misleading AI metric
Comparing AI models by price per 1M tokens can mislead teams. Tokenizer differences and chain-of-thought efficiency matter far more than the sticker price per token.
Token Inflation in LLM: A Black-Box Audit Reveals Risks
A new audit study on token inflation in LLM services reveals significant risks for users.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.com200K-Token LLM Serving on a 24 GiB Laptop
JustFit enables 200K-token LLM serving on a 24 GiB laptop, enhancing local capacity.
Mini-Jev: Typesafe's Jev Implemented on an LLM
Mini-Jev classifies using multiple-choice questions on a frozen model instead of JSON.
WebLLM: High-Performance In-Browser LLM Inference Engine
WebLLM offers a high-performance in-browser LLM inference engine. It is OpenAI API compatible and supports custom model integration.
Restore LLM Inference Capacity Quickly with Shadow Engine Recovery in NVIDIA Dynamo
NVIDIA Dynamo's shadow engine recovery allows rapid restoration of LLM processes after failures.
Cross-vendor byte-identical inference for a 72B LLM (AMD MI300X vs. Nvidia H100)
A new protocol enables byte-identical outputs from a 72B LLM using AMD MI300X and Nvidia H100.
Stealing Reasoning Traces from Proprietary LLM APIs
We found a way to extract reasoning traces from frontier AI APIs, highlighting significant security risks and potential data leaks for engineers.
Microsoft's Three-Layer LLM Routing Architecture for AI Agents
Microsoft has unveiled a three-layer LLM routing architecture for Azure Kubernetes Service, designed to efficiently manage agent traffic.