» Tag

llm

478 posts

Jul 7, 20268.00
Why price per 1M tokens is a misleading AI metric
Comparing AI models by price per 1M tokens can mislead teams. Tokenizer differences and chain-of-thought efficiency matter far more than the sticker price per token.
ai benchmarking cost optimization llm
Aug 26, 20267.90
Restore LLM Inference Capacity Quickly with Shadow Engine Recovery in NVIDIA Dynamo
NVIDIA Dynamo's shadow engine recovery allows rapid restoration of LLM processes after failures.
cloud gpu inference llm
Aug 25, 20267.90
Cross-vendor byte-identical inference for a 72B LLM (AMD MI300X vs. Nvidia H100)
A new protocol enables byte-identical outputs from a 72B LLM using AMD MI300X and Nvidia H100.
ai amd inference llm
Ad
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.com
Aug 11, 20267.90
Stealing Reasoning Traces from Proprietary LLM APIs
We found a way to extract reasoning traces from frontier AI APIs, highlighting significant security risks and potential data leaks for engineers.
ai data_leaks llm security
Jul 29, 20267.90
Microsoft's Three-Layer LLM Routing Architecture for AI Agents
Microsoft has unveiled a three-layer LLM routing architecture for Azure Kubernetes Service, designed to efficiently manage agent traffic.
ai architecture azure kubernetes
Jul 25, 20267.90
Persistent State Machine: Breaking the von Neumann Memory Wall for LLM Attention
The Persistent State Machine addresses the memory wall in LLMs, cutting energy use and data traffic significantly.
architecture automation llm memory
Jul 22, 20267.90
Making LLM Extraction Trustworthy Enough to Act On
Honeycomb addresses the verification processes needed to enhance LLM extraction trustworthiness.
data engineering llm trustworthiness
Jul 22, 20267.90
Statgate: Statistically Calibrated Gates for LLM Evaluations
Statgate offers statistically calibrated gates that reduce false positives in LLM evaluations.
ci-cd dev-tools llm software-engineering
Jul 21, 20267.90
Open-ultra: A Self-Training LLM Routing Proxy
Open-ultra is a self-training LLM routing proxy that reduces costs while maintaining high output quality.
cost-reduction llm machine-learning openai
Jul 16, 20267.90
Add a Free LLM to a Static Site with Cloudflare Workers AI
Learn how to integrate AI into a static site using Cloudflare Workers AI without a backend or API key, and get a PRD outline in 15 seconds.
ai cloudflare llm static-sites