» Tag
llm
48 postsSparSEEty: Extracting Tokens from Sparsity-Exploiting LLM Serving Systems
SparSEEty addresses token extraction attacks on LLM serving systems, achieving high reconstruction accuracy with minimal overhead.
The Commons: Experiments in Knowledge Inheritance Among LLM Agents
The Commons explores knowledge inheritance and errors among LLM agents through experimental research.
Benchmarking LLMs in File System Design and Implementation
The Bench framework evaluates LLM performance in file system development with 505 tasks, highlighting capabilities and failure mitigation.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comInside vLLM: Anatomy of a High-Throughput LLM Inference System
Explore the core components and features of vLLM's high-throughput LLM inference system.
Testing LLM Concurrency on Consumer Hardware (RTX 5060)
LLM concurrency tests on RTX 5060 yield crucial insights for engineers.
Model Genome: Fingerprinting Whether an LLM Was Trained from Scratch or Derived
Model Genome offers a method to determine if LLMs were trained from scratch or derived.
A Tool to Quickly Audit LLM-Generated Python Codebases
pyreplay aids in auditing Python code generated by LLMs, turning runs into explorable HTML pages.
Bursty Arrivals Accelerate LLM Inference Times
Bursty workloads have been found to unexpectedly speed up LLM inference.
Red-Team LLM Tools for Simulated Actions
The CoT Red Team Agent evaluates LLM and agent behavior. Use the open-source CLI and Python API for security testing.
Solving Moe Load Imbalance in LLM Training via Optimal Transport
TAOT method improves MoE training speed by 43% while reducing communication costs by 74%.