» Tag
benchmark
26 postsLoRA Speedrun: A Public Wall-Clock Leaderboard for Fine-Tuning
LoRA Speedrun is a public, verified wall-clock leaderboard for LoRA fine-tuning techniques on Qwen2.5-1.5B and GSM8K, run on a single L40S GPU.
AI Jailbreak Benchmark Reveals 100x Safety Gap Between Models
New benchmark shows up to 100x safety gaps among frontier AI models against jailbreak attacks; some models yield zero jailbreaks.
ExploitGym: A Benchmark for AI-Driven Exploit Development
ExploitGym is an open-source AI benchmark with 869 real-world exploit tasks spanning userspace software, V8, and the Linux kernel.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comApple's SpeechAnalyzer Beats Whisper Small in On-Device Speech Test
Benchmark shows Apple's SpeechAnalyzer outperforms Whisper Small and the legacy SFSpeechRecognizer on LibriSpeech word error rate and speed.
Intel Arc B70 32GB Runs Qwen3.6-35B-A3B at 130 t/s via Vulkan
Benchmark: Intel Arc Pro B70 32GB runs Qwen3.6-35B-A3B via llama.cpp Vulkan at ~130 t/s (4-bit, 262k ctx) and ~69 t/s hybrid 8-bit offload.
data-eng-bench: Snowflake's dbt benchmark for coding agents
Snowflake's data-eng-bench tests coding agents on 103 realistic dbt data-engineering tasks across DuckDB and Snowflake, via the open-source Harbor framework.
CVE-Bench: A Sandbox Benchmark for LLM Agents Fixing Real CVEs
CVE-Bench is an open-source benchmark testing LLM agents' ability to fix real Python CVEs inside sandboxed Docker containers.
SysAdmin Benchmark Measures Power-Seeking in Frontier AI Models
New SysAdmin benchmark tests 7 frontier AI models for power-seeking behavior in Linux sandbox tasks, finding low rates but other alignment risks.
ClickHouse vs Postgres: A 100 Million Row Showdown on a Mac Mini
A Mac Mini M4 benchmark pits ClickHouse against Postgres across 100 million LLM trace rows, comparing ingest speed, disk usage, and query latency.
Kyutai's Pocket TTS Clones Voices in 5 Seconds, on CPU, Under MIT
Kyutai's Pocket TTS delivers zero-shot voice cloning from a 5-second sample on CPU. See how it stacks up against Kokoro, Supertonic, and Inflect-Nano in a detailed benchmark.