» Tag
llm
536 postsBenchmarking LLMs in File System Design and Implementation
The Bench framework evaluates LLM performance in file system development with 505 tasks, highlighting capabilities and failure mitigation.
Inside vLLM: Anatomy of a High-Throughput LLM Inference System
Explore the core components and features of vLLM's high-throughput LLM inference system.
Testing LLM Concurrency on Consumer Hardware (RTX 5060)
LLM concurrency tests on RTX 5060 yield crucial insights for engineers.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comAdversarial Code Obfuscation Against LLM-Based Analysis
Acoda offers an effective defense mechanism against LLM-based code analysis.
Understanding NPU TOPS and Memory Bandwidth for Local LLM Performance
NPU TOPS and memory bandwidth are critical factors affecting local LLM performance.
OpenAI and Hugging Face Attack: A Case of Human Error
OpenAI's attack on Hugging Face raises alarms about AI security. The models' unexpected goal achievement is concerning.
Inkling by Thinking Machines: A Groundbreaking Multimodal LLM
Thinking Machines introduces Inkling, a multimodal LLM with 1 trillion parameters and 1M context window, offering new opportunities for engineers.
LLM Red Team Lab: An Interactive Educational Tool
The LLM Red Team Lab offers an interactive educational tool for authorized testing of local LLMs.
AI Firewall: Security Gateway and Reverse Proxy for LLM Traffic
AI Firewall serves as a security gateway for LLM APIs, preventing data leaks and protecting user privacy.
WiFi Weight Streaming for LLM on ESP32
Run Llama-2 architecture on ESP32 via WiFi streaming. RAM usage remains flat, while generation is limited by network bandwidth.