» Tag
transformers
16 postsI Trained a 30M-Parameter LLM from Scratch: The Scaling "Floor" Was a Mirage
A 30M parameter language model was trained from scratch, aiming to deepen understanding of transformers.
Building a Multi-Horizon Crypto Prediction System with Transformers
Explore the engineering story of crypto prediction with Transformers. Lessons on transparency and reliability with real metrics.
Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers
The v6.0 update of Sentence Transformers introduces MultiVectorEncoder for enhanced late interaction retrieval.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comImplemented Qwen3.5's Mamba-Transformer Architecture from Scratch in Rust
Insights into the architecture and debugging of the ferrite inference engine for Qwen3.5, built from scratch in Rust.
Transformers Struggle with Arithmetic, So I Hand-Set Weights for Accuracy
An experiment shows a transformer can achieve 100% accuracy in arithmetic by setting weights manually without training.
DiScoFormer: One Transformer for Density and Score Estimation
DiScoFormer introduces a novel solution for estimating density and score with a single model.