I trained a 113M-parameter earthquake LLM from scratch
A 113M-parameter earthquake LLM was trained from scratch. The entire process is explained step by step.
A small GPT model for earthquake science was trained, detailing the entire LLM lifecycle from data crawling to inference. The nanoGPT-Seis repository aims to make each stage of language model pretraining comprehensible. Training was conducted on two NVIDIA A30 GPUs, with results showing improved fluency through a mix of general and domain-specific texts.