mine:o1-nano: Full Technical Report — Architecture, Training, Benchmarks & Evaluation

Abstract

Full technical report for mine:o1-nano, a 124-million-parameter, GPT-2-stylelanguage model built from scratch by MineAI Technology using JAX and Flax NNX,trained on Kaggle TPU v5e-8 infrastructure. This report documents the complete model lifecycle: architecture (withdiagrams covering the full model, masked self-attention, and operator-leveltransformer block detail), training configuration and metrics, zero-shotperplexity benchmarks against OpenAI's GPT-2 124M (WikiText-2, WikiText-103,LAMBADA), a comparison against contemporary compressed GPT-2-scale models(TQCompressedGPT2, KnGPT-2, Krony-PT), inference speed optimization oncommodity CPU infrastructure, production deployment details, and a candidaccount of the model's known limitations. Results are reported in full, including outcomes unfavorable to the model,in the interest of accurate, verifiable public benchmarking rather thanpromotional framing. Keywords: language model, GPT-2, transformer, JAX, Flax, perplexity,benchmark, small language model, from-scratch training, Pakistan