Rabinarayan Dash

Information-Preserving Learned Compression for Compute-Efficient Language Model Training: A Proof of Concept and a Decisive Experimental Programme

This work investigates whether the learning-relevant information in a corpus can be represented in substantially fewer computational units through a learned, content-adaptive, information-preserving compact representation, and whether a language model can be trained directly on t...

Information-Preserving Learned Compression for Compute-Efficient Language Model Training: A Proof of Concept and a Decisive Experimental Programme

This work investigates whether the learning-relevant information in a corpus can be represented in substantially fewer computational units through a learned, content-adaptive, information-preserving compact representation, and whether a language model can be trained directly on t...