Yanzhao Gao

ATAC: Anchor-tail aware context parallelism for LLM training

Large language model training commonly relies on multidimensional parallelism, including data, tensor, pipeline, and context parallelism, to support long-context and large-scale workloads. However, real pretraining corpora consist of highly heterogeneous variable-length samples,...

PassBitNet: low-resource password guessing using 1-Bit quantized transformers

Large language models (LLMs) demonstrate exceptional accuracy in password-guessing tasks, but their massive memory cost, high computational demands, and high duplicate rates severely limit deployment in resource-constrained scenarios. This paper introduces the 1-bit/1.58-bit quan...