3

Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models
Diffusion Large Language Models (DLLMs) have emerged as a new paradigm of language modeling beyond autoregressive next-token …
Shutong Wu, Jiawei Zhang
Cite
Population-Proportional Preference Learning from Human Feedback: An Axiomatic Approach
Conventional preference learning methods often prioritize opinions held more widely when aggregating preferences from multiple …
Kihyun Kim, Jiawei Zhang, Asuman Ozdaglar, Pablo A Parrilo
Cite
MLorc: Momentum Low-rank Compression for Large Language Model Adaptation
With increasing size of large language models (LLMs), full-parameter fine-tuning imposes substantial memory demands. To alleviate this, …
Wei Shen, Zhang Yaxiang, Minhui Huang, Mengfan Xu, Jiawei Zhang, Cong Shen
Cite
On the convergence analysis of muon
The majority of parameters in neural networks are naturally represented as matrices. However, most commonly used optimizers treat these …
Wei Shen, Ruichuan Huang, Minhui Huang, Cong Shen, Jiawei Zhang
Cite
On the Iteration Complexity of Smoothed Proximal ALM for Nonconvex Optimization Problem with Convex Constraints
It is well-known that the lower bound of iteration complexity for solving nonconvex unconstrained optimization problems is …
Jiawei Zhang, Wenqiang Pu, Zhi-Quan Luo
Cite