Adapting large language models to downstream tasks and to human preferences has become the dominant cost in the model development pipeline. This thesis argues that the binding constraint in adaptation is not the volume of parameters, data, or computation available, but the fraction of that budget which carries usable learning signal – and that this holds in both parameterisation and post-training data. This fraction is small, and can be identified.

The thesis develops this claim in two parts. The first concerns \emph{parameter efficiency}. We show empirically that the soft prompt learned by prompt tuning exhibits a low intrinsic rank, with most of its singular values collapsing toward zero over the course of training.  Motivated by this observation, we propose Decomposed Prompt Tuning, which reparameterises the soft prompt as the product of two thin matrices and thereby constrains it to be low-rank by construction. On SuperGLUE, the method matches or exceeds vanilla and residual prompt tuning while training an order of magnitude fewer parameters.

The second and larger part concerns \emph{signal efficiency} in post-training, which we define as downstream gain per unit of training signal consumed. We establish this through three studies. First, we introduce the mean reward of sampled responses as a proxy for prompt difficulty, and show that the hardest prompts contribute little to self-play preference optimisation: retaining only the easiest $30\%$ outperforms training on the full set. Second, turning from which prompts to train on to which responses to pair, we show that the conventional practice of pairing the highest- and lowest-reward responses degrades as the sampling budget grows, and that selecting the rejected response at reward position $\mu – 2\sigma$ rather than at the minimum restores scalable improvement. Third, we eliminate supervision entirely, deriving verifiable rewards from the structure of raw documents through a reconstruction task, which improves long-context performance without any annotated data.

Taken together, these studies converge on a single observation: in every setting we examine, naive scaling of the available resource – parameters, samples per prompt, training prompts, annotated data – fails to help and often actively hurts, while a small, well-chosen fraction of it outperforms the whole. Efficient adaptation is therefore governed less by how much is spent than by what is selected.

Speaker’s profile

Yao Xiao received the BE degree from the College of Management and Economics, Tianjin University in 2017, the MS degree in computer science of Shanghai Jiao Tong university in 2022. He is currently working toward the PhD’s degree in ISTD, focusing on LLMs.