Afra Amini

Hi there! I am a doctoral student at ETH AI Center. I am supervised by Prof. Ryan Cotterell and Prof. Elliott Ash. I conduct research in natural language processing and machine learning. My goal is to understand biases of NLP models, and leverage this understanding to build more efficient and controllable systems.

Before starting my Ph.D., I completed a masters degree in computer science at ETH Zürich. I did my undergrad in computer engineering, software engineering major, at Sharif University of Technology.

I like to help Iranian students with their applications to graduate schools. If you are interested in studying NLP in graduate school, feel free to fill in this form.

news

Jul 3, 2024	I received the Qualcomm Innovation Fellowship Europe Reward for my research on Controllable Text Generation 🎉
Jul 7, 2023	We gave a tutorial at ACL 2023 on Generating Text from Language Models ✍️
Oct 7, 2022	I will spend a year (part-time) as a student researcher at Google Deepmind, Zurich
Oct 1, 2021	I started my Ph.D. at ETH AI Center 👩‍💻

selected projects

arXiv

Variational Best-of-N Alignment

Afra Amini, Tim Vieira, and Ryan Cotterell

In arXiv 2024

Abs PDF

Best-of-N (BoN) is a popular and effective algorithm for aligning language models to human preferences. The algorithm works as follows: at inference time, N samples are drawn from the language model, and the sample with the highest reward, as judged by a reward model, is returned as the output. Despite its effectiveness, BoN is computationally expensive; it reduces sampling throughput by a factor of N. To make BoN more efficient at inference time, one strategy is to fine-tune the language model to mimic what BoN does during inference. To achieve this, we derive the distribution induced by the BoN algorithm. We then propose to fine-tune the language model to minimize backward KL divergence to the BoN distribution. Our approach is analogous to mean-field variational inference and, thus, we term it variational BoN (vBoN). To the extent this fine-tuning is successful and we end up with a good approximation, we have reduced the inference cost by a factor of N. Our experiments on a controlled generation task suggest that while variational BoN is not as effective as BoN in aligning language models, it is close to BoN performance as vBoN appears more often on the Pareto frontier of reward and KL divergence compared to models trained with KL-constrained RL objective.
ACL 2024

Direct Preference Optimization with an Offset

Afra Amini, Tim Vieira, and Ryan Cotterell

In Findings of the Association for Computational Linguistics: ACL 2024 2024

Abs PDF

Direct preference optimization (DPO) is a successful fine-tuning strategy for aligning large language models with human preferences without the need to train a reward model or employ reinforcement learning. DPO, as originally formulated, relies on binary preference data and fine-tunes a language model to increase the likelihood of a preferred response over a dispreferred response. However, not all preference pairs are equal: while in some cases the preferred response is only slightly better than the dispreferred response, there can be a stronger preference for one response when, for example, the other response includes harmful or toxic content. In this paper, we propose a generalization of DPO, termed DPO with an offset (ODPO), that does not treat every preference pair equally during fine-tuning. Intuitively, ODPO requires the difference between the likelihood of the preferred and dispreferred response to be greater than an offset value. The offset is determined based on the extent to which one response is preferred over another. Our experiments on various tasks suggest that ODPO significantly outperforms DPO in aligning language models, especially when the number of preference pairs is limited.
ICML 2024

Principled Gradient-based Markov Chain Monte Carlo for Text Generation

Li Du, Afra Amini, Lucas Torroba Hennigen, Xinyan Velocity Yu, Jason Eisner, Holden Lee, and Ryan Cotterell

In International Conference on Machine Learning 2024

Abs PDF

Recent papers have demonstrated the possibility of energy-based text generation by adapting gradient-based sampling algorithms, a paradigm of MCMC algorithms that promises fast convergence. However, as we show in this paper, previous attempts on this approach to text generation all fail to sample correctly from the target language model distributions. To address this limitation, we consider the problem of designing text samplers that are faithful, meaning that they have the target text distribution as its limiting distribution. We propose several faithful gradient-based sampling algorithms to sample from the target energy-based text distribution correctly, and study their theoretical properties. Through experiments on various forms of text generation, we demonstrate that faithful samplers are able to generate more fluent text while adhering to the control objectives better.
EMNLP 2023

Linear-Time Modeling of Linguistic Structure: An Order-Theoretic Perspective

Tianyu Liu, Afra Amini, Mrinmaya Sachan, and Ryan Cotterell

In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing 2023
🏆 Outstanding Paper Award
Abs PDF

Tasks that model the relation between pairs of tokens in a string are a vital part of understanding natural language. Such tasks, in general, require exhaustive pair-wise comparisons of tokens, thus having a quadratic runtime complexity in the length of the string. We show that these exhaustive comparisons can be avoided, and, moreover, the complexity of such tasks can be reduced to linear by casting the relation between tokens as a partial order over the string. Our method predicts real numbers for each token in a string in parallel and sorts the tokens accordingly, resulting in total orders of the tokens in the string. Each total order implies a set of arcs oriented from smaller to greater tokens, sorted by their predicted numbers. The intersection of total orders results in a partial order over the set of tokens in the string, which is then decoded into a directed graph representing the desired linguistic structure. Our experiments on dependency parsing and coreference resolution show that our method achieves state-of-the-art or comparable performance. Moreover, the linear complexity and parallelism of our method double the speed of graph-based coreference resolution models, and bring a 10-times speed-up over graph-based dependency parsers.
NeurIPS 2023

Structured Voronoi Sampling

Afra Amini, Li Du, and Ryan Cotterell

In Proceedings Neural Information Processing Systems (NeurIPS) 2023

Abs PDF

Recently, there has been a growing interest in the development of gradient-based sampling algorithms for text generation, especially in the context of controlled generation. However, there exists a lack of theoretically grounded and principled approaches for this task. In this paper, we take an important step toward building a principled approach for sampling from language models with gradient-based methods. We use discrete distributions given by language models to define densities and develop an algorithm based on Hamiltonian Monte Carlo to sample from them. We name our gradient-based technique Structured Voronoi Sampling (SVS). In an experimental setup where the reference distribution is known, we show that the empirical distribution of SVS samples is closer to the reference distribution compared to alternative sampling schemes. Furthermore, in a controlled generation task, SVS is able to generate fluent and diverse samples while following the control targets significantly better than other methods.
ACL 2023

Hexatagging: Projective Dependency Parsing as Tagging

Afra Amini*, Tianyu Liu*, and Ryan Cotterell

In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: Short Papers 2023
🏆 Outstanding Paper Award
Abs PDF

We introduce a novel dependency parser, the hexatagger, that constructs dependency trees by tagging the words in a sentence with elements from a finite set of possible tags. In contrast to many approaches to dependency parsing, our approach is fully parallelizable at training time, i.e., the structure-building actions needed to build a dependency parse can be predicted in parallel to each other. Additionally, exact decoding is linear in time and space complexity. Furthermore, we derive a probabilistic dependency parser that predicts hexatags using no more than a linear model with features from a pretrained language model, i.e., we forsake a bespoke architecture explicitly designed for the task. Despite the generality and simplicity of our approach, we achieve state-of-the-art performance of 96.4 LAS and 97.4 UAS on the Penn Treebank test set. Additionally, our parser’s linear time complexity and parallelism significantly improve computational efficiency, with a roughly 10-times speed-up over previous state-of-the-art models during decoding.
EMNLP 2022

On Parsing as Tagging

Afra Amini, and Ryan Cotterell

In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing 2022

Abs arXiv PDF

There have been many proposals to reduce constituency parsing to tagging in the literature. To better understand what these approaches have in common, we cast several existing proposals into a unifying pipeline consisting of three steps: linearization, learning, and decoding. In particular, we show how to reduce tetratagging, a state-of-the-art constituency tagger, to shift–reduce parsing by performing a right-corner transformation on the grammar and making a specific independence assumption. Furthermore, we empirically evaluate our taxonomy of tagging pipelines with different choices of linearizers, learners, and decoders. Based on the results in English and a set of 8 typologically diverse languages, we conclude that the linearization of the derivation tree and its alignment with the input sequence is the most critical factor in achieving accurate taggers.