I am on the 2026-27 job market looking for industry Research Scientist positions. Please reach out via email if you see a good fit!

I am a final year PhD student in the Computer Science Department at the University of Texas at Austin. I am fortunate to be co-advised by Prof. David Harwath and Prof. Eunsol Choi. I am part of the broader UT NLP group. My research interests are in Speech and Natural Language Processing: my goal is to build speech and multimodal LLMs that understand and generate not just what is said, but how it is said.

I received my B.Tech (with Honors) degree in Computer Science and a Minor in Applied Statistics and Informatics from IIT Bombay in 2021, where I had a wonderful time working with Prof. Preethi Jyothi and Prof. Sunita Sarawagi.

I have also spent some time interning at Meta Superintelligence Labs (2026, with Kartik Audhkhasi and Bhuvana Ramabhadran), Google DeepMind (2023, with Yu Zhang and Ankur Bapna), Meta FAIR (2022, with Abdelrahman Mohamed, Wei-Ning Hsu and Ching-Feng Yeh), Adobe Research India (Summer 2020) and UCLouvain (Summer 2019).

In my spare time, I enjoy reading, quizzing, solving wordgames, and watching the latest movies and TV shows.

News

  • Oct 2026New paper under review on teaching speech assistants to act on vocal cues, from my Meta internship.
  • Sep 2026New paper under review on discovering interpretable acoustic features in voice-cloning TTS models.
  • Aug 2026ParaSpeechCLAP was shortlisted for the Best Student Paper Award at Interspeech 2026.
  • Jun 2026Passed my PhD proposal and became a PhD candidate.
  • Jun 2026ParaSpeechCLAP accepted at Interspeech 2026.
  • May 2026Started a Research Scientist internship at Meta Superintelligence Labs in New York.
  • Feb 2026CS-YODAS accepted at LREC 2026.
  • Sep 2025Gave an invited talk at TTIC on scaling rich style-prompted text-to-speech datasets.
  • Aug 2025ParaSpeechCaps and VoiceCraft-X accepted at EMNLP 2025.
  • Jul 2025Rhapsody accepted at COLM 2025.
  • May 2025CS-FLEURS accepted at Interspeech 2025.
  • Jan 2025Dynamic-SUPERB Phase-2 accepted at ICLR 2025.
  • Sep 2024Textless speech-to-speech translation with limited parallel data accepted at EMNLP 2024 Findings.
  • Jul 2024Gave an invited talk at AI4Bharat, IIT Madras on textless speech-to-speech translation.
  • Jun 2024US patent 12,014,217 granted for scheduling ML jobs on interruptible cloud instances, from my Adobe Research internship.
  • May 2023Started a Student Researcher position at Google DeepMind in Mountain View.
  • May 2023When to Use Efficient Self Attention? accepted at ACL 2023.
  • Feb 2023Continual learning for on-device speech recognition, from my Meta FAIR internship, accepted at ICASSP 2023.
  • Oct 2022Zero-shot video moment retrieval accepted at the TL4NLP workshop at NeurIPS 2022.
  • Oct 2022Why is Winoground Hard? accepted at EMNLP 2022 (oral).
  • May 2022Started a research internship at Meta FAIR in Seattle.
  • Aug 2021Started my PhD in Computer Science at UT Austin, co-advised by David Harwath and Eunsol Choi.
  • Aug 2021Graduated from IIT Bombay with a B.Tech (Honors) in Computer Science and Engineering.
  • Aug 2021Reduce and Reconstruct was shortlisted for the Best Student Paper Award at Interspeech 2021.
  • Jun 2021Three papers accepted at Interspeech 2021, including the MUCS 2021 challenge that I co-organized.

Scroll for more

Publications

Teaching Speech Assistants to Act on Vocal Cues

Anuj Diwan, Puyuan Peng, Haohe Liu, Abdelrahman Mohamed, Bhuvana Ramabhadran, Kartik Audhkhasi

Under review

Discovering Interpretable Acoustic Features in Autoregressive Voice-Cloning TTS Models

Anuj Diwan, David Harwath, Eunsol Choi

Under review

ParaSpeechCLAP: A Dual-Encoder Speech-Text Model for Rich Stylistic Language-Audio Pretraining

Anuj Diwan, Eunsol Choi, David Harwath

Interspeech 2026 Shortlisted for Best Student Paper Award

CS-YODAS: A Mined Dataset of In-the-Wild Code-Switched Speech

Brian Yan, Qingzheng Wang, Matthew Wiesner, Anuj Diwan, Olga Iakovenko, Alex Polok, Injy Hamed, Shuichiro Shimizu, Iris Emerman, Thomas Hain, David R. Mortensen, Peter Viechnicki, Shinji Watanabe

LREC 2026

Scaling Rich Style-Prompted Text-to-Speech Datasets

Anuj Diwan, Zhisheng Zheng, David Harwath, Eunsol Choi

EMNLP 2025

16K+ Hugging Face downloads (dataset and model) · 166 GitHub stars

VoiceCraft-X: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editing

Zhisheng Zheng, Puyuan Peng, Anuj Diwan, Cong Phuoc Huynh, Xiaohang Sun, Zhu Liu, Vimal Bhat, David Harwath

EMNLP 2025

Rhapsody: A Dataset for Highlight Detection in Podcasts

Younghan Park, Anuj Diwan, David Harwath, Eunsol Choi

COLM 2025

CS-FLEURS: A Massively Multilingual and Code-Switched Speech Dataset

Brian Yan, Injy Hamed, Shuichiro Shimizu, Vasista Lodagala, William Chen, Olga Iakovenko, Bashar Talafha, Amir Hussein, Alexander Polok, Kalvin Chang, Dominik Klement, Sara Althubaiti, Puyuan Peng, Matthew Wiesner, Thamar Solorio, Ahmed Ali, Sanjeev Khudanpur, Shinji Watanabe, Chih-Chen Chen, Zhen Wu, Karim Benharrak, Anuj Diwan, et al.

Interspeech 2025

118K+ dataset downloads on Hugging Face

Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks

Chien-yu Huang, Wei-Chih Chen, Shu-wen Yang, Andy T. Liu, Chen-An Li, Yu-Xiang Lin, Wei-Cheng Tseng, Anuj Diwan, et al.

ICLR 2025

Textless Speech-to-Speech Translation With Limited Parallel Data

Anuj Diwan, Anirudh Srinivasan, David Harwath, Eunsol Choi

EMNLP 2024 Findings

Continual Learning for On-Device Speech Recognition using Disentangled Conformers

Anuj Diwan, Ching-Feng Yeh, Wei-Ning Hsu, Paden Tomasello, Eunsol Choi, David Harwath, Abdelrahman Mohamed

ICASSP 2023

Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality

Anuj Diwan*, Layne Berry*, Eunsol Choi, David Harwath, Kyle Mahowald (*Equal contribution)

EMNLP 2022 (Oral)

Zero-shot Video Moment Retrieval With Off-the-Shelf Models

Anuj Diwan*, Puyuan Peng*, Raymond J. Mooney (*Equal contribution)

TL4NLP@NeurIPS 2022

Reduce and Reconstruct: ASR for Low-Resource Phonetic Languages

Anuj Diwan, Preethi Jyothi

Interspeech 2021 Shortlisted for Best Student Paper Award

Low Resource ASR: The surprising effectiveness of High Resource Transliteration

Shreya Khare*, Ashish Mittal*, Anuj Diwan*, Sunita Sarawagi, Preethi Jyothi, Samarth Bharadwaj (*Equal contribution)

Interspeech 2021

MUCS 2021: Multilingual and Code-Switching ASR Challenges for Low Resource Indian Languages

Anuj Diwan, Rakesh Vaideeswaran, Sanket Shah, Ankita Singh, Srinivasa Raghavan, Shreya Khare, Vinit Unni, Saurabh Vyas, Akash Rajpuria, Chiranjeevi Yarra, Ashish Mittal, Prasanta Kumar Ghosh, Preethi Jyothi, Kalika Bali, Vivek Seshadri, Sunayana Sitaram, Samarth Bharadwaj, Jai Nanavati, Raoul Nanavati, Karthik Sankaranarayanan

Interspeech 2021

Scheduling and Control of Executable Jobs over Compute Instances

Subrata Mitra, Sunav Choudhary, Shaddy Garg, Anuj Jitendra Diwan, Piyush Kumar Maurya, Arpit Aggarwal, Prateek Jain

US Patents and Trademarks Office (Filed 2021 | Issued 2024) | Adobe Inc.

Invited Talks

September 2025

Toyota Technological Institute at Chicago

"Scaling Rich Style-Prompted Text-to-Speech Datasets"

July 2024

AI4Bharat, Indian Institute of Technology Madras

"Textless Speech-to-Speech Translation With Limited Parallel Data"

Education

2021-2027 (expected)

PhD in Computer Science

University of Texas at Austin

Concentration: Speech and Natural Language Processing

Advisors: Prof. David Harwath and Prof. Eunsol Choi

2017-2021

B.Tech in Computer Science and Engineering (with Honors)

Indian Institute of Technology Bombay

Minor in Applied Statistics and Informatics

B.Tech Thesis: "Multilingual, Code-switching and Low-Resource NLP and ASR"

Experience

May-Oct 2026

Research Scientist Intern

Meta Superintelligence Labs, New York, NY

Post-trained conversational speech LLMs to act on users' vocal cues, in collaboration with FAIR.

May-Dec 2023

Student Researcher

Google DeepMind, Mountain View, CA

Improved out-of-domain generalization for TTS and speech translation via speech LLM chain-of-thought prompting.

May-Dec 2022

Research Intern (AI)

Meta FAIR, Seattle, WA

Worked on continual learning for on-device speech recognition.

Apr-Jul 2020

Research Intern

Adobe Research, Bangalore, India

Worked on reinforcement learning-based scheduling of ML jobs on interruptible cloud VMs (US patent).

May-Jul 2019

Research Intern

UCLouvain, Belgium

Worked on Riemannian optimization for graph-regularized matrix completion.

Honors & Awards

  • Shortlisted for the ISCA Best Student Paper Award at Interspeech 2026 (ParaSpeechCLAP) and Interspeech 2021 (Reduce and Reconstruct)
  • Excellence in Research Award, IIT Bombay CS Department (2021)
  • IIT Bombay: Institute Academic Prize (2017-18), Excellence in Teaching Assistantship Award (2021), top-1% grades in 7 courses
  • Among the top 35 in India selected for the International Mathematical Olympiad Training Camp (2016)
  • All India Rank 118 among ~160k candidates in JEE Advanced (2017)

Service

Reviewing

  • Reviewer for ICLR, ACL, EMNLP, ACL Rolling Review, Interspeech, ICASSP, IEEE TASLP, IEEE JSTSP and IEEE OJSP
  • Task editor for Dynamic-SUPERB Phase-2 (ICLR 2025), reviewing 23 community task proposals

Teaching

  • Teaching Assistant, Intro to Speech and Audio Processing (Prof. David Harwath), UT Austin, Spring 2024
  • Teaching Assistant, Automatic Speech Recognition (Prof. Preethi Jyothi), IIT Bombay, Spring 2021
  • Teaching Assistant, Logic for Computer Science (Prof. Ashutosh Gupta and Prof. Krishna S.), IIT Bombay, 2019-20
  • Teaching Assistant, Computer Programming and Utilization (Prof. Kameswari Chebrolu and Prof. Varsha Apte), IIT Bombay, Fall 2018

Mentoring

  • Younghan Park: mentored the project that became Rhapsody, published at COLM 2025
  • Rishi Salem: completed an undergraduate research project on speech negotiation models