Core member of the Omnilingual Team for 15 months.
Led the omnilingual extension of SONAR to 4,000+ language varieties via a two-stage teacher-student distillation framework, achieving 15x cross-lingual error reduction across 1,560 languages. arXiv paper
Also contributed to Omnilingual MT (first MT system for 1,600+ languages), BLASER 3 (most multilingual MT quality estimation model to date), and multilingual data curation pipelines at the scale of billions of examples. arXiv paper
Earlier, proposed a character-level encoder via teacher-student distillation achieving state-of-the-art zero-shot speech translation on FLEURS, surpassing SeamlessM4T. ACL 2025 paper
Proposed a sequential contrastive framework for audio-visual learning using multidimensional sequential distances over non-aggregated representations, achieving up to 3.5x relative improvement in retrieval recall over previous methods. ICASSP 2025 paper
Contributed to a masked generative video-to-audio model combining a full-band audio codec with sequence-to-sequence parallel generation, achieving both high audio quality and temporal synchronicity with visual actions. ECCV 2024 paper
FSDP, ZeRO, mixed-precision training, data pipelines at scale, multi-node GPU clusters, Slurm
Python, PyTorch, torchaudio, HuggingFace, fairseq1, fairseq2, Weights & Biases, polars
Multilingual & Multimodal Representation Learning, Large Language Models, Contrastive Learning, Self-supervised Learning, Teacher-Student Distillation, Knowledge Distillation
Cross-modal Alignment, Sentence Embeddings, Optimal Transport, Zero-shot Transfer, Dense Retrieval, RLHF, DPO, Mixture of Experts, Parameter-Efficient Fine-tuning
Speech Translation, Machine Translation, ASR, Audio-Visual Learning, Automatic Quality Estimation
Coordinated the lectures and the practical labs for the sequence-to-sequence modeling and NLP parts of the course. → Course Page
Supervision of practical labs, assisting students with python and ML questions, grading assignments and final exams. → Course Page
Supervision of practical sessions, grading assignments and final exams. → Course Page
BSc Thesis Supervisor for Pol Rosinés Pozo → Thesis PDF
BSc Thesis Supervisor for Miquel M. de Morentin Cardoner → Thesis PDF
Research Project Supervisor for Pau Lozano García
This project features an AI DJ system designed to perform a collaborative back-to-back set with a human artist. Uniquely, the system is sound-agnostic; it uses a language model to understand the sentiment of YouTube comments to find songs with a similar "feeling" to the recent track history. The project was showcased at the various events including S+T+ARTS Festival in Barcelona and Sonar+D in Lisbon.