We propose a character-based translation model to improve adaptability to new languages and modalities, particularly for low-resource scenarios. Our method achieves …
We introduce Sequential Contrastive Audio-Visual Learning (SCAV), a novel method that contrasts non-aggregated sequential representations to learn fine-grained audio-visual …
We introduce ZeroSwot, a zero-shot speech translation method that aligns a speech encoder with a multilingual MT model using only ASR data, achieving state-of-the-art results …