PitchSTAR💫: Self-Supervised Pitch Style Transfer with Complementary Temporal and Time-Agnostic Style Representations

Leonardo Boulitreau de M. M. Marques and Gaël Richard

LTCI, Télécom Paris, Institut Polytechnique de Paris, France

Audio Samples Paper (Under review) Code (Available Soon)

About

This is the accompanying page for the paper “PitchSTAR: Self-Supervised Pitch Style Transfer with Complementary Temporal and Time-Agnostic Style Representations”, submitted to ICASSP 2027 and currently under review. PitchSTAR is a method for arbitrary pitch style transfer (PST). The PST task is defined as: generating a new pitch curve given a reference ornamented pitch (style) and a source pitch containing the desired notes (content), as shown in the Figure 1.

Figure 1. The (Pitch Style Transfer) PST task.

PitchSTAR is based on a self-supervised flow-matching framework sculpted for the PST task. It models a full context style reference through note-relative pitch modulation, from which style representations are obtained through temporal and time-agnostic branches. The first is designed to capture phrase-level organization (ornament placement, ordering, etc), and the latter to capture local details (ornament slope, frequency, etc). Both enable complementary representations for style while bottlenecking trivial copying in the self-supervised training setting. The method is shown below on Figure 2.

Figure 2. Training scheme of PitchSTAR. The note-relative pitch modulation is encoded by complementary temporal and time-agnostic branches, which respectively capture the coarse and local characteristics of the reference ornaments while suppressing content leakage.

Audio Samples

We randomly select a style reference audio and a source content audio for each model and pitch ornament on GTSinger. Below we show their respective transferred audios and pitch curves as well as the selected style reference and source content. The audios were synthesized with a 44.1kHz based NSF-HiFiGAN vocoder by doing analysis and simply subsituting the original source pitch curve with the transferred one before synthesis. The notes extracted for each pitch are plotted as thicker horizontal lines.

In-Domain Transfer

Style references are taken from the training set and source contents are taken from the test set of our GTSinger split. Select one reference below to inspect.

Style Reference Content (Notes)
PitchSTAR	StylePitcher w/ Mod	StylePitcher	VEVO 2

Style Reference Content
PitchSTAR	StylePitcher w/ Mod	StylePitcher	VEVO 2

Style Reference Content
PitchSTAR	StylePitcher w/ Mod	StylePitcher	VEVO 2

Style Reference Content
PitchSTAR	StylePitcher w/ Mod	StylePitcher	VEVO 2

Out-of-Domain Transfer

Style references are taken from an external dataset and source contents are taken from the test set of our GTSinger split.

A brief upward pitch excursion to a neighboring note, immediately returning to the original pitch. Extracted from the AdoVoc Dataset.

Style Reference

Content

PitchSTAR

A gentle swing or oscillation that begins from a fixed note and reaches the periphery of another note. During these oscillations, it touches various microtones present between the notes. Extracted from the Raga Ornament Detection Dataset.

Style Reference