Computing Target from Frequency

[tbehrensprivate]

Hi there!

CREPE is great! I am doing my own experiments, trying to learn more.

I wonder, how exactly do you compute the target vector (360-cents) given a frequency?

Best,
Tristan

[jongwook]

Hi Tristan, thanks for your interest.

I'm in the process of rewriting the repo to make it easier to reuse/extend, so please stay tuned for updates!

Meanwhile, the code that I used to convert the pitch values to the 360-D vectors is:

import mir_eval
import numpy as np
from scipy.stats import norm

classifier_lowest_hz = 31.70
classifier_lowest_cent = mir_eval.melody.hz2cents(np.array([classifier_lowest_hz]))[0]
classifier_cents_per_bin = 20
classifier_octaves = 6
classifier_total_bins = int((1200 / classifier_cents_per_bin) * classifier_octaves)
classifier_cents = np.linspace(0, (classifier_total_bins - 1) * classifier_cents_per_bin, classifier_total_bins) + classifier_lowest_cent
classifier_cents_2d = np.expand_dims(classifier_cents, axis=1)
classifier_norm_stdev = 25
classifier_pdf_normalizer = norm.pdf(0)

def to_classifier_label(pitch, crossentropy=False):
    """
    Converts pitch labels in cents, to a vector representing the classification label
    Uses the normal distribution centered at the pitch and the standard deviation of 25 cents,
    normalized so that the exact prediction has the value 1.0.
    :param pitch: a number of ndarray of dimension 1
    pitch values in cents, as returned by hz2cents with base_frequency = 10 (default)
    :return: ndarray
    """
    if np.isscalar(pitch):
        vec = norm.pdf((classifier_cents - pitch) / classifier_norm_stdev)
        vec /= crossentropy and classifier_pdf_normalizer or np.sum(vec)
        return vec
    else:
        result = np.zeros((classifier_total_bins, len(pitch)))
        for i, p in enumerate(pitch):
            vec = norm.pdf((classifier_cents - p) / classifier_norm_stdev)
            vec /= crossentropy and classifier_pdf_normalizer or np.sum(vec)
            result[:, i] = vec
        return result

def to_weighted_average_cents(label):
    if label.ndim == 1:
        productsum = np.sum(classifier_cents * label)
        weightsum = np.sum(label)
        return productsum / weightsum
    if label.ndim == 2:
        productsum = np.sum(classifier_cents_2d * label, axis=0)
        weightsum = np.sum(label, axis=0)
        return productsum / weightsum
    raise Exception("label should be either 1d or 2d ndarray")

[tbehrensprivate]

You are the best!

Meanwhile I implemented this:

frequency_reference = 10
    c_true = 1200 * math.log(frequency / frequency_reference, 2)

    cents_mapping = np.linspace(0, 7180, 360) + 1997.3794084376191
    target = np.exp(-(cents_mapping - c_true)**2 / (2 * 25**2))

Seems to produce the same. What do you think?

[jongwook]

yeah, it seems right! mine got a bit complex to work with either 1D / 2D dimensions, and to deal with an alternative training setup where it uses a different normalization scheme.

[tbazin]

Hi! Quick question on this discussion:
The CREPE paper states that:

The bin corresponding to the ground truth fundamental frequency is given a magnitude of one.

But, with the final implementation discussed here, the bin is given a magnitude of one only if the target pitch happens to fall exactly on one of the values of the cents_mapping matrix, otherwise it's slightly lower.
Is that the actual behavior that was implemented for training the model, or should the target frequencies be rounded to the nearest bin value in order to properly assign magnitudes on the bin level?

[jongwook]

Hi! You're correct that it is not given a magnitude of one unless the target pitch falls exactly on the center value, when I trained the model as well. Maybe I should have described it more accurately in the paper.

This is a repo that contains the code I used for training (not so well maintained or documented), and the code I commented above is actually from this training code.

IMO this behavior is advantageous for the post-processing step that takes weighted average to find more precise predictions than the frequency bin resolution (details in README.md):

[symbiosdotwiki]

Hi I just wanted to leave a comment noting that C1 should be 32.7Hz (correctly stated in the pdf paper) but it implemented as 31.7Hz in the code leading to incorrect translation of bin to Hz if you are using information from the paper.

[jongwook]

Oops... thanks for letting me know.

I found that the training was also done assuming C1 is 32.7 Hz, so while the overall range is about half semitone lower than the paper, the predictions in Hz from the code should be accurate.