Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow NAM_SAMPLE_FLOAT to switch model input to float instead of double #48

Merged
merged 2 commits into from
Jun 23, 2023
Merged

Allow NAM_SAMPLE_FLOAT to switch model input to float instead of double #48

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
85dff1e
Allow NAM_SAMPLE_FLOAT to switch model input to float instead of double
mikeoliphant May 29, 2023
133cf43
Revert parameters to double
mikeoliphant Jun 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion NAM/convnet.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ convnet::ConvNet::ConvNet(const int channels, const std::vector<int>& dilations,
{
}

convnet::ConvNet::ConvNet(const double loudness, const int channels, const std::vector<int>& dilations,
convnet::ConvNet::ConvNet(const NAM_SAMPLE loudness, const int channels, const std::vector<int>& dilations,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I don't like loudness being of "sample" type--it's a parameter of the model.

But I get that it mainly serves to do arithmetic with samples, so I can live with it.

const bool batchnorm, const std::string activation, std::vector<float>& params)
: Buffer(loudness, *std::max_element(dilations.begin(), dilations.end()))
{
Expand Down
2 changes: 1 addition & 1 deletion NAM/convnet.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ class ConvNet : public Buffer
public:
ConvNet(const int channels, const std::vector<int>& dilations, const bool batchnorm, const std::string activation,
std::vector<float>& params);
ConvNet(const double loudness, const int channels, const std::vector<int>& dilations, const bool batchnorm,
ConvNet(const NAM_SAMPLE loudness, const int channels, const std::vector<int>& dilations, const bool batchnorm,
const std::string activation, std::vector<float>& params);

protected:
Expand Down
21 changes: 11 additions & 10 deletions NAM/dsp.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,15 @@ DSP::DSP()
{
}

DSP::DSP(const double loudness)
DSP::DSP(const NAM_SAMPLE loudness)
: mLoudness(loudness)
, mNormalizeOutputLoudness(false)
, _stale_params(true)
{
}

void DSP::process(double** inputs, double** outputs, const int num_channels, const int num_frames,
const double input_gain, const double output_gain,
void DSP::process(NAM_SAMPLE** inputs, NAM_SAMPLE** outputs, const int num_channels, const int num_frames,
const NAM_SAMPLE input_gain, const NAM_SAMPLE output_gain,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it looks weird having all of these parameters being called "samples". Can you point me to somewhere else that follows this convention? It just doesn't sound right.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure either, and I'm fine either way. "loudness" is an internal parameter anyway, and "input_gain" and "output_gain" should be probably be removed since the plugin just passes "1.0" and handles gain itself (as does my LV2 plugin).

const std::unordered_map<std::string, double>& params)
{
this->_get_params_(params);
Expand All @@ -58,7 +58,7 @@ void DSP::_get_params_(const std::unordered_map<std::string, double>& input_para
}
}

void DSP::_apply_input_level_(double** inputs, const int num_channels, const int num_frames, const double gain)
void DSP::_apply_input_level_(NAM_SAMPLE** inputs, const int num_channels, const int num_frames, const NAM_SAMPLE gain)
{
// Must match exactly; we're going to use the size of _input_post_gain later
// for num_frames.
Expand All @@ -83,13 +83,14 @@ void DSP::_process_core_()
this->_core_dsp_output[i] = this->_input_post_gain[i];
}

void DSP::_apply_output_level_(double** outputs, const int num_channels, const int num_frames, const double gain)
void DSP::_apply_output_level_(NAM_SAMPLE** outputs, const int num_channels, const int num_frames,
const NAM_SAMPLE gain)
{
const double loudnessGain = pow(10.0, -(this->mLoudness - TARGET_DSP_LOUDNESS) / 20.0);
const double finalGain = this->mNormalizeOutputLoudness ? gain * loudnessGain : gain;
const NAM_SAMPLE loudnessGain = (NAM_SAMPLE)pow(10.0, -(this->mLoudness - TARGET_DSP_LOUDNESS) / 20.0);
const NAM_SAMPLE finalGain = this->mNormalizeOutputLoudness ? gain * loudnessGain : gain;
for (int c = 0; c < num_channels; c++)
for (int s = 0; s < num_frames; s++)
outputs[c][s] = double(finalGain * this->_core_dsp_output[s]);
outputs[c][s] = (NAM_SAMPLE)(finalGain * this->_core_dsp_output[s]);
}

// Buffer =====================================================================
Expand All @@ -99,7 +100,7 @@ Buffer::Buffer(const int receptive_field)
{
}

Buffer::Buffer(const double loudness, const int receptive_field)
Buffer::Buffer(const NAM_SAMPLE loudness, const int receptive_field)
: DSP(loudness)
{
this->_set_receptive_field(receptive_field);
Expand Down Expand Up @@ -176,7 +177,7 @@ Linear::Linear(const int receptive_field, const bool _bias, const std::vector<fl
{
}

Linear::Linear(const double loudness, const int receptive_field, const bool _bias, const std::vector<float>& params)
Linear::Linear(const NAM_SAMPLE loudness, const int receptive_field, const bool _bias, const std::vector<float>& params)
: Buffer(loudness, receptive_field)
{
if (params.size() != (receptive_field + (_bias ? 1 : 0)))
Expand Down
22 changes: 14 additions & 8 deletions NAM/dsp.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@
#include "activations.h"
#include "json.hpp"

#ifdef NAM_SAMPLE_FLOAT
#define NAM_SAMPLE float
#else
#define NAM_SAMPLE double
#endif

enum EArchitectures
{
kLinear = 0,
Expand Down Expand Up @@ -40,7 +46,7 @@ class DSP
{
public:
DSP();
DSP(const double loudness);
DSP(const NAM_SAMPLE loudness);
virtual ~DSP() = default;
// process() does all of the processing requried to take `inputs` array and
// fill in the required values on `outputs`.
Expand All @@ -51,8 +57,8 @@ class DSP
// 3. The core DSP algorithm is run (This is what should probably be
// overridden in subclasses).
// 4. The output level is applied and the result stored to `output`.
virtual void process(double** inputs, double** outputs, const int num_channels, const int num_frames,
const double input_gain, const double output_gain,
virtual void process(NAM_SAMPLE** inputs, NAM_SAMPLE** outputs, const int num_channels, const int num_frames,
const NAM_SAMPLE input_gain, const NAM_SAMPLE output_gain,
const std::unordered_map<std::string, double>& params);
// Anything to take care of before next buffer comes in.
// For example:
Expand All @@ -66,7 +72,7 @@ class DSP

protected:
// How loud is the model?
double mLoudness;
NAM_SAMPLE mLoudness;
// Should we normalize according to this loudness?
bool mNormalizeOutputLoudness;
// Parameters (aka "knobs")
Expand All @@ -87,7 +93,7 @@ class DSP

// Apply the input gain
// Result populates this->_input_post_gain
void _apply_input_level_(double** inputs, const int num_channels, const int num_frames, const double gain);
void _apply_input_level_(NAM_SAMPLE** inputs, const int num_channels, const int num_frames, const NAM_SAMPLE gain);

// i.e. ensure the size is correct.
void _ensure_core_dsp_output_ready_();
Expand All @@ -98,7 +104,7 @@ class DSP
virtual void _process_core_();

// Copy this->_core_dsp_output to output and apply the output volume
void _apply_output_level_(double** outputs, const int num_channels, const int num_frames, const double gain);
void _apply_output_level_(NAM_SAMPLE** outputs, const int num_channels, const int num_frames, const NAM_SAMPLE gain);
};

// Class where an input buffer is kept so that long-time effects can be
Expand All @@ -108,7 +114,7 @@ class Buffer : public DSP
{
public:
Buffer(const int receptive_field);
Buffer(const double loudness, const int receptive_field);
Buffer(const NAM_SAMPLE loudness, const int receptive_field);
void finalize_(const int num_frames);

protected:
Expand All @@ -133,7 +139,7 @@ class Linear : public Buffer
{
public:
Linear(const int receptive_field, const bool _bias, const std::vector<float>& params);
Linear(const double loudness, const int receptive_field, const bool _bias, const std::vector<float>& params);
Linear(const NAM_SAMPLE loudness, const int receptive_field, const bool _bias, const std::vector<float>& params);
void _process_core_() override;

protected:
Expand Down
2 changes: 1 addition & 1 deletion NAM/lstm.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ lstm::LSTM::LSTM(const int num_layers, const int input_size, const int hidden_si
{
}

lstm::LSTM::LSTM(const double loudness, const int num_layers, const int input_size, const int hidden_size,
lstm::LSTM::LSTM(const NAM_SAMPLE loudness, const int num_layers, const int input_size, const int hidden_size,
std::vector<float>& params, nlohmann::json& parametric)
: DSP(loudness)
{
Expand Down
2 changes: 1 addition & 1 deletion NAM/lstm.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ class LSTM : public DSP
public:
LSTM(const int num_layers, const int input_size, const int hidden_size, std::vector<float>& params,
nlohmann::json& parametric);
LSTM(const double loudness, const int num_layers, const int input_size, const int hidden_size,
LSTM(const NAM_SAMPLE loudness, const int num_layers, const int input_size, const int hidden_size,
std::vector<float>& params, nlohmann::json& parametric);
~LSTM() = default;

Expand Down
2 changes: 1 addition & 1 deletion NAM/wavenet.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ wavenet::WaveNet::WaveNet(const std::vector<wavenet::LayerArrayParams>& layer_ar
{
}

wavenet::WaveNet::WaveNet(const double loudness, const std::vector<wavenet::LayerArrayParams>& layer_array_params,
wavenet::WaveNet::WaveNet(const NAM_SAMPLE loudness, const std::vector<wavenet::LayerArrayParams>& layer_array_params,
const float head_scale, const bool with_head, nlohmann::json parametric,
std::vector<float> params)
: DSP(loudness)
Expand Down
2 changes: 1 addition & 1 deletion NAM/wavenet.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ class WaveNet : public DSP
public:
WaveNet(const std::vector<LayerArrayParams>& layer_array_params, const float head_scale, const bool with_head,
nlohmann::json parametric, std::vector<float> params);
WaveNet(const double loudness, const std::vector<LayerArrayParams>& layer_array_params, const float head_scale,
WaveNet(const NAM_SAMPLE loudness, const std::vector<LayerArrayParams>& layer_array_params, const float head_scale,
const bool with_head, nlohmann::json parametric, std::vector<float> params);

// WaveNet(WaveNet&&) = default;
Expand Down