Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mismatch in Hyper Parameters provided in code and those specified in the paper #44

Open
nategr03 opened this issue Nov 2, 2019 · 0 comments

Comments

@nategr03
Copy link

nategr03 commented Nov 2, 2019

Not really an issue, more of a question.

I am carrying out some research on sentiment analysis and I was looking to reproduce the results of Subtask A, specified in the paper here. Now, I should add that with the source code as is provided, and with TensorFlow backend, I get very similar results. However, upon closer inspection, I noticed some inconsistincies between what's in the code and in the paper.

For example, the paper states that:

We train all of our networks to minimize the
cross-entropy loss, using back-propagation with
stochastic gradient descent and mini-batches of
size 128.

However, in the code, the batch size is 50:

history = nn_model.fit(training[0], training[1], validation_data=validation if not FINAL else testing, epochs=50, batch_size=50, class_weight=class_weights, callbacks=_callbacks)

There are few other inconsistencies as well. My question is, should I change the hyper parameters to the ones specified in the paper in order to exactly replicate the results? Or is exact replication not possible? I should also add that when I run the system multiple times, I get similar, but slightly different results for each epoch. Maybe this has something to with using TensorFlow as a backend? I've been reading that some people have had issues getting the same results when using TensorFlow backend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant