Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1st attempt at SIMD sigmoid #1200

Merged
merged 2 commits into from
Jan 29, 2018
Merged

1st attempt at SIMD sigmoid #1200

merged 2 commits into from
Jan 29, 2018

Conversation

pmichel31415
Copy link
Collaborator

OK I think I've figured out a way to put everything in one operation by basically having:

sigmoid(x)=max(1/2, 1/(1+e^(-x)) + min(1/2, e^x/(1+e^x)) - 1/2

The output is now

[5.960464477539063e-08, 5.960464477539063e-08, 5.960464477539063e-08, 5.960464477539063e-08, 5.960464477539063e-08]
[-5.960464477539063e-08, -5.960464477539063e-08, -5.960464477539063e-08, -5.960464477539063e-08, -5.960464477539063e-08]

for the example in #1090

I've found it to be slightly slower than the original logistic function but not by much (600ms/800ms). @neubig if you can test in your environment maybe and let me know the result that would help.

TODO:

  • confirm speed
  • Estimate cost (I have no idea how this works, maybe @redpony can help?)

@pmichel31415
Copy link
Collaborator Author

I think the tests are passing now (the travis error seems to be network-related)

@neubig
Copy link
Contributor

neubig commented Jan 29, 2018

LGTM!

@neubig neubig merged commit 2329b88 into master Jan 29, 2018
@pmichel31415 pmichel31415 deleted the stable_logistic_simd branch February 16, 2018 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants