-
Notifications
You must be signed in to change notification settings - Fork 352
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
do einops' operations account for contiguous memory layout? #88
Comments
Good question! Let's assume we are talking in pytorch terms. Einops operations can work with any memory layout and internally flatten array if that's required. There is afaik no need to provide contiguous tensors to other operations (besides |
I'll keep this question open for reference |
so I recently had the problem that pytorch would throw me a warning lile
and the line which causes the issueing of this warning was
it was gone after I changed it to
so I thougt you wanna know that. Maybe it is an option to call contiguous anywars within einops? |
Hi @CDitzel ,
All operations try to incur minimal overhead in terms of memory and time, returning views when possible is the right policy (e.g. transpose in pytorch/numpy works the same way). Returning contiguous always would result in higher memory consumption even when not necessary.
Line that causes warning is likely the next operation because that op is not optimized for non-cont layout but does not call Let me state - everything works how it should work
You also want to check that speed is improved after this addition. If not - you can skip Warning you observe looks new to me, but it's good news that pytorch starts reporting potentially non-optimal places. Non-informativeness of warning you see is disappointing though. |
thank you for your reply. I agree that this topic should be kept open for future references |
Upon heavy reshaping and dimension manipulations, it is necessary from time to time to call .contiguous() on the resulting tensors to straighten out the memory layout. Does einops account for this automatically? I dont see no call to contiguous() anywhere in the examples
The text was updated successfully, but these errors were encountered: