Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Pytorch] Add an adapter for c10::string_view #1392

Merged
merged 6 commits into from
Sep 14, 2023

Conversation

HGuillemet
Copy link
Collaborator

@HGuillemet HGuillemet commented Jul 25, 2023

See #1390

I didn't use the owner field of the adapter. I don't think there is a case where we want to free something when the adapter is deleted. Is there ? A string_view is supposed to be a not-owning view of some string.

@sbrunk, could you give that a try ?

@HGuillemet
Copy link
Collaborator Author

I mapped string_view with:

        new Info("c10::basic_string_view<char>", "c10::string_view").annotations("@StringView").valueTypes("String", "BytePointer))

While for std::string it's, by default:

        new Info("std::string").annotations("@StdString").valueTypes("BytePointer", "String").pointerTypes("BytePointer"))

The difference in order of valueTypes results in functions returning string_view return String in Java and functions returning std::string return BytePointer in Java. We'd better unify this.

Do you remember why it's this way for std::string and shall we do the same for string_view ?
Maybe this is to allow application of different encoding ?

@saudet
Copy link
Member

saudet commented Jul 26, 2023 via email

@sbrunk
Copy link
Contributor

sbrunk commented Jul 26, 2023

@sbrunk, could you give that a try ?

Did a quick test with the original einsum example after building locally. Looking good now :)

println(torch.einsum("ii", torch.randn(Seq(4, 4)))) // -2.5606

@saudet
Copy link
Member

saudet commented Aug 17, 2023

@sbrunk Please let us know if this sufficient for all your use cases! Thanks

@sbrunk
Copy link
Contributor

sbrunk commented Aug 17, 2023

I have to admit I haven't done much testing with other functions using c10::string_view arguments beyond einsum.

This is partly because it's much easier to test with the prebuilt CI snapshots. For instance, right now I'm traveling without access to my Linux build machine and I can't get the preset to build locally on macOS.

So if you merge this now, I can't promise that I won't find any issue once I've improved test coverage. I don't want to cause you any additional overhead though, so if you'd rather like to wait until I've done more comprehensive testing, I'll do it as soon as I'm back/able to build locally again.

@saudet
Copy link
Member

saudet commented Aug 17, 2023

Ok, in any case, the build fails on Windows, so that needs to be fixed before we can merge this:

2023-08-17T15:12:19.7007842Z [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project pytorch: Compilation failure: Compilation failure: 
2023-08-17T15:12:19.7008718Z [ERROR] /D:/a/javacpp-presets/javacpp-presets/pytorch/src/gen/java/org/bytedeco/pytorch/global/torch.java:[25536,119] cannot find symbol
2023-08-17T15:12:19.7009300Z [ERROR]   symbol:   class StringViewOptional
2023-08-17T15:12:19.7015054Z [ERROR]   location: class org.bytedeco.pytorch.global.torch

@HGuillemet
Copy link
Collaborator Author

The checks should pass now. It passed on linux because the parser is run, and not on windows. Is that on purpose ?

@saudet saudet merged commit 61a9b3b into bytedeco:master Sep 14, 2023
6 checks passed
@HGuillemet HGuillemet deleted the pytorch_add_stringview branch September 14, 2023 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants