Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read Large OAS file #84

Open
XDean opened this issue Mar 10, 2022 · 2 comments
Open

Read Large OAS file #84

XDean opened this issue Mar 10, 2022 · 2 comments
Labels
enhancement New feature or request

Comments

@XDean
Copy link

XDean commented Mar 10, 2022

I have .oas file that larger than 5GB with billions polygons.

When I use gdstk to read, it used minutes time and memory > 5GB.

Is there a way to handle the large oas without read it all into memory at first time? Something like numpy.memmap.

Or is there a way to read it as several parts?

@heitzmann
Copy link
Owner

@XDean at this point, there is no way to read it partially. In the future, I plan to add a filter functionality similar to read_gds, but there are other issues I'm working on before that.

@heitzmann heitzmann added the enhancement New feature or request label Mar 10, 2022
@MarkMa1990
Copy link

I think there is always memory problem while reading large oas files, it consumes very big memory resources suddenly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants