Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug][ARM] Missing re2 ARM build cause Internal Server Error #660

Closed
ananzh opened this issue Jul 22, 2021 · 9 comments · Fixed by #887
Closed

[Bug][ARM] Missing re2 ARM build cause Internal Server Error #660

ananzh opened this issue Jul 22, 2021 · 9 comments · Fixed by #887
Assignees
Labels
bug Something isn't working v1.2.0

Comments

@ananzh
Copy link
Member

ananzh commented Jul 22, 2021

Describe the bug

Currently, re2 doesn't have an ARM build. When run osd in graviton instance, we observe the following error:

node_modules/re2/build/Release/re2.node: cannot open shared object file: No such file or directory

Screen Shot 2021-07-21 at 10 15 20 AM

To Reproduce

  • Use a graviton2 instance (Linux aarch64)

  • logon to OSD

  • load sample web log data

  • go to visualize and select to create a timeline visualization

  • type in query .es(split=host.keyword:50, index=opensearch_dashboards_sample_data_logs).label("Host Name: $1", "^.:(.) >.*")

Expected behavior
Solve the above error

OpenSearch Version
1.0.0

Dashboards Version
1.0.0

Host/Environment (please complete the following information):

  • OS: Linux aarch64
@ananzh ananzh added the bug Something isn't working label Jul 22, 2021
@ananzh ananzh self-assigned this Jul 22, 2021
@ananzh ananzh added the v1.1.0 label Jul 22, 2021
@ananzh
Copy link
Member Author

ananzh commented Jul 22, 2021

To solve this issue, we need to modify the native modules here and add similar url and sha256 for linux-arm64

Some steps to solve:

  1. create linux-arm64-64.gz
npm install --save re2
cd node_modules/re2
gzip -c build/Release/re2.node > linux-arm64-64.gz
  1. upload to a storage place and provide url and sha256 in this issue
  2. we will then patch it in the code

need some here from infra @bbarani @peterzhuamazon

@kavilla
Copy link
Member

kavilla commented Jul 23, 2021

Should we close this issue: #648

@ananzh
Copy link
Member Author

ananzh commented Jul 23, 2021

Should we close this issue: #648

O sorry. I think it is duplicate. I will close this one.

@kavilla
Copy link
Member

kavilla commented Aug 27, 2021

Moving to 1.2 label since we still do not have a replacement artifact.

@kavilla kavilla added v1.2.0 and removed v1.1.0 labels Aug 27, 2021
@tmarkley tmarkley removed the v1.2.0 label Sep 28, 2021
@ananzh
Copy link
Member Author

ananzh commented Sep 29, 2021

@bbarani @peterzhuamazon @peternied

To solve this issue, we need to modify the native modules here and add similar url and sha256 for linux-arm64

Some steps to solve:

  1. create linux-arm64-64.gz
npm install --save re2
cd node_modules/re2
gzip -c build/Release/re2.node > linux-arm64-64.gz
  1. upload to a storage place and provide url and sha256 in this issue
  2. we will then patch it in the code

@bbarani @peterzhuamazon @peternied Haven't heard back and let me make the request more clearly. To solve this reported issue and allow re2 to have a ARM64 build, we need to have a re2.node uploaded and saved. I think the Infra team could help here to apply step 1 to create linux-arm64-72.gz and help to save it to a place that is able to be referred. Then Dashboards can update the patch_native_modules_task.ts to verify again whether it solves the problem.

@peterzhuamazon
Copy link
Member

We need more context on this one:

  1. What is this re2 used for?
  2. Is it a native plugin for Dashboards and you already have your code ready for arm64?
  3. How did you resolve the x64 cases and urls? I dont think Infra has done anything to provide the x64 one.
  4. Is this related to a plugin or anything? If not, then it should be part of the Dashboards build itself handled by your team I think.

Thanks.

@kavilla
Copy link
Member

kavilla commented Oct 21, 2021

Hi @peterzhuamazon

This for the timeline core plugin. This bug existed pre-fork with the error message including: node_modules/re2/build/Release/re2.node: cannot open shared object file: No such file or directory. It turns out the application uses node-re2 so that it can safely use the regular expressions specified by end-users within Timeline, but Node-RE2 does not contain a hosted build for ARM. So it fails while hosted on machines with ARM architecture since it doesn't exist. So the proposed solution would to be place where it is hosted but perhaps we can just download a version and throw it into the repo.

@ananzh
Copy link
Member Author

ananzh commented Oct 21, 2021

@peterzhuamazon
A bit more comments here:
Here is the list of NODE-RE2 hosted build and you could see there is no ARM. Kibana solved this issue by manually created it (could use my steps as reference) and host it here in https://storage.googleapis.com. So we are seeking some help to host it.

@peterzhuamazon
Copy link
Member

similar to chromium no arm64 version cause percy agent to complain when build on arm64?

ananzh added a commit to ananzh/OpenSearch-Dashboards that referenced this issue Oct 24, 2021
In Timeline, we use node-re2 for the regular expressions specified by
the end users. Currently, re2 doesn't have an ARM build and returns an
error. To solve this issue, we create an linux-arm64-64.gz and store it.

Issue Resolved:
opensearch-project#660

Signed-off-by: Anan Zhuang <ananzh@amazon.com>
ananzh added a commit to ananzh/OpenSearch-Dashboards that referenced this issue Oct 24, 2021
In Timeline, we use node-re2 for the regular expressions specified by
the end users. Currently, re2 doesn't have an ARM build and returns an
error. To solve this issue, we create an linux-arm64-64.gz and store it.

Issue Resolved:
opensearch-project#660

Signed-off-by: Anan Zhuang <ananzh@amazon.com>
ananzh added a commit to ananzh/OpenSearch-Dashboards that referenced this issue Oct 24, 2021
In Timeline, we use node-re2 for the regular expressions specified by
the end users. Currently, re2 doesn't have an ARM build and returns an
error. To solve this issue, we create an linux-arm64-64.gz and store it.

Issue Resolved:
opensearch-project#660

Signed-off-by: Anan Zhuang <ananzh@amazon.com>
@ananzh ananzh mentioned this issue Oct 24, 2021
5 tasks
ananzh added a commit to ananzh/OpenSearch-Dashboards that referenced this issue Oct 24, 2021
In Timeline, we use node-re2 for the regular expressions specified by
the end users. Currently, re2 doesn't have an ARM build and returns an
error. To solve this issue, we create an linux-arm64-64.gz and store it.

Issue Resolved:
opensearch-project#660

Signed-off-by: Anan Zhuang <ananzh@amazon.com>
@ananzh ananzh closed this as completed in d8d4aee Oct 25, 2021
ananzh added a commit to ananzh/OpenSearch-Dashboards that referenced this issue Oct 25, 2021
In Timeline, we use node-re2 for the regular expressions specified by
the end users. Currently, re2 doesn't have an ARM build and returns an
error. To solve this issue, we create an linux-arm64-64.gz and store it.

Issue Resolved:
opensearch-project#660

Backport PR:
opensearch-project#887

Signed-off-by: Anan Zhuang <ananzh@amazon.com>
ananzh added a commit that referenced this issue Oct 26, 2021
In Timeline, we use node-re2 for the regular expressions specified by
the end users. Currently, re2 doesn't have an ARM build and returns an
error. To solve this issue, we create an linux-arm64-64.gz and store it.

Issue Resolved:
#660

Backport PR:
#887

Signed-off-by: Anan Zhuang <ananzh@amazon.com>
@kavilla kavilla linked a pull request Oct 26, 2021 that will close this issue
5 tasks
@tmarkley tmarkley added the v1.2.0 label Nov 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working v1.2.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants