Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SYCL failures for fluids example on SunSpot #1603

Open
jrwrigh opened this issue Jun 11, 2024 · 0 comments
Open

SYCL failures for fluids example on SunSpot #1603

jrwrigh opened this issue Jun 11, 2024 · 0 comments
Assignees

Comments

@jrwrigh
Copy link
Collaborator

jrwrigh commented Jun 11, 2024

I've seen some failures on SunSpot with the fluids examples. The general behavior is:

  • /gpu/sycl/ref passes fine everytime
  • /gpu/sycl/shared fails about 90% of the time
  • /gpu/sycl/gen fails about 10% of the time

The failures are only present on a few tests (SunSpot is down for maintenance today, so I can't confirm which ones exactly right now, but I'm fairly certain the Gaussian wave tests are one of them), but the above behavior is pretty consistent. This is observed using the oneapi/release/2024.04.15.001.

The failure specifically is a non-linear solver divergence:

[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: TSStep has failed due to DIVERGED_NONLINEAR_SOLVE, increase -ts_max_snes_failures or make negative to attempt recovery

Given the relationships between the backends, I'm guessing the error is probably in the shared functions between the shared and gen backends.

Tagging @kris-rowe @uumesh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants