-
Notifications
You must be signed in to change notification settings - Fork 565
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DO NOT MERGE MueLu: Cut drop refactor #13282
base: develop
Are you sure you want to change the base?
Conversation
typename STS::magnitudeType x_aij = static_cast<SC>(std::fabs(static_cast<double>(rowView.value(x) * rowView.value(x)))); | ||
typename STS::magnitudeType y_aij = static_cast<SC>(std::fabs(static_cast<double>(rowView.value(y) * rowView.value(y)))); | ||
typename STS::magnitudeType x_aiiajj = static_cast<SC>(std::fabs(static_cast<double>(threshold * threshold * ghostedDiagValsView(rowView.colidx(x)) * ghostedDiagValsView(row)))); | ||
typename STS::magnitudeType y_aiiajj = static_cast<SC>(std::fabs(static_cast<double>(threshold * threshold * ghostedDiagValsView(rowView.colidx(y)) * ghostedDiagValsView(row)))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
std::fabs
might need to be replaced with Kokkos::ArithTraits<impl_scalar_type>::magnitude
. We need to think through this a bit for complex scalar type. Same for the other spots where we cast things around.
ea7bad3
to
3a60359
Compare
Original code within ORIGINAL ifdef. New code within NEW ifdef. DropTol structure marked with KOKKOS_INLINE_FUNCTION and default values are hard coded. Default Algorithm and Cut Drop Algorithm split into separate for loops in NEW code. Cut Drop converted to use Kokkos nested parallel loops. Timers placed in new code and are commented out. Code passes current unit tests. Saw a speedup of about 1.5x with Cuda and 1.2x with Serial when running unit tests with 10,000,000 rows. Signed-off-by: Ian Halim <ihalim@ascicgpu031.sandia.gov>
DropTol structure in algorithm replaced with new, smaller DropTolKokkos structure. Computations are now done on the fly. Code passes current unit tests. No significant change in speed. Signed-off-by: Ian Halim <ihalim@ascicgpu031.sandia.gov>
Per Christian's request. DropTolKokkos structure removed and replaced with view indices and view of drop flags. ORIGINAL code removed. BuildKokkos removed. Removed commented out timers. Added comments. Signed-off-by: Ian Halim <ihalim@ascicgpu031.sandia.gov>
22d2e36
to
bdc6d3d
Compare
Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; this inspection will remain valid until a new commit to source branch is performed. |
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
Using Repos:
Pull Request Author: jhux2 |
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
|
1520bc5
to
64e64b8
Compare
Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; this inspection will remain valid until a new commit to source branch is performed. |
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
Using Repos:
Pull Request Author: jhux2 |
typename implATS::magnitudeType y_aij = implATS::magnitude(rowView.value(y) * rowView.value(y)); | ||
typename implATS::magnitudeType x_aiiajj = implATS::magnitude(threshold * threshold * ghostedDiagValsView(rowView.colidx(x)) * ghostedDiagValsView(row)); | ||
typename implATS::magnitudeType y_aiiajj = implATS::magnitude(threshold * threshold * ghostedDiagValsView(rowView.colidx(y)) * ghostedDiagValsView(row)); | ||
return x_aij / x_aiiajj > y_aij / y_aiiajj; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return x_aij / x_aiiajj > y_aij / y_aiiajj; | |
return (x_aij / x_aiiajj) > (y_aij / y_aiiajj); |
maybe?
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
|
64e64b8
to
e2fcfd1
Compare
e2fcfd1
to
8eb75e6
Compare
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
@trilinos/framework Does this indicate an error that slipped through? |
@jhux2 I am not familiar enough to say for certain if this should have caused a failure for the test, but I did run a brief query on this test and I did not find any similar runs with that produced that same message |
I should have done a little googling first. The message indicates that MPI isn't CUDA aware. In fact, that's printed at the top of the output in the link I posted:
But since the build isn't enabling CUDA, it's weird that a library is looking for CUDA. |
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
3 similar comments
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
2 similar comments
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
else { | ||
A_t->insertGlobalValues(gblRow, Teuchos::tuple<GO>(gblRow - 1, gblRow, gblRow + 1), Teuchos::tuple<SC>(negOne, two, negOne)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Row 1 has diagonal entry 2 and two off-diagonal -1 entries.
else { | ||
rowID += 2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we drop one of the off-diagonal entries in row 1?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@NyquilDreams I'm confused about the test, would you mind having a look?
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
I've seen this a couple of times, we'll let a RETEST stand here but I'll keep it in mind if we start seeing it again. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The DistanceLaplacianCutSym test claims to be set up specifically to drop one of the entries. But when we look at what it actually checks we see that nothing is being dropped. So either the test is bad, or there is a bug in CutDrop.
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
11 similar comments
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
All Jobs Finished; status = PASSED, However PR is now STALE, and must be retested. Set the AT: RETEST Label to force retest.... |
@trilinos/muelu
Motivation
This is part of Ian Halim's summer project. Converts an aggregation thresholding algorithm to use Kokkos to make it performance portable. A bonus is that the host-based implementation is now faster.
Related Issues
Stakeholder Feedback
N/A
Testing
Covered by existing unit tests.