Shylu: allow fastilu to work with block CRS matrices #12124

jgfouca · 2023-08-10T22:28:41Z

@trilinos/shylu
@trilinos/tpetra

Also, add some BlockCRS helper functions to Tpetra

Motivation

A customer wants to be able to use shylu::fastilu with blockCrs matrices

Testing

Correctness:

Enable determinism and fine-granularity fills (FASTILU_ONE_TO_ONE_UNBLOCKED). Then run soak tests on both
CPU and GPU machines with 2 fill levels. Specifics:
- for N = 4, 6, 8, 9
  - for block_size of each int that evenly divides N
    - for nnz % 10, 20, 50, 100
      - Generate 10 random matrices with these properties
        
        For each random matrix:
        
        Run Ifpack_tif_belos with these inputs
        
        Run blockcrs, crs, and original (a copy of original fastilu source) FastILU impls
        
        All class members (A, L, U) should uncompress to equivalent matrices
        
        Didn't worry about convergence details
Run valgrind with some of these
Repeat the basic 6w test a bunch of times, both CPU and GPU

Performance:

For 10000x10000, 100 nnz per row, 4x4 blocks, block dense

CPU:
JGF initialization of blocked took: 0.0351184
JGF initialization of unblocked took: 0.805753
JGF initialization of orig took: 0.820278
 Time Init: 4.16738
JGF compute of blocked took: 0.0162119
JGF compute of unblocked took: 0.0473241
JGF compute of orig took: 0.0492717
 Finished computing Ifpack2 preconditioner
  Time (s): 0.11435
JGF apply of blocked took: 0.00204011
JGF apply of unblocked took: 0.00214537
JGF apply of orig took: 0.00173839

GPU:
JGF initialization of blocked took: 0.0791826
JGF initialization of unblocked took: 1.95681
JGF initialization of orig took: 1.93094
 Time Init: 8.77923
JGF compute of blocked took: 0.003213
JGF compute of unblocked took: 0.00254731
JGF compute of orig took: 0.00244249
 Finished computing Ifpack2 preconditioner
  Time (s): 0.0123409
JGF apply of blocked took: 0.00140019
JGF apply of unblocked took: 0.000446767
JGF apply of orig took: 0.000458988

…level=0

jgfouca · 2023-08-21T17:56:13Z

@lucbv , I think I addressed your modification requests.

trilinos-autotester · 2023-08-21T21:29:09Z

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request.

trilinos-autotester · 2023-08-21T21:31:06Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Build Num: 2692
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

Build Num: 1214
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

Build Num: 1204
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_clang-11.0.1

Build Num: 1203
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_python3

Build Num: 2419
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	ascic
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

Build Num: 2206
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	GPU
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_intel-2021.3

Build Num: 845
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Using Repos:

Repo: TRILINOS (jgfouca/Trilinos)

Pull Request Author: jgfouca

trilinos-autotester · 2023-08-22T00:41:16Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Build Num: 2692
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

Build Num: 1214
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

Build Num: 1204
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_clang-11.0.1

Build Num: 1203
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_python3

Build Num: 2419
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	ascic
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

Build Num: 2206
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	GPU
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_intel-2021.3

Build Num: 845
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`4fa1f41`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

CDash Test Results for PR# 12124.

trilinos-autotester · 2023-08-22T00:41:33Z

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
THE LAST COMMIT TO THIS PULL REQUEST HAS NOT BEEN REVIEWED YET!

trilinos-autotester · 2023-08-22T00:41:40Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

trilinos-autotester · 2023-08-23T00:44:20Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

trilinos-autotester · 2023-08-24T01:04:05Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

srajama1

@jgfouca Some minor things. Otherwise this is fine.

srajama1 · 2023-08-24T01:25:36Z

packages/ifpack2/src/Ifpack2_Details_CrsArrays.hpp

    if(Acrs)
    {
      getStructureCrs(Acrs, rowptrsHost, rowptrs, colinds);
      return;
    }
+    if (Abrs) {
+      getStructureBrs(Abrs, rowptrsHost, rowptrs, colinds);


Minor nitpick: I would have preferred bcrs (which is the custom)

srajama1 · 2023-08-24T01:26:42Z

packages/ifpack2/src/Ifpack2_Details_FastILU_Base_def.hpp

 #include <KokkosKernels_Utils.hpp>
 #include <Kokkos_Timer.hpp>
 #include <Teuchos_TimeMonitor.hpp>
 #include <stdexcept>
+#include <shylu_fastilu.hpp> // Remove this later


Remove it later?

srajama1 · 2023-08-24T02:59:06Z

packages/shylu/shylu_node/fastilu/src/shylu_fastilu.hpp

-            #ifdef FASTILU_INIT_TIMER
-            host_space().fence();
-            std::cout << " Sort time : " << timer.seconds() << std::endl;
-            timer.reset();


Not sure if this is an artifact of GH's diff. I thught all this depp_copy was needed, no? Did we actually remove it?

@srajama1 , I moved this code to symbolicILU_common. I noticed there was nearly identical code at the end of both versions of symbolicILU, so I encapsulated it in a new function.

srajama1 · 2023-08-24T03:31:09Z

packages/shylu/shylu_node/fastilu/src/shylu_fastilu.hpp

+        }
+
+        KOKKOS_INLINE_FUNCTION
+        void functor_bcrs_impl(const Ordinal start, const Ordinal end) const


I am not sure which is better, to match unblocked numerics or get blocked performance either.

srajama1 · 2023-08-24T05:01:28Z

packages/shylu/shylu_node/fastilu/src/shylu_fastilu.hpp

+        }
+
+        KOKKOS_INLINE_FUNCTION
+        void functor_bcrs_impl(const Ordinal start, const Ordinal end) const


I would have gone with the later, you went with the former, that is ok. At some point we might want to implement the later and compare.

lucbv

Thanks for the small fixes

trilinos-autotester · 2023-08-24T08:52:14Z

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ srajama1 lucbv ]!

trilinos-autotester · 2023-08-24T08:52:21Z

Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR...

trilinos-autotester · 2023-08-24T18:01:56Z

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request.

trilinos-autotester · 2023-08-24T18:03:55Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Build Num: 2710
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

Build Num: 1230
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

Build Num: 1220
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_clang-11.0.1

Build Num: 1219
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_python3

Build Num: 2435
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	ascic
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

Build Num: 2222
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	GPU
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_intel-2021.3

Build Num: 861
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Using Repos:

Repo: TRILINOS (jgfouca/Trilinos)

Pull Request Author: jgfouca

trilinos-autotester · 2023-08-24T21:15:26Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Build Num: 2710
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

Build Num: 1230
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

Build Num: 1220
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_clang-11.0.1

Build Num: 1219
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_python3

Build Num: 2435
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	ascic
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

Build Num: 2222
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	GPU
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

Build Information

Test Name: Trilinos_PR_intel-2021.3

Build Num: 861
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS	AT: AUTOMERGE
PULLREQUESTNUM	12124
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	jgfouca/fastilu_blockcrs
TRILINOS_SOURCE_REPO	https://github.com/jgfouca/Trilinos
TRILINOS_SOURCE_SHA	`3ca2cdd`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`0234ff1`

CDash Test Results for PR# 12124.

trilinos-autotester · 2023-08-24T21:15:44Z

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ srajama1 ]!

trilinos-autotester · 2023-08-24T21:15:51Z

Status Flag 'Pull Request AutoTester' - Pull Request will be Automerged

trilinos-autotester · 2023-08-24T21:15:54Z

Merge on Pull Request# 12124: IS A SUCCESS - Pull Request successfully merged

jgfouca added 30 commits May 15, 2023 16:50

Ifpack2 FastILU has BlockCrs available via Param

5d0f979

Remove unused code

59715ea

Use macros to reduce cpp usage in fastilu

5732bde

Forgot to add this file

4e3dced

Minor fixes

2883402

Reduce code duplication for symbolicILU

8f6e91c

Reduce duplicate in intialize

b3b17cd

Most of this goes away

b12b0ed

Progress. FastILUPrec_Functor functors converted but not verified

30dfd52

Progress. More functors converted

58c5b8b

Remove unused member from Filu

b27239a

Progress

97b8d44

Progress

f4b80e4

Struggling with ILU

62b8b7a

Progress on block fastilu functor. Fix bugs in CRS filler

fcc1a8a

progress

3cc52ba

Algo switch

66afd1b

Progress. FastILUFunctor appears to be working

9a80890

Minor fixes

c060a86

Add a bit more printing

4a4f253

U numeric broken, even at level0

ab458b3

Fixes

dad6aa6

Add orig verification. Everything through numericILU is working with …

8f9d116

…level=0

Unblocked and keep sentinels works for level 0

25cec5d

Fix bug in orig

da75d7f

Everything working through compute

7e55596

Things appear to be fully working

027ef71

Remove debugging stuff

d3bd299

More debugging removed

c309462

GPU progress

5a53733

GitHub feedback: improve Filu blockcrs handling. Revert Frosh fix

4fa1f41

jgfouca requested a review from lucbv August 21, 2023 17:43

trilinos-autotester removed the AT: STALE Added by the PR autotester if too much time has elapsed since the last successful PR test iteration label Aug 21, 2023

srajama1 previously approved these changes Aug 24, 2023

View reviewed changes

lucbv previously approved these changes Aug 24, 2023

View reviewed changes

More minor fixes based on gh feedback

3ca2cdd

jgfouca dismissed stale reviews from lucbv and srajama1 via 3ca2cdd August 24, 2023 15:43

jgfouca requested a review from srajama1 August 24, 2023 15:43

jgfouca added the AT: AUTOMERGE Causes the PR autotester to automatically merge the PR branch once approvals are completed label Aug 24, 2023

srajama1 approved these changes Aug 24, 2023

View reviewed changes

trilinos-autotester merged commit 4a119a3 into trilinos:develop Aug 24, 2023
3 checks passed

trilinos-autotester removed the AT: AUTOMERGE Causes the PR autotester to automatically merge the PR branch once approvals are completed label Aug 24, 2023

jgfouca deleted the jgfouca/fastilu_blockcrs branch September 20, 2023 17:54

Shylu: allow fastilu to work with block CRS matrices #12124

Shylu: allow fastilu to work with block CRS matrices #12124

Conversation

jgfouca commented Aug 10, 2023

Motivation

Testing

jgfouca commented Aug 21, 2023

trilinos-autotester commented Aug 21, 2023

trilinos-autotester commented Aug 21, 2023

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_clang-11.0.1

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_python3

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_intel-2021.3

Jenkins Parameters

Using Repos:

trilinos-autotester commented Aug 22, 2023

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_clang-11.0.1

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_python3

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_intel-2021.3

Jenkins Parameters

trilinos-autotester commented Aug 22, 2023

trilinos-autotester commented Aug 22, 2023

trilinos-autotester commented Aug 23, 2023

trilinos-autotester commented Aug 24, 2023

srajama1 left a comment

Choose a reason for hiding this comment

srajama1 Aug 24, 2023

Choose a reason for hiding this comment

srajama1 Aug 24, 2023

Choose a reason for hiding this comment

srajama1 Aug 24, 2023

Choose a reason for hiding this comment

jgfouca Aug 24, 2023

Choose a reason for hiding this comment

srajama1 Aug 24, 2023

Choose a reason for hiding this comment

srajama1 Aug 24, 2023

Choose a reason for hiding this comment

lucbv left a comment

Choose a reason for hiding this comment

trilinos-autotester commented Aug 24, 2023

trilinos-autotester commented Aug 24, 2023

trilinos-autotester commented Aug 24, 2023

trilinos-autotester commented Aug 24, 2023

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Jenkins Parameters