This directory contains tutorials and examples of Cabana (https://github.com/ECP-copa/Cabana) for Fortran.
Explanations of using the basic data structures and parallel capabilities of Kokkos and Cabana with Fortran
- Hello World
- Tuples
- Struct-of-Arrays
- AoSoA
- Parallel for
- Kokkos::parallel_for
Kernels that can be readily adopted for users wanting to build a Fortran PIC code or incorporate Cabana kernels into an existing Fortran PIC code.
- parallel_for
- Push
- Scatter
- Unmanaged aosoa
This file contains all of the Kokkos/Cabana code, which is in C++. It contains an macro enabling usage of the Kokkos parallel_for both generally and specifically to a particle AoSoA. Users must specify the particle properties here.
This include file contains macros used in the Fortran code:
ATTR_DEVICE
: Adds "attributes(device)" in the GPU version to specify execution space. Other Cuda options available:ATTR_HOST, ATTR_GLOBAL, ATTR_SHARED, ATTR_CONSTANT, ATTR_PINNED
COPY_TO_DEVICE(err,icount,d_array,h_array)
: Copies from host to device if GPU version, otherwise simple copyCOPY_TO_HOST(err,icount,h_array,d_array)
: Copies from device to host if GPU version, otherwise simple copyCABANA_COUNT_KIND
: Variable type needed for icount in the above macrosFCABANA_DEVICE
: Specifies that a variable or array is on the device if GPU versionCABANA_DEF_THREAD_NUM
: DeclaresOMP_GET_THREAD_NUM
in OpenMP versionCABANA_GET_THREAD_NUM(x)
: Gets OMP thread numberCABANA_ADD(x,y)
: Normal addition if on CPU; AtomicAdd if on GPU.CABANA_DEF_MAX_THREADS
: Declates OMP_GET_MAX_THREADS in OpenMP versionCABANA_GET_MAX_THREADS(x)
: Gets OMP number of threadsPARTICLE_OP_INTERFACE(C_FUNC)
: Interfaces to the corresponding C++ routine incabana_cpp_interface.cpp
orparticle_ops.h
MISC_OP_INTERFACE(C_FUNC)
: Interfaces to the corresponding C++ routine inmisc_ops.h
Contains Fortran wrappers for Cabana/Kokkos C++ routines:
cabana_initialize
: Cabana::initializecabana_finalize
: Cabana::finalizekokkos_fence
: Kokkos::fence
Users must specify the particle properties here and they must correspond to those in cabana_cpp_interface.cpp. The particle initialization and some utilities are also here.
This is a list of particle operations. In these operations, the fortran kernel receives the particle data, number of structures in the SoA, and the parallel_for index as arguments. Users can add their own operations to the list and easily create custom kernels by analogy with existing ones.
An example grid module that a user might have.
The user-specified physics of the push.
The push kernel. Currently does nothing interesting besides "subcycling" - performing the push multiple times per time step. Could be expanded to include multiple algorithms to choose from.
The scatter kernel. Currently decides which cell to write to arbitrarily, and deposits a "1" in the specified cell.