-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BiCGStab OMP issue #1563
Comments
From the code you linked, I could see that you are using a block Jacobi preconditioner, with block size 32 for OMP and reference. Have you tried using a block size of 1 and see if any differences appear? Nevertheless, it could still hint at some issue in our OMP implementation. |
It is indeed fixing the issue. |
I've run our benchmarks with the provided matrices. If I used the same residual reduction as in your code
Perhaps just for clarification, by specifying only the reduction factor, the iteration will be stopped when Interestingly, the difference disappears if |
Ok thank you, I indeed though I was using an absolute norm critera, whereas the "residual reduction" terminology was quite clear. I let you close the issue depending if you think it would require an improvement from your side or not. Thank you for the quick answer. |
@MarcelKoch how did you "run your benchmarks with the provided matrices" ? I see you have some benchmark using SuiteSparse but I see no way to use local data. Should I upload my matrix in SuiteSparse ? You did it already ? |
No I didn't upload anything and it's not necessary to run our benchmarks. Our benchmarks are only documented as part of reproducibility for some paper. How to use them individually is not really documented. To give you a broad summary: First you need to define the input data in a json file. The json file has to consist of exactly one array, and within that array the test cases are defined. For example:
The files have to be in matrix market format. Some benchmarks require extra some extra fields. For example the solver benchmarks requires the field Then you can call a benchmark and pass in the input via stdin, i.e.
You can get the options that are available for a given benchmark by using The output of our benchmarks is again json, and it is printed to stdout, while our status messages are printed to stderr. So, you can store the output with
|
Great. It will be very usefull. So, coming back to the current issue, stop using the preconditionner indeed solved the problem with the matrices I provided at first, but we still have some difficulties with OMP. So preconditionner was just part of the problem. I made a minimal reproducer for it: https://github.com/blegouix/gko_omp_reprod/tree/main Note that using the benchmarks on the matrices does not reproduces the problem, maybe because of the lower precision of numbers in the |
@MarcelKoch sorry to ping you but can you confirm you have seen my message ? As I did not open a new issue and this one is quite old |
If you apply the following patch and export your files using diff --git a/benchmark/solver/solver_common.hpp b/benchmark/solver/solver_common.hpp
index 19e718e08..541a9f662 100644
--- a/benchmark/solver/solver_common.hpp
+++ b/benchmark/solver/solver_common.hpp
@@ -288,7 +288,7 @@ struct SolverGenerator : DefaultSystemGenerator<> {
{
if (config.contains("rhs")) {
std::ifstream rhs_fd{config["rhs"].get<std::string>()};
- return gko::read<Vec>(rhs_fd, std::move(exec));
+ return gko::read_generic<Vec>(rhs_fd, std::move(exec));
} else {
gko::dim<2> vec_size{system_matrix->get_size()[0], FLAGS_nrhs};
if (FLAGS_rhs_generation == "1") { |
Great. I can now reproduce the problem in your benchmark:
I also have updated the reproducer to export the binary file. We have noticed the problem looks absent with Gmres |
Hi @blegouix, I just wanted to let you know that I'm able to reproduce your issue with the files you provided, but I might not find time this week to investigate this further. |
Hello, is there any progress on this issue ? |
Hello,
This issue is the continuation of #1561 with more details. We use BiCGStab to solve a quasi-tridiagonal pds problem with different executors (Serial, OMP and CUDA). We observe a numerical issue with OMP. In our test bench build upon those solvers (which involve more than just Ginkgo) we evaluate an error to
1e-16
with Serial or CUDA, but~1e-10
with OMP even withOMP_NUM_THREADS=1
.The problem disappears just by replacing the OMP executor with a Serial one.
I link the matrix and the rhs which is problematic. Note that the size of the matrix has an impact (no problem with n=10 or n=100 ie.)
csr.txt
rhs.txt
Please let me know if it is enough for you to reproduce the problem.
I tried with gcc and clang, with Ginkgo 1.7.0 and current develop branch.
Valgrind does not reveals anything and both cases (with serial or omp executor) "converge" in 3 iterations.
Issue from our side: CExA-project/ddc#332
Problematic ginkgo apply call (all ginkgo stuff is in this file): https://github.com/CExA-project/ddc/blob/cc2942283213cc700b3bd8c09fb00346621997e3/include/ddc/kernels/splines/matrix_sparse.hpp#L207
(
create_gko_exec<Kokkos::Serial>()
is agko::ReferenceExecutor::create();
whilegko_exec
is agko::OmpExecutor::create();
.)Regards
The text was updated successfully, but these errors were encountered: