-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simulate prod test #875
Simulate prod test #875
Conversation
Signed-off-by: Victor Barbosa Martins <victorbarbosamartins@yahoo.com.br>
Signed-off-by: Victor Barbosa Martins <victorbarbosamartins@yahoo.com.br>
Signed-off-by: Victor Barbosa Martins <victorbarbosamartins@yahoo.com.br>
I've looked into
and files (although very small ones are generated for me at The log files look reasonable and it runs through without errors. There are issues, many statements like
in the log file. |
Hi @GernotMaier, thanks for checking. This also works for me when running pytest in the container as you suggested. However, it does not work when running the application in the terminal (even in the master branch):
Could you please try to reproduce this? |
Also the file you pointed out that exists is already being checked for in the integration test in this branch, which lead to a failure in the test. I am not entirely sure what is going on. https://github.com/gammasim/simtools/actions/runs/8723725721/job/23932598219?pr=875#step:5:554 |
Signed-off-by: Victor Barbosa Martins <victorbarbosamartins@yahoo.com.br>
@VictorBarbosaMartins , I can run Are you sure you are using a recent installation of simtools? (there are some warnings regarding observation levels, which are related to something else) |
@VictorBarbosaMartins - everything runs fine with me locally without any errors. Can you check again? Also please check the integration test which is failing on github ( |
Hi @GernotMaier. The problem is not visible at first, one has to dig in. Naturally there is no error printed out, and that is the problem(!) The application runs smoothly (
To worsen even more, I found out that the integration test of the output file ( I will look into these issues now but I will need help with the issue with the parameters. Please follow my suggestion above and check the corsika log file. |
And this error is related to the other one. Here the simtel file is not produced, and an attempt to pack the output for registering on the grid fails. Given that this was working before, I assume that the error is not on the application/module but rather on the fact that simtel/corsika is not running properly. It is better to fix the error with running corsika/simtel_array before checking this failing integration test. I am looking into this rn. |
The message above is quite clear on where the errors are coming. Wrong names of parameters. I also could not find those parameters in the DB. I am convinced these names were added/changed in the last updates (note also at the effective focal length error, which we also just changed the code there): Unknown configuration item 'site_config_name'. I found some of these variables in the They seem not to be part of the model and are written to the config file anyways. Do you see any straightforward solution @GernotMaier ? |
The command How is your setup? I am running it in a container (podman or docker; doesn't matter):
With a container from Not sure what else we can check. |
Signed-off-by: Victor Barbosa Martins <victorbarbosamartins@yahoo.com.br>
Signed-off-by: Victor Barbosa Martins <victorbarbosamartins@yahoo.com.br>
Ok, I think we are slowly clarifying things here. I fixed the |
simtools/applications/production.py
Outdated
for key, value in primary_data.items(): | ||
if key in ["showers", "array"]: | ||
continue | ||
config_showers[primary][key] = value | ||
config_arrays[primary][key] = value | ||
|
||
# Grabbing showers entries | ||
# showers entries |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shower entries
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed this typo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All good. One minor comment.
Signed-off-by: Victor Barbosa Martins <victorbarbosamartins@yahoo.com.br>
Signed-off-by: Victor Barbosa Martins <victorbarbosamartins@yahoo.com.br>
# Conflicts: # tests/resources/prod_multi_config_test.yml
@tobiaskleiner, same here :D I guess we are ready to go? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All good now. Thanks!
Thanks again! |
Signed-off-by: Victor Barbosa Martins <victorbarbosamartins@yahoo.com.br>
I did some additional changes and will, therefore, wait until someone has another look. Thanks! |
Merging here as well as this is approved and it is complementing the #906 |
Done:
Here I am including some missing tests to check whether the files for either corsika or simtel_array output were generated by the production tool. We are constantly breaking this tool because the tests are not checking whether the output files exist. The integration tests for simulate_prod.py will fail at the moment.
I removed the test flag for
sim_showers_for_trigger_rates
because it was preventing us to simulate and de facto test the tool. To minimize computational time of the simulation, I reduced the number of events to 2.To do:
The best would be to make the corrections to the tool before merging this and have the integration tests passing.