-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mirage has an issue with the linearized dark names when "segmentation" is done. #652
Comments
Can you send me the yaml file you were using for this? I just ran a NIRISS imaging case with 2 integrations and 10 groups with the raw dark you mentioned, and Mirage ran successfully. I thought that the results you are seeing were because you're starting with a raw dark and when that is processed, it is segmented because the version of the code you are using still has the extra low segmentation threshold. But I just tried running that scenario as well and Mirage finished successfully. |
You can see the .yaml files where this issue happened in directory /ifs/jwst/wit/niriss/kevin/new_car1094_simulations. The files where the problem happened are jw01094001002_01101_00061_nis.yaml jw01094003002_01101_00061_nis.yaml |
I just ran jw01094001002_01101_00061_nis.yaml with the file splitting threshold low enough to segment the file, and Mirage ran successfully. In this case, self.linDark was set to the list of dark files here: https://github.com/spacetelescope/mirage/blob/master/mirage/imaging_simulator.py#L85 And then seed_dict had the expected mapping of seed image files to dark files. |
OK so I am not clear about what was wrong in my run that caused the issue. I do not see a file splitting threshold in the .yaml file. Where is this set? |
The file splitting limit is set in constants.py. It's large enough (160 full frame groups) that basically only TSO observations should be split. https://github.com/spacetelescope/mirage/blob/master/mirage/utils/constants.py#L100 And in fact I just learned that in flight, only TSO mode observations can possibly be split. So it's not clear to me why your run failed. But if you update the version of the code you are using, you should get the large file splitting limit, which will then mean that your observations won't be split. |
Which version of Mirage did you use to get it to work? When I looked at the constants.py file for my installation of 30 March 2021 the splitting limit was set to 38.*2048.*2048, but I have run things with 2 times 20 frames successfully including some of the runs done in the group in the directory where I ran into the issue. Or did you need to chance constants.py to get it to work? |
I merged the increased file splitting limit on Feb 12. For the tests today I brought the limit down to 2020482048 in order to make sure the files would be split. But you should see a limit of 16020482048 for code from 30 March. If you're installation is using a local copy of the code, then you can be sure that you have an updated version by going into the directory where the code is and 1 .Be sure your mirage environment is activated
Assuming you originally installed the code using |
I just changed the value manually. But I am puzzled as to why the file had a value of 38*2048.*2048. I certainly did not edit the file. Something must have gotten messed up in the installation somehow. This is rather curious. But I just ran one of the models successfully so that is good. |
When running a Mirage model for NIRISS imaging with 2 integrations in the exposure the Mirage code crashes. The issue is that the code creates two linearized dark files
jw01094001002_01101_00061_nis_uncal_seg001_linear_dark_prep_object.fits
jw01094001002_01101_00061_nis_uncal_seg002_linear_dark_prep_object.fits
from the original dark file
/ifs/jwst/wit/mirage_data/niriss/darks/raw/NISNIRISSDARK-172500017_15_496_SE_2017-09-07T05h28m22_dms_uncal.fits
but in the obs_generator.py routine when the code line 1212
seed_files = seed_dict[linDark]
is executed it has an empty list and then in the statements
if isinstance(seed_files[0], str):
print('\nSeed files:')
print(seed_files)
the code crashes because seed_files[0] is out of range for the indexing.
I inserted a few print statements into the code at this point
and the log plus the error messages look like this:
Creating output file name with segment number.
Diagnostic: jw01094001002_01101_00061-seg001_nis_uncal.fits
Diagnostic: /ifs/jwst/wit/niriss/kevin/new_car1094_simulations/jw01094001002_01101_00061_nis_uncal_seg001_linear_dark_prep_object.fits
Diagnostic seed_dict: {'/ifs/jwst/wit/niriss/kevin/new_car1094_simulations/jw01094001002_01101_00061_nis_uncal_seg001_linear_dark_prep_object.fits': [], '/ifs/jwst/wit/niriss/kevin/new_car1094_simulations/jw01094001002_01101_00061_nis_uncal_seg002_linear_dark_prep_object.fits': []}
Diagnostic: []
Traceback (most recent call last):
File "./runmodels.py", line 64, in
t1.create()
File "/home/kvolk/anaconda3/envs/mirage_30march2021/lib/python3.7/site-packages/mirage/imaging_simulator.py", line 82, in create
obs.create()
File "/home/kvolk/anaconda3/envs/mirage_30march2021/lib/python3.7/site-packages/mirage/ramp_generator/obs_generator.py", line 1183, in create
if isinstance(seed_files[0], str):
IndexError: list index out of range
The seed_dict variable does not have the file names in it as expected.
One can simply force the seed_dict values to equal the key names, but I hesitate to do this without knowing what the seed_dict variable is actually supposed to be doing.
The text was updated successfully, but these errors were encountered: