Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

differences in NaN's in pipeline after rearranging correction order #80

Open
falkben opened this issue Sep 28, 2022 · 1 comment
Open

Comments

@falkben
Copy link
Member

falkben commented Sep 28, 2022

After rearranging the steps to move the corrections before summing (see: #61), we observe differences in NaN's in the outputs. Mostly these are differences in the edges and, when run with a large dataset, are not extremely noticeable. With a small amount of data (a few observations) the differences are amplified. Below, I discuss differences with a large dataset (NGC628).

A few potential causes:

  • when working on individual frames instead of the summed frames, there's less signal in each frame, which leads to the corrections containing more zeros or NaN's
  • the corrections are done normalized frames and then converted back to counts for summing, before being normalized again. And being done on frames instead of summed areas, the additional normalization calculation might lead to frame data being turned into NaN's by dividing by zero.

Summary of the differences per data type:

  1. (Primary) Slight difference along edges. There are fewer NaN's in the rearranged step order (signal goes further out to the edge).
  2. (Coincidence Loss Correction Factor) There are some additional NaN's, especially around the edges, in the rearranged step order
  3. (coincidence loss correction uncertainty) There are some additional NaN's around the edges in the rearranged step order
  4. (zero point corr. factor) Additional NaN's around edges
  5. (Poisson noise) additional NaN's around edges.

Example (zero point corr. factor, zoom 1)

Left is before rearrange steps and right is after rearrange steps

image

@mdecleir
Copy link
Collaborator

For this issue, we also need to check how masks are being used in every step in the pipeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants