Crossmatch produces duplicates in catalog_creation.get_all_catalogs()

[vandalt]

Hello!

I'm not sure if this is a known issue, but the get_all_catalogs() sometimes seems to return duplicates of the same target.
I noticed it when trying to simulate GTO program 1200. 3 of the 4 targets show up twice in the catalogue (and in each case they are the only target in the FOV of the SUB80 subarray).

In this specific case, it is easy to manually verify that the targets are indeed duplicates based on the IDs in each survey and to manually edit the catalogue, as there is only one target per catalogue. It might be less straightforward to fix when generating catalogues for a wider FOV (e.g. using the "FULL" NIRISS array).

For two of the targets (HD 95086, HD 115600), the WISE-Gaia d2d is smaller than 0.4, but the "DESIGNATION" match fails. I think this is because the uppercase DESIGNATION in gaia_wise_crossref is the WISE name and Gaia is lowercase.

For these targets, the 2MASS-Gaia crossmatch also fails, but because the distance is greater than the 0.5 arcsec threshold.

For HR 8799 (HD 218396), there is no 2MASS target found, and the WISE-Gaia crossmatch fails because d2d is greater than 0.4 arcsec (~ 0.6).

Potential solutions

From what I understand the solution for the first two WISE duplicates is to change gaia_wise_crossref['DESIGNATION'] to gaia_wise_crossref['designation'] in line 1435 of create_catalog. This fixes it for my use case. However, it still generates a second entry in the catalog (with exactly the same magnitudes and coordinates), potentially because of the failed 2MASS crossmatch?

For the cases where 2MASS or WISE does not match with Gaia because of separations, I would see the following options:

• Let users fix manually or with a separate script when it matters that a source is duplicated
  • (And potentially issue a warning if the match fails but IDs are the same)
• Skip the "distance" match if the IDs/designations match (are there cases where the IDs could match with the entries corresponding to different targets?)
• Make the distance match more flexible (e.g. allow wider separations, but verify the star used is the nearest neighbour and has the same ID)?

I'm happy to open a PR with some of these suggestions (or another fix) if it can help.

Thank you!

Sample code used to debug

Click to unfold code

# %%
import os
from mirage.catalogs import create_catalog

catalog_dir = "catalogs_dbg/"
if not os.path.exists(catalog_dir):
    os.makedirs(catalog_dir)

# RA and DEC values taken from the pointing file
targets = {
    "HD-218396": [346.87040, 21.13393],
    "HD-95086": [164.26184, -68.66727],
    "HD-115600": [199.83099, -59.47246],
    "HD-93649": [161.70019, -69.20977],
}

# %%
# Run lower level functions on single target to check results
outra, outdec = targets["HD-218396"]
box_width = 5.5
filter_list = ["F480M", "F430M", "F380M"]
wise_catalog = "allwise"

# %%
(
    gaia_cat,
    gaia_mag_cols,
    gaia_2mass,
    gaia_2mass_crossref,
    gaia_wise,
    gaia_wise_crossref,
) = create_catalog.query_GAIA_ptsrc_catalog(outra, outdec, box_width)
twomass_cat, twomass_cols = create_catalog.query_2MASS_ptsrc_catalog(
    outra, outdec, box_width
)
wise_cat, wise_cols = create_catalog.query_WISE_ptsrc_catalog(
    outra, outdec, box_width, wise_catalog=wise_catalog
)

# %%
print(gaia_cat[["solution_id", "DESIGNATION", "ra", "ra_error", "dec", "dec_error"]])
print(gaia_wise)
print(wise_cat)
print(gaia_wise_crossref)

# %%
# Run on all targets and save to produce catalogs
for target in targets:
    ra, dec = targets[target]
    cat, mag_col_names = create_catalog.get_all_catalogs(
        ra, dec, box_width, filters=filter_list
    )

    source_for_fname = target.replace("-", "").lower()
    file_name = f"sources_{source_for_fname}.cat"
    file_path = os.path.join(catalog_dir, file_name)
    cat.save(file_path)