MACRO-calibration incomplete cleanup for set("cat_node")

[OFR-IIASA]

When applying the function add_macro to a scenario, the set cat_node is not removed, prior to being re-populated.

The issue in more detail:

• An R11 model version, which has already been calibrated to macro, is split to a new region version R17.
• The regions CPA, FSU, PAO, PAS have been split into new regions.
• The 4 above regions remain in the scenario as these still have the land-use emulator implemented, as GLOBIOM doesnt have all the new regions. The regions have NO energy-system elements and therefore no demands; therefore they also DO NOT need to be calibrated to macro.
• The macro-input excel file has NO mention of the 4 regions listed above; they are however still part of the set cat_node.
• Calling the calibration process via scen.add_macro() results in UNDF values for the 4 regions, e.g. for demand_scale above and calibration fails. This will be logged in the file MACRO_run.lst.

How to resolve:

• A current work around is to remove the set cat_node prior to sunning the macro calibration process.

with scen.transact("Remove set cat_node"):
    df = scen.set("cat_node", filters={"type_node": "economy"})
    scen.remove_set("cat_node", df)

A possible fix can be added here in the calibration process

Versions

ixmp:        3.6.1.dev38+g43562c1
     43562c1 (HEAD, upstream/main) Merge pull request #466 from iiasa/reuse-workflows
message_ix:  3.6.1.dev49+g3b6651f.d20221205
     7d8ac40 (HEAD, upstream/issue/723, issue/723) Revise duality test setup

[khaeru]

Thanks for the detailed explanation. This is one clear and specific real-world case of #320. (There may be others, but I think it is hard to hypothesize what those might be, and maybe better that we identify them one-by-one like this and fix them as they come.)

I agree that add_structure() would be the right place for the fix, and it would also be simple to write a test:

1. Create a base scenario.
2. Add some problematic entries to cat_node.
3. Run .add_macro().
4. Check that the problematic entries are removed so the process completes successfully.