Configuration/state information is not managed correctly when the library is upgraded (TZ-433)

[nomis]

Answers checklist.

• I have read the documentation ESP Zigbee SDK Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) and ESP Zigbee libs (esp-zboss-lib and esp-zigbee-lib) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v5.3-dev-422-ga7fbf452fa

esp-zigbee-lib version.

1.0.5

esp-zboss-lib version.

1.0.5

Espressif SoC revision.

ESP32-C6

What is the expected behavior?

When I upgrade the library from 1.0.0 to 1.0.5 on an existing device, it should continue to work correctly with the existing configuration and state.

What is the actual behavior?

The group cluster state (and possibly other information) is now wrong because the format of the data stored in flash has changed in some way that isn't managed correctly.

Steps to reproduce.

1. Set up a device with 1.0.0 and add it to some groups (Group membership incorrectly reported for all endpoints (TZ-249) #83)
2. Upgrade to 1.0.5
3. Observe that the group information is now wrong

More Information.

Before upgrade:
2023-10-18 12:25:02.578 DEBUG (MainThread) [zigpy.zcl] [0x3828:11:0x0004] Decoded ZCL frame: Groups:get_membership_response(capacity=5, groups=[0x0006, 0x0005, 0x0002])

After upgrade:
2023-11-25 20:20:00.202 DEBUG (MainThread) [zigpy.zcl] [0x3828:12:0x0004] Decoded ZCL frame: Groups:get_membership_response(capacity=29, groups=[0x0006, 0x04A6, 0x07FF])

It looks like groups 0x0002 and 0x0005 have become 0x04A6 and 0x07FF.

This was probably caused by the increase in maximum number of supported groups. That will be useful but it's inconvenient when it breaks the existing group configuration.

[nomis]

I can't remove group 0x07FF. Nothing I try (including adding it to all the endpoints and then removing it) works.

Filling up all the available group table entries on one of the endpoints (including one that is group 0x07FF) and then removing all of them worked.