Deleting a single event from recurring events does not work correct

[bege10]

Hello,
I sync contacts and calendars on my Android phones with Thunderbird on my Windows computer via WiFi and a local Radicale server on the computer. Thus changes on my phones are not synced immediately but only when I connect them with my computer when I am at home. Thunderbird is always connected to the Radicale server when the computer is running.

When I delete a single event of a recurring event on a phone and later sync it with the computer this single event does not disappear from Thunderbird. When I sync the phone again the deleted event returns on the phone.
I tested this while the phone was connected to the computer and synced it right after deleting such an event. The result is the same.

But the other way round, deleting a single event from a recurring event in Thunderbird and syncing it with the phone works correct.

What might be the cause of the issue and how can I fix it?

[bege10]

Can anybody confirm this issue when deleting a single event from a recurring event on Android?

I have /e/OS-T (based on LineageOS for microG, Android 13) on two phones, both with the same issue.

[rfc2822]

Does it only happen with Thunderbird?

Is it possible that the event actually gets modified on the server, like this?

1. Delete a single event from a recurring event on the phone
2. For some reason, event is modified on Radicale, for instance because another client has touched the event in the meanwhile or Radicalke changed the ETag
3. Sync with DAVx5
4. The deleted event is back on the phone (Seems like DAVx5 writes the state from the Radicale server to the phone instead of in the other direction – which would be the correct behavior if the event would have a modified ETag on the server)
5. Sync the Thunderbird calendar
6. Nothing changes in the Thunderbird calendar

[bege10]

...

2. **For some reason, event is modified on Radicale, for instance because another client has touched the event in the meanwhile or Radicalke changed the ETag**

...

The respective events have definitely not been changed on Thunderbird in the meantime. What is the exact change you suspect in the instance on Radicale that nevertheless might have happened on Radicale that I can look for before I sync with DAVx5? What is the ETag?

Could you add an option to have DAVx5 ask which change to sync (server or client) if both sides have changed, as Thunderbird does? Maybe first in a debug version so that I get a notice if there is a change on both sides to get more info about this issue.

[devvv4ever]

The ETag is a very important mechanic to instantly see if a resource has changed. It is a hash stored both on the client and server for every resource. On successful upload from a client the server generates a new etag, which is stored with the changed/uploaded resource. It tells the client this etag, and DAVx5 saves it locally. On next sync these etags are compared. If the one on the server is not the one on the client DAVx5 downloads the resource, because "it seems to have changed" since the last time DAVx5 has dealt with this resource.

[bege10]

@devvv4ever Thank you for the information. Is the ETag set for the whole database or per entry?
It is very likely that changes have happened on both devices when I sync them, but never to the same entry. So yes, the ETag has probably changed. But that would not be a reason to restore the old state of a not changed entry.
But for such cases where an entry has changed on both sides again my question whether DAVx5 could do it the same way as Thunderbird does and ask which side to take - server or client.

I continue testing currently with a different calendar app on the Android device to see whether that shows different results.

[devvv4ever]

@devvv4ever Thank you for the information. Is the ETag set for the whole database or per entry?

Per entry

[bege10]

I edited the time of two instances of a recurring event with a different calendar app and the problem remains.
After syncing the next day one instance was reverted, one instance is displayed twice: the original recurring instance and the changed instance.
Then I edited the reverted instance again and deleted the double instance and synced again right away. Now both are displayed correct on both sides.
What can I do to narrow down the problem? Where and how can I examine the ETag in Android and Windows?

[rfc2822]

Can you set the radicale log level to the maximum and send us the Radicale log that is written while you reproduce the problem?

[bege10]

Obviously I forget to save my answer. Once again:
I could not configure the server log. I run Radicale with a NSSM service. There a log file is defined. I tried to set log configuration according to the Radicale manual but then the service starts with an "unknown error". I will send a new log via e-mail. If that is not enough, please, tell me how to configure the Radicale log.

[Jero6090]

Hi There thanks for working on that issue. Just like to mention that I experiance same without synchronization just getting recurring event (send by outlook) ->Fine.... If one event out of series is canceled.... then firstly its deleted in calender =>fine.....BUT after restart Thunderbird it automatically returns in calender... -grateful for any solution- Greets
TB_version: 115.12.1 32Bit

[rfc2822]

Can you please provide exact steps to reproduce (so that I can reproduce your problem from scratch), the VEVENT, server version and verbose logs from DAVx5 (and the server)?

[rfc2822]

The logs show:

18:17:04 Event created (PUT) by DAVx5 → 201, State A
18:17:25 Event modified (PUT) by Thunderbird → 201, State B
18:19:26 Attempt to modify event that is expected to be in State A (PUT) → 412 because event is already in State B, so this is a conflict, and the server wins → DAVx5 downloads event in State B

So there was a write access from Thunderbird to the event that was not yet synchronized to DAVx5 before changes on the mobile phone were made. This is called a conflict and in case of a conflict, DAVx5 lets the server win (to guarantee consistency on the server).

Please make sure to sync the latest changes from the server with DAVx5 before editing an event that has been changed on the server in the meanwhile.

[bege10]

Great you found that out. Thank you!
The thing is I did not edit these events in Thunderbird.

1. I created a new event on the phone
2. I synced with DAVx5
3. I synced the Thunderbird calendar
4. I edited an instance on the phone
5. I synced with DAVx5

That means that Thunderbird marked the new event as modified when it only read it from the server and created it in the Thunderbird calendar.
Do you think this is a Thunderbird or a Radicale issue? How can I find that out?

[rfc2822]

I have tried with Thunderbird and couldn't reproduce the behavior. Maybe you have an extra extension that does things with events in the background? Or did you set attendence information or something like that?

[bege10]

What is "attendance information"?

These are the "extras" here:

• An old issue with DAVx5 is that I must sync twice in a row to transfer all changes to the server. In the other direction once is enough.
• Because of this issue on the computer I have a background process running (NirSoft's FolderChangesView) that gives a sound if a change has happened to any of the Radicale directories. That "rings" at the first two syncs. At a third sync it remains silent. Afterwards I manually sync with Thunderbird.
• When I close a reminder notification of Thunderbird this process also rings. Does Thunderbird mark this event as changed then? All the affected events have reminders. But when I did the tests no reminder popped up. So there must be something else.

[rfc2822]

What is "attendance information"?

On a group-scheduled event (with more than 1 attendee), you can set whether you participate in the event. This info also modifies the event.

These are the "extras" here:

* An old issue with DAVx5 is that I must sync twice in a row to transfer all changes to the server. In the other direction once is enough.

That should definitely not be the case. I didn't hear of that yet. This could be related to your problem, as it is a thing that is not normal.

Are you using the latest version of DAVx5?

* When I close a reminder notification of Thunderbird this process also rings. Does Thunderbird mark this event as changed then? All the affected events have reminders. But when I did the tests no reminder popped up. So there must be something else.

Yes, such things can modify the event (and thus change the ETag).

[bege10]

What is "attendance information"?

On a group-scheduled event (with more than 1 attendee), you can set whether you participate in the event. This info also modifies the event.

I don't use that feature, there are no attendees.

These are the "extras" here:

* An old issue with DAVx5 is that I must sync twice in a row to transfer all changes to the server. In the other direction once is enough.

That should definitely not be the case. I didn't hear of that yet. This could be related to your problem, as it is a thing that is not normal.

It was discussed quite some time ago without a solution. It should be solved with a change in sync process of DAVx5 but was not.

Are you using the latest version of DAVx5?

Yes, currently 4.4.2-ose

* When I close a reminder notification of Thunderbird this process also rings. Does Thunderbird mark this event as changed then? All the affected events have reminders. But when I did the tests no reminder popped up. So there must be something else.

Yes, such things can modify the event (and thus change the ETag).

Would you, please, test and see whether you can reproduce the issue with recurring events with reminders being active between syncs?

[bege10]

Thanks a lot! I will create an issue at Thunderbird. But with respect to my experience with solving Thunderbird issues it will take very long. And perhaps there are Thunderbird's internal reasons why this behavior shall not be changed.

Thunderbird would "forget" that the user has already acknowledged the event and show a popup again.

Indeed, I saw this behavior when I edited a respective event on the phone again.

I never got an answer here at DAVx5 to my repeated question whether the app could get a feature Thunderbird has, which is to ask which change shall win if there are changes on both sides. I always read "server always wins". Maybe most users will not need a different behavior because they always sync online instantly. This feature - if activated in app settings - would leave it to the user how to deal with such situations.
Thanks again.

[rfc2822]

Thanks a lot! I will create an issue at Thunderbird. But with respect to my experience with solving Thunderbird issues it will take very long. And perhaps there are Thunderbird's internal reasons why this behavior shall not be changed.

Yes, it makes sense that they store this information in the iCalendar. I wouldn't consider this as a Thunderbird bug.

I never got an answer here at DAVx5 to my repeated question whether the app could get a feature Thunderbird has, which is to ask which change shall win if there are changes on both sides. I always read "server always wins". Maybe most users will not need a different behavior because they always sync online instantly. This feature - if activated in app settings - would leave it to the user how to deal with such situations.

Well it would change the way DAVx5 works fundamentally, so this can't be implemented "along the way". Currently we don't have resources to work on this. Also, "client wins" would in your case drop the x-moz-lastack and Thunderbird would forget that you have clicked on "Close". So we would just have "moved" the problem from one property to another.

I think there's no real solution in CalDAV for this except:

1 Synchronize in DAVx5 before you edit such events (events with reminders that have been changed by Thunderbird in the meanwhile) on the phone. This is the most simple way to deal with that.
2. We're currently trying to bring CalDAV-Push to DAVx5 (and servers). This would allow DAVx5 to synchronize the event as soon as it's changed on the server – in this case as soon as you click on "Close" in Thunderbird. If you then edit the event on the phone, it should work. Basically the same option as 1., but automatically.
3. Some option of "merging" iCalendars like vCard PID merging – although there's no such thing for iCalendar as far as I know, and no client uses it.

So I don't really have something better to offer but I'm open to ideas.

[bege10]

Well it would change the way DAVx5 works fundamentally, so this can't be implemented "along the way". Currently we don't have resources to work on this. Also, "client wins" would in your case drop the x-moz-lastack and Thunderbird would forget that you have clicked on "Close". So we would just have "moved" the problem from one property to another.

I think there's no real solution in CalDAV for this except:

1 Synchronize in DAVx5 before you edit such events (events with reminders that have been changed by Thunderbird in the meanwhile) on the phone. This is the most simple way to deal with that. 2. We're currently trying to bring CalDAV-Push to DAVx5 (and servers). This would allow DAVx5 to synchronize the event as soon as it's changed on the server – in this case as soon as you click on "Close" in Thunderbird. If you then edit the event on the phone, it should work. Basically the same option as 1., but automatically. 3. Some option of "merging" iCalendars like vCard PID merging – although there's no such thing for iCalendar as far as I know, and no client uses it.

So I don't really have something better to offer but I'm open to ideas.

Thank you for this verbose explanation. Option 1 and 2 will work if I always remember to first sync (option 2: connect) the phone with the server before I start Thunderbird. That seems not to be too high an expectation. But with respect to my own experience... 🙄
I'd prefer option 4 as Thunderbird does: When DAVx5 notices changes for an event on both sides - server and client - ask the user for each affected event which side shall win. In my case this is easy to answer because I rarely edit events in Thunderbird but only create new ones.
If this feature needs to be activated in the settings it will not kill the nerves of users who don't need it.

[rfc2822]

What do you mean with "connect" in option 2? You wouldn't have to do anything as soon Push is set up once.

Option 4: But then Thunderbird would then still forget that you have already acknowledged the event (if you let DAVx5 win)…

[bege10]

What do you mean with "connect" in option 2? You wouldn't have to do anything as soon Push is set up once.

The Radicale server is on my notebook and this is running only when I work with it. The server and the phone are connected via my Windows hotspot, not via Internet. If I edit a recurring event on my phone while I am not at home Thunderbird will not get that information at home right away. I wonder which push notification will be first when Thunderbird starts: The change on the phone or the immediate change when Thunderbird shows the reminder.

Option 4: But then Thunderbird would then still forget that you have already acknowledged the event (if you let DAVx5 win)…
You are right. 🤔
This will only be the case if I forget to sync the phone before starting Thunderbird. And a second reminder is less bad than a wrong event.