Importing event without UID makes khal crash

[doronbehar]

I exported an event and a todo list item using calcurse to ics format with the following command:
calcurse --export > mytest.ics
It created the following:

BEGIN:VCALENDAR
PRODID:-//calcurse//NONSGML v4.2.2//EN
VERSION:2.0
BEGIN:VEVENT
DTSTART:20170421T220000
DURATION:P0DT1H0M0S
SUMMARY:test calendar item created in calcurse
END:VEVENT
BEGIN:VTODO
PRIORITY:0
SUMMARY:New todo list item created with calcurse
END:VTODO
END:VCALENDAR

And running khal import -r mytest.ics gives me the following error

Traceback (most recent call last):
  File "/home/doron/.local/bin/khal", line 11, in <module>
    load_entry_point('khal==0.9.6.dev1+g72e6761', 'console_scripts', 'khal')()
  File "/home/doron/.virtualenv/khal/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/doron/.virtualenv/khal/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/doron/.virtualenv/khal/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/doron/.virtualenv/khal/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/doron/.virtualenv/khal/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/doron/.virtualenv/khal/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/doron/.virtualenv/khal/lib/python3.6/site-packages/khal/cli.py", line 443, in import_ics
    env={"calendars": ctx.obj['conf']['calendars']},
  File "/home/doron/.virtualenv/khal/lib/python3.6/site-packages/khal/controllers.py", line 546, in import_ics
    vevents = utils.split_ics(ics, random_uid, conf['locale']['default_timezone'])
  File "/home/doron/.virtualenv/khal/lib/python3.6/site-packages/khal/utils.py", line 616, in split_ics
    events_grouped[item['UID']].append(item)
  File "/home/doron/.virtualenv/khal/lib/python3.6/site-packages/icalendar/caselessdict.py", line 45, in __getitem__
    return super(CaselessDict, self).__getitem__(key.upper())
KeyError: 'UID'

[geier]

The issue here is that the VEVENT you are trying to has no UID which is required by relevant RFC (RFC 5545) and the icalendar library we are using here and there really is no sensible way for us in khal to work around that.

I have opened an issue for icalendar, but don't expect icalendar to accept this any time soon, if at all.

I would suggest opening an issue with calcurse as well, this might lead to faster results.

As long as you only want to import this single event, you might want to insert a line like this UID:somethingunique in between the start and end of the event.

Also, thank you for the well formed issue and I'm sorry for not being able to help you.

[doronbehar]

What about the -r flag? I read it's documentation and I understand it doesn't suppose to help with a situation like this but perhaps we should change it's behavior so that it will create a random UID if it doesn't exist?

[geier]

The problem lies with the underlying icalendar library, which throws an exception when it encounters a vevent without a uid. This cannot be fixed without changing icalendar (or building or using another icalendar parser).

[amandabee]

Just wanted to add, since this helped me solve my problem, that it does work to add a random UID (I used my password generator) but if you're importing a flight confirmation there might be two events in a single .ics which might necessitate adding two UIDs to the .ics file.

Then khal import Downloads/flight_info.ics works great.

[dopefishh]

I've encountered the same problem and used the following bash script to automatically add UID's to ics files.

#!/bin/bash
set -e
while read line
do
	echo $line
	if [ "$line" = "BEGIN:VEVENT" ]
	then
		echo "UID:$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 36)"
	fi
done

[madduck]

@dopefishh for next time:

sed -e "/^BEGIN:VEVENT$/aUID:$(uuid 2>/dev/null || uuidgen 2>/dev/null || python -c 'import uuid; print(uuid.uuid1())')'

scnr

[WhyNotHugo]

I'd encourage you to reach out to whomever sends those iCalendar files, and ask for them to be fixed.

One of the issues of just automatically adding a UID, is that it's impossible to detect duplicates, or updates to a same event (eg: if the UID is the same, it's an update, if it's a new one, it's a new event).

Also, no harm is asking for them to actually generate valid iCalendar files.

[madduck]

@WhyNotHugo good luck, I've tried. Sites, such as Eventbrite, don't even bother to reply. That is if you find a way to even get in touch with them.

[WhyNotHugo]

It might make sense to use the file's sha256 as a UID for these cases.

It'll avoid duplication if it gets imported more than once. If you import an updated version though, you'll get a duplicate, but tough luck, that's what UIDs are for.

[madduck]

That sounds like a good idea, thanks!

[Ram-Z]

Could khal at the very least catch the error and provide some useful error message rather than crash?

I got this message on the very first event I wanted to import using this software and seeing it crash is not great user experience. A message stating that the file is invalid would be a lot better.

[WhyNotHugo]

Isn't this fixed by #1082 ?

[Ram-Z]

Ah possibly, looks like it should have.

Sorry. I just found this issue with no recent activity suggesting recent changes, so I didn't bother checking HEAD, latest release doesn't have #1082. My bad.

[WhyNotHugo]

No worries, forgot to update this issue anyway.

Can you confirm that it works fine for you on master? If so, I'll go ahead and close this.

[Ram-Z]

Yeah, seems to work. Thanks.