Add support for dynamic defaults

[lognaturel]

Add support for dynamic defaults by expanding what the exiting default column does. See the forum thread for discussion.

If the contents of the default column are not a literal value, the expression is passed through to the value attribute of a setvalue action triggered by the odk-instance-first-load event. In the case of nodes that are not in repeats, this action is added as a child of the model and ideally right after the bind for the node of the value it sets (this is not a hard requirement -- it can be anywhere in the model).

In the case of repeats, the action is added as a child of the repeat form control (in the body) and the triggering event is odk-instance-first-load odk-new-repeat.

Given the following row from a survey sheet:

type	name	label	default
date	event_date	Event date	today()

The following XForm content should be generated in the model:

<bind nodeset="/data/event_date" type="date"/>
<setvalue ref="/data/event_date" value="today()" event="odk-instance-first-load"/>

Given the following rows from a survey sheet:

type	name	label	default
begin repeat	event	Event
date	event_date	Event date	today()
end repeat	event

The following XForm content should be generated in the body:

<repeat nodeset="/data/event">
  <setvalue event="odk-instance-first-load odk-new-repeat" ref="/data/event/event_date" value="today()"/>
  <input ref="/data/event/event_date">
    <label>Event date</label>
  </input>
</repeat>

I believe the best approach for identifying static vs. dynamic defaults will be to use a regular expression. It should be fine for static values to end up as the value for the value attribute so not a big deal if the regex catches some things that are not actually intended to be evaluated. However, there is the possibility that a user will want to literally use text that could be evaluated. For example, a user could want the literal string "today()" as a default and that would no longer be possible. We may want to add some kind of escape strategy for that case but I propose we wait until a user requests it because it seems unlikely to me.

[lognaturel]

@MartijnR, could you please double-check that I captured everything discussed about this feature?

[MartijnR]

This all looks good to me. Thanks!

[nribeka]

Looking at this one next.

[nribeka]

@lognaturel is there existing regular expression to check for expression in the xlsform code that you're aware of?

[lognaturel]

None that I'm aware of and that's definitely the tricky part. Perhaps there's a better strategy you can think of.

I think it would be great to answer some of these questions about approach now. We probably should have a discussion about how/when to release, though. I don't believe Enketo has support for events and actions yet. Collect will only be getting odk-new-repeat support in v1.24 coming out soon. We may either want to hold off on releasing and/or provide a warning that dynamic defaults are only supported in certain clients.

The "good" news is that forms with this feature would entirely fail to load in old versions of Collect so at least there's no risk of the user thinking they're going to get a default but not getting one. Note also that using odk-new-repeat will not pass the current version of Validate.

[nribeka]

Looking at the code, I was thinking of expanding the question_type_dictionary.py and using the action field. But this would potentially means every element will ended up getting set-value action and element that have action already, might not getting the action in the form (start-geopoint).

Would it be a good approach to expand that dictionary and add something like

"default": {
  "name": "set-value", "event": "odk-instance-first-load"
}

Thoughts @lognaturel ?

Update:
Or maybe dynamic-default instead of default.

[nribeka]

@lognaturel added first pass of the implementation. The conversion throw validation error which is expected, right?

[joeflack4]

Wow, I just came here to say that pyxform should throw an error if it detects a dynamic default. But it looks like support will be added, which is even better!

[lognaturel]

Yay, @joeflack4! Out of curiosity, what kind of dynamic default were you hoping to set? That is, is it top level, in a repeat? What type of value is it? Date, ...?

Yes, @nribeka, the Validate failure is expected. It will need to be updated to JavaRosa 2.17.0.

I propose that we add a warning like "not all form filling software support dynamic defaults so you should test the form with the software you plan to fill it with" or something like that. What do you think, @MartijnR, @joeflack4?

[joeflack4]

Oh gosh, I wish I could think of an example right now. This has come up a number of times. I believe both date and text, maybe integer. I don't think it's come up in a repeat group before, but it's possible. I'll see if my PMA colleagues can comment on this.

Does any client support dynamic defaults? If so, I'm fine with just a warning. If not, I imagine that the conversion should just error out. That is, until it's well supported.

[MartijnR]

I propose that we add a warning like "not all form filling software support dynamic defaults so you should test the form with the software you plan to fill it with" or something like that. What do you think, @MartijnR, @joeflack4?

Generally, I think it is the responsibility of the troublesome client to alert the user for something like this upon loading the form. However, in this case, detection is a little tricky in Enketo, so this warning would be very helpful!

[joeflack4]

Well of course, the old tenet is something like 'fail early and loudly'. So in this case, the conversion validator used by the conversion tool is an ideal place. But of course people might not use that converter, or might use old software, so I think it's also good to catch on the client if possible. That also handles any other edge cases.

[lognaturel]

However, in this case, detection is a little tricky in Enketo, so this warning would be very helpful!

@MartijnR, I just noticed that no warning was added by #385. Would you still like one?

Just kidding!! Sorry about that. I'll add a test.

[MartijnR]

Actually Enketo added support for this in December! 😀 🎉

[lognaturel]

Oh, fantastic! Then should we just not include a warning at all, then, @MartijnR? It could affect other clients or downstream tools so maybe we leave it in for one release as a courtesy?

[MartijnR]

I think we might be alright without a warning here (though wouldn't really hurt to be sure). Note, it was Esri that sponsored this feature in Enketo, so they'll be welcoming this pyxform change (and wouldn't need the warning) (@tedrick).