-
Notifications
You must be signed in to change notification settings - Fork 67
2.49 How It Works Dataref Encoding and Decoding
This represents, to the best of my research, how datarefs are encoded into aa lookup dictionary, bone names, and game properties, then decoded and used in .obj files.
Not exhaustive, or sorted in any particular way. Lines may not match completely with official source.
- XPlaneAnimObject.py
def doapply(evt,val)def getvals(bonename, dataref, index)
- XPlaneExport8_util.py
-
def getdataref(self, object, child, name, suffix, first, count)and it's uses in Anim.init def getcustomdataref(object, child, thing, names)
-
- XPlaneUtils.py
def getDatarefs()def make_short_name(fullpath) #fullpath is a lie, it can be any textdef remove_vowels(s)
- Datarefs.txt (which will not necessarily only have laminar research content.)
-
Dataref, Dataref path*, full dataref: A full dataref, unshortened, and not a component of it. Has a tail and head. For example:
sim/aircraft/acf_struct-acf_structis the tail,sim/aircraftis the head (notice the lack of trailing '/') - Known Dataref: A dataref that came from Datarefs.txt
- Unknown Dataref: A dataref that came from some other place, such a lua script
- Ambiguous Dataref: A dataref which requires additional steps and bookkeeping to look up and to a bone names, and game properties. Known and unknown datarefs can be ambiguous with other datarefs
-
datarefs dictionary: The dictionary of datarefs from parsing Datarefs.txt. Annoyingly it is sometimes referred to in code as
datarefswhich is a pretty generic name for a very very important concept - sname, short name: A full dataref that has been run through make_short_name.
- tail-name: The tail of a dataref, referring to its use as a type of encoding information for bone names and the datarefs dictionary
- "[idx]", brackets, idx portion: bone names, game props, and outputted OBJ directives can include an array index as part of the text saying to X-Plane what index in the chosen dataref's array. A lot of string munging involves adding and removing this (as needed). "[1]" is the brackets portion of "my/example/ref[1]". There is one weird exception to this rule involving some of sim/weather props where a tail-name could include this.
- Bone Name: Bone names are a short dataref to be looked up in the datarefs dict, or the tail-name to be looked up in a more complex manner (datarefs dict or in getcustomdataref)
- Parent Armature: Each bone has a parent armature where game properties are stored
- Anim-Value Game Properties: Game properties that connect animation values like v1,v2,...,loop to a dataref. Anim-Value refers to "Dataref Animation Value", not the value side of the property [key,value].
- Disambiguating Key:For the lookup process, ambiguous and unknown values need a key-value pair that can reconstruct the full dataref without the help of Datarefs.txt (aka the contents of the datarefs dictionary)
One last example: my/example/ref int[20] y, and we're animating with my/example/ref[1]. We're assuming that it is found in Datarefs.txt
| Text | Name |
|---|---|
| my/example/ref | Full Dataref |
| "my","example","ref" | Components |
| my/example | Head aka Head of Dataref |
| ref | Tail of dataref, tail-name. Found in Datarefs Dictionary |
| ref[1] | Tail + [idx]. Not found in Datarefs Dictionary. This is not tail-name. |
| ref[1] | Bone name. Since "ref" is so short, we're using the tail-name + [idx] |
| me_ref | sname (hint: look at make_short_name for how its made). Found in Datarefs Dictionary |
| me_ref[1] | sname + [idx] |
me_ref[1]_v2:.75
|
Anim-Value Game Prop on the parent armature: At frame 2 of my/example/ref[1] the dataref value will be .75 |
{ 'me_ref':('my/example/ref', 20),ref:('my/example/ref', 20)} |
The contents of the datarefs dictionary (excluding other keys). Two keys point to the same full dataref. |
If my/other/example/ref int[20] y were also in the dictionary, we'd have duplicate tail-names representing an ambiguity.
{'moe_ref': ('my/other/example/ref', 20), 'me_ref': ('my/example/ref', 20), 'ref': None}
Notice both datarefs have sname keys pointing towards them, but there is only one tail-name key pointing to None.
If my/example/ref was not in Datarefs.txt we'd have a disambiguation key in any parent armature who had bones who used my/example/ref.
| Text | Name |
|---|---|
ref:my/example
|
Disambiguation Key, the tail component (never shortened) with Head value. Disambiguation Key != (Bone Name - [idx]) when Bone Name is of type sname! (This edge case is discussed later) |
Got it? Try to keep up, we've got pages more of this.
Generated by XPlaneUtils.getDatarefs, this dictionary represents the contents of your chosen Datarefs.txt. In code it is usually called datarefs or lookup.
Each dataref in your Datarefs.txt will have two keys (shortened full dataref and unshortened tail) attempting to point at a record of the full dataref and it's array size (1 if scalar).
There will always be exactly 1 sname and 1 tail-name attempting to pointing at the same full dataref.
Using the datarefs dictionary with an sname is never None, using it with a tail-name is possibly None.
Any attempts to use a duplicate sname will be ignored and the current parsed value won't be stored.
Any attempts to use a duplicate tail-name will set
lookup[tail_name]to None (representing an ambiguity).
The Datarefs Dictionary as a Python 3.4 type annotation might be getDatarefs() -> Dict[Union[ShortenedDataref,tail-name],Optional[Tuple[FullDataref,ArraySize]]]
The idea of the lookup dictionary is: "Given some input (sname, or tail-name) we can find the full dataref or say that there is an ambiguity. All of a valid Datarefs.txt file will be read in and perfectly usable, even trying to handle datarefs that resolve to duplicate snames or have duplicate tail-names".
-
Only dataref path is considered in detecting dictionary collisions. Other data like type, array size, units, or description are ignored.
- sim/whatever int n
- sim/whatever int[250] n
In this example, you will get {
"whatever":(sim/whatever, 1),whatever:None}, not two datarefs as expected. A warning is given about ambiguous short names. -
sname collision is particularly dangerous
- "sim/bad_choice/mychoice"
- "sim/boring_choice/mychoice"
In this example, you will get {
sb_mychoice:(sim/bad_choice/mychoice),mychoice:None}, not two datarefs as expected. -
Array indexes in dataref path are kept
- There are several known datarefs (
sim/weather/cloud_type,sim/weather/wind_altitude) which have the array index in their names, as insim/weather/wind_altitude_msl_m[0]. This is legal. While laminar datarefs do not appear to have this AND be an array type like int[25], it could be possible. The spec here (which there isn't one) is undefined. While not breaking, it is a large potential for bugs and confusion, as there is ambiguity between which is the right answer:(sim/weather/wind_altitude_msl_m[2],1)vs(sim/weather/wind_altitude_msl_m[2],3). According the the dataref as defined, it has no array. "[2]" is simply text in the path. - An .obj with a dataref like this gets exported fine. The game properties used are
sw_cldbsmslm[0]_v1 and v2, bone name iscloud_base_msl_m[0]. Even with an ambiguous tail with "[idx]" still works. - This is the point of getdataref's use of
seq=[ref, name]we think, to check "wind_altitude_msl_m" and "wind_altitude_msl_m[0]". The laterseq.append(make_short_name(dataref))is actually important!refcould come from a bone whose name came from an sname or a tail-name.
- There are several known datarefs (
-
We fail on 'sim/weather/wind_direction_degt', 'sim/weather/wind_direction_degt[0]' and bone name is wind_direction_degt[0]
This fails for us because our implementation's lookup won't know how to choose between the options. Currently there is a manual hard-coded fix for this one edge case. Other user's custom DataRefs.txt that have datarefs like this will be told to change it or they can open the source themselves and add an exception. See 249: sim/weather/wind_direction_degt vs sim/weather/wind_direction_degt[n] is a problem for more details.
Matching a bone name to game properties is done with getdataref and getcustomdataref.
-
Using the bone name, attempt to get the full dataref path.
a.) If it is a known unambiguous key (
(bone_name in datarefs and datarefs[bone_name]) is True,use datarefs[bone_name].*b.) If it is a known but ambiguous key (
(bone_name in datarefs and datarefs[bone_name] is None) is True)or unknown**, search the properties for the disambiguating key (using bone_name - "[idx]")***.Combine tail-name + value to get dataref. -
Using your dataref, make an sname of it (to make sure we can access Dataref Value For Frame and Loop properties). Using [bone_name, bone_name + "[idx]" (if needed), short_name_of_dataref], iterate and match each property to its dataref.
*If the datarefs dictionary is queried using a bone name that is of type tail-name, and
datarefs[bone_name]is None it is counted as ambiguous even if the sname lookup would have worked
**In the unknown case, you'd better hope that the bone name used tail-name form and NOT sname, because the disambiguating key is always
tail-name:head! If the unknown key's bone name is usingsnameyou'll never be able to match the bone name to any disambiguating key! There is no validation for this
***In the known but ambiguous case, you are guaranteed, to be using a tail-name (otherwise, you'd be following code path a!).
****You had also better hope that the tail is short enough because the prop keys also have a size limit of 31 characters and will silently cut off extra characters. "AB_CDFGHJKLMNPQRSTVWXYZ012345" only has room for "_v".
Game properties are used to represent the saved input from the AnimObject menu. The process involves appending extra text to an sname or tail-name. The 2.49 automatically trims space around property and bone names, remove .001,.002, etc from bone names, and will ignore a trailing / on the value of a disambiguating key. Any key that is too long gets truncated without warning.
| Anim-Value | Encoding Type | Pattern | Notes |
|---|---|---|---|
| Disambiguating Key | tail-name |
tail-name:head
|
See XPlaneAnimObject, line 503 |
| Dataref Value for Frame N | sname + any "[idx]" | "name_vN" or "name[idx]_vN" where N is the frame number | Unless specified, the first two values are assumed to be 0.0 and 1.0 and not necessary to define. |
| Loop | sname + any "[idx]" | "name_loop" or "name[idx]_loop" | |
| Hide/Show | tail-name + any "[idx]" | "name_hide/show_vN" or "name[idx]_hide/show_vN" | Unlike Dataref Value for Frame N, there are always two properties per Hide/Show. Also, it is not tied to a bone's name! |
| Concept | 2.49 | 2.78 |
|---|---|---|
| Action | Dict[channel_name,Ipo] | List[FCurves] but also has groups of channels to organize it |
| Channel | Uses bone name, semanti | List[FCurves], semantics, subsection of Action's list |
| Group | Optional | Mandatory, List[Channels], name is bone name |
| Curve | IPOCurve | FCurve |
| Anim Data | in bezierPoints | in keyframe_points |
During read, 2.7x turns channels into groups and IPOCurves into FCurves.
During export, 2.49 replaces missing dataref values with 0 (if unspecified) and 1 for deleted middle keys and the last key (if unspecified) so that every Blender keyframe has a pair.
An ex 7 frame animation:
| nth Frame | Value | In Game Props? | As In OBJ |
|---|---|---|---|
| 1 | 0 | No | ..._key 0 |
| 2 | 0.0 | Yes | ..._key 0.0 |
| 3 | 2001.0 | Yes | ..._key 2001.0 |
| 4 | -3999.0 | Yes | ..._key -3999.0 |
| 5 | 1 | No (deleted) | ..._key 1 |
| 6 | 1.0 | Yes | ..._key 1.0 |
| 7 | 1.0 | No | ..._key 1 |
(List from first read to last. None of these appear in the list of ambiguous tail-names) Notice the difference between engine vs enigne (Misspelling is marked as deprecated)
| SName | Dataref |
|---|---|
| scs_ntcnlthtprngn | sim/cockpit/switches/anti_ice_inlet_heat_per_enigne |
| scs_ntcnlthtprngn | sim/cockpit/switches/anti_ice_inlet_heat_per_engine |
Notice the difference between _value and _avail! What a coincidence! Fortunately, we can use tail-name to look them up, and they won't be shortened
| SName | Dataref |
|---|---|
| sj_jymppdxsvl | sim/joystick/joy_mapped_axis_avail |
| sj_jymppdxsvl | sim/joystick/joy_mapped_axis_value |
Notice the difference between failures and failures
| SName | Dataref |
|---|---|
| sof_nblrndmflrs | sim/operation/failures/enable_random_falures |
| sof_nblrndmflrs | sim/operation/failures/enable_random_failures |
-
sim/joystick/yoke_roll_ratioandsim/cockpit2/controls/yoke_roll_ratio
-
sim/flightmodel2/engines/engine_rotation_speed_rad_sec(Can't be used with Show/Hide because tail is so long)
sim/aircraft/acf_struct int[10] n
-
laminar/c172/fuel/fuel_tank_selector, has values at 1, 2, 3, and 4 - `laminar/c172/knob_OATS, has values at 0 and 1
-
A/B/CDEFGHIJKLMNOPQRSTUVWXYZ123(Unknown dataref of length 30, with tail of 27. sname can be a bone name, the tail triggers shortening causing a later comparison of ,AB_CDFGHJKLMNPQRSTVWXYZ123andCDEFGHIJKLMNOPQRSTUVWXYZ123)
-
sim/weather/cloud_type[0], easily testable with changing the weather sim/weather/cloud_base_msl_m[0]sim/weather/wind_altitude_msl_m[1]
The double trouble double/trouble/idx[0] int[10] y case will not export, saying it is deformed.
- If a name is > 26 characters long, it must be shortened