Add VmomiSupport.VmomiJSONEncoder for encoding pyVmomi objects as JSON

[jeking3]

This fixes #21
This closes #669

This allows for deep JSON encoding of pyVmomi Managed Objects by providing a JSONEncoder. Tested with Datacenter, Datastore, HostSystem, Network, VirtualMachine objects against a VCSA version 6.7. A sample python script is provided allowing any object to be dumped by type and MOID.

Example: Serialize a VirtualMachine

    data = json.dumps(vm, cls=VmomiJSONEncoder)

Example: Serialize a list of VirtualMachines and their snapshots

    data = json.dumps([vm1, vm2], cls=VmomiJSONEncoder,
                      explode=[templateOf('VirtualMachine'),
                               templateOf('VirtualMachineSnapshot')])

Example: Serialize a vim.Network using the dumpjson example

Command Line

Assuming you have already run tox:

~/pyvmomi$ PYTHONPATH=.:.tox/py27/lib/python2.7/site-packages python sample/dumpjson.py \
    --host vcenter --user my_user --pass my_pass --type Network --id network-16

Code

Or perhaps you just want to see the code:

   obj = VmomiSupport.templateOf('Network')('network-16', si._stub)
   print(json.dumps(obj, cls=VmomiSupport.VmomiJSONEncoder,
                    sort_keys=True, indent=2))

Result

Note this is perhaps a poor example, since the encoder handles deep structures completely, but the other examples are rather huge.

{
  "_vimid": "network-16",
  "_vimref": "vim.Network:network-16",
  "_vimtype": "vim.Network",
  "alarmActionsEnabled": true,
  "availableField": [],
  "configIssue": [],
  "configStatus": "green",
  "customValue": [],
  "declaredAlarmState": [],
  "disabledMethod": [],
  "effectiveRole": [
    -1
  ],
  "host": [
    "vim.HostSystem:host-14",
    "vim.HostSystem:host-25"
  ],
  "name": "VM Network",
  "overallStatus": "green",
  "parent": "vim.Folder:group-n6",
  "permission": [],
  "recentTask": [],
  "summary": {
    "accessible": true,
    "dynamicProperty": [],
    "dynamicType": null,
    "ipPoolId": null,
    "ipPoolName": "",
    "name": "VM Network",
    "network": "vim.Network:network-16"
  },
  "tag": [],
  "triggeredAlarmState": [],
  "value": [],
  "vm": [
    "vim.VirtualMachine:vm-17",
    "vim.VirtualMachine:vm-18",
    "vim.VirtualMachine:vm-19",
    "vim.VirtualMachine:vm-20",
    "vim.VirtualMachine:vm-21",
    "vim.VirtualMachine:vm-22",
    "vim.VirtualMachine:vm-27",
    "vim.VirtualMachine:vm-28",
    "vim.VirtualMachine:vm-29",
    "vim.VirtualMachine:vm-32",
    "vim.VirtualMachine:vm-36",
    "vim.VirtualMachine:vm-42",
    "vim.VirtualMachine:vm-69",
    "vim.VirtualMachine:vm-86",
    "vim.VirtualMachine:vm-87",
    "vim.VirtualMachine:vm-111",
    "vim.VirtualMachine:vm-112"
  ]
}

[vmwclabot]

@jeking3, you must sign our contributor license agreement before your changes are merged. Click here to sign the agreement. If you are a VMware employee, read this for further instruction.

[vmwclabot]

@jeking3, VMware has approved your signed contributor license agreement.

[jeking3]

I didn't change anything relative to SSL, so it's odd that two SSL tests are failing. @tianhao64 this also happens locally for me. I'll take another look as to why.

[jeking3]

Updated for v6.7.1.

[jeking3]

@pgbidkar when will the next release be?

[tianhao64]

@jeking3 Looks like the vcrpy latest release is not working well with existing SSL tests. If you pull the latest code or update vcrpy==1.12.0 in test-requirements.txt, the tests should pass.

[jeking3]

@tianhao64 I did that (actually set it to vcrpy<2.0.0 which seems to work just fine, since 1.13.0 is out and worked for me).

I have pull requests in other projects that depends on this change getting merged and released. I think it provides significant value. Let me know if anything is blocking this from getting in.

[pgbidkar]

Hi,
I tried testing your changes in my Local test env

Command I ran is
python .\sample\dump_all_json.py -s -u
-p -t 'VirtualMachine' -i 'vm-id'

Getting this error
Traceback (most recent call last):
File ".\sample\dump_all_json.py", line 80, in
main()
File ".\sample\dump_all_json.py", line 77, in main
sort_keys=True, indent=4))
File "C:\Python27-13\Lib\json_init_.py", line 251, in dumps
sort_keys=sort_keys, **kw).encode(obj)
File "C:\Python27-13\Lib\json\encoder.py", line 209, in encode
chunks = list(chunks)
File "C:\Python27-13\Lib\json\encoder.py", line 442, in _iterencode
o = _default(o)
File "C:\Work\venvs\venv-2713\lib\site-packages\pyVmomi\VmomiSupport.py", line 345, in default
super(VmomiJSONEncoder, self).default(obj)
File "C:\Python27-13\Lib\json\encoder.py", line 184, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: 'vim.VirtualMachine:vm-id' is not JSON serializable

Please can you retest and confirm?

[pgbidkar]

Indentation in pyVmomi code is 3 space. Looks like you did 4 space indentation.
Please can you check and change that?

[jeking3]

@pgbidkar I do not see what you are seeing. This works fine for me:

pyvmomi$ PYTHONPATH=.:.tox/py27/lib/python2.7/site-packages/ python sample/dumpjson.py -s 10.0.0.4 -u my_user -p my_pass -t VirtualMachine -i vm-22

There must be something different in your environment. Are you running against a real vCenter or a simulator?

[jeking3]

@pgbidkar 3 space indents are non-standard, you should consider standardizing. I made the changes you requested.

https://lintlyci.github.io/Flake8Rules/rules/E111.html

[pgbidkar]

Hi @jeking3 , I am running sample with Real vCenter

[jeking3]

Not sure where you got "dump_all_json.py" from; the sample included is called "dumpjson.py". Line 345 is not the proper line for calling to default, line 352 is. Do you have your virtual environment set up with the latest copy of this PR?

https://github.com/vmware/pyvmomi/pull/732/files#diff-3d209b7e252f047d63b9b948cb74282cR352

[pgbidkar]

@jeking3 Created fresh Virtual env. Cloned json-encode branch. Sample is running fine now.
I should be done with review by tomorrow EOD.
Thanks

[jeking3]

Any thoughts?

[pgbidkar]

Merging the change. Thanks

[jeking3]

Thanks. Now the only thing blocking upstream changes is a pip release.

[jeking3]

pyvmomi > 6.7.1 will acquire this feature now that it has been released in v6.7.1.2018.12. Thank you!