Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation generator updates #624

Merged
merged 4 commits into from
May 7, 2020
Merged

Documentation generator updates #624

merged 4 commits into from
May 7, 2020

Conversation

mosra
Copy link
Collaborator

@mosra mosra commented May 7, 2020

Motivation and Context

Assorted updated to the documentation generator.

  • Handles issues like in Doc build crashes habitat-lab#385 more gracefully. Unfortunately I wasn't able to reproduce locally (maybe it's something specific to a particular Python version, I'm on 3.8 here), so if this happens again it should print a descriptive warning and continue.
  • Fixes non-deterministic output for default arguments containing function pointers, which means useless changes like in https://github.com/facebookmicrosites/habitat-website/commit/9cbead7f85aad0c30d86514e0bd790fe4dda8c1c won't happen anymore
  • Creates output directory for Doxygen because Doxygen is unable to do that on its own.
  • Adjusts itself for new bugs and regression in Doxygen 1.8.17 and 1.8.18.
  • Properly includes CONTRIBUTING.md in the Doxyfile
  • updates Doxyfile to latest Doxygen (it only removes a bunch of outdated options, should still keep working on older versions)

How Has This Been Tested

Docs build correctly locally

Types of changes

  • Docs change / refactoring / dependency upgrade
  • Bug fix (non-breaking change which fixes an issue)

@mosra mosra requested a review from dhruvbatra May 7, 2020 22:08
@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label May 7, 2020
@mosra mosra requested a review from mathfac May 7, 2020 22:08
@mosra mosra merged commit dfe7a9a into master May 7, 2020
@mosra mosra deleted the docs4 branch May 7, 2020 22:21
@dhruvbatra
Copy link
Contributor

@mosra -- now the documentation build for habitat-sim is also breaking:

(habitat)
dbatra at dbatra-mbp in ~/projects/habitat-sim on master
$ rm -rf build; python setup.py install --bullet
...
<successful>
dbatra at dbatra-mbp in ~/projects/habitat-sim on master [?]
$ cd docs/
(habitat)
dbatra at dbatra-mbp in ~/projects/habitat-sim/docs on master
$ mkdir -p ../build/docs
(habitat)
dbatra at dbatra-mbp in ~/projects/habitat-sim/docs on master
$ ./bu
build-public.sh* build.sh*
(habitat)
dbatra at dbatra-mbp in ~/projects/habitat-sim/docs on master
$ ./build-public.sh
warning: Tag 'TCL_SUBST' at line 270 of file 'Doxyfile' has become obsolete.
         To avoid this warning please remove this line from your configuration file or upgrade it using "doxygen -u"
/Users/dbatra/projects/habitat-sim/docs/bullet.tag:1: warning: Internal inconsistency: scope for class btSoftBody::AJoint::IControl not found!
/Users/dbatra/projects/habitat-sim/docs/bullet.tag:1: warning: Internal inconsistency: scope for class btSoftBody::AJoint::Specs not found!
/Users/dbatra/projects/habitat-sim/src/esp/sensor/Sensor.h:69: warning: Compound esp::sensor::ObservationSpace is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/sensor/Sensor.h:63: warning: Compound esp::sensor::Observation is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/SuncgSemanticScene.h:33: warning: Compound esp::scene::SuncgObjectCategory is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/SuncgSemanticScene.h:22: warning: Compound esp::scene::SuncgSemanticRegion is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/Mp3dSemanticScene.h:27: warning: Compound esp::scene::Mp3dRegionCategory is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/GibsonSemanticScene.h:12: warning: Compound esp::scene::GibsonObjectCategory is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/nav/PathFinder.h:102: warning: Compound esp::nav::NavMeshSettings is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/gfx/WindowlessContext.h:12: warning: Compound esp::gfx::WindowlessContext is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/gfx/Renderer.h:15: warning: Compound esp::gfx::Renderer is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/gfx/RenderCamera.h:15: warning: Compound esp::gfx::RenderCamera is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/gfx/MaterialData.h:19: warning: Compound esp::gfx::PhongMaterialData is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/core/spimpl.h:102: warning: Compound spimpl::impl_ptr is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/core/spimpl.h:84: warning: Compound spimpl::details::default_copier is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/core/random.h:12: warning: Compound esp::core::Random is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/core/Configuration.h:17: warning: Compound esp::core::Configuration is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/core/Buffer.h:28: warning: Compound esp::core::Buffer is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/PTexMeshData.h:24: warning: Compound esp::assets::PTexMeshData::MeshData is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/GenericInstanceMeshData.h:23: warning: Compound esp::assets::GenericInstanceMeshData::RenderingBuffer is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/GenericInstanceMeshData.h:21: warning: Compound esp::assets::GenericInstanceMeshData is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/sensor/PinholeCamera.h:21: warning: Compound esp::sensor::PinholeCamera is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/Attributes.h:400: warning: Compound esp::assets::CylinderPrimitiveAttributes is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/Attributes.h:390: warning: Compound esp::assets::CubePrimitiveAttributes is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/Attributes.h:363: warning: Compound esp::assets::ConePrimitiveAttributes is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/agent/Agent.h:83: warning: Compound esp::agent::Agent is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/SceneManager.h:21: warning: Compound esp::scene::SceneManager is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/Attributes.h:426: warning: Compound esp::assets::IcospherePrimitiveAttributes is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/gfx/PTexMeshDrawable.h:18: warning: Compound esp::gfx::PTexMeshDrawable is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/ObjectControls.h:19: warning: Compound esp::scene::ObjectControls is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/gfx/PTexMeshShader.h:24: warning: Compound esp::gfx::PTexMeshShader is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/SuncgSemanticScene.h:49: warning: Compound esp::scene::SuncgRegionCategory is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/GenericInstanceMeshData.h:74: warning: Compound esp::assets::GenericInstanceMeshData::PerObjectIdMeshBuilder is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/sim/Simulator.h:63: warning: Compound esp::sim::Simulator is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/SuncgSemanticScene.h:12: warning: Compound esp::scene::SuncgSemanticObject is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/nav/PathFinder.h:22: warning: Compound esp::nav::HitRecord is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/SceneGraph.h:20: warning: Compound esp::scene::SceneGraph is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/PTexMeshData.h:31: warning: Compound esp::assets::PTexMeshData::RenderingBuffer is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/core/spimpl.h:433: warning: Compound std::hash< spimpl::impl_ptr< T, D, C > > is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/agent/Agent.h:20: warning: Compound esp::agent::AgentState is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/gfx/GenericDrawable.h:15: warning: Compound esp::gfx::GenericDrawable is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/agent/Agent.h:36: warning: Compound esp::agent::ActionSpec is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/sensor/Sensor.h:77: warning: Compound esp::sensor::Sensor is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/SceneNode.h:33: warning: Compound esp::scene::SceneNode is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/agent/Agent.h:52: warning: Compound esp::agent::AgentConfiguration is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/PTexMeshData.h:22: warning: Compound esp::assets::PTexMeshData is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/core/spimpl.h:76: warning: Compound spimpl::details::default_deleter is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/SceneConfiguration.h:14: warning: Compound esp::scene::SceneConfiguration is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/Attributes.h:456: warning: Compound esp::assets::UVSpherePrimitiveAttributes is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/assets/Mp3dInstanceMeshData.h:25: warning: Compound esp::assets::Mp3dInstanceMeshData is not documented.
<unknown>:1: warning: Detected potential recursive class relation between class Magnum::DebugTools::ResourceManager and base class ResourceManager< Magnum::Implementation::ResourceManagerLocalInstance, GL::AbstractShaderProgram, GL::Buffer, GL::Mesh, GL::MeshView, DebugTools::ForceRendererOptions, DebugTools::ObjectRendererOptions >!

<unknown>:1: warning: Detected potential recursive class relation between class Magnum::DebugTools::ResourceManager and base class ResourceManager< Magnum::Implementation::ResourceManagerLocalInstance, GL::AbstractShaderProgram, GL::Buffer, GL::Mesh, GL::MeshView, DebugTools::ForceRendererOptions, DebugTools::ObjectRendererOptions >!

/Users/dbatra/projects/habitat-sim/src/esp/geo/OBB.h:14: warning: Compound esp::geo::OBB is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/ReplicaSemanticScene.h:12: warning: Compound esp::scene::ReplicaObjectCategory is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/scene/Mp3dSemanticScene.h:11: warning: Compound esp::scene::Mp3dObjectCategory is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/sensor/Sensor.h:124: warning: Compound esp::sensor::SensorSuite is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/sensor/VisualSensor.h:20: warning: Compound esp::sensor::VisualSensor is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/sim/Simulator.h:36: warning: Compound esp::sim::SimulatorConfiguration is not documented.
/Users/dbatra/projects/habitat-sim/src/esp/sensor/Sensor.h:40: warning: Compound esp::sensor::SensorSpec is not documented.
Traceback (most recent call last):
  File "./m.css/documentation/python.py", line 2655, in <module>
    run(os.path.dirname(os.path.abspath(args.conf)), config, templates=os.path.abspath(args.templates))
  File "./m.css/documentation/python.py", line 2428, in run
    hooks_post_run=state.hooks_post_run)
  File "/Users/dbatra/projects/habitat-sim/docs/m.css/documentation/../plugins/m/dox.py", line 174, in register_mcss
    tagfiles=mcss_settings.get('M_DOX_TAGFILES', []))
  File "/Users/dbatra/projects/habitat-sim/docs/m.css/documentation/../plugins/m/dox.py", line 69, in init
    tree = ET.parse(os.path.join(input, tagfile))
  File "/anaconda3/envs/habitat/lib/python3.6/xml/etree/ElementTree.py", line 1196, in parse
    tree.parse(source, parser)
  File "/anaconda3/envs/habitat/lib/python3.6/xml/etree/ElementTree.py", line 586, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/Users/dbatra/projects/habitat-sim/docs/../build/docs/habitat-cpp.tag'

@mosra
Copy link
Collaborator Author

mosra commented May 8, 2020

Nice, more Doxygen bugs. Looking into that.

Until I pile on more workarounds, can you switch the used Doxygen version from 1.8.18 to 1.8.16 or 17? Locally I was on a forked version with my own patches in order to be able to build magnum docs, not on the stable 1.8.18, and the bug zoo is apparently very different between those two.

EDIT: Just checked and for the automatic doc build everything worked correctly -- this is done with 1.8.16 IIRC: https://github.com/facebookmicrosites/habitat-website/commit/08277936ad60836bbd2dd59ab98e47dd947668f7

@mosra
Copy link
Collaborator Author

mosra commented May 8, 2020

Apparently 1.8.18 doesn't produce any output at all for Habitat, for no apparent reason, with no log output indicating why. I'll need to bisect the 280 new commits that happened since the last working state on my fork, until then you have to pin to 1.8.17.

@dhruvbatra
Copy link
Contributor

Confirming that

  1. Doc autobuilding in CI is indeed done with doxygen 1.8.16.

    conda install -y -c conda-forge doxygen==1.8.16

  2. With doxygen 1.8.16, I am able to build docs for h-sim. Consider this thread closed. Moving onto other issues now.

cd docs
mkdir -p ../build/docs
Copy link
Contributor

@dhruvbatra dhruvbatra May 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mosra -- Minor note -- doc building on h-sim (master) is still crashing without this:

(habitat) 
dbatra at dbatra-mbp in ~/projects/habitat-sim/docs on master
$ ./build-public.sh 
...
...
WARNING:root:classesp_1_1assets_1_1ResourceManager.xml: function parameter description doesn't match parameter names: {'shaderType': ('Indentifies the desired shader program for rendering the <a href="classesp_1_1gfx_1_1Drawable.html" class="m-doc">gfx::<wbr />Drawable</a>.', ''), 'meshID': ('Optional, the index of this mesh component stored in meshes_', ''), 'texture': ('Optional texture for the mesh.', ''), 'color': ('Optional color parameter for the shader program. Defaults to white.', '')}
Traceback (most recent call last):
  File "./m.css/documentation/python.py", line 2668, in <module>
    run(os.path.dirname(os.path.abspath(args.conf)), config, templates=os.path.abspath(args.templates))
  File "./m.css/documentation/python.py", line 2441, in run
    hooks_post_run=state.hooks_post_run)
  File "/Users/dbatra/projects/habitat-sim/docs/m.css/documentation/../plugins/m/dox.py", line 174, in register_mcss
    tagfiles=mcss_settings.get('M_DOX_TAGFILES', []))
  File "/Users/dbatra/projects/habitat-sim/docs/m.css/documentation/../plugins/m/dox.py", line 69, in init
    tree = ET.parse(os.path.join(input, tagfile))
  File "/anaconda3/envs/habitat/lib/python3.6/xml/etree/ElementTree.py", line 1196, in parse
    tree.parse(source, parser)
  File "/anaconda3/envs/habitat/lib/python3.6/xml/etree/ElementTree.py", line 586, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/Users/dbatra/projects/habitat-sim/docs/../build/docs/habitat-cpp.tag'

Following this error up with mkdir succeeds.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sigh, Doxygen. Opened #630 for this.

dhruvbatra pushed a commit that referenced this pull request May 13, 2020
* master:
  Interactive RigidObject python tutorial page (#611)
  Update README.md
  Separate Scene And Object functionality into different classes (#628)
  Updated m.css submodule. (#627)
  Update agent state set state to be less confusing (#614)
  Direct Navmesh/Meshdata handling; PluginManager instance variable (#623)
  Documentation generator updates (#624)
  Update corrade submodule. (#618)
  Physics primitives (non-colliding) (#622)
  save_nav_mesh python bindings (#619)
  --Renamed GltfMeshData -> GenericMeshData; (#617)
  Update README.md
  Update README.md
Ram81 pushed a commit to Ram81/habitat-web-sim that referenced this pull request Dec 10, 2020
* doc: properly include the CONTRIBUTING.md file.

* doc: update Doxyfile for 1.8.18.

* doc: clarify doc building README.

* doc: update m.css submodule.
Ram81 pushed a commit to Ram81/habitat-web-sim that referenced this pull request Dec 10, 2020
* master:
  Interactive RigidObject python tutorial page (facebookresearch#611)
  Update README.md
  Separate Scene And Object functionality into different classes (facebookresearch#628)
  Updated m.css submodule. (facebookresearch#627)
  Update agent state set state to be less confusing (facebookresearch#614)
  Direct Navmesh/Meshdata handling; PluginManager instance variable (facebookresearch#623)
  Documentation generator updates (facebookresearch#624)
  Update corrade submodule. (facebookresearch#618)
  Physics primitives (non-colliding) (facebookresearch#622)
  save_nav_mesh python bindings (facebookresearch#619)
  --Renamed GltfMeshData -> GenericMeshData; (facebookresearch#617)
  Update README.md
  Update README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants