Documenting Honeybee model schema

[mostaphaRoudsari]

Now that the repository is cleaned up it should be a good time to restructure the repository to address two challenges:

1. We need to package the schema so we can use it in other modules and in Pollination model-service
2. We need to automate the process of generating the openapi schema from the Pydantic objects.

The challenge for second step is how to do it without having to set-up the endpoints. We have to dig into FastAPI and see how it does it probably.

@AntoineDao do you have any inputs for item 2?

[AntoineDao]

This is a tricky one. There are three contexts for using the models:

1. Within the pollination backend services
2. As objects sent to the pollination microservices
3. For applications completely separate from the Pollination project

The first two are relatively easy to automate through the mechanism we are currently using:

1. Create backend service
2. Document backend service schemas through FastAPI
3. Generate client libraries through OpenAPI generators

The latter use case is tricky as I think we want to give it independence from the Pollination platform. As such we could still generate an OpenAPI schema from the pollination model-service and extract the result manually or we could stick with the JSON schema objects created by Pydantic.

I guess the best way to decide is to explain how we can best facilitate the last use case I described above (ie: non pollination related applications using the model schema).

[mostaphaRoudsari]

I have looked around for a good JSON schema visualization platform similar to redocly but I couldn't find any. I think we can modify get_openapi utility of FastAPI to generate the documentation without generating an end point: https://fastapi.tiangolo.com/tutorial/extending-openapi/#generate-the-openapi-schema

It has to be mixed with this Redocly/redoc#681 to document the objects itself.

It is one of those that might end up being very easy or be a rabbit hole. 🐰 🥕

[mostaphaRoudsari]

Turned out to be pretty hack-able. See here for a live version.

Here is the code to generate it:

import json
from pydantic.schema import schema
from typing import Dict

# TODO: We should move this module to be the main module and remove end-points
from app.models import Model

_base_open_api = {
    "openapi": "3.0.2",
    "servers": [],
    "info": {
        "description": "This is the documentation for Honeybee model schema.",
        "version": "1.0.0",
        "title": "Honeybee Model Schema",
        "contact": {
            "name": "Honeybee Data Model Support",
            "email": "info@ladybug.tools",
            "url": "https://github.com/ladybug-tools/honeybee"
        },
        "x-logo": {
            "url": "https://www.ladybug.tools/assets/img/honeybee-large.png",
            "altText": "Honeybee logo"
        },
        "license": {
            "name": "BSD",
            "url": "https://github.com/ladybug-tools-in2/energy-model-schema/blob/master/LICENSE"
        }
    },
    "externalDocs": {
        "description": "See how to use this model in action.",
        "url": "https://api.pollination.cloud/"
    },
    "tags": [],
    "x-tagGroups": [
        {
            "name": "Models",
            "tags": []
        }
    ],
    "paths": {},
    "components": {"schemas": {}}
}


def get_openapi(
    *,
    title: str = None,
    version: str = None,
    openapi_version: str = "3.0.2",
    description: str = None,
    ) -> Dict:

    open_api = dict(_base_open_api)

    open_api['openapi'] = openapi_version

    if title:
        open_api['info']['title'] = title

    if version:
        open_api['info']['version'] = version

    if description:
        open_api['info']['description'] = description

    definitions = schema([Model], ref_prefix='#/components/schemas/')

    # goes to tags
    tags = []
    # goes to x-tagGroups['tags']
    tag_names = []

    schemas = definitions['definitions']
    schema_names = list(schemas.keys())
    schema_names.sort()
    for name in schema_names:
        model_name = '%s_model' % name.lower()
        tag_names.append(model_name)
        tag = {
            'name': model_name,
            'x-displayName': name,
            'description': '<SchemaDefinition schemaRef=\"#/components/schemas/%s\" />\n' % name
        }
        tags.append(tag)

    tag_names.sort()
    open_api['tags'] = tags
    open_api['x-tagGroups'][0]['tags'] = tag_names

    open_api['components']['schemas'] = schemas

    with open(r"C:\Users\Mostapha\Dropbox\Public\openapi.json", 'w') as outf:
        json.dump(open_api, outf, indent=2)


if __name__ == '__main__':
    get_openapi()

I will restructure the repository and send a PR.

[mostaphaRoudsari]

@chriswmackey, I decided to push the refactored code to a different repository mainly because these schemas will be a dependency of the model service and shouldn't be in a repository that's itself a fork of the model service repository.

The repository is here: https://github.com/ladybug-tools-in2/honeybee-model-schema and the documentation is auto-generated here: https://ladybug-tools-in2.github.io/honeybee-model-schema/

There are a number of improvements to be done but I wanted to first make sure moving this repository to a new place is fine with you and @tanushree04 before spending more time.

Fixes are:

• favicon should be changed to honeybee
• openapi version should changed based on release version

We can also add better examples to the objects as discussed here:
https://pydantic-docs.helpmanual.io/usage/schema/#schema-customization

[tanushree04]

@mostaphaRoudsari The new repository and documentation look good.