v1.10.0 does not support local files in $ref

[eivindga]

Describe the bug

The latest version of the parser does not support '$ref' used anywhere in asyncapi-files, referencing local files when generating doc.

How to Reproduce

Try to include a local file using ref in:

• message example
• description field

Example of message:

  'aggregate.vehicle.journey.v2':
    description:
      $ref: markdown/VehicleJourneyV2.md
    subscribe:
      message:
        name: VehicleJourneyV2
        schemaFormat: application/vnd.apache.avro;version=1.9.0
        payload:
          $ref: schema/VehicleJourneyV2
        examples: [
          { $ref: examples/VehicleJourneyV2_deadrun.json },
          { $ref: examples/VehicleJourneyV2_servicejourney.json }
        ]
        bindings:
          kafka:
            compact: false
           

Expected behavior

Latest version should work in the same way as v1.9.18.

[github-actions]

Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

[derberg]

@eivindga can you please paste the error that you're getting from the system?

[eivindga]

We are encountering multiple different error-messages:

• referencing is not allowed
• schema is invalid
• a line break is expected

Something went wrong:
Error: Input is not a corrent AsyncAPI document so it cannot be processed.
    at Generator.generateFromString (/home/yes/.nvm/versions/node/v18.13.0/lib/node_modules/@asyncapi/generator/lib/generator.js:264:19)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /home/yes/.nvm/versions/node/v18.13.0/lib/node_modules/@asyncapi/generator/cli.js:157:9
Errors:
[
  {
    "code": "asyncapi-document-unresolved",
    "message": "Referencing in this place is not allowed",
    "path": [
      "channels",
      "entity.profile.key.v2",
      "subscribe",
      "message",
      "examples",
      "0"
    ],
    "severity": 0,
    "source": "/home/some/path/asyncapi/pd/asyncapi.yml",
    "range": {
      "start": {
        "line": 54,
        "character": 10
      },
      "end": {
        "line": 55,
        "character": 62
      }
    }
  },

  {
    "code": "asyncapi2-message-examples",
    "message": "schema is invalid: data/type must be equal to one of the allowed values, data/type must be array, data/type must match a schema in anyOf",
    "path": [
      "payload"
    ],
    "severity": 0,
    "source": "/some/path/asyncapi/pd/examples/kunde/CancelRequestForRevocationV2Example.json",
    "range": {
      "start": {
        "line": 2,
        "character": 13
      },
      "end": {
        "line": 24,
        "character": 3
      }
    }
  },

 {
    "code": "parser",
    "message": "A line break is expected",
    "severity": 0,
    "range": {
      "start": {
        "line": 1,
        "character": 2
      },
      "end": {
        "line": 1,
        "character": 2
      }
    },
    "path": [],
    "source": "/some/path/asyncapi/pd/markdown/kunde/CreateLegalBasisV2.md"
  },

Unfortunately, I am unable to share the complete asyncapi-file and the resource files, as this is an internal documentation site.

But from my limited testing it seems it is no longer possible to include external:

• example files
• markdown text
• schemas (?)

when using local references.

[jonaslagoni]

The reason you are getting these errors is that you are not allowed to have references there, i.e. what you have is indeed an invalid AsyncAPI document (this was a bug (but a feature for you I guess 😅) in the earlier version of the parser), see more here: asyncapi/parser-js#405

You are only allowed to have references where the type is marked as xx | Reference Object in the spec: https://www.asyncapi.com/docs/reference/specification/v2.6.0#channelItemObject

Otherwise, you will need some kind of pre-processing 🙂

[eivindga]

Yikes! This will cause us a lot of work, not to mention a big headache maintaining this enormous asyncapi-file. 😕

Is there a known pre-processor we could use that would let us continue having resource files outside the asyncapi-yaml?

[jonaslagoni]

This sounds like the perfect reason to suggest that we change the spec, and allow you to do this 🙂 Because you have the use-case for it. Because it would be a non-breaking change, it should be relatively easy to champion, what do you think about that? At least won't hurt to suggest it 🙂

There is no pre-processor, no, cause well, we haven't had this issue before now 😅

Maybe you can use the bundler: https://github.com/asyncapi/bundler 🤔 I, for example, use it myself as part of the release flow for AsyncAPI documents. So whoever uses the AsyncAPI document down the line (documentation, HTML generator, react component, etc) doesn't need to handle local references (I am using valid AsyncAPI documents however, not sure if it works for your case).

But yea, I would definitely suggest the change in the spec itself anyway, cause then it will no longer be an invalid AsyncAPI document 😄

[derberg]

Closing as it is not generator related. It must be first fixed in the spec, then by default it will be fixed in parser, and automatically will be fixed in generator without any special action