Documentation generation for GDScript

[ThakeeNathees]

Describe the project you are working on:
Working on documentation system for GDScript as part of GSoC 2020
The implementation : godotengine/godot#39359 godotengine/godot#41095

Describe the problem or limitation you are having in your project:
The in-engine documentation is very useful as we could browse and search the whole API offline, without leaving the editor. But for those who design plugins and library for godot are lack of this feature, the users of it have no way to understand the API and how to use it.

previous discussion on this topic:
[0] godotengine/godot#4370
[1] #177
[2] godotengine/godot#35716
[3] godotengine/godot#23188
[4] #408

---

Describe the feature / enhancement and how it helps to overcome the problem or limitation:
Implementing documentation support for GDScript with annotations(@vnen is working on it for the new GDScript parser #828).

• The parse tree has all the information about a property/function (name, data type, return type, setter, getter, default value, value of a const, ...) which will be extract to the DocData of a script and from the annotation It'll gain the the description, brief_description, tutorials, links, etc. and then the DocData will be updated with the Engine's helper window in real-time.

• With the documentation property of a GDScript, the code editor can show a brief description when a property/method is hovered and a description when auto-completing for the property/method. The description of an exported var could also be used as a tooltip[2].

• Since the Godot helper window supports bbcode, the documentations could contain images, colored text, alignments, bold, underlines, etc.

• An XML version of the documentation could also be generated for the plugin/library and it could be exported to HTML by the designer for his/her website.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

[WIP] doc comment (EDIT initial proposal change from annotation to doc comment)

[WIP] note: description and tutorial-link was hard coded since it isn't integrated with annotation yet.

[WIP] description when auto completing

any suggestions, modification, ideas, related features to include with this are welcome.

---

If this enhancement will not be used often, can it be worked around with a few lines of script?:
No, It was a wanted feature for a long time and there is no simple work around

Is there a reason why this should be core and not an add-on in the asset library?:
there is no way to implement this with an add-on

[YuriSizov]

I'm not sure I like the proposed syntax. It's pretty verbose, not easy to read and it has a downside of replacing comments completely for code documentation

I'd argue, that an inline documentation should be completely readable without any external tools. Currently, text is all over the place, aligned randomly and spread into several annotation statements. It'd be better to use specially formatted comments instead of such annotations, like shown here, for example. This way the descriptions are perfectly readable without any syntax highlight.

[willnationsdev]

@pycbouh

Well, first of all, he's saying that he can get the necessary information directly from the parse tree. And for that matter, you could also get it from the generic Script reflection methods (like get_method_list() which will also tell you the same information. If you're going to have a doc-comment like the one in the example, then you can just use static typing since the doc information would become implied from the given type hints.

I don't really see a significant difference between...

func foo(bar):
    """
    @method foo
    @param bar extends Node
    @return extends Reference
    """
    return bar

and what could already be there in vnen's current version:

func foo(bar: Node) -> Reference:
    return bar

Cluttering the code with annotations that don't actually contribute any meaning would be wasteful. In fact, this is better, since it catches the error that is ignored in your sample, i.e. can't return a Node if expecting a Reference. It's more concise too.

If you really did want to have custom information associated with each detail, then I'd recommend vnen supporting a syntax that allows you to switch to a block that directly supplies the information as bare words, and the syntax highlighter can just auto-interpret it as string, optionally in an indented sub-block.

@param bar:
    some description
@return:
    some description
func foo(bar):
    return bar

Here it is for the class definition:

@brief_description:
    An image processing library for Godot
@description:
    This library contains multiple image processing algorithms which
    works with [Texture] node. Call [method ImageLib.to_gray] to convert
    image texture to gray scale.
    Here is a demo screenshot [img]res://addons/image_lib/demo.jpg[/img]
    [codeblock]
    func _ready():
        var im = ImageLib.ImageRGB.new()
        var gray = im.to_gray()
    [/codeblock]
@tutorial:
    http://gd-image-lib/tutorial.html
class_name ImageLib

WDYT?

Also, personally, I would much prefer it if the information by which we derive the DocData information comes from the core Script interface, that way we can have a single framework by which to generate class documentation that is consistent between all programming languages. That is one of the goals of #22 anyway which has gotten a lot of support from the community.

[aaronfranke]

Where does "Warning: don't use ImageLib.some_old_method" come from? This text does not exist in the example code, but it does exist in the example output. Also, your example doesn't follow the GDScript style guide since it uses ' for strings instead of ".

Aside from that, I think it would be nice to have something similar to the discussion here.

# Called once per frame.
# @param delta - Time passed since last frame.
func _process(delta):

This way, existing comments above methods would automatically be usable as documentation, and it would be very simple to write. The first part of the text would be interpreted as the description (or maybe it should be the brief description), until an @something changes the context.

Additionally, I would love if there was a macro for easy generation of documentation blocks, similar to /// in C#. Perhaps typing ## before a function should create something like this, and then users would be expected to fill out the lines with the description:

##
## @param delta
func _process(delta):

[willnationsdev]

Is vnen planning on having a distinct separation between language annotations and comment annotations? I mean, if that's something the community is willing to do, then I'd be good with @aaronfranke's proposal too, since it allows people to differentiate between comments (#) and doc comments (##). But I'd prefer that as much information be inferred from the type hints in the actual code so people don't have to re-iterate information between their comments and their code.

[YuriSizov]

@willnationsdev

If you're going to have a doc-comment like the one in the example, then you can just use static typing since the doc information would become implied from the given type hints.

I've only linked the example to share the look I'd go for, not the features that it describes. When I think of inline documentation in GDScript I think about descriptions, and not class and type hints. We don't need those, we are not JavaScript, we have a type system and a proper way to declare things.

If you really did want to have custom information associated with each detail, then I'd recommend vnen supporting a syntax that allows you to switch to a block that directly supplies the information as bare words, and the syntax highlighter can just auto-interpret it as string, optionally in an indented sub-block.

Yes, that would work as well. Though vnen has not yet confirmed that such syntax will be available (as far as I remember the annotations issue). And the downside here is that this will still completely replace the existing comments, instead of relying on them. It would be nice to have existing, simple comments be interpreted as class, property and method descriptions out of the box.

[Calinou]

GDScript Docs Maker is worth a look at too.

[vnen]

The thing is that comments are a hassle to parse. The tokenizer currently can simply discard all the comments. It can be done but I think it might be an overcomplication of the system.

The only advantage of comments is that doing:

# Player speed in pixels per second.
var speed = 100

Is a bit easier in the eyes than:

@description("Player speed in pixels per second.")
var speed = 100

But then there are trickier things: brief/long description for class, tutorial links, probably something else that I'm missing.

When it comes to functions, this could potentially be solved with docsrings:

func clamp_speed(speed: float) -> float:
    """Clamps the speed value into the game limits and returns the result

    Parameters:
    speed: The speed value to be clamped.
    """

Which also needs a special parser for this, but it's more convenient because the parser is seeing the string there, so it can be extracted while parsing the function. (I believe that the docs currently don't support description for parameters, so the whole string could be used as description). But then this would only work for methods (maybe classes) and not anything else.

Using annotations is the "easy way out" because they are already parsed and their contents can be easily retrieved.

---

If you really did want to have custom information associated with each detail, then I'd recommend vnen supporting a syntax that allows you to switch to a block that directly supplies the information as bare words, and the syntax highlighter can just auto-interpret it as string, optionally in an indented sub-block.

Yes, that would work as well. Though vnen has not yet confirmed that such syntax will be available (as far as I remember the annotations issue).

Note that the syntax proposed there would conflict with this proposed here. There it applies the annotation to all statements in the block while here it uses the block as a string argument for the annotation. Those cannot coexist.

---

I still think annotations is the better compromise because they're much easier to parse. We can probably improve the look somewhat, maybe even with the highlighter, but it's simpler to use annotations than something else.

@ThakeeNathees if you have some suggestions to improve this based on feedback (and you are comfortable implementing them), feel free to update the proposal. I'm personally okay with it so far.

[ThakeeNathees]

• comments as documentations are only for some thirdparty tools like Doxygen. bringing it to the language parser will simply break everything. for an example (without ignoring the comments)

func _ready():
	if true:
		print("true", ## not the second arg
			## still not the second arg
			"i'm the second arg")
	## now the if doesn't have an else
	## and there is an else below without an if
	else: ## this comment breaks the indentation of the next print
		print("false")

I'd argue, that an inline documentation should be completely readable without any external tools.

yeah, we can't go with the external tools, so can't use comments.

• Using string literal is easier to implement and would work for function and maybe for classes (even no way to distinct between description, brief_description, tutorials) but what to do with member variables, signals, enums, constants ?? using docstring for functions and something else for enums brings inconsistency, (if you have any idea on how to resolve this, a mock up pice of code would be helpfull).

• for those reasons annotation is the better deal, and the documentation can become the part of the reflection. like @vnen said we could improve the look of the annotations, may be use editor code folding, sinppet generation, etc.

[YuriSizov]

@vnen

I still think annotations is the better compromise because they're much easier to parse.

I understand what you are standing for here, but your argument is completely about how easy or hard each proposal is to deal with internally. While a small, beautiful, easy to maintain implementation is what any programmer would want, we shouldn't make every decision about language based exclusively on how easy it is to implement.

Regarding complicating the parser, though, usually documentation comments use some special tokens, that can be parsed as both comments and something else. While it would complicate some things, it shouldn't make parser unbearable to deal with.

@ThakeeNathees

yeah, we can't go with the external tools, so can't use comments.

You completely missed my point. Any code editor with its syntax highlighting and IDE-lite features is what is here referred to as "external tools". This includes the built-in Script editor and anything attached to the GDScript language server. My point is that you shouldn't need to use any editor to read those docs when dealing with code, and without a powerful tool by your side the syntax proposed by you is hard to parse by a human.

Also your example is not how comments are usually used for documentation. Look at JSDoc

/**
 * Represents a book.
 *
 * @constructor
 * @param {string} title - The title of the book.
 * @param {string} author - The author of the book.
 */
function Book(title, author) {
}

and documentation comments in C#

/// <summary>
///    This method changes the point's location by
///    the given x- and y-offsets.
/// <example>
///    For example:
///    <code>
///       Point p = new Point(3,5);
///       p.Translate(-1,3);
///    </code>
///    results in <c>p</c>'s having the value (2,8).
/// </example>
/// </summary>

public void Translate(int xor, int yor) {
    X += xor;
    Y += yor;
} 

In my opinion you can read those easily without any tools, given that they are properly formatted (I've disabled syntax highlighting on purpose). And what is proposed here is too busy to be used outside of Godot Editor, if even there.

---

To both of you, guys, let's make sure that the implementation is easy to parse for a human first and foremost, and for a machine only second. We shouldn't sacrifice our ability to read code for a prospect of documentation 🙂

[YuriSizov]

Note that the syntax proposed there would conflict with this proposed here. There it applies the annotation to all statements in the block while here it uses the block as a string argument for the annotation. Those cannot coexist.

What if we allow annotations to apply to comments, though?

@description:
  # Player speed in pixels per second.
var speed = 100

So it still is applied to the block and not uses whatever comes next as its string content.

Would that be easier to parse? Because to me it would be easier to read, probably.

[willnationsdev]

Note that the syntax proposed there would conflict with this proposed here. There it applies the annotation to all statements in the block while here it uses the block as a string argument for the annotation. Those cannot coexist.

If the normal behavior is something like this...

@export:
    var a := ""
    var b := 0

...resulting in a and b both being exported, and for that reason, we cannot support this...

@description:
    Some information.
    Some more information.

...because it would be equivalent to...

@description("Some information.")
@description("Some more information.")

...then having commented-out results wouldn't make a difference since multiple lines of comments would still generate multiple annotations. For example...

@description("# Player speed in pixels per second.")
@description("# Example: [code]velocity = delta * speed[/code]")

However, couldn't you resolve the issue by just saying that, for these types of documentation-related annotations, you instead build the results of multiple same-named annotations by joining them with newlines? In which case, the color block style would be consistent again. However, you'd still need to have the block use the comment syntax that @pycbouh proposed in order to keep it from registering the text as full-on property declarations and whatnot. Either that or allow the block to just be a big docstring.

@description:"""
    Player speed in pixels per second.
    Example: [code]velocity = delta * speed[/code]
"""
var speed = 100

@description:
    # Player speed in pixels per second.
    # Example: [code]velocity = delta * speed[/code]
var speed = 100

The below example looks a lot cleaner.

Edit: removed reference to needing backslashes in docstrings since...you don't need them (forgot for a sec). In which case, either strategy really works, honestly.

[vnen]

However, couldn't you resolve the issue by just saying that, for these types of documentation-related annotations, you instead build the results of multiple same-named annotations by joining them with newlines? In which case, the color block style would be consistent again. However, you'd still need to have the block use the comment syntax that @pycbouh proposed in order to keep it from registering the text as full-on property declarations and whatnot. Either that or allow the block to just be a big docstring.

Well, adding a feature (block annotations) with many exceptions baked in right from the start sounds like the wrong way to go IMO. Either we do one or another (though I'm not convinced by either yet).

Also, I don't see a lot of difference between this:

@description:"""
    Player speed in pixels per second.
    Example: [code]velocity = delta * speed[/code]
"""
var speed = 100

@description:
    # Player speed in pixels per second.
    # Example: [code]velocity = delta * speed[/code]
var speed = 100

And this:

@description("""
    Player speed in pixels per second.
    Example: [code]velocity = delta * speed[/code]
""")
var speed = 100

You can also close the string in the last line:

@description("""
    Player speed in pixels per second.
    Example: [code]velocity = delta * speed[/code]""")
var speed = 100

We can also change the annotation name to be shorter and less noticiable:

@doc("""
    Player speed in pixels per second.
    Example: [code]velocity = delta * speed[/code]""")
var speed = 100

Which is not very far from a comment and it's much easier to deal with code-wise.

For example C# uses the XML tags, which is some required structure to deal with docs. In GDScript we can use annotations for the same effect.

[YuriSizov]

Another point to consider: languages using pure comments for inline documentation allow for things like attributes and code-affecting annotations to be visually different.

Compare this:

/// <summary>
///    This method changes the point's location by
///    the given x- and y-offsets.
/// <example>
///    For example:
///    <code>
///       Point p = new Point(3,5);
///       p.Translate(-1,3);
///    </code>
///    results in <c>p</c>'s having the value (2,8).
/// </example>
/// </summary>
[System.Runtime.InteropServices.DllImport("user32.dll")]
[CustomAttribute("argument", Flag=true, AnotherFlag=false)]
public void Translate(int xor, int yor) {
    X += xor;
    Y += yor;
} 

To this:

@description("""
    Set player's speed in pixels per second.
    Example: [code]set_speed(velocity * delta)[/code]""")
@tutorial("https://example.com/set_speed_tutorial.html")
@connect(input_control, "value_changed")
@custom_annotation("argument", true, false)
func set_speed(value : float) -> void:
    speed = value

It may not be the worst case, but it's an important idea to consider.

[Zylann]

I dislike the use of annotations for this. See my comment about it: godotengine/godot#20318 (comment)

Also the fact this is only about GDScript, while Godot supports C#, NativeScript and other, it means something more should still be provided for Godot to have script docs like this.

I started an doc extraction addon recently but Calinou made me realize a hidden feature (access to AST through language server?) which allowed to implement this https://github.com/GDQuest/gdscript-docs-maker

Unrelated note: the examples imply the use of class_name to document a script, but class_name is optional. That means to document a class you would have to annotate... in the void? Unless script-wide annotation already works without, but I'm doubtful about it cuz you could start a script with nothing but a sub class definition or a member, which makes such annotation ambiguous. Perhaps that should also be mentionned on the annotation GIP, but such a problem would arise only for doc so far.

[willnationsdev]

Also the fact this is only about GDScript, while Godot supports C#, NativeScript and other, it means something more should still be provided for Godot to have script docs like this.

This is really important imo. It won't do much good if the only language that can take advantage of documentation generation is GDScript.

I dislike the use of annotations for [documentation].

In this sense, I think it comes down to whether enough people absolutely don't want annotation-based in-language documentation such that it's worth the effort to implement an alternative, more complex parsing workflow for comments.

So, we pick from one of the options below, unless someone else has another idea, yeah? Should probably send out a poll or something (which highlights how the decision is one between implementation simplicity/maintainability and usability/readability).

@doc("""
Player speed in pixels per second.
Example: [code]velocity = delta * speed[/code]""")
var speed = 100

# Player speed in pixels per second.
# Example: [code]velocity = delta * speed[/code]
var speed = 100

[ThakeeNathees]

In this sense, I think it comes down to whether enough people absolutely don't want annotation-based in-language documentation such that it's worth the effort to implement an alternative, more complex parsing workflow for comments

yeah, I've discuses with @vnen and planed to implement it with comments by collecting comments separately and when compiling it, apply the comments as documentation string to the properties.

so the documentation will look like

## Player speed in pixels per second.
## Example: [code]velocity = delta * speed[/code]
var speed :int= 100

## Set player's speed in pixels per second.
## Example: [code]set_speed(velocity * delta)[/code]
func set_speed(val:int)->void:
    speed = val

## this signal emitted when ... 
signal time_out

not sure how class description , brief description, and class tutorials would be. any mock piece of code with those would be helpful.

@willnationsdev @Zylann @pycbouh @aaronfranke

[willnationsdev]

@ThakeeNathees If it were following conventions from other comment-based dynamic programming languages, (PHP, Python, etc.), then I'm thinking you'd have @-based annotations within comments that separate fields, and then if none are given, it just all defaults to a summary/description value.

# Full version
## @brief A summary here
## @description Blah blah something descriptive here and
## continues between newlines until another doc annotation.
## @tutorial https://website.com/tut
var speed: int = 100

# Abbreviated version with only description
## Blah blah something descriptive here and
## continues between newlines until another doc annotation.
var speed: int = 100

# Combo version with description and other custom doc comments.
## Blah blah something descriptive here and
## continues between newlines until another annotation.
## @tutorial https://website.com/tut
var speed: int = 100

Edit: Also, in a scenario such as this, it would be easier on the eyes if doc comments had a slightly different syntax highlight color than regular comments. Also helpful would be if doc annotations within doc comments were additionally highlighted

Edit 2: This article has an example of syntax highlighting for PHPDoc.

[YuriSizov]

Handling indentations on continued lines correctly also helps with visibility, like in this case:

## @description Blah blah something descriptive here and
##              continues between newlines until another doc annotation.

[ThakeeNathees]

@pycbouh currently I've planed to strip edges unless it's inside a [codeblock] and the first lines of comments starting with "##_" before any variable, functions, subclasses, constants, signals are considered class description and they are treated like this. any suggestions?

-------------------------------------
## this treated as brief description of the class
## as it doesn't have any @ annotation
-------------------------------------
## @brief: this is also brief description
## @desc: now this is description and
##        the edges are striped to make it as a whole paragraph
## @tutorial: http://www.mysite.com/tutorial/
-------------------------------------
## this is brief description
## @tutorial: http://www.mysite.com/tutorial/
## so this must be description
-------------------------------------
## @desc: the description
## @tutorial: http://www.mysite.com/tutorial/
## so this must be brief description
-------------------------------------
## @brief: brief description
## @desc: description
## @tutorial: http://www.mysite.com/tutorial/
## this line is not a part of the documentation <-----------

[YuriSizov]

@ThakeeNathees I think implicit description should only be generated based on commented lines before any @ keywords are used. So examples 3 and 4 are just as invalid as the 5th. Less ambiguity this way.

I think we need to drop either @desc or @brief in favor of fully relying on implicit descriptions. Probably should make implicit lines be @brief, while a full @description field can be used for a more detailed documentation, if one is required. From my experience it would be natural to do something like this:

## Get all character data from a string
##
## @usage: get_chars("My example string")
## @description: For an arbitrary string value generate a complete set of character data.
##               Returned value is an array of dictionaries, following this structure:
##               [codeblock]
##               {
##                 "character": "M",
##                 "utf8_code": "004D",
##               }
##               [/codeblock]
## @seealso: [method get_chars_unicode]
func get_chars(value : String) -> Array:
  pass

[aaronfranke]

@pycbouh That is alignment, not indentation, since it depends on each extra line being offset the same number of characters to match the first one. I think it's much better to do this, since it doesn't depend on the user's tab width but still allows tabs:

## @description
##	Blah blah something descriptive here and
##	continues between newlines until another doc annotation.

[YuriSizov]

I don't think that handling both cases is mutually exclusive. 🙂

[Zylann]

Just thought a bit about the script-wide doc block, and I think it can simply be solved by docs being separated by a line.

If a script contains this:

## @description This is the main class

class SubClass: # Happens to be undocumented
    # ...

or

## @description This is the main class

## @description This is the sub class
class SubClass:
    # ...

Because at least one space (or other non-doc-able symbol, like extends or class_name) separates the doc and whatever symbol that follows, we can fairly assume it's not about the subclass, but the script itself. So the defining condition would be, being before any doc-able symbol, and not directly above one.

[ThakeeNathees]

the main class description is inside the class block itself, IMO subclass doc should be like that

extends Node

## the brief description of the main class
## @desc: and the description of the main class
##        they are inside the class

class SubClass:

    ## brief description of the sub class
    ## @desc: similarly for sub class descriptions
    ##        should go inside it's class

    ## but other properties description
    ## will be above the declaration
    var x

@Zylann

[Zylann]

@ThakeeNathees but that doesn't solve the problem, because you have the same situation between the class description and the var in you snippet. So it should still be possible to have docs be just above the symbol they are documenting.

[ThakeeNathees]

@Zylann Sorry didn't get it. every documentation of a property should be right above it without any spaces and a big block of comments in the beginning of the file applies to the class (brief description, and the description). But if it touches a variable (no empty line in-between), then the description applies to the variable instead.

[Zylann]

@ThakeeNathees so since line separation is handled, that means we don't have to put docs inside sub-classes, they can be on top like everything else. The only natural exception is the script itself, where it can be at the very top.

[ThakeeNathees]

every class doc (including main script class and it's subclasses) can omitted if it's unnecessary, but to have them they must be written before all variable, functions, subclasses (and if it's a sub class then it's subclasses), constants, signals but the class_name doesn't considered since it's optional and it need an empty line after that ( or except class_name, non-doc comment are ok)

[YuriSizov]

Try to imagine those with syntax highlighting, the existing one plus some color for the @tags (maybe different shades per tag?

There is your problem with your first suggestion, though. We are getting annotations in 4.0's version of GDScript, and those tags syntactically mix with them. Without syntax highlighting it would be harder to read, and for syntax highlighting to work properly, parser would need to understand if it's an annotation or a documentation "tag".

I also don't see a point in adding comments to the documentation or in commenting out parts of it.

[NathanLovato]

I also don't see a point in adding comments to the documentation or in commenting out parts of it.

Agreed.

The proposal above makes writing docs more complicated than it needs to be. If you write actual API docs, you'll see you don't need that much functionality or special highlighting. You spend way less time to write and update code docs than to design code in the first place, so you don't need much functionality there.

I'd invite everyone who's commenting on this proposal to write some real-world API docs in at least two different languages. You'll quickly realize that a simple solution works well, that it doesn't make the code hard to read or to manage.

Python and Emacs Lisp, where the convention is to document every function or module, sometimes with extensive inline code documentation, just have docstrings, pretty much no special markup, and it works well. JSDocs works great too, and it's just regular comments with some @keyword - . Same for C#, although it uses XML and text editors typically give you a template with the tags. But IIRC besides that, it's regular comments.

Again, there should be a need to justify more complex syntax or alternatives, like simpler solutions causing concrete problems in production.

[nonunknown]

guys I was testing here and variables starting with _ are still shown in the Property Description section, and this shouldnt happen

I think a special word should be make, because there's some situation where the user wants the "private" variables to be shown
so something like this at the top of the script:

extends Node
class_name Test

## ignore _

[Calinou]

I think a special word should be make, because there's some situation where the user wants the "private" variables to be shown
so something like this at the top of the script:

The documentation generation system is intended to document public APIs, so private properties and methods shouldn't be displayed there. I would just exclude properties whose name start with an underscore. However, we can't do that for methods as virtual methods start with an underscore.

[YuriSizov]

Well, GDScript doesn't have virtual methods, so we indeed can do that for methods. We could also add something like an annotation to denote GDScript methods as virtual, if that's an important designation. Then we could include them in the docs regardless.

[NathanLovato]

Documenting virtual functions is pretty important yes. For instance, we worked on some AI framework and you need virtual functions for users to plug into the system. We write them with a leading underscore for consistency with Godot. We do have something similar where we mark functions as virtual with a comment above the definition, to distinguish them from pseudo-private stuff.

[vnen]

@nonunknown please open an issue on godot/godotengine about this. Comments in a closed proposal will likely be missed and forgotten.

[Calinou]

For virtual annotations, there's already a proposal: #1631

[nonunknown]

@vnen there's a proposal for that already create by @ThakeeNathees which is #4370 So that makes no sense! unles I report as bug instead!

[vnen]

@nonunknown but it is a bug: variables starting with _ are considered private and should not show in the help window. This is what the docs say: https://docs.godotengine.org/en/latest/tutorials/scripting/gdscript/gdscript_documentation_comments.html#documenting-script-members

If it's not working like that then it's a bug.

[nonunknown]

ok awesome then, I'll report! thanks.

[marcinn]

Why not markdown? It is highly readable as a pure text. I started documenting with bbcode, but comments are unreadable in source files.

[YuriSizov]

@marcinn Because we already have a format for documentation markup, we're just reusing it, and markdown also doesn't work well with GDScript comments syntax.

[marcinn]

Thanks for the explanation, @YuriSizov.

Adding the markdown parser library shouldn't be difficult. The syntax is a problem. So... how about introducing python-style docstrings? They will not clash with the markdown syntax, and docstrings will gain readability. Both styles (bbcode and markdown) can live together until G5 or G6 (until bbcode is removed).

[YuriSizov]

BBCode is not going to be removed. And docstrings are not going to be added. You can still use multiline strings to document your code if you want, but this syntax that is implemented here is for the built-in help, so it must be written in a way compatible with that. There isn't much benefit in adding multiple ways to describe classes with multiple parses to maintain.

[marcinn]

There is a huge benefit from markdown - readability. As a human you aren't forced do parse and render bbcode tags in your head when reading and maintaining sources. EDIT: Also note that bbcode is a markup used in fairly old forum engines, while devs use markdown mostly.

[YuriSizov]

There is a huge benefit from markdown - readability. As a human you aren't forced do parse and render bbcode tags in your head when reading and maintaining sources.

Yes, and this is not for human readability. It's for the built-in documentation generation. A different medium, that has specific inner workings. We need special markup that is designed to work for the purposes of the built-in documentation. Even if changing all of our markup language to (a more limited) markdown would be up to consideration, that's not something we can change now, for Godot 4, without introducing excess and confusion with a secondary system as you propose.

That said, most of the documentation can be written with simple ## Description syntax. You only lose readability if you start to add more elaborate formatting to it.

Also note that bbcode is a markup used in fairly old forum engines, while devs use markdown mostly.

Prejudice aside, markdown is a limited language, and its only expansion is to embed HTML. Which makes it even harder to parse and more prone to vulnerabilities. Godot uses bbcodes for rich text markup throughout its rich text controls, and you can create your own tags for custom effects and extended functionality. It serves different purposes. Documentation is built on top of that and leverages its powers.

[marcinn]

Yes, and this is not for human readability. It's for the built-in documentation generation

I'm trying to say that Markdown (properly extended) will solve both things.

that's not something we can change now, for Godot 4,

That's why I proposed a period of N years, where two markups will live together, then depreciate the old one. I believe that most devs would pick Markdown.

You only lose readability if you start to add more elaborate formatting to it.

That's my problem. I focused on documenting my addon, and I lost a lot of readability. The square brackets syntax used for classes, enums, etc, is pretty fine. But the problem is with tags like [code], [b], newlines with [br], missing paragraphs, and lists, which never worked for me (note that plain text asterisks and dashes are rendered in a single line). I have mess in the doc, or mess in the source file. And I can't have both good enough until something like Markdown will be available (because of bbcode syntax).

Godot uses bbcodes for rich text markup throughout its rich text controls

Ah yes, I read about this yesterday. But it's a different purpose than writing docs, don't you think?

Documentation is built on top of that and leverages its powers.

Markdown syntax can be extended in a similar way, I think.

---

Yuri, my goal is not to force you to do things you don't want. I'm just asking and suggesting solutions from a dev perspective (and as far I understand, the engine and editor are created for devs). There are issues with bbcode in docstrings. Some can be solved by fixing bugs (like extra whitespaces in docs which comes from [br]), but others will never be solved because of the bbcode syntax. I can create proposal and some prototype, if there is a chance to improve Godot on that field.

Thanks for your time.

[YuriSizov]

I'm trying to say that Markdown (properly extended) will solve both things.

It will not, for the reasons I provided.

That's why I proposed a period of N years, where two markups will live together, then depreciate the old one.

We do not practice having two competing systems that do the same thing. This is not user-friendly, and it creates unnecessary maintenance burden. And that also assumes that we want to remove "the old one". You propose a long-term commitment to a hacky solution, while creating an extra point of friction for users who would have to choose between different systems. That's not to mention that we would have to either parse MD ourselves, which is not a trivial task, or integrate yet another library for a very minimal feature.

Ah yes, I read about this yesterday. But it's a different purpose than writing docs, don't you think?

It's relevant, because this is the basis for the entire system at hand. We can natively and efficiently support documentation comments with the same capabilities that are available in Godot's own documentation and in Godot's own rich text controls. And it gives us tools for customization that Markdown doesn't have.

Markdown syntax can be extended in a similar way, I think.

It cannot, it doesn't have a capacity for extension in the same way.

There are issues with bbcode in docstrings. Some can be solved by fixing bugs (like extra whitespaces in docs which comes from [br]), but others will never be solved because of the bbcode syntax.

These are not issues. These are personal preferences. Verbosity of the syntax is hardly a blocker for most devs — some other systems are even more verbose (C# for example, which uses XML for their documentation generation). Just because you prefer Markdown and are used to Markdown, doesn't mean it's the best option that we have here. And I'm providing you rationale for the current system (if the preceding discussion is not enough), and explain technical limitations of Markdown that we would have to overcome or live with.

You can open a proposal, but there is a number of reasons why any change is unlikely, and your stop-gap solution would be even worse. Just to reiterate:

• The existing system comes at a relatively low cost for the engine, leveraging the power of the existing engine systems for rich-text content. This allows us generate rich documentation pages on par with built-in documentation, without compromises or limitations for users.
• This feature has been discussed for a long time, and various syntax options have been considered. It's naïve to come to the discussion at this point with "Why not markdown" as if no other solution has been considered, but I'm fine discussing this anyway so you can have a form of closure instead of the question hanging forever answered.
• The implementation is still very much readable, even though it can be unfamiliar to some. I've used quite a few programming languages, and they all have different syntaxes for documentation comments. Most don't have any formatting capabilities, actually. So IMO for most developers it will be sufficient to just write plain text descriptions, with very occasional formatting.
• Having two systems at the same time is a bad solution to any problem. It creates more frictions to users, who end up arguing which system is the best instead of writing code and making games. You can see examples of this in practically any programming language, starting with banal tabs vs spaces. Godot 4 only just released, so with your suggestion the entire lifecycle of this major release would have two competing implementations, distracting users from their work with rather pointless conflicts.
• Having two systems also multiplies several times the amount of maintenance. That's even ignoring the complexity of full Markdown support, whether first part or third party. We have to practice restrain when it comes to adding maintenance costs, and when it comes to adding third-party dependencies for that matter. This applies to absolutely any proposal, and is beyond this conversation.
• All that on a premise that Markdown is superior, while it clearly lacks customization options that we need with our "domain-specific" markup language. And while it conflicts with the comment syntax in GDScript (which only has single-line comments, by the way). I would say, that RST from Python's docstrings would have a higher chance of adoption, as we already use this for online documentation. But then again, it requires a parser.

In short, it's too late. And you don't propose a clearly superior solution. And your stop-gap would be all sorts of problematic. And it's all based on a personal dislike or unfamiliarity for bbcodes.

[marcinn]

It is not about personal preferences. It is about user experience and using right tools to do the job. Bbcode is not the right tool for docstrings created by devs. That's all, and it is quite simple. This is a markup ceeated for bboards/forums, as an safe alternative to html. As a result docs will not use bbcode and will not be good enough (readability of generated doc), or will be unreadable in sources.

Your answers are based on wrong assumptions. Markdown will solve issues mentioned by me, there is always good time to discuss about improvements, and nobody here wants to force these changes in 4.x.

And it is not about liking/disliking/knowing the syntax of bbcode, but about readability of the sources (and generated docs at the dame time). I wrote about this few times.

Regards,
Marcin

PS. To be clear - bbcode in richedittext is the right tool for the job. Markdown can introduce problems here. Docstrings are different, they aren't part of the runtime, so the engine may have no support of MD. Just the editor.

[vnen]

There is no point discussing when a PR for this was merged over 2 years ago. The time for suggesting changes for this implementation is already passed.

Anyone can open a discussion (or an actual new proposal) to bring forth their suggestions and arguments.