GDScript: Fix doc comment parsing (newlines and spaces)

[dalexeev]

• Do not add an extra space after [br].
• Do not add an extra space at the end of codeblock.
• Fix text overlapping after [/codeblock].
• More correct processing of codeblocks.
• @tutorial and blank line no longer resets in_codeblock in class doc comment.

## Line
## Continue[br]
## New Line
## [codeblock]
## if true:
##     print(42)
## [/codeblock]
## Test
func test():
    pass

Before:

After:

Another test:

## Line
## Continue[br]
## New Line[br]
## Test[br]Test
## [codeblock]
## test
## test[br]
## test[br]test
##
## @tutorial
## [/codeblock]
## test

Closes #76815.

[akien-mga]

CC @anvilfolk

[anvilfolk]

I keep meaning to give this one a review but everyone in my household is currently sick and I'm not super familiar with this part of the code. Some comments would be lovely, explaining what the functions do, why some ints used in for start at 2, etc :)

I didn't know [codeblock] could be used in docstrings, that's very cool!

[dalexeev]

The main problem that this PR solves is the correct detection of line join, depending on whether the current line is inside [codeblock][/codeblock] or not, and also whether the previous line ends with [br] or not.

• Outside codeblock:
  • If the text is empty, line join is empty.
  • If the previous line ends with [br], line join is empty.
  • Otherwise line join is one space.
  • Extra spaces must be removed.
• Inside codeblock:
  • All spaces and newlines must be kept as is (except for the initial indent after ##).

For example, the following comment

## Line
## Continue[br]
## New Line[br]
## Test[br]Test
##   Test
## [codeblock]
## test
## test[br]
##     test[br]test
## [/codeblock]
## test

should be converted to

Line Continue[br]New Line[br]Test[br]Test Test
[codeblock]
test
test[br]
    test[br]test
[/codeblock]
test

Also, although this is not a recommended syntax, the following comment

## aaa[codeblock]bbb[/codeblock]ccc[codeblock]ddd[/codeblock]eee

should be converted to

aaa
[codeblock]
bbb
[/codeblock]
ccc
[codeblock]
ddd
[/codeblock]
eee

I recommend "Hide whitespace" to make diff easier.

[dalexeev]

Added check for [code][/code] (for example [code][codeblock][/code]).

[akien-mga]

LGTM, though TIWAGOS. I haven't reviewed in depth but it's also touching a non-critical part of the parser (only affects editor behavior) so it seems fairly safe to me to merge.

[anvilfolk]

My apologies - I said I'd try to take a look but didn't. I am absolutely swamped with childcare and nightschool. I am definitely too tired to make sense of this right now.

The only thing I would say is that my personal preference is for a few more in-code comments. For instance, I would love to know ta a glance what the difference is between DOC_LINE_IN_CODE and DOC_LINE_IN_CODE_BLOCK. I assume it's the back-quote vs the actual [codeblock]?, thus allowing back-quotes to have multiline strings?

I'm taking a super quick look now to look for minor things since I definitely can't currently make sense of the logic, but I wouldn't really stop that from merging.

[dalexeev]

For instance, I would love to know ta a glance what the difference is between DOC_LINE_IN_CODE and DOC_LINE_IN_CODE_BLOCK. I assume it's the back-quote vs the actual [codeblock]?, thus allowing back-quotes to have multiline strings?

DOC_LINE_IN_CODE - in [code][/code]
DOC_LINE_IN_CODEBLOCK - in [codeblock][/codeblock]

[anvilfolk]

For instance, I would love to know ta a glance what the difference is between DOC_LINE_IN_CODE and DOC_LINE_IN_CODE_BLOCK. I assume it's the back-quote vs the actual [codeblock]?, thus allowing back-quotes to have multiline strings?

DOC_LINE_IN_CODE - in [code][/code] DOC_LINE_IN_CODEBLOCK - in [codeblock][/codeblock]

You're right, once I finally got through parsing some of the code it wasn't so hard to find :)

[akien-mga]

Thanks!