Parameters in Various Contexts

[tajmone]

@pml-lang, I'm currently focusing on how to correctly implement parameters in the syntax (see pml-lang/pml-userman#2).

I've been giving it some though in the past four days, and I came to the conclusion that it's important to get the implementation of parameters straight, from the beginning since they can occur in so many contexts. I fear that postponing it might create problems once the syntax becomes too entangled, because correct handling of parameters occurrences might influence the way nodes and attributes context are defined.

This is going to be a rather delicate process, requiring different approaches and extensive testing; so I'll probably keep the work in a dedicated dev branch since I'm happy with the results, and only then merge them into master.

But most of all, I'll like your opinion and guidance in the process, in terms of potential future usage of parameters, and which you think would be the best editing experience in their regard.

Tweaked Parameters Symbols Indexing

To begin with, I've just changed the way parameters and their definitions Symbols get indexed:

• Definitions are now indexed with the [const prefix (e.g. [const myParam)
• Occurrences are now indexed within angle brackets (e.g. <<myParam>>)

this way, in the Symbol panel it's easy to visually spot a parameter or its definition, since they appear mimicking their native syntax. This is useful when dealing a long list of symbols, because by simply typing [con or << in the Fuzzy search control the list will immediately be reduced to showing just parameters (either definitions or occurrences).

But the other reason I've done it was to enforce some consistency in the way they're indexed, because if a [ch title containing a parameter is indexed, it will be indexed with the brackets (since it's indexed verbatim), so I though that parameters should follow the same format also when indexed individually.

Implementing Correct Parameters Support

Now the challenge is to correctly handle all the various nodes and attributes that might contain a parameter inside them, to ensure that the parameter is also matched and scoped correctly.

Quoting you from pml-lang/pml-userman#2:

There are three valid contexts:

• the value of an attribute (e.g. title="foo<<p1>>bar")
• text contained in a node, including nodes that contain raw text, such as code or html. (e.g. [div foo<<p1>>bar])
• the value of a constant (i.e. a parameter declaration can use the value of a previously defined parameter in its value (e.g. [const p2 = foo<<p1>>bar]))

so the three examples you've provided are a good starting point.

In case of title="foo<<p1>>bar", what should happen is:

1. The foo<<p1>>bar string is indexed as a Chapter title in the Symbols index (i.e. as = foo<<p1>>bar).
2. p1 is also indexed as a parameter occurrence (i.e. as <<p1>>).

As far as coping goes:

1. The foo<<p1>>bar string is scoped as a Chapter title string (quoted in this example), including <<p1>>.
2. <<p1>> is also scoped as a parameter occurrence, albeit inside a string.

This is similar to how escape sequences inside string are handled — they're part of the string, but treated specially by the editor.

Themes Concerns

Usually most ST3 themes will cover proper highlighting of elements inside strings, like escapes, interpolations, etc.; but in the case of PML there are no guarantees that any given theme will be able to highlight parameters properly, because they are scoped as support.function.call (had to, to achieve Goto Symbol Definition) which is odd, since function calls are not expected inside literal strings.

So, we'll probably have to ship our own custom theme(s) to provide the best editing experience, like you suggested (and indeed, the Markdown package ships with more than four themes, because markdown sources simply look bad with generic ST3 themes).

In any case, I would focus on functionality over aesthetics, because proper scoping is the base for any additional functionality and plug-in — the more precise the scoping, the easier it will be to manipulate source files for all sort of operations (linting, migrating to newer PML versions, etc.).

[tajmone]

Parameters in Chapter Titles: Initial Results

I've started working on some preliminary test in the dev-params branch.

As it turns out, the syntax supports out-of-the-box parameters inside Chapter titles, which are correctly scoped, as the syntax-tests/syntax_test_Constant.pml test files shows:

[ch title = "PML v<<PMLVer>>" ]
!   ^^^^^^^^^^^^^^^^^^^^^^^^^   meta.annotation.node-attributes.chapter
!           ^^^^^^^^^^^^^^^^^   entity.name.section.quoted
!           ^^^^^^^^^^^^^^^^^   markup.heading meta.toc-list.heading
!                 ^^^^^^^^^^    meta.function-call
!                   ^^^^^^      support.function.call
!                 ^^            punctuation.definition.generic.begin
!                         ^^    punctuation.definition.generic.end

(it even looks good in terms of colours highlighting)

Nested Parameters Are Not Being Indexed

The problem is that the PMLVer parameter is not being indexed, and I have no idea why.

I'll have to carry out some tests, to see if the problem is due to the fact that there are two overlapping symbols, or if its the symbol transformations that are interfering. Or maybe I should just ask on the Sublime Text forum about this.

The problem here is that we need to capture the whole title string as a heading symbol, because we want to index it as a single symbol. If the problem is due to the parameter being nested into the heading (i.e. that ST can't handle overlapping/nested symbols) then we might have a problem — in the worst case, we'll have to renounce to indexing the parameter occurrence in order to save the heading, I guess.

[celtic-coder]

Yes, being aware of new versions in a straightforward way would be a great feature! 🙂👍

[pml-lang]

inability of being automatically notified when a new PML version is released

Yes, that's a pity and needs to be changed.

add to the root of the PML Converter repository a text file named pml_version.txt

Good idea. Done with version.txt.

I usually rely on RSS feeds

I was thinking already about setting up a Mailchimp account, so that people can register and receive news about PML. But some people would possibly prefer RSS.

[tajmone]

I was thinking already about setting up a Mailchimp account, so that people can register and receive news about PML.
But some people would possibly prefer RSS.

The advantage of RSS feeds is that they don't get blocked by SPAM filters.

Today email SPAM filters have become really nasty, the risk is not only that your newsletter doesn't get through, but that your IP and domain get blacklisted.

PS: If you ever need to setup a newsletter, let me know and I'll share with you some gems on how to do it without getting blocked or blacklisted. I've worked for a number of years in a company were I was in charge of handling websites, web advertisements and newsletters, and have learned how to handle lists with over 20.000 subscribers and successfully send as many emails without getting blocked. But I won't share these tricks of the trade publicly, only via private email exchanges. (We even tried Mailchimp, but wasn't as effective as we wanted it to be, so we dropped it in favor of our in-house solutions.)

[celtic-coder]

Hi Tristano (@tajmone),

What do you think of Buttondown as a possible subscription platform for a newsletter?

Kind Regards,
Liam

[tajmone]

What do you think of Buttondown as a possible subscription platform for a newsletter?

I don't know this particular service, so I can't really say. My general experience has been that the best solutions are the in-house solutions, and unless we're dealing with an open source community project I would tend to avoid relying on third party services for handling sensitive data as customers and subscribers emails addresses.

I also believe that none of these services can grant 100% delivery to all subscribers (and they will never achieve that goal). A good service probably delivers around 60% of all mails, at its best.

I'm subscribed to many newsletters, and I regularly see them getting trapped in GMail SPAM folder, regardless of the service they're using. It's possible to avoid this from happening, but the solution is not using similar newsletter online services, but well planned emailing strategies and the use of dedicated tools. Every single email sent requires planning and quite some work, if you want it to reach all subscribers and not getting blacklisted.

But in general, for open source community projects, we've been using newsgroup services like Yahoo! and Google Groups — especially the latter, since GMail doesn't block as SPAM other Google services, and GMail is definitely a very popular email service. Open source developers tend to publish their email address all over the place (from LICENSE files, forums, bug trackers, etc.) so usually you don' really have the concern of keeping emails private, unlike with other type of newsletters.