Support Rouge Formatters with other method signatures

[jonsgreen]

While working on a plugin for Bridgetown I realized that it was not possible to use some of the newer Rouge Formatters like Rouge::Formatters::HTMLLineHighlighter.new(formatter, highlight_lines: [3, 5]) for at least two reasons. First of all the first argument is a delegate for another formatter. Secondly, there is no way to pass the highlight_lines option through to the Formatter without doing some monkey patching.

Is there any interest in supporting these other Formatters from within Kramdown (rather than resorting to external patches)? Are you open to PRs on this and is there the bandwidth from someone with a deeper knowledge of the codebase to advise?

[gettalong]

You can use the :formatter option for this, see https://kramdown.gettalong.org/syntax_highlighter/rouge.html. All you need to do is write a class that respects the interface (see the link) expected the rouge syntax highlighter of kramdown (also see https://github.com/gettalong/kramdown/blob/master/lib/kramdown/converter/syntax_highlighter/rouge.rb#L33-L34).

[jonsgreen]

@gettalong Thanks for your prompt response! I am actually aware of the formatter configuration options and that part of the code. In fact that is what I am trying to deal with in this patch. The challenge is that from what I can tell you can only really pass in general configuration options to your formatter which is fine in most cases but for this newish line highlighting feature you need to pass in the highlight_lines options specific to a code block.

Ideally it would be possible to merge some of the call_opts here into the opts when appropriately needed by the formatter.

Does that make sense to you?

[gettalong]

I see, thanks for the explanation.

The call_opts are more for passing information back out to the converter, so this wouldn't help.

How would you represent the to-be-highlighted lines in a kramdown document?

[jonsgreen]

I am trying to port over the markdown blog from our website to Bridgetown and we are currently using Gatsby and this plugin to handle the line highlighting so I am trying to support the same line representation (i.e. ```ruby{1,3-5}) used there. I actually have a rough parser for that working in the same PR I shared above. It converts the line numbers into an array which is what the formatter expects. Of course we are happy to have this be supported from within Kramdown but did not want to make assumptions about what was possible.

Is that what you were asking?

[gettalong]

Yes, exactly. So in this case I would suggest you write a syntax highlighter that can parse that line representation and then delegate the rest to the rouge syntax highlighter.

You should, however, insert a question mark after the language name so that kramdown extracts the language correctly into the class attribute:

kramdown -o hash_ast
~~~ruby?{1,3-5}
test
~~~
^D
{:type=>:root, :options=>{:encoding=>#<Encoding:UTF-8>, :location=>1, :options=>{}, :abbrev_defs=>{}, :abbrev_attr=>{}, :footnote_count=>0}, :children=>[{:type=>:codeblock, :attr=>{"class"=>"language-ruby"}, :value=>"test\n", :options=>{:location=>1, :fenced=>true, :lang=>"ruby?{1,3-5}"}}]}

The full string is available in el.options[:lang] that is passed further to the syntax highlighter.

[jonsgreen]

@gettalong I am not sure what you are suggesting. I apologize if I am misunderstanding you but are you saying that we should write our own version of Kramdown::Converter::SyntaxHighlighter::Rouge ?

[gettalong]

@jonsgreen No, I suggested you write a syntax highlighter that uses the built-in rouge highlighter to do the highlighting. But the parsing of the highlighting numbers and instantiating the necessary formatter would be the job of your highlighter.

[jaredcwhite]

Just jumping in here as @jonsgreen and I am working on this together. So…if I can reiterate what you're saying @gettalong to make sure we understand: we can write some code to parse the ?{1,3-5} style syntax into the highlight_lines: [1, 3, 4, 5] option which Rouge expects, and we would do that by writing our own formatter class (wrapping Rouge::Formatters::HTMLLegacy and Rouge::Formatters::HTMLLineHighlighter) which then gets passed to Kramdown via the syntax_highlighter_opts.formatter config option?

[jaredcwhite]

@gettalong So it doesn't seem like the value of lang gets passed to the formatter. In this line:

kramdown/lib/kramdown/converter/syntax_highlighter/rouge.rb

Line 33 in bd678ec

formatter = formatter_class(opts).new(opts)

ruby?{1,3-5} isn't actually available within opts.

So we'd need to write a new syntax highlighter which in turn calls Kramdown::Converter::SyntaxHighlighter::Rogue with the right opts, and then we'd additionally have to write a formatter as described above?

That seems like a rather complex custom solution to have to maintain outside of Kramdown. 😕 Is this something you can envision supporting within Kramdown core?

[sudeeptarlekar]

@jaredcwhite were you able to find solution for this issue? Even I am facing problems in parsing.

[jaredcwhite]

@sudeeptarlekar Not yet unfortunately. I'm essentially working on a third-party plugin system for Kramdown at this point (to better support things like GFM, mark tags, and other additions). Hopefully the line highlighting functionality makes its way in there soon.