Replace tipue search with lunr.js and lunr.py #275

codecachet · 2019-02-07T03:05:00Z

I am distinctly getting the impression that tipue search is no longer being maintained, or is otherwise less than desirable. I suggest that we replace tipue search with lunr - lunr.py to create the index, and lunr.js to query the index when a search is launched. Both projects seem alive and well, and both Jekyll and Hugo seem to recommend using lunr for search.

Just a thought - like others, I have been very frustrated trying to get search to work for Elegant. One thing about tipue is I am not crazy about the jQuery dependency. I know jQuery is a dependency of Elegant currently, but if we get rid of Bootstrap, we could get rid of jQuery.

talha131 · 2019-02-10T16:43:55Z

I too want to get rid of jquery. It's on our roadmap.

As for tipue. When elegant was first released, tipue was the only available search solution for static sites. Now we have much better and powerful options like

Down the line, we should replace or add support for other search solutions too.

iranzo · 2019-06-09T09:49:36Z

@codecachet do you want to create a PR for this?

talha131 · 2019-06-09T16:10:35Z

@iranzo this is a big change. It should be handled by the core team. More importantly, we need to compare different solutions first.

talha131 · 2019-07-05T01:09:11Z

An issue noticed with tipue is that it treats plural differently from singular

Search for "variable" and then "variables" in the docs site. Result is different.

We can use stem words option to fix it but then how many words are we going to add in the list. Tipue should have this feature by default.

iranzo · 2020-01-28T19:22:20Z

I'm looking at lunr.js for other aspects, seems that the data it uses for the search can be contained/generated on publish time without plugin, just a jinja loop (at least on jekyll). I'll provide some information once I finish my investigation.

talha131 · 2020-01-29T11:37:24Z

@iranzo I appreciate that you have taken up this task. Before you go ahead and look deeply into lunr.js, can you please step back and decide which library we should use.

In terms of Github stars, Fuse is way ahead of lunr.js, which is followed by flex search.

Github stars are not a reliable metric but they do give a pretty good idea about the community interest and support, which we are going to need.

Then there is performance factor. Flex claims that it is the most performant.

Finally, we have to consider the ease of use for the end user.

Can you please investigate these three libraries and let me know how they fare on the criteria I mentioned?

iranzo · 2020-01-29T11:41:28Z

Hi, No worries, just wanted to mention that I was testing it as a website I'm using has it integrated (but it was pretty old version and using jekyll) about features. Lunr is not that bad, but my concern here is that it, as with tipue, pregenerates a javascript array with all website content... that means that for a big website, it will have to preload all information in the html generated and then perform the search.

I don't mind looking at flex directly to see how it performs, but probably doing perf test will require lot of articles and I was thinking that some one here in the team had a busy one that could be used for testing if the website source is available somewhere.

talha131 · 2020-01-29T11:41:36Z

This is the result of flex search benchmark that I got.

Flex is much more performant than other libraries. How well it is supported and how actively it is under development is to be investigated.

talha131 · 2020-01-29T11:43:22Z

that means that for a big website, it will have to preload all information in the html generated and then perform the search.

You hit the nail right on the head.

If we can avoid it using these new libraries then it would become a high priority feature to implement and ship.

talha131 · 2020-01-29T11:53:09Z

Flex search is fast but development has seem to have lost momentum.

Also, the documentation is not friendly. We will have a hard time making a head and tail of it.

talha131 · 2020-01-29T12:07:48Z

that means that for a big website, it will have to preload all information in the html generated and then perform the search.

I just tested lunrjs demo. Even if we have to ship a JSON object with all the contents of the site, just like TipueSearch, lunarjs has better search results.

Try searching for "assignment" in the demo here
It shows "assigned" in the results too, which is desirable.

Tipue cannot handle plural and singular words correctly.

I think lunrjs is a better option.

talha131 · 2020-01-29T12:09:23Z

Questions and work items for whoever pick up this task.

Do we need lunr.py because we already have a tipue search plugin, that can be forked to work with lunr.js
How do we show the search results? Tipue generates the search result page for us. With lunrjs, we will have to do it ourselves.

iranzo · 2020-01-29T12:15:43Z

From what I've seen about lunr.js: The result is generated by jekyll when the page for 'search' is created, it contains array for all the text in the website.

Search page replaces text in the 'search results' area with the output of the script

Right now the tipue_search plugin does the generation of the search results as lunr just uses a backend for the index that needs to be populated it could also probably be done with current tipue_search plugin output or even try to avoid that plugin to get one 'easier' by no longer requiring ap plugin (if possible), as I've tested this with jekyll and not with pelican

talha131 · 2020-01-29T13:28:45Z

or even try to avoid that plugin to get one 'easier' by no longer

requiring ap plugin (if possible) That's not a good idea. A plugin improves search performance. is what lunrjs docs recommends, https://lunrjs.com/guides/index_prebuilding.html

…

talha131 · 2020-01-31T15:57:26Z

How to generate search result from lunr.js

talha131 · 2020-02-02T20:43:18Z

The json generated by Tipue Search plugin can be used by lunrjs without any changes.

All we need is to generate the search results page.

If anyone is willing to step up to write the code for search results page, then please let us know. I will work with you to save your time and effort, if needed.

iranzo · 2020-02-03T14:10:19Z

@talha131 review #573

Note, the code just 'works', sytling should be used on the results functions and probably remove the tipue search javascript from website (and maybe put the lunr one)

I count on you for the 'styling' , in the current form for the PR, it works as a drop-in replacement of tipue with same requirements (for plugin, etc)

talha131 · 2020-02-03T14:12:04Z

Love it. Will do the rest.

iranzo · 2020-02-03T14:14:41Z

Signed-off-by: Pablo Iranzo Gómez <Pablo.Iranzo@gmail.com> Update #275

# [5.2.0](V5.1.0...V5.2.0) (2020-02-03) ### Bug Fixes * regression introduced due to cache busting ([b953a38](b953a38)) * **archives:** at smaller screen layout does not make the best use of available space ([a289e9d](a289e9d)) * **syntax:** code blocks with line numbers do not have the correct border radius ([3a34a3e](3a34a3e)) * **syntax:** improve the font size and height of download button ([07595b3](07595b3)) * **typography:** fix font size and style of TOC ([2d597ab](2d597ab)), closes [#509](#509) * Categories and Tags pages do not have space below the header ([d78f4e5](d78f4e5)) * remove invalid CSS rule ([368cb2c](368cb2c)) * use consistent color for border ([c85b7b8](c85b7b8)) * use darker color for the site name ([19f37ca](19f37ca)) * **article:** use loclate_date attribute instead of hardcoding the date format ([1da552c](1da552c)), closes [#552](#552) * **bootstrap:** WIP. add Bootstrapv2.3.2 sources file to the project ([43ee8eb](43ee8eb)), closes [#429](#429) * **seo:** regression introduced in PR [#556](#556) ([28bcb85](28bcb85)), closes [#505](#505) * **typography:** about me and my projects heading is not on one line ([6a7989c](6a7989c)) * **typography:** font size of superscript number in categories and tags page ([d0a7162](d0a7162)) * **typography:** override Bootstrap base font ([f6a83a6](f6a83a6)), closes [#429](#429) * **typography:** top menu bar does not have correct height ([d30ce82](d30ce82)) * **typography:** use consistent font and color for project list ([477a467](477a467)) * **typography:** use consistent font in the footer ([eeb0e30](eeb0e30)) ### Features * add support cache busting ([cde0dc5](cde0dc5)) * **categories:** change background color of uncollapsed Category ([6cc11b8](6cc11b8)) * **gallery:** add support for PhotoSwipe image gallery using raw HTML ([ad1bcea](ad1bcea)), closes [#567](#567) * **js:** combine and minify all JS files into one to improve load speed ([f5047d4](f5047d4)) * **search:** improve search results page look ([73fa743](73fa743)), closes [#573](#573) [#275](#275) * **search:** replace tipue_search with lunr.js ([9d60af1](9d60af1)), closes [#275](#275) * **SEO:** Add header and documentation for Claiming Website on Yandex ([2bb691d](2bb691d)) * **SEO:** add SEO and SMO meta keywords in site's home page ([c5be0eb](c5be0eb)) * **sharing:** Enable customizing the sharing links ([f3e262f](f3e262f)) * **syntax:** add copy to clipboard button for every code snippet ([6f73317](6f73317)), closes [#574](#574) [#525](#525) * **syntax:** add support for highlighting lines in code snippets ([ba2de8c](ba2de8c)), closes [#520](#520) * **syntax:** add support for liquid_tags.include_code plugin ([0550a3a](0550a3a)), closes [#518](#518) * **syntax:** use Gruvbox syntax highlighting theme ([0b50ab1](0b50ab1)) * **typography:** use consistent fonts ([8b7a8bd](8b7a8bd)), closes [#429](#429) * **typography:** use consistent style for timestamps in tags, categories and archives pages ([a421dcd](a421dcd)) * **typography:** use dns-prefetch and preconnect to improve font load time ([e4bd582](e4bd582)) * **typography:** use responsive fonts ([39acdbf](39acdbf))

talha131 · 2020-02-03T19:00:41Z

🎉 This issue has been resolved in version 5.2.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

dclong · 2020-12-13T20:33:26Z

Does elegant still uses the tipue_search pelican plugin and only replaces the underlying tipue with lunr.js? Is configuration still the same?

talha131 · 2020-12-14T18:24:32Z

yeah. underlying configuration is same

talha131 added this to the Roadmap milestone Feb 10, 2019

iranzo added enhancement pull request welcome Not a priority right now, but if you provide a pull request, we would merge it. labels Jun 9, 2019

talha131 self-assigned this Jun 9, 2019

talha131 mentioned this issue Jul 5, 2019

TipueSearch treats plural differently from singular #298

Closed

talha131 mentioned this issue Feb 3, 2020

Improve Image gallery #567

Closed

iranzo mentioned this issue Feb 3, 2020

Use LUNR for search engine reusing tipue_search pelican plugin #573

Closed

5 tasks

talha131 pushed a commit that referenced this issue Feb 3, 2020

feat(search): replace tipue_search with lunr.js

9d60af1

Signed-off-by: Pablo Iranzo Gómez <Pablo.Iranzo@gmail.com> Update #275

talha131 closed this as completed in 73fa743 Feb 3, 2020

talha131 added the released label Feb 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace tipue search with lunr.js and lunr.py #275

Replace tipue search with lunr.js and lunr.py #275

codecachet commented Feb 7, 2019

talha131 commented Feb 10, 2019

iranzo commented Jun 9, 2019

talha131 commented Jun 9, 2019

talha131 commented Jul 5, 2019

iranzo commented Jan 28, 2020

talha131 commented Jan 29, 2020

iranzo commented Jan 29, 2020

talha131 commented Jan 29, 2020

talha131 commented Jan 29, 2020

talha131 commented Jan 29, 2020

talha131 commented Jan 29, 2020

talha131 commented Jan 29, 2020

iranzo commented Jan 29, 2020

talha131 commented Jan 29, 2020 via email

talha131 commented Jan 31, 2020 •

edited

Loading

talha131 commented Feb 2, 2020

iranzo commented Feb 3, 2020

talha131 commented Feb 3, 2020

iranzo commented Feb 3, 2020

talha131 commented Feb 3, 2020

dclong commented Dec 13, 2020

talha131 commented Dec 14, 2020

Replace tipue search with lunr.js and lunr.py #275

Replace tipue search with lunr.js and lunr.py #275

Comments

codecachet commented Feb 7, 2019

talha131 commented Feb 10, 2019

iranzo commented Jun 9, 2019

talha131 commented Jun 9, 2019

talha131 commented Jul 5, 2019

iranzo commented Jan 28, 2020

talha131 commented Jan 29, 2020

iranzo commented Jan 29, 2020

talha131 commented Jan 29, 2020

talha131 commented Jan 29, 2020

talha131 commented Jan 29, 2020

talha131 commented Jan 29, 2020

talha131 commented Jan 29, 2020

iranzo commented Jan 29, 2020

talha131 commented Jan 29, 2020 via email

talha131 commented Jan 31, 2020 • edited Loading

talha131 commented Feb 2, 2020

iranzo commented Feb 3, 2020

talha131 commented Feb 3, 2020

iranzo commented Feb 3, 2020

talha131 commented Feb 3, 2020

dclong commented Dec 13, 2020

talha131 commented Dec 14, 2020

talha131 commented Jan 31, 2020 •

edited

Loading