tantivy-vibrato

A Tantivy tokenizer using Vibrato.

Usage

let tokenizer = VibratoTokenizer::new("/path/to/dictionary")?;
let analyzer = TextAnalyzer::from(tokenizer).filter(LowerCaser);
index.tokenizers().register("lang_ja", analyzer);

You need to specify a path to the Vibrato's dictionary file.