This repository contains empirically determined approximate expected Levenshtein distances between random strings over alphabets of different sizes, as well as simple python code to generate them.
To use the code, you will need numpy
and numba
.
Simply clone this repo:
git clone https://github.com/nickmachnik/expected-levenshtein.git [TARGET DIR]
and then install via pip
pip install [TARGET DIR]
or install directly from PyPI (this won't include unreleased changes as specified in the changelog):
pip install expected-levenshtein
Test the cloned package:
cd [TARGET DIR]
python -m unittest
This package comes with precomputed models for certain alphabet sizes k and string lengths n. Currently the following models are available:
- k = 20, 25 ≤ n ≤ 6000
Note: A model for a specific value of n only fits values for m (the length of the second string) such that m ≤ n.
The following example shows how a models can be loaded and used to compute the expected levenshtein distances for k = 20, n = 5000:
import expected_levenshtein.fit as efit
import numpy as np
# load all models for k = 20
row_indices, coefficients, mean_squared_deviations = efit.load_precomputed(20)
# get the specific model for n = 5000. Here we consider an index row offset.
coeff_5k = coefficients[5000 - row_indices[0]]
# predict expected distance for n=5000, m=876
single_distance = efit.poly(876, coeff_5k)
# predict expected distances for n=5000, m ≤ 5000
range_distances = efit.poly(np.arange(5000), coeff_5k)
To compute the approximate expected Levenshtein distances of random strings of lengths 1 ≤ lengths ≤ n, use random_average_levenshtein
in sample.py
.
This example shows how to compute the distances of random strings up to length 100 over a 4-letter alphabet, averaged over 1000 replicates.
from sample import random_average_levenshtein
import numpy as np
random_average_levenshtein(100, 1000, np.arange(4))
For long sequences, the distance matrix returned by random_average_levenshtein
can get quite large.
If you prefer not to load and query a large matrix object every time you need an expected distance,
you can use fit.model_average_levenshtein
. This function generates a polynomial model for each row in
the distance matrix. That way, the information that needs to be stored to compute approximate
expected levenshtein distances is reduced to the coefficients of the polynomials. Once computed,
these can be used to predict expected distances with fit.poly
.
This example shows how to generate and use such models for random strings from length 25 to length 50.
from sample import random_average_levenshtein
from fit import poly, model_average_levenshtein
import numpy as np
# sample distances
average_distances = random_average_levenshtein(50, 1000, np.arange(4))
# make models
row_indices, coefficients, mean_squared_deviations = model_average_levenshtein(
average_distances, model_rows=np.arange(25, 51))
# predict expected distance for n=50, m=44
coeff_n_50 = coefficients[-1]
predicted_expected_distance = poly(44, coeff_n_50)
MIT license (LICENSE or https://opensource.org/licenses/MIT)