Adding more characters to the dictionary in the typo function

[abheesht17]

In decepticonlp/transforms/perturb.py, in the function typo, we have defined the Python dictionary a certain way, with the keys as all the characters, and their corresponding values as the characters close to the respective key on the QWERTY keyboard. But we haven't taken digits (0-9) into account. Also, we might have missed out on a few alphabetic characters as well.
For example,
1.
Our implementation: "e": ["w", "s", "d", "r"]
Their implementation: "e": ["2","@","3","#","4","$","w","r","s","d","f"]
2.
Our implementation: "h": ["g", "y", "u", "j", "n", "b"]
Their implementation: "h":["t","y","u","g","j","b","n","m"]

For details, have a look at this (under the section QWERTY):
https://towardsdatascience.com/data-augmentation-library-for-text-9661736b13ff

They have used "One Keyword Distance Error" while deciding which characters are in proximity on the QWERTY Keyboard.

I am a bit doubtful about special characters though, since users tend to remove them during text pre-processing. So, I leave that to your discretion.

Even if we ignore the extra alphabetic characters, I think numeric characters must be added.

[parantak]

I believe I could add a feature to map digits to their adjacents, and special characters to their adjacents. But I suppose replacing alphabets with special characters won't make a lot of sense, as logically the perturbation won't even make sense to us. What do you think?
@abheesht17 , @someshsingh22 , @rajaswa

[someshsingh22]

We are already considering special look alike chars, I think we are getting confused between typos and visually similar chars. Keep them separate

[abheesht17]

@parantak, I don't think there is a problem with replacing alphabetic characters with special characters. Here, we are talking about typos (as @someshsingh22 mentioned). Typos are decided by their proximity to the correct character on the keyboard; basically, characters which replace the correct one due to an accidental key press.

The one problem that can arise with special characters is that users tend to remove them during pre-processing, which is why I said that I leave it to your discretion.

What we can do is this:

1. Map digits to keys on the keyboard which are close by (as you said).
2. Update the dictionary of the alphabetic characters with updated key-value pairs (as given in the blog). The lists in the values can include digits as well (and possibly special characters).

[parantak]

Yeah, I agree. That's what I meant. Even, without preprocessing, even if were to simulate typos per se, any user would explicitly take note of the typo, if it was a special character in place of an alphabet.

[parantak]

I'll make the updates soon.

[someshsingh22]

@parantak If ur making any updates now you just have to edit the json