-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement ngram-based blacklist filter #54
Conversation
Come to think of it we can extend the effective time window by reducing the ngram addition rate via stochastic sampling. Instead of adding add ngrams in a completion we can simply select a subset. Right now around 12,000 n-grams are added per step (from 50 completions combined). We can likely reduce this by a factor of 5-10x by randomly selecting n-grams. |
|
||
# Check if any n-grams have significance above the boundary | ||
for ngram in ngrams: | ||
if scores.get(ngram) > self.boundary: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be of interest to include a self.blacklist
attribute which contains all the ngrams with significance>self.boundary
. It should not be a very large object (10s-100s items). This is something we could log to wandb regularly to introspect the blacklist.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Things to be done:
- Track raw score values (being done on Feature/track raw score #63 )
- Identify why we have some n-grams of 3
- Fix branch conflicts
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, I would be ok to merge it once the branch conflicts have been fixed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some minor comments, then it's good to go
Co-authored-by: Steffen Cruz <steffenjcruz@gmail.com>
…sor/text-prompting into features/ngram-blacklist
Co-authored-by: Steffen Cruz <steffenjcruz@gmail.com>
Co-authored-by: Steffen Cruz <steffenjcruz@gmail.com>
Co-authored-by: Steffen Cruz <steffenjcruz@gmail.com>
Co-authored-by: Steffen Cruz <steffenjcruz@gmail.com>
Co-authored-by: Steffen Cruz <steffenjcruz@gmail.com>
Co-authored-by: Steffen Cruz <steffenjcruz@gmail.com>
Co-authored-by: Steffen Cruz <steffenjcruz@gmail.com>
Co-authored-by: Steffen Cruz <steffenjcruz@gmail.com>
…or/text-prompting into feature/track_raw_score
Introduces a ngram-based blacklist. The blacklist is based on an n-gram counter with a sliding window (deque). Conceptually, when completions contain similar subsequences the associated n-grams accumulate counts and significance. When the significance exceeds a threshold (
Blacklist.boundary
), all completions containing that n-gram are flagged and recieve no reward. The sliding window allows the blacklist to adapt to the changing network.Runtime
Estimated runtime for adding to the counter is 25ms per step (50 completions).
Estimated runtime for rewarding completions is unknown right now but it requires a full pass over a dict of
max_size
(~1M) itemsHyperparameters
There is a tradeoff between speed and memory consumption. I think 1M is okay for the queue size but 100k works too.
The length of the queue and throughput of the network (words per unit time) determine how rapidly the blacklist learns/forgets n-grams. I estimate that one step contains around 3400 completion words so a queue of length 1M should be completely flushed in 300 steps. This means that there will be a record of an n-gram for at most 300 steps or ~2.5 hours (@30s per step).
n_min = 5 and n_max = 15 seem reasonable to me. If you make n_min too large then exploits will contain small phrases.
Notes
Long n-grams end up creating multiple entries in the blacklist as their subsequences are also necessarily frequent too. Eg.
('this','is','an','example','long','sentence')
will also guarantee the presence of the n-gram('an','example','long','sentence')
(and all the other subsequences) with at least as many counts. Because of this, a common 14-gram also has a common 13-gram, 12-gram ... which tend to fill of the queue quite rapidly.TODO