Add ability to clear ALL data associated with an index #179
Conversation
There was a problem hiding this comment.
Maybe we should let them control the batching parameter but defaulted to 500?
There was a problem hiding this comment.
I think if someone's goal is to just clear all of the data from an index, we should implement the best practice under the hood and not make the user worry about it? But it certainly woudln't be hard to add an optional arg in the future if folks need it in the future!
|
|
||
| # Paginate using queries and delete in batches | ||
| for batch in self.paginate( | ||
| FilterQuery(FilterExpression("*"), return_fields=["id"]), page_size=500 |
There was a problem hiding this comment.
Unlikely scenario, but what if I want to get the (some) data out before destroying the index, e.g. return_fields being configurable also?
There was a problem hiding this comment.
I agree with the notion that someone might want to be able to export data from an index. However, I think that should be it's own clear feature. For example, an index.export(file_path="file.json") method or something similar?
I think clear() should just be a simple method for clearing data out of your index
This PR introduces the
clear()method to the core SearchEngine class in RedisVL. This now allows for the ability to clear out all data, while leaving the index in place.Useful for manual cache eviction, manual session clearing, and more.
This PR also updates the extension classes to use the new
clear()method as opposed to the SCAN ITER approach.