-
Notifications
You must be signed in to change notification settings - Fork 218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Aborting OutOfMemoryError caused by too many results from ItemSearcher called from UFEDChatParser #2038
Comments
The user reported commit above fixed this OOME issue, so I'll merge it into master soon. |
fix #2038: use safer ItemSearcher searchIterable() instead of search()
@lfcnassif, while checking an issue related to the HTML generated by UFEDChatParser (reported by another user to @felipecampanini), I observed a different behavior comparing master and 4.1.5. |
Clarifying: an infinite loop, if more than one item was found, and a lost message, if a single item was returned by the query, as hasNext() will return false after the next(). |
Thank you @wladimirleite and sorry for my fault! |
Fix a possible infinite loop (#2038)
An user reported OOME with a 80GB heap. Analyzing a smaller 32GB heap I asked for, there are many parsing threads using up to 1GB each:
Taking a look at them, most heap is being used by large
ArrayList<Item>
objects:It took me a while to find from where those large Item lists come from. Looking those Threads stacktrace, those Lists are returned by
ItemSearcher.search(query)
calls fromUFEDChatParser
:Maybe there is some problem with the query, but it would be better to use the safer
ItemSearcher.searchIterable(query)
instead ofItemSearcher.search(query)
where possible, which returns an Iterable instead of an ArrayList.The text was updated successfully, but these errors were encountered: