Fix for race conditions in presistence plugins #1543
Conversation
|
@Horusiath test failures
|
|
@Horusiath saw that the tests were re-run - is there a known issue with the |
|
I'd like to know it too. I've re-runed them to check if this are racy tests or repeatable issue associated with changes I've applied. |
Horusiath
force-pushed
the
persistence-race-fix
branch
from
ee31816
to
b8545a7
Compare
|
As the previous attempt was not 100% thread safe, I've changed it to implementation which is more in line with JVM version - now plugin holders are synced through atomic references on immutable dictionaries. |
| var plugin = new Lazy<PluginHolder>(() => CreatePlugin(configPath, _ => DefaultPluginDispatcherId), LazyThreadSafetyMode.ExecutionAndPublication); | ||
| pluginContainer = _snapshotPluginExtensionIds.AddOrUpdate(configPath, plugin, (key, old) => plugin); | ||
| pluginContainer = new Lazy<PluginHolder>(() => CreatePlugin(configPath, _ => DefaultPluginDispatcherId), LazyThreadSafetyMode.ExecutionAndPublication); | ||
| _snapshotPluginExtensionIds.CompareAndSet(extensionIdMap, extensionIdMap.AddOrUpdate(configPath, pluginContainer)); |
There was a problem hiding this comment.
Looks good to me - immutable copy + CAS
|
I think this approach should work. Going to merge it in and give it a try. |
Fix for race conditions in presistence plugins
This PR should fix race problems with lazy-loaded persistent plugins (#1522). It turns out that lazy loading isn't actually needed. Also plugins cache has been refactored to work in safer manner.
[EDIT:30.12.2015] More description
Actual problem seemed to lay in the way the plugin handlers were registered in
Persistenceextension. Because of concurrent access, all parties (i.e. persistent actors) were trying to lazily initialize plugin handler andAddOrUpdateit to concurrent dictionary - this part was handled correctly. However when there are no entry in dictionary, new plugin was initialized and overriding concurrently updated entry, that may appeared in the meantime, as the default dictionary conflict resolution was to take latest plugin created. However one of the plugin handler actions was new journal actor creation - that means, the latest plugin handler would trigger actor creation failure as that actor was already created by earlier plugin handlers already registered.I think there were two solutions for this problem:
ConcurrentDictionary.GetOrAddmethod - this also seems to solve an issue, but lack of good example to recreate the problem on the first place seems a more riskier to me. Also turning back from concurrent dictionary into combination of atomic + immutable collection seems to be better option.