I have lots of multiprocessing processes which have to add to and search in a dict. Deletion of values is not needed.
Atm I am using multiprocessing.Manager() and
dict = manager.dict()
This works pretty well, but I think that the manager is a huge bottleneck here. Any ideas? It has to run on older Python 3 versions, otherwise I would use this cool thing I found: https://github.com/ronny-rentner/UltraDict
Have you tested the throughput of
dict.manager
?If you need big scale, there is always redis.