On Wed, Jan 17, 2018 at 6:03 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Tue, 16 Jan 2018 17:18:06 -0800 Nathaniel Smith <njs@pobox.com> wrote:
On Tue, Jan 16, 2018 at 5:06 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
I think it would be a very fragile thing In practice: if you have even one variable in the context that isn't pickleable, your code that uses a ProcessPool would stop working. I would defer Context pickleability to 3.8+.
There's also a more fundamental problem: you need some way to match up the ContextVar objects across the two processes, and right now they don't have an attached __module__ or __qualname__.
They have a name, though. So perhaps the name could serve as a unique identifier? Instead of being serialized as a bunch of ContextVars, the Context would then be serialized as a {name: value} dict.
One of the points of the ContextVar design is to avoid having unique identifiers requirement. Names can clash which leads to data being lost. If you prohibit them from clashing, then if libraries A and B happen to use the same context variable name—you can't use them both in your projects. And without enforcing name uniqueness, your approach to serialize context as a dict with string keys won't work. I like Nathaniel's idea to explicitly enable ContextVars pickling support on a per-var basis. Unfortunately we don't have time to seriously consider and debate (and implement!) this idea in time before the 3.7 freeze. In the meanwhile, given that Context objects are fully introspectable, users can implement their own ad-hoc solutions for serializers or cross-process execution. Yury