Hello,
Brief problem statement: Let's say I have a custom file type (say,
with extension .foo) and these .foo files are included in a package
(along with other Python modules with standard extensions like .py and
.so), and I want to make these .foo files importable like any other
module.
On its face, importlib.machinery.FileFinder makes this easy. I make a
loader for my custom file type (say, FooSourceLoader), and I can use
the FileFinder.path_hook helper like:
sys.path_hooks.insert(0, FileFinder.path_hook((FooSourceLoader, ['.foo'])))
sys.path_importer_cache.clear()
Great--now I can import my .foo modules like any other Python module.
However, any standard Python modules now cannot be imported. The way
PathFinder sys.meta_path hook works, sys.path_hooks entries are
first-come-first-serve, and furthermore FileFinder.path_hook is very
promiscuous--it will take over module loading for *any* directory on
sys.path, regardless what the file extensions are in that directory.
So although this mechanism is provided by the stdlib, it can't really
be used for this purpose without breaking imports of normal modules
(and maybe it's not intended for that purpose, but the documentation
is unclear).
There are a number of different ways one could get around this. One
might be to pass FileFinder.path_hook loaders/extension pairs for all
the basic file types known by the Python interpreter. Unfortunately
there's no great way to get that information. *I* know that I want to
support .py, .pyc, .so etc. files, and I know which loaders to use for
them. But that's really information that should belong to the Python
interpreter, and not something that should be reverse-engineered. In
fact, there is such a mapping provided by
importlib.machinery._get_supported_file_loaders(), but this is not a
publicly documented function.
One could probably think of other workarounds. For example you could
implement a custom sys.meta_path hook. But I think it shouldn't be
necessary to go to higher levels of abstraction in order to do
this--the default sys.path handler should be able to handle this use
case.
In order to support adding support for new file types to
sys.path_hooks, I ended up implementing the following hack:
#############################################################
import os
import sys
from importlib.abc import PathEntryFinder
@PathEntryFinder.register
class MetaFileFinder:
"""
A 'middleware', if you will, between the PathFinder sys.meta_path hook,
and sys.path_hooks hooks--particularly FileFinder.
The hook returned by FileFinder.path_hook is rather 'promiscuous' in that
it will handle *any* directory. So if one wants to insert another
FileFinder.path_hook into sys.path_hooks, that will totally take over
importing for any directory, and previous path hooks will be ignored.
This class provides its own sys.path_hooks hook as follows: If inserted
on sys.path_hooks (it should be inserted early so that it can supersede
anything else). Its find_spec method then calls each hook on
sys.path_hooks after itself and, for each hook that can handle the given
sys.path entry, it calls the hook to create a finder, and calls that
finder's find_spec. So each sys.path_hooks entry is tried until a spec is
found or all finders are exhausted.
"""
def __init__(self, path):
if not os.path.isdir(path):
raise ImportError('only directories are supported', path=path)
self.path = path
self._finder_cache = {}
def __repr__(self):
return '{}({!r})'.format(self.__class__.__name__, self.path)
def find_spec(self, fullname, target=None):
if not sys.path_hooks:
return None
for hook in sys.path_hooks:
if hook is self.__class__:
continue
finder = None
try:
if hook in self._finder_cache:
finder = self._finder_cache[hook]
if finder is None:
# We've tried this finder before and got an ImportError
continue
except TypeError:
# The hook is unhashable
pass
if finder is None:
try:
finder = hook(self.path)
except ImportError:
pass
try:
self._finder_cache[hook] = finder
except TypeError:
# The hook is unhashable for some reason so we don't bother
# caching it
pass
if finder is not None:
spec = finder.find_spec(fullname, target)
if spec is not None:
return spec
# Module spec not found through any of the finders
return None
def invalidate_caches(self):
for finder in self._finder_cache.values():
finder.invalidate_caches()
@classmethod
def install(cls):
sys.path_hooks.insert(0, cls)
sys.path_importer_cache.clear()
#############################################################
This works, for example, like:
>>> MetaFileFinder.install()
>>> sys.path_hooks.append(FileFinder.path_hook((SourceFileLoader, ['.foo'])))
And now, .foo modules are importable, without breaking support for the
built-in module types.
This is still overkill though. I feel like there should instead be a
way to, say, extend a sys.path_hooks hook based on FileFinder so as to
be able to support loading other file types, without having to go
above the default sys.meta_path hooks.
A small, but related problem I noticed in the way FileFinder.path_hook
is implemented, is that for almost *every directory* that gets cached
in sys.path_importer_cache, a new FileFinder instance is created with
its own self._loaders attribute, each containing a copy of the same
list of (loader, extensions) tuples. I calculated that on one large
project this alone accounted for nearly 1 MB. Not a big deal in the
grand scheme of things, but still a bit overkill.
ISTM it would kill two birds with one stone if FileFinder were
changed, or there were a subclass thereof, that had a class attribute
containing the standard loader/extension mappings. This in turn could
simply be appended to in order to support new extension types.
Thanks,
E
The recent thread on variable assignment in comprehensions has
prompted me to finally share
https://gist.github.com/ncoghlan/a1b0482fc1ee3c3a11fc7ae64833a315 with
a wider audience (see the comments there for some notes on iterations
I've already been through on the idea).
== The general idea ==
The general idea would be to introduce a *single* statement local
reference using a new keyword with a symbolic prefix: "?it"
* `(?it=expr)` is a new atomic expression for an "it reference
binding" (whitespace would be permitted around "?it" and "=", but PEP
8 would recommend against it in general)
* subsequent subexpressions (in execution order) can reference the
bound subexpression using `?it` (an "it reference")
* `?it` is reset between statements, including before entering the
suite within a compound statement (if you want a persistent binding,
use a named variable)
* for conditional expressions, put the reference binding in the
conditional, as that gets executed first
* to avoid ambiguity, especially in function calls (where it could be
confused with keyword argument syntax), the parentheses around
reference bindings are always required
* unlike regular variables, you can't close over statement local
references (the nested scope will get an UnboundLocalError if you try
it)
The core inspiration here is English pronouns (hence the choice of
keyword): we don't generally define arbitrary terms in the middle of
sentences, but we *do* use pronouns to refer back to concepts
introduced earlier in the sentence. And while it's not an especially
common practice, pronouns are sometimes even used in a sentence
*before* the concept they refer to ;)
If we did pursue this, then PEPs 505, 532, and 535 would all be
withdrawn or rejected (with the direction being to use an it-reference
instead).
== Examples ==
`None`-aware attribute access:
value = ?it.strip()[4:].upper() if (?it=var1) is not None else None
`None`-aware subscript access:
value = ?it[4:].upper() if (?it=var1) is not None else None
`None`-coalescense:
value = ?it if (?it=var1) is not None else ?it if (?it=var2) is
not None else var3
`NaN`-coalescence:
value = ?it if not math.isnan((?it=var1)) else ?it if not
math.isnan((?that=var2)) else var3
Conditional function call:
value = ?it() if (?it=calculate) is not None else default
Avoiding repeated evaluation of a comprehension filter condition:
filtered_values = [?it for x in keys if (?it=get_value(x)) is not None]
Avoiding repeated evaluation for range and slice bounds:
range((?it=calculate_start()), ?it+10)
data[(?it=calculate_start()):?it+10]
Avoiding repeated evaluation in chained comparisons:
value if (?it=lower_bound()) <= value < ?it+tolerance else 0
Avoiding repeated evaluation in an f-string:
print(f"{?it=get_value()!r} is printed in pure ASCII as {?it!a}
and in Unicode as {?it}"
== Possible future extensions ==
One possible future extension would be to pursue PEP 3150, treating
the nested namespace as an it reference binding, giving:
sorted_data = sorted(data, key=?it.sort_key) given ?it=:
def sort_key(item):
return item.attr1, item.attr2
(A potential bonus of that spelling is that it may be possible to make
"given ?it=:" the syntactic keyword introducing the suite, allowing
"given" itself to continue to be used as a variable name)
Another possible extension would be to combine it references with `as`
clauses on if statements and while loops:
if (?it=pattern.match(data)) is not None as matched:
...
while (?it=pattern.match(data)) is not None as matched:
...
== Why not arbitrary embedded assignments? ==
Primarily because embedded assignments are inherently hard to read,
especially in long expressions. Restricting things to one pronoun, and
then pursuing PEP 3150's given clause in order to expand to multiple
statement local names should help nudge folks towards breaking things
up into multiple statements rather than writing ever more complex
one-liners.
That said, the ?-prefix notation is deliberately designed such that it
*could* be used with arbitrary identifiers rather then being limited
to a single specific keyword, and the explicit lack of closure support
means that there wouldn't be any complex nested scope issues
associated with lambda expressions, generator expressions, or
container comprehensions.
With that approach, "?it" would just be an idiomatic default name like
"self" or "cls" rather than being a true keyword. Given arbitrary
identifier support, some of the earlier examples might instead be
written as:
value = ?f() if (?f=calculate) is not None else default
range((?start=calculate_start()), ?start+10)
value if (?lower=lower_bound()) <= value < ?lower+tolerance else 0
The main practical downside to this approach is that *all* the
semantic weight ends up resting on the symbolic "?" prefix, which
makes it very difficult to look up as a new Python user. With a
keyword embedded in the construct, there's a higher chance that folks
will be able to guess the right term to search for (i.e. "python it
expression" or "python it keyword").
Another downside of this more flexible option is that it likely
*wouldn't* be amenable to the "if expr as name:" syntax extension, as
there wouldn't be a single defined pronoun expression to bind the name
to.
However, the extension to PEP 3150 would allow the statement local
namespace to be given an arbitrary name:
sorted_data = sorted(data, key=?ns.sort_key) given ?ns=:
def sort_key(item):
return item.attr1, item.attr2
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
I don't know...to me this looks downright ugly and an awkward special case.
It feels like it combines reading difficulty of inline assignment with the
awkwardness of a magic word and the ugliness of using ?. Basically, every
con of the other proposals combined...
--
Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone
elsehttps://refi64.com/
On Feb 15, 2018 at 8:07 PM, <Nick Coghlan <ncoghlan(a)gmail.com>> wrote:
The recent thread on variable assignment in comprehensions has
prompted me to finally share
https://gist.github.com/ncoghlan/a1b0482fc1ee3c3a11fc7ae64833a315 with
a wider audience (see the comments there for some notes on iterations
I've already been through on the idea).
== The general idea ==
The general idea would be to introduce a *single* statement local
reference using a new keyword with a symbolic prefix: "?it"
* `(?it=expr)` is a new atomic expression for an "it reference
binding" (whitespace would be permitted around "?it" and "=", but PEP
8 would recommend against it in general)
* subsequent subexpressions (in execution order) can reference the
bound subexpression using `?it` (an "it reference")
* `?it` is reset between statements, including before entering the
suite within a compound statement (if you want a persistent binding,
use a named variable)
* for conditional expressions, put the reference binding in the
conditional, as that gets executed first
* to avoid ambiguity, especially in function calls (where it could be
confused with keyword argument syntax), the parentheses around
reference bindings are always required
* unlike regular variables, you can't close over statement local
references (the nested scope will get an UnboundLocalError if you try
it)
The core inspiration here is English pronouns (hence the choice of
keyword): we don't generally define arbitrary terms in the middle of
sentences, but we *do* use pronouns to refer back to concepts
introduced earlier in the sentence. And while it's not an especially
common practice, pronouns are sometimes even used in a sentence
*before* the concept they refer to ;)
If we did pursue this, then PEPs 505, 532, and 535 would all be
withdrawn or rejected (with the direction being to use an it-reference
instead).
== Examples ==
`None`-aware attribute access:
value = ?it.strip()[4:].upper() if (?it=var1) is not None else None
`None`-aware subscript access:
value = ?it[4:].upper() if (?it=var1) is not None else None
`None`-coalescense:
value = ?it if (?it=var1) is not None else ?it if (?it=var2) is
not None else var3
`NaN`-coalescence:
value = ?it if not math.isnan((?it=var1)) else ?it if not
math.isnan((?that=var2)) else var3
Conditional function call:
value = ?it() if (?it=calculate) is not None else default
Avoiding repeated evaluation of a comprehension filter condition:
filtered_values = [?it for x in keys if (?it=get_value(x)) is not None]
Avoiding repeated evaluation for range and slice bounds:
range((?it=calculate_start()), ?it+10)
data[(?it=calculate_start()):?it+10]
Avoiding repeated evaluation in chained comparisons:
value if (?it=lower_bound()) <= value < ?it+tolerance else 0
Avoiding repeated evaluation in an f-string:
print(f"{?it=get_value()!r} is printed in pure ASCII as {?it!a}
and in Unicode as {?it}"
== Possible future extensions ==
One possible future extension would be to pursue PEP 3150, treating
the nested namespace as an it reference binding, giving:
sorted_data = sorted(data, key=?it.sort_key) given ?it=:
def sort_key(item):
return item.attr1, item.attr2
(A potential bonus of that spelling is that it may be possible to make
"given ?it=:" the syntactic keyword introducing the suite, allowing
"given" itself to continue to be used as a variable name)
Another possible extension would be to combine it references with `as`
clauses on if statements and while loops:
if (?it=pattern.match(data)) is not None as matched:
...
while (?it=pattern.match(data)) is not None as matched:
...
== Why not arbitrary embedded assignments? ==
Primarily because embedded assignments are inherently hard to read,
especially in long expressions. Restricting things to one pronoun, and
then pursuing PEP 3150's given clause in order to expand to multiple
statement local names should help nudge folks towards breaking things
up into multiple statements rather than writing ever more complex
one-liners.
That said, the ?-prefix notation is deliberately designed such that it
*could* be used with arbitrary identifiers rather then being limited
to a single specific keyword, and the explicit lack of closure support
means that there wouldn't be any complex nested scope issues
associated with lambda expressions, generator expressions, or
container comprehensions.
With that approach, "?it" would just be an idiomatic default name like
"self" or "cls" rather than being a true keyword. Given arbitrary
identifier support, some of the earlier examples might instead be
written as:
value = ?f() if (?f=calculate) is not None else default
range((?start=calculate_start()), ?start+10)
value if (?lower=lower_bound()) <= value < ?lower+tolerance else 0
The main practical downside to this approach is that *all* the
semantic weight ends up resting on the symbolic "?" prefix, which
makes it very difficult to look up as a new Python user. With a
keyword embedded in the construct, there's a higher chance that folks
will be able to guess the right term to search for (i.e. "python it
expression" or "python it keyword").
Another downside of this more flexible option is that it likely
*wouldn't* be amenable to the "if expr as name:" syntax extension, as
there wouldn't be a single defined pronoun expression to bind the name
to.
However, the extension to PEP 3150 would allow the statement local
namespace to be given an arbitrary name:
sorted_data = sorted(data, key=?ns.sort_key) given ?ns=:
def sort_key(item):
return item.attr1, item.attr2
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
_______________________________________________
Python-ideas mailing list
Python-ideas(a)python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Hello, talking about this syntax :
[y+2 for x in range(5) let y = x+1]
*Previous talks*
I've had almost exactly the same idea in June 2017, see subject "variable
assignment in functional context here: https://mail.python.org/
pipermail/python-ideas/2017-June/subject.html.
Currently this can easily be done by iterating over a list of size 1:
[y+2 for x in range(5) for y in [x+1]]
(Other ways exist, see section "current possibilities" below).
This comparison would answer a lot of questions like "can we write [x+2 for
x in range(5) for x in [x+1]], so [x+2 for x in range(5) let x = x+1]" the
new variable would indeed shadow the old one.
In June 2017 I introduced the "let" syntax using the existing "for"
keyword, using the "=" instead of a "in" like this:
[y+2 for x in range(5) for y = x+1]
The only difference I introduced was that it would be logical to accept the
syntax in any expression:
x = 5
print(y+2 for y = x + 1)
or with the "let" keyword:
x = 5
print(y+2 let y = x + 1)
*Previous talk: Pep*
In the June conversation one conclusion was that someone wrote a pep (
https://www.python.org/dev/peps/pep-3150/) in 2010 that's still pending
(not exactly the same syntax but the same idea, he used a "given:" syntax)
that looked like that :
print(y+2 given:
y=x+1)
*Previous talk: GitHub implementation on Cython*
Another conclusion was that someone on GitHub implemented a "where"
statement in Cython (url: https://github.com/thektulu/cpython/commit/
9e669d63d292a639eb6ba2ecea3ed2c0c23f2636) where one could write :
print(y+2 where y = x+1)
So in a list comprehension:
[y+2 where y = x + 1 for x in range(5)]
As the author thektulu said "just compile and have fun".
*New syntax in list comprehension*
However, we probably would like to be able to use this "where" after the
for:
[y+2 for x in range(5) where y = x+1]
This would allow the new variable to be used in further "for" and "if"
statement :
[y+z for x in range(5) where y = x+1 for z in range(y+1)]
*Choice of syntax*
Initially I thought re-using the "for" keyword would be a good idea for
backward comptability (a variable named "where" in old code wouldn't be a
problem), however some people pointed out that the python Grammar wouldn't
be LL1, when reading the "for" token it wouldn't be able to choose directly
if the rest would be a "for in" or "for =" so actually introducing a
dedicated keyword is probably better, like this :
[y+2 for x in range(5) where y = x+1]
print(y+2 where y = x+1)
The "where" keyword is very readable in my opinion, very close to English
sentences we use. Sometimes we introduce a new variable before using it,
generally using "let" or after using it using "where". For example "let y =
x+1; print(y+2)". Or "print(y+2 where y = x+1)".
The first syntax is chosen in the "let in" syntax in haskell :
print(let y = x+2 in y+2)
Or chained:
print(let x = 2 in let y = x+1 in y+2)
But Haskell user would probably line break for clarity :
print(let x = 2 in
let y = x+1 in
y+2)
A postfix notation using "where" would probably be less verbose in my
opinion. Another example of "postfix notation" python has is with the "a if
condition else b" so adding a new one wouldn't be surprising. Furthermore,
the postfix notation is preferred in the context of "presenting the result
first, then the implementation" (context discussed already in the 2010
pep), the "presenting the result first" is also a goal of the list
comprehension, indeed one does write [x+3 for x in range(5)] and not [for x
in range(5): x+3], the latter would be more "imperative programming" style,
and would be translated to a normal loop.
The problem of chaining without parenthesis, how to remove the parenthesis
in the following statement ?
print((y+2 where y = x+1) where x = 2)
We have two options :
print(y+2 where x = 2 where y = x+1)
print(y+2 where y = x+1 where x = 2)
The first option would be probably closer to the way multiple "for" are
linked in a list comprehension:
[y+2 for x in range(5) for y in [x+1]]
But the second option would be more "present result first" and more close
to the parenthesized version, the user would create new variables as they
go, "I want to compute y+2 but hey, what is y ? It's x+1 ! But what is x ?
It's 5 !)). However, keeping the same order as in "multiple for in list
comprehension" is better, so I'd choose first option.
In the implementation on GitHub of thektulu the parenthesis are mandatory.
Another syntax issue would probably surprise some users, the following
statement, parenthesized :
[(y+2 where y = x+1) for x in range(5)]
Would have two totally legal ways to be done:
[y+2 where y = x+1 for x in range(5)]
[y+2 for x in range(5) where y = x+1]
The first one is a consequence of the "where" keyword usable in an
expression and the second one is a consequence of using it in a list
comprehension.
However I think it doesn't break the "there is only one obvious way to do
it", because depending on the case, the "y" variable would be a consequence
of the iteration or a consequence of the computation.
*Goals of the new syntax*
Personally, I find it very useful to do an assignment in such context, I
use list comprehension for example to generate a big json with multiple for
and if.
I don't have here a list of big real world example thar would be simplified
using this syntax but people interested could search arguments here.
People could argue only the "list comprehension" case would be useful and
not the "any expression" case:
[y+2 for x in range(5) where y = x+1]
would be accepted but not :
print(y+2 where y=x+1)
Because the latter could be written:
y = x + 1
print(y+2)
However, the idea of having an isolated scope can be a good idea.
*Current possibilities*
Currently we have multiple options :
[y+2 for x in range(5) for y in [x+1]]
[y+2 for (x+1 for x in range(5))]
[(lambda y:y+2)(y=x+1) for x in range(5)]
The first one works well but it's not obvious we just want to assign a new
variable, especially when the expression is long, or multiple, or both:
[y+z for x in range(5) for y,z in [((x + 1) * 2, x ** 2 - 5)]]
The second one makes it impossible to reuse the "x" variable and the y =
x+1 relation is not obvious.
The third example is what a functional programmer would think but is really
too much complex for a beginner and very verbose.
*Proposed syntax : Conclusion*
As I said, I like the "where" syntax with the "where" keyword.
[y+2 for x in range(5) where y = x+1]
Also usable in any expression :
print(y+2 where y = x+1)
*Conclusion*
Here is all the talk/work/argument I've already found about this syntax.
Apparently it's been a while (2010) since such an idea was thought but I
think having a new pep listing all the pros and cons would be a good idea.
So that we can measure how much the community would want this concept to be
introduced, and if it's refused, the community would have a document where
the "cons" are clearly written.
Robert Vanden Eynde
This idea is inspired by Eric Osborne's post "Extending __format__
method in ipaddress", but I wanted to avoid derailing that thread.
I notice what seems to be an inconsistency in the ipaddress objects:
py> v4 = ipaddress.IPv4Address('1.2.3.4')
py> bin(v4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'IPv4Address' object cannot be interpreted as an integer
But that's surely not right: we just need to explicitly do so:
py> bin(int(v4))
'0b1000000100000001100000100'
IP addresses are, in a strong sense, integers: either 32 or 128 bits.
And they can be explicitly converted losslessly to and from integers:
py> v4 == ipaddress.IPv4Address(int(v4))
True
Is there a good reason not to give them an __index__ method so that
bin(), oct() and hex() will work directly?
py> class X(ipaddress.IPv4Address):
... def __index__(self):
... return int(self)
...
py> a = X('1.2.3.4')
py> bin(a)
'0b1000000100000001100000100'
I acknowledge one potentially undesirable side-effect: this would
allow using IP addresses as indexes into sequences:
py> 'abcdef'[X('0.0.0.2')]
'c'
but while it's weird to do this, I don't think it's logically wrong.
--
Steve
Folks-
The ipaddress library returns an IP address object which can represent
itself in a number of ways:
In [1]: import ipaddress
In [2]: v4 = ipaddress.IPv4Address('1.2.3.4')
In [3]: print(v4)
1.2.3.4
In [4]: v4
Out[4]: IPv4Address('1.2.3.4')
In [6]: v4.packed
Out[6]: b'\x01\x02\x03\x04'
In [9]: str(v4)
Out[9]: '1.2.3.4'
In [10]: int(v4)
Out[10]: 16909060
In [13]: bin(int(v4))
Out[13]: '0b1000000100000001100000100'
In [14]: hex(int(v4))
Out[14]: '0x1020304'
In [15]: oct(int(v4))
Out[15]: '0o100401404'
There are IPv6 objects as well:
In [6]: v6 =
ipaddress.IPv6Address('2001:0db8:85a3:0000:0000:8a2e:0370:7334')
In [7]: int(v6)
Out[7]: 42540766452641154071740215577757643572
and what I'm proposing will work for both address families.
In either case, bin/hex/oct don't work on them directly, but on the integer
representation. This is a little annoying but not such a big deal. What
is a big deal (at least to me) is that the binary representation isn't
zero-padded. This makes it harder to compare two IP addresses by eye to
see what the differences are, i.e.:
In [16]: a = ipaddress.IPv4Address('0.2.3.4')
In [30]: bin(int(a))
Out[30]: '0b100000001100000100'
In [31]: bin(int(v4))
Out[31]: '0b1000000100000001100000100'
It would be nice if there was a way to have an IP address always present
itself in fully zero-padded binary (32 bits for IPv4, 128 bits for IPv6).
I find this particularly convenient when putting together training
material, as it's easier to show subnetting and aggregation if you point at
the binary than if you give people dotted-quad addresses and ask them to do
the binary conversion in their head. Hex is also handy when you're
comparing a dotted-quad IP address to a hex sniffer trace. It's possible
to do this in a one-liner (thanks to Eric Smith): f'{int(v4):#0{34}b}'. But
this is a little cryptic.
I opened bpo-32820 (https://github.com/python/cpython/pull/5627) to
contribute a way to do this. I started with an __index__ method but Issue
15559 (
https://github.com/python/cpython/commit/e0c3f5edc0f20cc28363258df501758c1b…)
rules this out. I instead added a bits() class method so that v4.bits
would return the fully padded string. This was not terribly pretty, but it
mirrored packed(), at least.
Nick Coghlan suggested I instead extend __format__, which is what the
diffs in the current pull request do. This allows a great deal more
flexibility: the current code takes 'b', 'n', or 'x' types, as well as the
'#' option and support for the '_' separator. I realize now I didn't add
'o' but I certainly can for completeness. I debated adding rfc1924
encoding for IPv6 addresses but decided it was entirely too silly.
This is just a convenience function, but IMO fills a need. Is this worth
pursuing?
eric
Let s be a str. I propose to allow these existing str methods to take
params in new forms.
s.replace(old, new):
Allow passing in a collection of olds.
Allow passing in a single argument, a mapping of olds to news.
Allow the olds in the mapping to be tuples of strings.
s.split(sep), s.rsplit, s.partition:
Allow sep to be a collection of separators.
s.startswith, s.endswith:
Allow argument to be a collection of strings.
s.find, s.index, s.count, x in s:
Similar.
These methods are also in `list`, which can't distinguish between
items, subsequences, and subsets. However, `str` is already inconsistent
with `list` here: list.M looks for an item, while str.M looks for a
subsequence.
s.[r|l]strip:
Sadly, these functions already interpret their str arguments as
collections of characters.
These new forms can be optimized internally, as a search for multiple
candidate substrings can be more efficient than searching for one at a
time. See
https://stackoverflow.com/questions/3260962/algorithm-to-find-multiple-stri…
The most significant change is on .replace. The others are simple enough to
simulate with a loop or something. It is harder to make multiple
simultaneous replacements using one .replace at a time, because previous
replacements can form new things that look like replaceables. The easiest
Python solution is to use regex or install some package, which uses (if
you're lucky) regex or (if unlucky) doesn't simulate simultaneous
replacements. (If possible, just use str.translate.)
I suppose .split on multiple separators is also annoying to simulate. The
two-argument form of .split may be even more of a burden, though I don't
know when a limited multiple-separator split is useful. The current best
solution is, like before, to use regex, or install a package and hope for
the best.
