[Python-Dev] Standard library vs Standard distribution?

On 29Nov2018 1330, Nathaniel Smith wrote:
> On Thu, Nov 29, 2018 at 10:22 AM Antoine Pitrou <antoine at python.org> wrote:
>>
>> Le 29/11/2018 ? 19:07, Steve Dower a ?crit :
>>> On 29Nov2018 0923, Antoine Pitrou wrote:
>>>> I think the whole argument amounts to hand waving anyway. You are
>>>> inventing an extended distribution which doesn't exist (except as
>>>> Anaconda) to justify that we shouldn't accept more modules in the
>>>> stdlib. But obviously maintaining an extended distribution is a lot
>>>> more work than accepting a single module in the stdlib, and that's why
>>>> you don't see anyone doing it, even though people have been floating the
>>>> idea for years.
>>>
>>> https://anaconda.com/
>>> https://www.activestate.com/products/activepython/
>>> http://winpython.github.io/
>>> http://python-xy.github.io/
>>> https://www.enthought.com/product/canopy/
>>> https://software.intel.com/en-us/distribution-for-python
>>> http://every-linux-distro-ever.example.com
>>>
>>> Do I need to keep going?
>>
>> I'm sure you could. So what? The point is that it's a lot of work to
>> maintain if you want to do it seriously and with quality standards that
>> would actually _satisfy_ the people for whom PyPI is not an option.
>
> Yeah, I draw two conclusions from the list above:
>
> - Paul expressed uncertainty about how many people are in his position
> of needing a single download with all the batteries included, but
> obviously he's not alone. So many people want a
> single-box-of-batteries that whole businesses are being built on
> fulfilling that need.
>
> - Currently, our single-box-of-batteries is doing such a lousy job of
> solving Paul's problem, that people are building whole businesses on
> our failure.
I agree with these two conclusions, and for what it's worth, I don't
think we would do anything to put these out of business - they each have
their own value that we wouldn't reproduce.
Python's box of batteries these days are really the building blocks
needed to ensure the libraries in these bigger distributions can
communicate with each other. For example, asyncio has its place in the
stdlib for this reason, as does pathlib (though perhaps the __fspath__
protocol makes the latter one a little less compelling). If the stdlib
was to grow a fundamental data type somehow shared by
numpy/scipy/pandas/etc. then I'd consider that a very good candidate for
the stdlib to help those libraries share data, even if none of those
packages were in the standard distro.
At the same time, why carry that data type into your embedded app if it
won't be used? Why design it in such a way that it can't be used with
earlier versions of Python even if it reasonably could be? We already do
backports for many new stdlib modules, so why aren't we just installing
the backport module in the newer version by default and running it on
its own release schedule? I forget which one, but there was one backport
that ended up being recommended in place of its stdlib counterpart for a
while because it got critical bugfixes sooner. Installing pip in this
manner hasn't broken anyone's world (that I'm aware of).
There's plenty of space for businesses to be built on being "the best
Python distro for <X>", and if the stdlib is the best layer for enabling
third-party libraries to agree on how to work with each other, then we
make the entire ecosystem stronger.
(That simple agreement got longer than I intended :) )
Cheers,
Steve