Download count is important, but it's certainly not the only criteria
one can use to evaluate a package.

> it spreads misinformation on two counts:
>
> 1) The number of downloads cannot be established since cdecimal
> is not hosted on PyPI.

pythonpackages.com considers this to be "zero downloads". if there was
an API for off-site downloads to report back to PyPI then maybe we could
use it to report those statistics on pythonpackages.com. AFAIK there is
currently no such thing. (I used to have a tool tip in place that
explained this; I'll put it back ASAP. crate.io has something similar in
place IIRC.)

>
> 2) "Now supports Python 3!" is false since cdecimal has supported
> Python3 from the very beginning.

Fair enough, I changed the message to "Supports Python 3". Is that
better? FWIW there is an issue tracker here for this sort of thing:

Re: What is the point of pythonpackages.com?

On Feb 5, 2012 1:48 PM, "Alex Clark" <[hidden email]> wrote:
> pythonpackages.com considers this to be "zero downloads". if there was an API for off-site downloads to report back to PyPI then maybe we could use it to report those statistics on pythonpackages.com. AFAIK there is currently no such thing. (I used to have a tool tip in place that explained this; I'll put it back ASAP. crate.io has something similar in place IIRC.)

Perhaps instead of zero with a tooltip, you could just say "N/A" for releases that don't list any files -- or just exclude the download line altogether.

You do, after all, know whether a release has any files from its PyPI data, so you do know the difference between "zero downloads" and "unknown downloads".

Re: What is the point of pythonpackages.com?

On 2/5/12 7:01 PM, PJ Eby wrote:

>
> On Feb 5, 2012 1:48 PM, "Alex Clark" <[hidden email]> <mailto:[hidden email]>> wrote:
> > pythonpackages.com <http://pythonpackages.com> considers this to be
> "zero downloads". if there was an API for off-site downloads to report
> back to PyPI then maybe we could use it to report those statistics on
> pythonpackages.com <http://pythonpackages.com>. AFAIK there is currently
> no such thing. (I used to have a tool tip in place that explained this;
> I'll put it back ASAP. crate.io <http://crate.io> has something similar
> in place IIRC.)
>
> Perhaps instead of zero with a tooltip, you could just say "N/A" for
> releases that don't list any files -- or just exclude the download line
> altogether.

Re: What is the point of pythonpackages.com?

On 02/05/2012 07:12 PM, Terry Reedy wrote:
> On 2/5/2012 8:37 AM, Kai Diefenbach wrote:
>> Why not? Packages, which are not hosted on PyPi suck.
>
> This is a technical discussion list, not a flame list.
> That comment is both wrong and unhelpful.

'suck' is not the right way to express the problem, and it's the
original poster's choice to host somewhere else, but it can indeed be
inconvenient to quite a few users of PyPI if a package is not hosted on
PyPI.

This because setuptools (and thus, easy_install, pip, buildout) for
better or for worse uses a "trawl the web" approach to find download
links, and multiple sites to download from create multiple potential
points of failure besides PyPI itself.

This makes setuptools work for a range of cases and that's nice, but
it's also a drawback, because on a fairly regular basis I at least have
had the issue that a package wasn't hosted on PyPI and that the site
hosting the package was suddenly down or had changed, breaking the
setuptools-based automatic download. If the package were hosted on PyPI
I wouldn't have had this issue, as PyPI itself is actually tolerably
reliable (especially with mirroring in place; but these external
packages are also not mirrored).

Of course the response I'll undoubtedly get is that I should host these
packages myself or include them in my version control system and all
that. And yes, I can do this, and sometimes I do. But doing that is in
this subjective user's opinion actually an inconvenience. Any 'pip' user
that installs a package from PyPI that has dependencies listed in
setup.py can run into this problem.

So the original poster could at least consider uploading their package
on PyPI to lessen his complaint. Besides the web UI, they'll find handy
tools available to help automate things, such as 'setup.py sdist upload'
and for more power, zest.releaser. But of course they can choose not to
do so at all too - that's the way things work [1].

Regards,

Martijn

[1] I suspect an alternate timeline in which setuptools had never done
this web trawling and would only download from PyPI would have lead to a
more pleasant situation for users, though I'm not sure: setuptools being
able to download dependencies might've retarded adoption of setuptools
instead.

Re: What is the point of pythonpackages.com?

> This because setuptools (and thus, easy_install, pip, buildout) for better
> or for worse uses a "trawl the web" approach to find download links, and
> multiple sites to download from create multiple potential points of failure
> besides PyPI itself.
>
> This makes setuptools work for a range of cases and that's nice, but it's
> also a drawback, because on a fairly regular basis I at least have had the
> issue that a package wasn't hosted on PyPI and that the site hosting the
> package was suddenly down or had changed, breaking the setuptools-based
> automatic download. If the package were hosted on PyPI I wouldn't have had
> this issue, as PyPI itself is actually tolerably reliable (especially with
> mirroring in place; but these external packages are also not mirrored).
>
> Of course the response I'll undoubtedly get is that I should host these
> packages myself or include them in my version control system and all that.
> And yes, I can do this, and sometimes I do. But doing that is in this
> subjective user's opinion actually an inconvenience. Any 'pip' user that
> installs a package from PyPI that has dependencies listed in setup.py can
> run into this problem.
>
> So the original poster could at least consider uploading their package on
> PyPI to lessen his complaint. Besides the web UI, they'll find handy tools
> available to help automate things, such as 'setup.py sdist upload' and for
> more power, zest.releaser. But of course they can choose not to do so at all
> too - that's the way things work [1].
>
> Regards,
>
> Martijn
>
> [1] I suspect an alternate timeline in which setuptools had never done this
> web trawling and would only download from PyPI would have lead to a more
> pleasant situation for users, though I'm not sure: setuptools being able to
> download dependencies might've retarded adoption of setuptools instead.

I agree 100% with Martijn. Maybe there was a time when hosting your
package off of PyPI was a good idea. I think if that time existed, it
has now passed. Normally I prefer giving package authors more control,
but this is one of those places where the users of the service ought
to be able to expect packages to all be found in one location.

Re: What is the point of pythonpackages.com?

Hi

On 2/6/12 11:15 AM, Daniel Greenfeld wrote:

> On Mon, Feb 6, 2012 at 7:32 AM, Martijn Faassen<[hidden email]> wrote:
>
>> This because setuptools (and thus, easy_install, pip, buildout) for better
>> or for worse uses a "trawl the web" approach to find download links, and
>> multiple sites to download from create multiple potential points of failure
>> besides PyPI itself.
>>
>> This makes setuptools work for a range of cases and that's nice, but it's
>> also a drawback, because on a fairly regular basis I at least have had the
>> issue that a package wasn't hosted on PyPI and that the site hosting the
>> package was suddenly down or had changed, breaking the setuptools-based
>> automatic download. If the package were hosted on PyPI I wouldn't have had
>> this issue, as PyPI itself is actually tolerably reliable (especially with
>> mirroring in place; but these external packages are also not mirrored).
>>
>> Of course the response I'll undoubtedly get is that I should host these
>> packages myself or include them in my version control system and all that.
>> And yes, I can do this, and sometimes I do. But doing that is in this
>> subjective user's opinion actually an inconvenience. Any 'pip' user that
>> installs a package from PyPI that has dependencies listed in setup.py can
>> run into this problem.
>>
>> So the original poster could at least consider uploading their package on
>> PyPI to lessen his complaint. Besides the web UI, they'll find handy tools
>> available to help automate things, such as 'setup.py sdist upload' and for
>> more power, zest.releaser. But of course they can choose not to do so at all
>> too - that's the way things work [1].
>>
>> Regards,
>>
>> Martijn
>>
>> [1] I suspect an alternate timeline in which setuptools had never done this
>> web trawling and would only download from PyPI would have lead to a more
>> pleasant situation for users, though I'm not sure: setuptools being able to
>> download dependencies might've retarded adoption of setuptools instead.
>
> I agree 100% with Martijn. Maybe there was a time when hosting your
> package off of PyPI was a good idea. I think if that time existed, it
> has now passed. Normally I prefer giving package authors more control,
> but this is one of those places where the users of the service ought
> to be able to expect packages to all be found in one location.

+1. And if you want to host your packages off-site I think that is
perfectly reasonable. But it would make everyone's life easier if we
knew that: for every release of a Python package on earth, there is a
corresponding package on PyPI.

E.g. In Plone-land we currently encourage dual-releasing to both PyPI
and plone.org/products. This has several benefits:

Re: What is the point of pythonpackages.com?

Stefan Krah wrote:
> Martijn Faassen <[hidden email]> wrote:
>> original poster's choice to host somewhere else, but it can indeed
>> be inconvenient to quite a few users of PyPI if a package is not
>> hosted on PyPI.
>
> I don't see any inconvenience since bytereef.org has a comparable
> uptime to python.org.

Not an argument. It is in the interest of all serious Python developers
that Python packages are maintained in a proper way on PyPI
(documentation, hosting, metadata etc.). Having a package on a private
server is often a single-point-of-failure and not acceptable for
professional deployments. My point about this: if a person does not want
to host its package on PyPi than it should stay away from PyPI. Package
hygiene and a certain level of professional package repository is more
important and personal reasons for not hosting packages on PyPI.

Re: What is the point of pythonpackages.com?

On 02/06/2012 09:08 PM, Stefan Krah wrote:
> Martijn Faassen<[hidden email]> wrote:
>> original poster's choice to host somewhere else, but it can indeed be
>> inconvenient to quite a few users of PyPI if a package is not hosted on
>> PyPI.
>
> I don't see any inconvenience since bytereef.org has a comparable
> uptime to python.org.

I've experienced a site which was hosting a Python package which had
awesome uptime, but then something was screwed up about the security of
the host at some point and while it remained up, it took forever
(months? years?) to get resolved.

So anyway, it's great bytereef.org has great uptime, but it's also clear
relying on 10 sites, even if each have a great uptime and network
reachability is going to give me worse uptime than having to rely on
one, if that one has a reasonable uptime. Unless of course the content
is mirrored, in which case reliability goes up.

Interesting, and thank you for the reference. Your reasons make sense of
course, though they don't tip the balance for me personally. I notice
all your reasons (besides the uptime one) are about control over your
package as the author. The arguments I've made from the perspective of
package users. I'm a package author too; plenty of my stuff on PyPI, but
I'm an even bigger user of other people's code.

Re: What is the point of pythonpackages.com?

Andreas Jung <[hidden email]> wrote:
> > I don't see any inconvenience since bytereef.org has a comparable
> > uptime to python.org.
>
> Not an argument. It is in the interest of all serious Python developers
> that Python packages are maintained in a proper way on PyPI
> (documentation, hosting, metadata etc.). Having a package on a private
> server is often a single-point-of-failure and not acceptable for
> professional deployments.

Martijn Faassen has predicted that this would come up, so here it goes:

If that's a point of failure then you are simply not doing a professional
deployment.

People who need guarantees like that should maintain their own package
repository or try Enthought, ActiveState, etc.

Re: What is the point of pythonpackages.com?

Martijn Faassen <[hidden email]> wrote:
> On 02/06/2012 09:08 PM, Stefan Krah wrote:
>> I don't see any inconvenience since bytereef.org has a comparable
>> uptime to python.org.
>
> I've experienced a site which was hosting a Python package which had
> awesome uptime, but then something was screwed up about the security of
> the host at some point and while it remained up, it took forever
> (months? years?) to get resolved.

And? I'm not exactly unreachable and I doubt there will be a security problem.
Furthermore I'm posting the sha256sums of the packages in the announcements,
so they are archived on several mailing lists.

For the general case I'd suggest that PyPI gives an author the option to
tie an sha256sum to a package version *once*. This leaves an opportunity
to correct a release (recent discussion), but as soon as the checksum is
published it cannot be altered.

If a package is removed entirely, any version numbers that have been used
would need to be stored intenally to prevent a re-upload with the same name
but a different checksum.

The download tools would need to get the capability to verify the checksum.

Re: What is the point of pythonpackages.com?

> Andreas Jung <[hidden email]> wrote:
>> > I don't see any inconvenience since bytereef.org has a comparable
>> > uptime to python.org.
>>
>> Not an argument. It is in the interest of all serious Python developers
>> that Python packages are maintained in a proper way on PyPI
>> (documentation, hosting, metadata etc.). Having a package on a private
>> server is often a single-point-of-failure and not acceptable for
>> professional deployments.
>
> Martijn Faassen has predicted that this would come up, so here it goes:
>
> If that's a point of failure then you are simply not doing a professional
> deployment.
>
> People who need guarantees like that should maintain their own package
> repository or try Enthought, ActiveState, etc.

I've been told that 'professional deployments should not been done
from PyPI for years. That's always irked me quite a bit, and I think
that things should change. In fact, I contend that as PyPI is the
canonical place for package listings, then that sentence is incredible
dismaying/shocking for new users of Python. How do Fedora/Ubuntu/Perl
and other systems work? Are their systems also modeled the same way?

So why can't PyPI become the best place for package downloads? The
technical obstacles are being overcome with mirrors and improved
architecture. As that occurs, I think a requirement for PyPI listing
should be that a copy of the posted version is on the site.

Re: What is the point of pythonpackages.com?

On 2/6/2012 3:17 PM, Andreas Jung wrote:
> My point about this: if a person does not want
> to host its package on PyPi than it should stay away from PyPI.

The Python Package Index was originally just that: a package *INDEX*,
aiming to be a complete index. It did not originate the idea of such an
index, but has pretty much superseded previous 'unofficial' efforts.

Now you want to censor it to meet *your* needs, to only list packages
that *you* are interested in.

If I remember correctly, the Cheeseshop/PyPI was originally *just* an
index. The hosting-repository service was added later -- as a
convenience firstly to authors. I now believe that the repository should
have been and should be kept separate, as the Python Package Repository
-- PyPaR. Then repository issues would be clearly separate from index
issues.

Re: What is the point of pythonpackages.com?

Daniel Greenfeld wrote:
> I've been told that 'professional deployments should not been done
> from PyPI for years. That's always irked me quite a bit, and I think
> that things should change.

I completely agree.

I'm one of those people who's told you that you can't reply on PyPI for
repeatable, bullet-proof deployments. That's a statement of fact, but
it's not a situation I'm particularly happy with. It'd be a pretty great
thing for our community if we could improve PyPI to the point that it
has sufficient reliability.

It's not a particularly hard problem technically, but it is going to
require us to stop being content with the status quo and push PyPI forward.