Discuss: should the gallery allow complete package deletion

In the old NuGet gallery, it was possible for the package owner to completely delete some versions of a package. In the new gallery (which was a complete rewrite), we remove that option, and instead implemented a new
unlisting feature. When a package is unlisted, it doesn't show up in any package lists, either on nuget.org or in the VS extension. But the package can still be installed by explicit version.

The reason we did that is that people were increasingly using our workflow that allows you
not commit packages to source control, and instead have them come down from nuget.org as needed (this workflow became even simpler in 1.6). This workflow is high reliant on packages continuing to be available on the feed. If the author removes a package
that someone depends on, their build instantly becomes broken, which is a very bad thing. But if the package is instead unlisted, everything continues to work, even though no one will be able to install the unlisted version in a new project (at least not without
going out of their way).

Generally, we think that is the correct behavior, as allowing packages to be too easily deleted was causing some real breaks and that had to be addressed. However, a few people have raised cases what they might really want to delete a version of a package.
Some potentially valid cases to do this:

The package was just uploaded and found to be bad. Since no one has likely installed it yet, deleting it is harmless.

The package binaries are ok, but the metadata is incorrect (e.g. the description or author name). For this, we are planning to allow modifying the metadata, or potentially re-upload the package with identical content but modified metadata.

The package mistakenly contains some sensitive information and needs to be taken down asap.

It should be noted that the NuGet team is willing to delete packages on a case by case basis via explicit owner request (we probably didn't advertise this option sufficiently). But today there is no automatic way for the owner to do this (e.g.
via nuget.org or the nuget.exe console app).

Question to discuss: do you think authors be able to freely delete any package version, or does that lead to more problems than it solve?

To a certain extent, I see your point; however, you do not have superior knowledge of my users and their use cases. Also, as the owner/maintainer of the package(s) in question, I need to retain the ability to control their distribution, at the very
least through the official distribution channels (i.e. obviously random people are free to distribute copies of whatever version). My personal view is that NuGet provides a [valuable] service in this regard; however, I do not accept the idea that I must
relinquish my ability to control what versions are retained [forever] by said service.

But don't you think the harm to the people using the 'restore' workflow is too great? e.g. take this scenario:

User A writes a site which uses one of your beta packages, and hosts the sources on github using the restore workflow

The author deletes that package from nuget.org

User B pulls user A's site from github, and tries to build it. They get a build break because the reference can't be retrieved. Note that User B knows nothing about your package, which is an implementation detail of the site. They may not even
know anything about NuGet! All they see is that the site doesn't build, and that's plain bad.

So the issue is that if too many packages get deleted, the restore workflow becomes too dangerous and no one will use it, which is a shame as the alternative (commiting binaries) is a nasty thing to do.

Also, note that when people commit binaries, they will continue to use your beta package just as much as with the restore worlflow, which would leave you in the same place anyway.

@hightechrider: yes definitely, that case is not controversial. We could even allow up to 5 downloads, to account for your own testing. And maybe some time delay as well. It's just a matter of finding the dev time to implement this.

No. Package deletion should never be possible. The reasoning behind my hard "no" is that it will indeed break user's restore workflows. On the other hand, my hard "no" is only valid if the user is using package restore (which I hope 99% of people are doing!).
The problem is: with NuGet you don't know. It's a general purpose, well-known feed of packages. This means the stiuff that breaks users should be kept out as much as possible: no package deletes!

The problem shifts a bit if you create your own feeds. With MyGet, for example, we chose to make it possible to delete a package. The reasoning? MyGet feeds should either contain production packages (which you should enforce by setting permissions and only
allow few contributors). Beta/alpha feeds can have packages deleted.

There's a related question as well: should it be possible to modify a package once its published? Should it be possible to add a file in a package and republish? In essence, that is a change and requires another semantic version number.

Package authors have become a mayor player in software development. They are responsible for (sometimes a large) part of your codebase and build process! You don't allow your team members to delete a source tree from source control (or at least: since it's
source control you can restore it). The same goes with NuGet packages. It should be possible to receive the exact same input package when a combination of package identifier and version is specified.

The *only* reason where I see packages can be deleted is because of copyrights. But that is an edge case and probably the infringed package shouldn't have been on the feed in the first place.

(and on the spam side: everyone, mirror well-known packages on your
www.myget.org feed and have your feed where your workflow is supported)

It was always possible before. As far as I'm concerned, this is a regression. The old behavior was correct. The current behavior is broken. The new "policy" of retaining package versions forever against the wishes of the author is
contrary to both logic and common sense.

Also, given the highly dynamic [and evolving] nature of NuGet (not to mention the Internet itself), any expectation that a given package and/or version will be available forever is pure folly, at best.

Being frank: no, it isn't comon sense. I don't want you to be responsible for my build to be green or red. If you decide to delete a package that I am using, my build breaks. And possibly the build of a few thousand others as well. You have a very large
impact on people's software at the moment you delete a package! (note this can be solved by mirroring the package which is something I would encourage teams to do, but still)

Situation 1: you delete a package and a team member still has it on his PC. I'm lucky and can use thet package.
Situation 2: you delete a package that no one has. If I'm lucky I can find the open-source project somewhere. If not, I have a big problem since part of my project is gone and has to be recreated. Imagine what a cost impact this potentially has!

If deletion does come back to NuGet, I vote in favor of having a feed that contains all deleted packages. I would then use that one so I at least can build my software. But I'm still opposed to deleting packages. And as you can see in my previous comment:
I'm even opposed to changing packages without updating the package version number.

Also, as David Ebbo has pointed out, deleting a package from the NuGet.org site does *NOT* remove it from existing projects; therefore, your entire point about "don't allow your team members to delete a source tree" is completely without merit.

Also, given the highly dynamic [and evolving] nature of NuGet (not to mention the Internet itself), any expectation that a given package and/or version will be available forever is pure folly, at best.

True. It is folly. I totally agree. You should pull the responsability of a package being available or not in your own hands. Unfortunately only 500+ feeds exist on MyGet and probably a number of feeds exist in people's own environments. Still, looking at
the high number of package downloads at stats.nuget.org, a large majority is not mirroring the packages they use and therefore face doom if packages can be updated/deleted from the official feed.

Again, it's up to the package author to decide on a "policy" of package version retention. Either check the version you desire into your source tree OR double-check that the package author does not have a "policy" of deleting old versions of the package(s)
you are using in your project(s).

And as you can see in my previous comment: I'm even opposed to changing packages without updating the package version number.

An exception to this is my case #2 is my initial post. Correcting package metadata does not affect its content, and forcing authors to increase the version for a simple metadata fix (e.g. a description typo) is unnecessary pain.

Also, as David Ebbo has pointed out, deleting a package from the NuGet.org site does *NOT* remove it from existing projects; therefore, your entire point about "don't allow your team members to delete a source tree" is completely without merit.

You misread my point. When using the popular Restore workflow, it does prevent existing projects from using it. I was just saying that if we allowed deletions, everyone would stop using the Restore workflow and end up committing packages.
But this is not something we'd like to happen.

Situation 2: you delete a package that no one has. If I'm lucky I can find the open-source project somewhere. If not, I have a big problem since part of my project is gone and has to be recreated. Imagine what a cost impact this potentially has!

If you are relying that heavily on a particular package (or the corresponding open-source project) then you need to mirror it locally somewhere... Wait, I know, how about the source tree (i.e. in someplace like "Externals").

Also, as David Ebbo has pointed out, deleting a package from the NuGet.org site does *NOT* remove it from existing projects; therefore, your entire point about "don't allow your team members to delete a source tree" is completely without merit.

You misread my point. When using the popular Restore workflow, it does prevent existing projects from using it. I was just saying that if we allowed deletions, everyone would stop using the Restore workflow and end up committing packages.
But this is not something we'd like to happen.

Your feature appears to be misguided in both concept and implementation:

1. Typically, if I am using a particular package, I want the latest stable version, not whatever version I happen to have added when I first created the project.

2. If I end up deploying a project with an obsolete package with no upstream support, then what?

I have another suggestion... Have a checkbox on a per-package basis, controlled by the package author, that allows deletion when enabled. Another effect of this checkbox would be to automatically add a very stern warning when somebody goes to
add a reference to said package (and on the package web page).

@mistachkin: You have misread my point: I totally agree you should mirror packages you depend on. That said: I think 99% of NuGet users don't do that. If these 99% are using package restore, a means of not polluting your source control with dependencies
(after all: that's what NuGet tries to solve), chances of breaking software are high.

Also: I don't want to blindly update to the latest version of a package. Reason for that is: I use the package version I know. If I have code base A with package B 1.0, in theory I have a different build when source code base remains A but package becomes B
1.1. Same input (= my code) should always yield the same output, regardless of dependencies having updated somewhere. I should be responsible for updating them, not any package author out there as every package would exponentially add a chance of breaking
my build.

1. Typically, if I am using a particular package, I want the latest stable version, not whatever version I happen to have added when I first created the project.

2. If I end up deploying a project with an obsolete package with no upstream support, then what?

I don't necessarily agree with that. First off, a newer version might not be compatible with your project. Secondly, it's the responsibility of the project owner to update a dependency, not the package owner.

Your feature appears to be misguided in both concept and implementation:

1. Typically, if I am using a particular package, I want the latest stable version, not whatever version I happen to have added when I first created the project.

2. If I end up deploying a project with an obsolete package with no upstream support, then what?

No, the feature works exactly as it should. The point of the feature is to end up with the same behavior as if you had committed your packages, but without doing that. If the package was magically updated to a newer versions, all kind of things could randomly break.
And two users may end up with different bits just from building 10 minutes apart!

NuGet has full support for updating packages in a project, but this is always an explicit action, and not something that just happens automatically when restoring packages.

@hhariri: Exactly and I agree. But let me copy-paste from a previous comment:

"True. It is folly. I totally agree. You should pull the responsability of a package being available or not in your own hands. Unfortunately only 500+ feeds exist on MyGet and probably a number of feeds exist in
people's own environments. Still, looking at the high number of package downloads at stats.nuget.org, a large majority is not mirroring the packages they use and therefore face doom if packages can be updated/deleted from the official feed."

This means either it should be made extremely difficult to break other people's builds -or- guidance/tooling to mirror packages should be in place
*ahem* www.myget.org *ahem*. NuGet is too point-and-click today and beginning & enthousiast users should not be crippled by package authors.

@mistachkin: You have misread my point: I totally agree you should mirror packages you depend on. That said: I think 99% of NuGet users don't do that. If these 99% are using package restore, a means of not polluting your source control with dependencies
(after all: that's what NuGet tries to solve), chances of breaking software are high.

Also: I don't want to blindly update to the latest version of a package. Reason for that is: I use the package version I know. If I have code base A with package B 1.0, in theory I have a different build when source code base remains A but package becomes B
1.1. Same input (= my code) should always yield the same output, regardless of dependencies having updated somewhere. I should be responsible for updating them, not any package author out there as every package would exponentially add a chance of breaking
my build.

I cannot speak for "99% of NuGet users"; however, I suspect they are more savvy than you believe they are.

As far as "blindly updating" to the latest version of a package, I never suggested that; however, it would be very nice to at least give developers the option of doing that OR in the event that the "restore" functionality is being used it can be interpreted
as "give me the latest stable version because I did not bother to mirror the exact version that I wanted".

Your feature appears to be misguided in both concept and implementation:

1. Typically, if I am using a particular package, I want the latest stable version, not whatever version I happen to have added when I first created the project.

2. If I end up deploying a project with an obsolete package with no upstream support, then what?

No, the feature works exactly as it should. The point of the feature is to end up with the same behavior as if you had committed your packages, but without doing that. If the package was magically updated to a newer versions, all kind of things could randomly break.
And two users may end up with different bits just from building 10 minutes apart!

NuGet has full support for updating packages in a project, but this is always an explicit action, and not something that just happens automatically when restoring packages.

If things "randomly break" when you update to a newer version of a package, then perhaps using that package should be rethought?

I cannot speak for all package authors; however, I never break compatibility in any non-obvious or undocumented way.

Just to clarify: I am very aware of the fact that by using NuGet I'm responsible for the possibility of my build breaking because of any package author, but I can mitigate that risk by mirroring. I'm just affraid of people who are unaware starting to yell
"NuGet broke my builds"

Anyway: back to the discussion. Should package deletions be possible? (I'm shutting up from now on)

If things "randomly break" when you update to a newer version of a package, then perhaps using that package should be rethought?

I cannot speak for all package authors; however, I never break compatibility in any non-obvious or undocumented way.

Have you tried using the package restore workflow? I think if you play with it a bit, you'll understand why it works the way it does. I don't think any argument can be made to automatically update packages on Restore. It is not just about breaking
changes, but more simply about predictability. Updating packages on Restore would be analogous to someone choosing to update your committed external references without telling you. It is just wrong.

Interesting discussion! One Maarten and I could not tackle in less than a chapter in our upcoming book. Looking at the length of the answers here, it was probably not a bad decision to dedicate a chapter to this no-commit strategy :-)

NuGet.org is a general public feed: relying on this in your CI environment is
not a better practice. Simply because you don't control it, nor its packages.

Relying on your own package repository (or repositories), such as a network share, internal NuGet server, MyGet etc., and
mirror packages onto yours, shields you from package deletions on NuGet.org

Listing/Unlisting as a soft deletion mechanism is definitely a good thing: it's reversible and has the same effect (from user point of view) as deleting it from the feed.

As with anything: communication is key. If a package owner is about to stop support for a certain (version of a) package, he should communicate this to the consumers of that package somehow. This is only easily achievable in a controlled
environment. (cfr.
MyGet's use case explained here)

As for NuGet.org, I'd go for the unlisting/listing strategy and a very strict, controlled deletion mechanism for:

packages with 0 or 1 downloads

packages deletion requests to the NuGet.org team (preferably with some sort of
pending delete notification to its consumers such as a warning on the site, or a custom http header in the response etc.)

As I said, depending on the NuGet.org feed is adding a risk to your build environment. If there is a strict and well-known package deletion process, the risk is properly mitigated, and you fall-back on the acceptable risk of potentially losing connectivity
to the NuGet feed. (if it isn't acceptable, host your own, and make sure you can blame your IT department :)).

I think the way it currently works is better than the suggested alternative. The potential valid case 1 seems kind of irrelevant to me, since there is no harm in not deleting the package in this case. Since case 2 is going to be "solved" soon without ability
to delete packages it seems also to be a non-issue too. The case 3 is the only one that appears 100% valid to me, and I'm guessing that it would be rare enough so that the package with the accidental sensitive information could be deleted by request.

At modern age the argument "I do not accept the idea that I must relinquish my ability to control what versions are retained [forever] by said service" does not seem to be quite convincing. Once your "stuff" is out there in internet you already relinquished
control of it, any one can copy/retain it as they please. People even can end-up writing mirroring crawlers just so they can feel confident, that the stuff they rely on to be there won't disappear suddenly. You can make it difficult for people to find a copy
but you can't make it impossible.

I think the way it currently works is better than the suggested alternative. The potential valid case 1 seems kind of irrelevant to me, since there is no harm in not deleting the package in this case. Since case 2 is going to be "solved" soon without ability
to delete packages it seems also to be a non-issue too. The case 3 is the only one that appears 100% valid to me, and I'm guessing that it would be rare enough so that the package with the accidental sensitive information could be deleted by request.

At modern age the argument "I do not accept the idea that I must relinquish my ability to control what versions are retained [forever] by said service" does not seem to be quite convincing. Once your "stuff" is out there in internet you already relinquished
control of it, any one can copy/retain it as they please. People even can end-up writing mirroring crawlers just so they can feel confident, that the stuff they rely on to be there won't disappear suddenly. You can make it difficult for people to find a copy
but you can't make it impossible.

For stuff that is "out there", you are correct; however, there is no reason that NuGet.org should maintain (or be required to maintain) obsolete versions of a given package forever. What if I needed to produce a new version of the package every day
(I don't, but that is beside the point). Keeping package versions forever is an anti-pattern, in my opinion. Perhaps an alternative should be to allow "delayed" deletions (e.g. after 5 days)?

Ah, now the real reason behind this change is finally revealed. Internal politics. I suspected as much.

How so?

Removing the ability to delete is purely to protect some subset of developers from their own ill-advised decisions.

Not following you. The Restore workflow works exactly as it should, and relies on packages remaining available. It is not widely used by NuGet users, and breaking it as you suggest would be a disservice to the community. What are the ill-advised decisions
you refer to, and where do 'internal politics' come into play?

Ah, now the real reason behind this change is finally revealed. Internal politics. I suspected as much.

How so?

Removing the ability to delete is purely to protect some subset of developers from their own ill-advised decisions.

Not following you. The Restore workflow works exactly as it should, and relies on packages remaining available. It is not widely used by NuGet users, and breaking it as you suggest would be a disservice to the community. What are the ill-advised decisions
you refer to, and where do 'internal politics' come into play?

The "community" you mention also includes package authors and as near as I can tell they were not consulted at all prior to this "policy" change taking place that prevented them from removing any obsolete versions of their package(s).

Not seen any convincing arguements here for allowing the deletion of packages, the exception being the accidental inclusion of security info but that is covered by placing a request to the team.

Not deleting packages is a cornerstone of the package restore functionality, allowing delrtions would make the feature worthless. It's all very well saying that people should be working of the latest stable binaries, but if the obsolete version
supports all the use cases then there is no reason to upgrade. It seems that the person arguing for this change thinks the package owner can decide if a previous version is obsolete. That is wrong, it is only the consumer of your package who can decide
that, based on the functionality they actually use.

To force someones build to break, then to upgrade their app to support a newer binary and finally to retest for no benefit whatsoever is plain wrong to me.

The package restore functionality is a very recent "innovation" that if I am understanding it correctly is fragile by it's very nature (i.e. on a fresh developer machine, it must re-download all the packages referenced by a given project). Not being
able to delete obsolete versions of my own packages just makes it less likely that I will continue to support NuGet as an official distribution channel for the code I own and/or maintain.

I think the three cases you highlight David are good. In general I think it would be good to make package deletion hard (require an email) even if NuGet didn't have package restore functionality. If
you make sure people know this ("Package uploads are permanent") then they will be more careful not to publish a dud package in the first place. Having to delete a package isn't good for anyone involved.

Ability to edit metadata without republishing is a nice idea, I've had to do that a few times. Having a short period of time/#downloads to allow deletion of accidental packages would be appreciated but could also be handled by explicit request (then you can
evaluate how many of these you get to decide if its worth implementing).

In the old NuGet gallery, it was possible for the package owner to completely delete some versions of a package. In the new gallery (which was a complete rewrite), we remove that option, and instead implemented a new
unlisting feature. When a package is unlisted, it doesn't show up in any package lists, either on nuget.org or in the VS extension. But the package can still be installed by explicit version.

The reason we did that is that people were increasingly using our workflow that allows you
not commit packages to source control, and instead have them come down from nuget.org as needed (this workflow became even simpler in 1.6). This workflow is high reliant on packages continuing to be available on the feed. If the author removes a package
that someone depends on, their build instantly becomes broken, which is a very bad thing. But if the package is instead unlisted, everything continues to work, even though no one will be able to install the unlisted version in a new project (at least not without
going out of their way).

Generally, we think that is the correct behavior, as allowing packages to be too easily deleted was causing some real breaks and that had to be addressed. However, a few people have raised cases what they might really want to delete a version of a package.
Some potentially valid cases to do this:

The package was just uploaded and found to be bad. Since no one has likely installed it yet, deleting it is harmless.

The package binaries are ok, but the metadata is incorrect (e.g. the description or author name). For this, we are planning to allow modifying the metadata, or potentially re-upload the package with identical content but modified metadata.

The package mistakenly contains some sensitive information and needs to be taken down asap.

It should be noted that the NuGet team is willing to delete packages on a case by case basis via explicit owner request (we probably didn't advertise this option sufficiently). But today there is no automatic way for the owner to do this (e.g. via nuget.org
or the nuget.exe console app).

Question to discuss: do you think authors be able to freely delete any package version, or does that lead to more problems than it solve?

I don't believe this feature of never delete was misguided. I think it was guided very much by other package management tools that have been out there for awhile. They have more experience than any of us on what is best for the community. With this decision
to not delete packages, we are standing on the shoulders of giants. Look at ruby gems. You can only yank a gem (which means soft-delete)
http://help.rubygems.org/kb/gemcutter/removing-a-published-rubygem (I urge you to read the reasons why, note the similarities). Debian, the package management system that has been
around for years, tried and tested and the de facto statndard, you cannot directly delete a package.
http://www.debian.org/doc/manuals/developers-reference/pkgs.html#removing-pkgs

The NuGet team is not trying to decide what is superior, only to stand on what works and has worked in other communities for years.

So I ask, how is this misguided?
____
Rob
"Be passionate in all you do"

If it were to really come to allowing packages to be deleted from the feed, our advice would be to mirror packages as soon as you set the feature up. It makes for a worse user experience than the current one, but it would in effect negate allowing hard package
deletions.

What I find interesting here is unless it's an exceptional scenario (for instance package contains hardcoded credentials etc), what benefit do you get by pulling down a package?

While I could probably concoct some reason why a library author might feel the need to be able to absolutely control deleting of old packages I agree completely with the current soft-delete option plus manual delete by request approach now in place. Given
the nice semantic versioning support in place in Nuget now having older packages stay available in normal cases is great.

I do think that it's prudent for a commercial enterprise to either mirror the packages they depend on locally or simply check them into source control. There's no way I'm going to allow my business' build process be susceptible to outside
control/interference. And that's easy to handle with Nuget.

But the flip side of this applies as well. If a library author feels they need more control for some reason, then they shouldn't have external dependencies either. So if
mistachkin wants complete control, he should host his packages himself so he can delete at will (and suffer the consequences of breaking his user's builds unexpectedly). If it's truly
critical for some reason, he should take control of that.

The only disadvantages I can see to having the old packages stick around are that it takes up some extra disk space on the Nuget Server and that's not my problem. :)

Incidentally, on David's point 3 about sensitive information, a hard delete capability isn't really going to help much. Once the information is out there, it's out there. If you accidentally publish your password or a private key you really need to change
it immediately anyway as you can't assume that it won't be propagated out to the "bad guys". About the only thing that deleting really solves with would be something embarrassing or a racial epithet.

This discussion is getting really heated, and Rob asked me to supply an external perspective. I'm the creator of rubygems.org, and we deal with the "package deletion problem" all of the time. Rubygems.org does *not* allow users to remove packages. The simple
reason why is that it could cause others' code to break without the original author knowing. We've had incidents where people in the community take down all of their work, utterly preventing others from doing their jobs (deploying, running code, etc). It's
the service's responsibility to make sure that doesn't happen. Rubygems.org (and NuGet) are all about sharing your code and making other developer's lives easier, let's please not forget that.

However, we do remove packages manually from time to time, on a case-by-case basis. Here's the two common cases I've seen:

1) Package was mistakenly published, contains internal/sensitive code. Usually they have "yanked" (hidden from being able to be installed/downloaded) these releases, and we just manually clear the uploaded data they've sent us.

2) There was a "bad release". You found a bug. Maybe it does something bad, a file was forgotten, or something. In this case we refuse to delete the gem since it's not *extremely* critical to remove it permanently. It costs barely anything to host it. It
costs more for me to deal with it, and someone else may have actually depended on that release. Just yank it, fix the problem, bump the version number, and push a new release.

Now given all of this, I've been leaning towards changing our policy to perma-deleting gems within the first ~15 minutes after releasing, if they're yanked. I haven't implemented this yet but the support load me and my fellow maintainers has to deal with
kind of sucks (1-3 deletion requests a week, but not all fall under #1).

I hope this helps. Also, give these project maintainers a break, I wish I could have worked with .NET with a decent package management system in place (or heck, any OSS code at all). Good luck folks.

@qrush: thanks Nick, it's great to get some wisdom from the rubygems side!

Seems the general consensus is that allowing deletion is bad, and there is essentially no drawback from keeping old versions around.

For now, we'll stay with a manual process to request deletion in exceptional scenarios. If this becomes too much of a burden, we'll try to add deletion support for the few cases where it makes sense (mostly, just uploaded packages with almost no downloads).

@qrush: thanks Nick, it's great to get some wisdom from the rubygems side!

Seems the general consensus is that allowing deletion is bad, and there is essentially no drawback from keeping old versions around.

For now, we'll stay with a manual process to request deletion in exceptional scenarios. If this becomes too much of a burden, we'll try to add deletion support for the few cases where it makes sense (mostly, just uploaded packages with almost no downloads).

Thanks everyone for your input!

Somehow, I knew this entire discussion was moot (which is why I originally refused to participate in it); however, I ended up participating against my better judgement. Clearly, I see now that your mind has already been made up all along because of your
new pet feature and the associated internal politics, which I still think is just begging to have more problems than it solves. But, I digress. Microsoft seems to believe they know how to handle my own packages better than I do? My past personal
experience would suggest otherwise. Also, I find it rather amazing that basically none of my original concerns/questions have actually been addressed (by anyone). Apparently, these forums are not a good venue for actually getting questions answered.

While I could probably concoct some reason why a library author might feel the need to be able to absolutely control deleting of old packages I agree completely with the current soft-delete option plus manual delete by request approach now in place. Given
the nice semantic versioning support in place in Nuget now having older packages stay available in normal cases is great.

I do think that it's prudent for a commercial enterprise to either mirror the packages they depend on locally or simply check them into source control. There's no way I'm going to allow my business' build process be susceptible to outside
control/interference. And that's easy to handle with Nuget.

But the flip side of this applies as well. If a library author feels they need more control for some reason, then they shouldn't have external dependencies either. So if
mistachkin wants complete control, he should host his packages himself so he can delete at will (and suffer the consequences of breaking his user's builds unexpectedly). If it's truly
critical for some reason, he should take control of that.

The only disadvantages I can see to having the old packages stick around are that it takes up some extra disk space on the Nuget Server and that's not my problem. :)

Incidentally, on David's point 3 about sensitive information, a hard delete capability isn't really going to help much. Once the information is out there, it's out there. If you accidentally publish your password or a private key you really need to change
it immediately anyway as you can't assume that it won't be propagated out to the "bad guys". About the only thing that deleting really solves with would be something embarrassing or a racial epithet.

Ok, so every time I upload a new version of my package I can simply submit a request to David to have him delete the old one? Somehow, I don't think he's going to like that idea.

Clearly, I see now that your mind has already been made up all along because of your new pet feature and the associated internal politics, which I still think is just begging to have more problems than it solves. But, I digress. Microsoft seems
to believe they know how to handle my own packages better than I do?

First of all, this isn't "Microsoft", this is the NuGet team. While some of those people are Microsoft employees (and some aren't), this isn't some company policy run amok. It's a reasonable reaction to a real problem, as @qrush has amply illustrated, IMO.

Just as important, though, is that I think you need to accept that the NuGet server is a service for the benefit of all of its users, not just you (and not just developers or publishers). When you publish a package on the NuGet server, there is an implied
contract that that such a package will exist in perpetuity. This new release of the gallery software better codifies that implicit contract between the NuGet team and the end users who are downloading packages. That you are unhappy with this contract says
that you will probably be happier hosting your own server where you can set your own rules. As an end user who consumes packages, I'm thrilled with the NuGet team's decision; as a developer who publishes packages, I'm also quite pleased, even if it does mean
a little more effort on my part if something unexpected should arise.

Clearly, I see now that your mind has already been made up all along because of your new pet feature and the associated internal politics, which I still think is just begging to have more problems than it solves. But, I digress. Microsoft seems
to believe they know how to handle my own packages better than I do?

First of all, this isn't "Microsoft", this is the NuGet team. While some of those people are Microsoft employees (and some aren't), this isn't some company policy run amok. It's a reasonable reaction to a real problem, as @qrush has amply illustrated, IMO.

Just as important, though, is that I think you need to accept that the NuGet server is a service for the benefit of all of its users, not just you (and not just developers or publishers). When you publish a package on the NuGet server, there is an implied
contract that that such a package will exist in perpetuity. This new release of the gallery software better codifies that implicit contract between the NuGet team and the end users who are downloading packages. That you are unhappy with this contract says
that you will probably be happier hosting your own server where you can set your own rules. As an end user who consumes packages, I'm thrilled with the NuGet team's decision; as a developer who publishes packages, I'm also quite pleased, even if it does mean
a little more effort on my part if something unexpected should arise.

jm2c

As an author of NuGet packages, you were always free to "not delete" your obsolete package versions before.

To look at other package management systems and understand why they do things a certain way is an important step in the right direction.

I am with Nick though, we don't delete packages just because. They get unlisted. If you don't like it, you are not required to use nuget.

There are other package management systems in .net (open wrap) that may be more allowing. Or you can host your own packages on your own nuget server.

I went through the pain of packages getting deleted (Rx anyone?) and before you say anything, I was checking the packages in. My packages got corrupted in source control so I needed to repull them. That fiasco alone taught us a lesson we should have learned
from the experience of those wiser than us.

As an author of NuGet packages, you were always free to "not delete" your obsolete package versions before.

Yes. you're right, I was free not to delete old versions. And I don't delete old versions. There's no reason to, unless you are abusing the system for some purposes other than its intended purpose. I'll reiterate that you're free to run your own server so
that you can set your own policies. Putting your packages on the NuGet team's server means you're subject to their policies. As near as I can tell, you're the only person in this thread who seems to disagree with them, and I haven't personally heard any persuasive
reason why you should be exempt from the policies.

For the Microsoft people: Has anybody tried to buy a retail copy of Windows 2000, Office 2000, or Visual Studio 6 lately? No? By the logic of this thread, they *MUST* be available forever because people might be depending on them and do
not want to upgrade. How's this for a strawman argument?

As an author of NuGet packages, you were always free to "not delete" your obsolete package versions before.

Yes. you're right, I was free not to delete old versions. And I don't delete old versions. There's no reason to, unless you are abusing the system for some purposes other than its intended purpose. I'll reiterate that you're free to run your own server so
that you can set your own policies. Putting your packages on the NuGet team's server means you're subject to their policies. As near as I can tell, you're the only person in this thread who seems to disagree with them, and I haven't personally heard any persuasive
reason why you should be exempt from the policies.

Now, back to lurking...

Characterizing a legitimate function of the system (as it was) as abuse is counter-productive and untrue.

>> every time I upload a new version of my package I can simply submit a request to David to have him delete the old one

Can you explain more about the scenario that forces you to delete the old package
every time you push a new one? I can’t think of any popular NuGet packages that needs to do this, and it shouldn’t be necessary. What is it about your scenario that requires this?

While I could probably concoct some reason why a library author might feel the need to be able to absolutely control deleting of old packages I agree completely with the current soft-delete option plus manual
delete by request approach now in place. Given the nice semantic versioning support in place in Nuget now having older packages stay available in normal cases is great.

I do think that it's prudent for a commercial enterprise to either mirror the packages they depend on locally or simply check them into source control. There's no way I'm going to allow my business' build process
be susceptible to outside control/interference. And that's easy to handle with Nuget.

But the flip side of this applies as well. If a library author feels they need more control for some reason, then they shouldn't have external dependencies either. So if
mistachkin wants complete control, he should host his packages himself so he can delete at will (and suffer the consequences of breaking his user's builds unexpectedly). If it's truly critical
for some reason, he should take control of that.

The only disadvantages I can see to having the old packages stick around are that it takes up some extra disk space on the Nuget Server and that's not my problem. :)

Incidentally, on David's point 3 about sensitive information, a hard delete capability isn't really going to help much. Once the information is out there, it's out there. If you accidentally publish your password
or a private key you really need to change it immediately anyway as you can't assume that it won't be propagated out to the "bad guys". About the only thing that deleting really solves with would be something embarrassing or a racial epithet.

Ok, so every time I upload a new version of my package I can simply submit a request to David to have him delete the old one? Somehow, I don't think he's going to like that idea.

To look at other package management systems and understand why they do things a certain way is an important step in the right direction.

I am with Nick though, we don't delete packages just because. They get unlisted. If you don't like it, you are not required to use nuget.

There are other package management systems in .net (open wrap) that may be more allowing. Or you can host your own packages on your own nuget server.

I went through the pain of packages getting deleted (Rx anyone?) and before you say anything, I was checking the packages in. My packages got corrupted in source control so I needed to repull them. That fiasco alone taught us a lesson we should have learned
from the experience of those wiser than us.

Question #1: What if I want to push a new build every week or every day (I don't)?

Question #2: Why was this "policy" change made without consulting any of the existing package authors in the system?

Question #3: Since there are clearly multiple failure modes for the new "restore" feature, why insist on retaining obsolete package versions forever?

RE Question #1: The look of the OctoPack site is irrelevant to the point of this discussion. When we get a web designer to contribute to the gallery, we can clean that up.

But, suppose we don't change the design, older packages can be removed from that list today by simply unlisting them.

RE Question #2: We discussed this in open forums that *anyone* could read, but we could have done a better job of communicating it even so. Agreed. You are correct there. We'll take that into consideration next time. I take full fault
in this. However, that doesn't make the decision wrong, so again, completely irrelevant to the debate. As Rob pointed out, we looked at the examples of other package managers. We don't suppose ourselves to be "superior" to the examples who've been around for
many years so we tend to follow their lead until we have a really really good reason to deviate.

RE Question #3: Disk space is cheap. Why shouldn't we keep them around? I haven't been convinced that we need to remove older ones.

So let me pose a question to you as this conversation has veered in odd directions. So far, the existing feature for unlisting packages addresses the main concerns you've raised. Unlisting packages removes them from all NuGet UI. The only way unlisted
packages can get installed are in the minor cases when someone is installing a package that has a dependency on the unlisted package or someone is using the package restore feature. In cases where unlisting isn't good enough (such as the conditions Ebbo described
earlier), the NuGet gallery team will delete the actual package contents for those who contact us. You can easily contact us through the website. So at worst, you suffer a minor delay.

So given all that, I think the crux of the issue is this: Why should the website provide a way to automatically and fully delete a package given we have the ability to unlist packages? What do you know that Ruby Gems, Debian, and we have all missed?

I think bringing all these extraneous issues into this distracts from the core discussion. If we can focus on this one question, I think we'll have a more productive discussion. Thanks!

I should add, that the majority of folks in this discussion are not Microsoft employees. So pointing fingers at a MS conspiracy isn't helpful. I should also add that we've had our minds changed before on items we were fiercely committed to based on solid
reasoning and examples we hadn't considered before. I was fully convinced everyone should commit packages to source code but I've been convinced otherwise.

If you have a great argument that provides a really really good reason to deviate the from the lead set by other package managers, then I'm all for it. In fact, if I heard such an argument, I'd probably turn around and provide it to them as well. I don't
see how NuGet should differ from them in this particular case, but I always keep an open mind.

I should add, that the majority of folks in this discussion are not Microsoft employees. So pointing fingers at a MS conspiracy isn't helpful. I should also add that we've had our minds changed before on items we were fiercely committed to based on solid
reasoning and examples we hadn't considered before. I was fully convinced everyone should commit packages to source code but I've been convinced otherwise.

If you have a great argument that provides a really really good reason to deviate the from the lead set by other package managers, then I'm all for it. In fact, if I heard such an argument, I'd probably turn around and provide it to them as well. I don't
see how NuGet should differ from them in this particular case, but I always keep an open mind.

I never mentioned a conspiracy. I was simply pointing out that the people running the NuGet project are Microsoft employees with the associated internal politics that goes along with that.

It seems that the "unlist" feature as implemented in somewhat buggy. I unlisted the previous version of Eagle and it shows me with 4 published packages and one unlisted (i.e. it thinks Eagle is completely unlisted, which is not the case).

My only other problem with NuGet (to change the topic slightly) is that the 32x32 icon I provided is being stretched to fit various sizes on the NuGet.org web site with very ugly results. It might be nice to have a logoUrl that is different from the
iconUrl to prevent this type of issue (i.e. the web site would use the logo and the VS plugin would continue to use the icon).

Anyhow, in the future it would be nice to know about major policy changes in advance.

Ok, "conspiracy" was too strong a word. :) Even so, internal politics wasn't a factor in this decision. I'm not sure where you're getting that idea from. But lets move on.

Regarding the bug in the unlist feature, I think the UI is a bit clunky. You can unlist a specific version or unlist all versions. You might have done the latter on accident. The text of the UI should clearly state what you are doing. To be absolutely sure,
visit the actual package (while logged in) and click on the package version in the list at the bottom. That should take you to a page to unlist just that package.

If you don't mind, could you log an issue here: https://github.com/NuGet/NuGetGallery/issues/ ideally with screenshots and the exact steps you took so we can look at it. If it's a bug, we want
to fix it. If it's the UI that's a problem, we want to improve it.

As someone who argued passionatly against the need to check packages into source control (I believe I may have called it 'dumb') I feel it only right that I run to the defence of the new policies.

Firstly, to echo what others have said here - if you're using NuGets in a decent sized business (I have about 60 devs) the first thing you do (unless you are an idiot) is set up your own NuGet server, I think it took me less than an hour. We only use
this server internally, we go as far as disabling nuget.org on the majority of our dev machines, and then we copy any NuGets we've decided to use (after much testing) to it. We back this server up nightly, so if nuget.org ever 'goes away' we're gold.

When we publish, we publish to the internal server. If we ever get happy enough to OSS our own NuGets then we'll publish to nuget.org after we've had them on our internal server for a few days.

Now I fully appreciate that for hobbyists this workflow is a step to far, but considering the other difficulties involved in NuGet production on a bigger scale, creating your own NuGet server is really one of the easier things to do.

There are no real downsides to not being able to automatically remove NuGets, where as the downsides to NuGet deletion are huge in the situation where you are dependent on them. Ironically, having our own server we delete NuGets all the time - but
I know all the users. I wouldn't support this on the public feed.

One thing no ones really talked about though is the concept of beta feeds. This came up during the semversion discussions if I remember. There is an argument for a hosted pre-release feed. The idea being that you can only post to the pre-release
feed, and then test from there. That feed would support deletion, etc. once happy you can push to the 'live' feed where deletion isn't possible. In a way this is similar to the NuGet Server 'mirror' idea most enterprises are using, but just providing
it as a service.

Its not a priority for me, but maybe it would be something the community would like?

One thing no ones really talked about though is the concept of beta feeds. This came up during the semversion discussions if I remember. There is an argument for a hosted pre-release feed. The idea being that you can only post to the pre-release
feed, and then test from there. That feed would support deletion, etc. once happy you can push to the 'live' feed where deletion isn't possible. In a way this is similar to the NuGet Server 'mirror' idea most enterprises are using, but just providing
it as a service.

Its not a priority for me, but maybe it would be something the community would like?

Interesting you ask this question, as this is exactly what you can do with
MyGet.org. The idea of for instance a CI, QA and PROD feed, each with its own target audience, to support a continuous package workflow (also described
here).

As you mention, this is indeed an enterprise scenario (controlled environment). Not sure this is something NuGet.org should provide, as each team might take a different approach, depending on its needs. IMO NuGet's support for SemVer pre-release packages
is sufficient to cover the needs of the general (open source) public. Consider NuGet.org as just another (PROD) feed, and use whatever else you need to suit your needs. There's really no difference between any PROD feed and NuGet.org, besides target audience.

Once released, it is there to stay (except for those extremely rare cases where unlisting is not good enough).

@davidebbo - I agree with you. I love the package restore, but if I open some solution down the road it would not be good if that package had disappeared.

What if you were allowed to delete the package in such a way that it no longer shows up in the searches or lists of packages available? If a direct request for the package came in - it would be allowed though. That essentially _helps_ each side
of the camp here doesn't it?

@waynebrantly That is exactly the behavior we have today with the unlisted packages. They don't show in in search results. You can only install it from direct request (if you know the exact version number) or if it's a dependency for another package.

@mistachkin: Not to bring this thing back up, but I am now one of your users (of which you earlier mentioned something about understanding our needs).
http://nuget.org/packages/System.Data.SQLite is a package I use and I am just now setting up enable package restore for it. Turns out that the version I was using 1.0.75.0 has been deleted?

If it was unlisted and package restore is not working, then it's a bug in nuget. Otherwise I would request that you do not delete older versions of this package.

Not a huge deal, I just rolled back to the particular revision I still had it in and uploaded it to
myget.org along with all of my other packages.config files and I am good to go. MyGet FTW!

But I also might be a little more savvy with the system than other people that may be wanting to enable package restore later than the time they downloaded the packages. Asking them to upgrade to the latest just to enable package restore seems unreasonable.
And then they need to find the right solution for package location.

So coming from a user of yours, please unlist and not delete. Thanks!
____
Rob
"Be passionate in all you do"

Looks like there are no unlisted versions. Only 1.0.66.1, 1.0.76.0 and 1.0.77.0. I suspect that it was deleted before we moved to the new gallery. The new gallery prevents deletions so, this break would not have happened.

For our nuget package [ and symbols + sources symbols.nupkg ] work i have two user stories where having //nuget.org/ package delete support is desirable.

1. I published a package to get familiar with eco system and then upon realizing how things work and security controls over access to published packages I wanted to remove packages, # downloads sitting at < 10-20 for this case. This in
theory this user story could be mitigated if I knew up front about the
http://preview.nuget.org/ environment mentioned below.

2. I published a package version and found immediate bugs doing end2end test of new package and wanted to delete package so I can publish an update of the package with the same version number with those issues resolved, # downloads sitting at <
10-20 for this case. The //symbolsource.org/ environment added delete functionality a while back which allows teams to not have to constantly increment the package version when changes were not worthy of work needed to increment version
in source control and other areas given the number of users that would have accessed/pulled that version is known to be a small controlled group.

As i understand it this doesn't align with nuget recommended versioning scheme
http://semver.org but for us it aligns with how we have been doing things so as to minimize the number of times we take the hit incrementing nuget package version settings and associated automated build output post processing steps until such time as we
have reached a final build for a given version. I also now understand that if we need to test or experiment with interim build package publishing we can use
http://preview.nuget.org where data on that server is reset each night, which shouldn't interfere with your testing or experimentation.

To restate the above points the main area where not having delete right now is affecting things for us is that during development of a nuget package we’ve used the model of only incrementing the package version number when we’ve added a set of
features versus just bug fixes. Naturally this behavior would change once we published the version, and listed it, that was intended to be an official version specific release. Taking this approach has cut down on the number of times, during
version prerelease dev/test, that we have to modify nuget package nuspec settings prior to change set submission and automated build processing. This versioning scheme is also currently supported when publishing symbols + sources *.symbols.nupkg packages
on //symbolsource.org/.

We do plan to allow package deletion when the right requirements are met. We haven't settled on what those requirements will be. It's also lower priority compared to other work we have queued for the next few iterations.