Archive upload process changes

Recommended Posts

Just a note to inform you of changes in the way I am going to be handling uploads in the future (and recent past--effective 1 Sep 2013). I've modified the README.INCOMING etc. files on the archive but probably no one will notice that so this may make it easier for all of us.

If you upload a file to the archives, you will be asked to reply to an email. That email might end up in your SPAM folder (especially on overzealous GMail) so if you do upload something keep an eye out. It will usually get there within a day or so of the time you upload, and as part of that process your file will be moved out of Incoming temporarily.

Why all this, you ask? Or you didn't ask but I'll tell you anyway. It's slightly depressing but that's the way it goes.

I think quoting the email is probably the simplest way to tell you the rest:

Due to the large number of uploaded DOOM levels that are either junk from an invalid email account, or uploaded with someone else's name on it, I have had to resort to checking the validity of every upload via email.

If you uploaded one or more files in the last week, then you need to reply to this email with the name of the file or files. That will be enough to allow it in. If I either get a reply that you aren't the uploader, or no reply in 7 days, the file(s) will be rejected. They have meanwhile been moved to a folder underneath the incoming folder which you (and others) cannot see.

If you indeed uploaded several files at once, please identify them all in your response. I will only send one of these requests per uploader email address regardless of the number of files involved.

Share this post

Link to post

Is this all done manually on Ty's part? I can't imagine how frustrating that might be for him. :(

Anyhow, this is probably better than the alternative, but it does throw a monkey wrench into the already-difficult process of uploading to /idgames. I wonder if any folks would be willing to consider the idea of developing some sort of upload frontend for DW /idgames that's tied to forum accounts (similar to how t/nc does it) which folks can use as an alternative.

Share this post

Link to post

To be honest, I'd rather do this than have to deal with people overwriting others' files (the worst part) and I was still having to evaluate suspicious email addresses by sending a test message. This way even if the email is correct, but that's not who uploaded it, the original author's aware of what's going on and has to in effect approve of the upload.

It's semiautomated now; I have several scripts I run to shuffle files and such. I'll probably add more automation to it as time goes on but it still sometimes takes a human eye to look at things. Let's face it, I shouldn't reject "fred [at] gmail [dot] com" just because Fred wants to avoid bot harvesting of his address. And though I could code for that, I don't really get that many uploads.

There are some folks who upload frequently enough that I "know" them and won't go through all this every time (think rslxxxxx.zips in combos) but it helps keep me more actively involved too. I would sometimes end up with 7-10 days between logging on but now I tend to get on daily-ish so throughput is better for everyone.

And if anyone chooses to pre-empt all this by emailing me ahead of the upload that they're going to do so, that simplifies it more. Worst thing is that this doesn't keep the crap away; I'll just know that the uploader didn't lie about their email. But I've never used quality as a criterion for allowing something into the archive, as discussed elsewhere and as is obvious when you look through the files...

Share this post

Link to post

Just a quick question, would you be able to recruit someone to help you manage the archives? I dunno who you could ask, but I'm pretty sure maintaining an archive of this magnitude shouldn't be done by only just one person.

Share this post

Link to post

Just a quick question, would you be able to recruit someone to help you manage the archives? I dunno who you could ask, but I'm pretty sure maintaining an archive of this magnitude shouldn't be done by only just one person.

This one person's been doing it since 1998 and that was when there was an active Quake community too, so I think I'll be okay. Thanks for the thought though.

Share this post

Link to post

This is great news for sure. Hopefully it will stop a lot more junk and spam wads from been uploaded.

I got that email yesterday after my Zone 300 upload and I replied to Ty. The good thing is he can add the final, fixed version to the archives. I uploaded the original release on Sep 23 but then uploaded the final version the other day.

Share this post

Link to post

Are you talking about idiotic wads or idiotic comments on the frontend? The latter is entirely Doomworld's domain. What do you propose for the former? Any sort of quality control implementation is just asking for drama, anyone should be able to see that.

Ty, echoing everyone else's positive sentiments. Most of us appreciate the work you do maintaining the archive. I don't even want to think of how splintered the Doom community would be without it, how many wads would be lost.

Share this post

Link to post

Matthewdoomer ran a successful test to whitelist my verification emails in Gmail, and as Gmail is the "worst offender" at thinking I'm an evil emailer, I wanted to share the steps with the rest of you in case it helps.

Log onto your account in Gmail

At the top you have a search bar. Click the (nearly invisible) down arrow by the blue magnifying glass.

Be sure the Search box says All Mail or Spam (click to select a folder)

Down in the lower right, click the link Create filter with this search >>

Check the box [x] Never send it to Spam
(can also apply a label, mark important, etc. if you want)

Optionally click the "Also apply filter..." checkbox if available to apply to existing messages

Click the blue Create filter button

Hope this helps a few folks.

BTW several recent uploaders have taken to emailing me right when they upload; that helps expedite the process considerably so thanks to those. And I'm not sure if I mentioned it anywhere but when I do get a verification and move files into the archive, I touch them to make their timestamp current to allow the most time in Newstuff even if it's been several days since the upload.

Share this post

Link to post

I actually had to follow those Gmail steps for my own Gmail account a year or two back because our forum's mailer over at Slick Productions sends things using my email address, and anything forum-related started ending up in my spam. I usually appreciate Gmail's spam filter, but it can definitely be overzealous when it comes to automated messages. Doubly so when those messages are coming from my account.

Anyway, set up the filter but I'll continue mailing you when I upload something. Thanks for the timestamp stuff (you touch our wads?!) too!

Share this post

Link to post

And if anyone chooses to pre-empt all this by emailing me ahead of the upload that they're going to do so, that simplifies it more.

It might be a bit late now but I'd put that in the email you send so more people know it.

There is one thing that's always confused me about the archive ever since I found it was allowed. As far as I'm aware maps that clear cut don't even have an exit are accepted (I'm not talking about boss death exits of course). I know quality control isn't a part of the archive, but I don't know what the point is of allowing something which doesn't have an exit.

Could someone enlighten me?

Edit: And I'd like to say thank you to Ty for all the countless time that's been spent and will continue to be spent on the archive. Dedication like yours is a rare thing indeed.

Share this post

Link to post

Not doing any quality control is not just a matter of principle. Checking to make sure levels have an exit would be a huge time commitment increase. Even with automated tools that could check for exit linedefs, there are all sorts of false-positive situations that could happen and so the rejects would need some kind of manual checking.

And to what end? The point of the archives is not to curate, but, well, to act as an archive.