Referring to Referers

Leoville went down earlier today for a few hours. It was out yesterday for about an hour, too. I contacted my excellent web host, Nacio, and they brought it back. Here’s what they said…

We have noticed that 2 IP addresses are consistently opening and not closing connections to your website: 12.xx.xx.xxx and 172.xx.xx.xxx. On average, each IP will have 60 open TCP sessions on a 24/7 basis. Are you familiar with these IPs? If not, we may look in to blocking them, as their irregular activity may be part of the problem.

Also, we have been monitoring the disk usage on your site. Currently you are using 1.5GB total–roughly 1.4GB being your discussion board. As you are on a shared webserver–geared toward smaller 40 – 80MB sites–we were hoping that you could remove some content from your site. Would 500MB be sufficient, or do you need more?

I asked them to block the two IP addresses and that’s seemed to help. I’m going to have to cut back on the disk usage, too. Obviously the boards have gotten way out of control. I’m pruning messages older than 90 days and I’ll probably cut the max file size to 50kb. Sorry to have to do that, but I really would like to keep this site running!

On another note, I’ve been playing with Dave Winer’s new Radio blogging software and I have to say it’s wonderful. I’m sticking with Movable Type, but for anyone who wants to create their own web site without having to struggle with the tech this is it. It finally fulfills the web’s promise to be the people’s publishing platform. You can read the temporary blog I set up to play with it at weblogs.com. It literally took me 10 minutes to get it up and running. And it’s free for 30 days – $40 for a year including the web hosting. If you’ve been thinking about blogging try Radio.

While playing with Radio, I noticed that the link to referrers is misspelled on the admin page. No blame to Dave Winer for this. The misspelling dates back to the original HTTP spec which also misspells referrers as “referers.”

This ancient error caused me endless confusion when I was writing my own Perl referers routine (it’s running on Leoville now. To see the most recent 20 referring pages click here). The program failed at first because I kept spelling referrers correctly. It took me a while to figure out where I was going wrong.

But the misspelling poses an interesting problem. Do you perpetuate it, as Dave has done, in public, or do you continue to spell it correctly while using the non-traditional spelling inside your programs? I chose the latter route on the front page of Leoville, but I might be in the minority.

In fact, this is exactly how a language evolves. I suppose, in time, “referers” will become the correct spelling, all thanks to a small spelling error at the W3C. Even though programmers are notoriously bad spellers, I can’t think of another instance where a misspelling has become enshrined in a spec. Can you?

Dood, I nevr speelchek my stuf be4 I send, yo.
So, by trimming the message boards, does that mean we’re going to lose the “Calling Carol” thread(s)? That would be most heinous.
Oh, I went through my entire day, wondering when I’d be able to use the word “heinous”. Thanks, Leo! 😉

I feel so far behind. What happened to Greymatter? Not that I have any problems with Movable. I’m just thinking about setting up my own and I’m bombarded with choices.
Well enough “noise” from U.P. North {Upper Peninsula of Michigan}.
weso
“I like my sugar with coffee and cream” – Beastie Boys – Intergalatic

Leo, whoa! 1.4 gig in the bases alone? Whoa!
Hey, have a “going out of business sale” on the images? Say you will kill all of them at a set time so people can go in and grab their favorites to their own harddrives.
Just a thought. 🙂

Greymatter and Movable Type are both excellent choices for folks who want to run a blog on their own server (and have access to CGI and Perl 5 or later).
If you don’t have a server Blogger and Radio are really great since they provide the space with the service. Both can post to a website using FTP, too. Radio is a little different since the software resides on your computer, Windows or Mac, and propagates the posts to the ‘net in the background.
I’m sticking with Movable Type because it seems to me to be the most robust and flexible solution. I have been very happy with it.http://www.noahgrey.com/greymatterhttp://www.movabletype.org/http://www.blogger.com/http://radio.userland.com/

Too late Antony. I had to delete the threads ASAP. They’ve been pruned mercilessly and I was able to get to around 450 MB. But all the old stuff is in the bit bucket alas.
Werd to the wise: save anything you like cuz the boards are going to be rolling off pretty quickly from now on.

Ah!!
So Leo, you will be purging a lot of images and pictures from the Images section, especially Calling Carol. So, bascially, we will not be able to look back far at all. Everyday gets about 45 messages.
Wow, I can’t believe 1.4 gigs!
Yes, like Antony said, I tottaly agree you should give us a due date when you will be purging all the pictures and files. So we can put it all on our hard drives. Leo, your too popular, thats the main problem.
Haha,
Take Care
Rachel

Leo, I noticed. I was going in there and to get some stuff. I’m glad I wasn’t wired or the other person would have heard me say, “D’oh! Too late!” hehe But that’s understandable, Leo.
Here’s a word of advice to the image people. I have found that Microsoft’s old Photo Editor I got back with one of the versions of Office 97 I had (had both Pro and Small Business so not sure what one gave it to me) does the job great. It makes most images under 15k as long as it’s within the 320 x 240 range. So even with Leo putting the new limit at 50k, I’d still be able to put 3 or 4 in one post if I wanted/needed to.
Last thing we need to remember is the size and make it as small as possible. It’s not hard to do that in most picture software but sometimes I think people want to be the first to post something they forget to put it in to shrink the size of the file.

No worries, Leo. Pruning can be a GOOD thing. 😉 Interesting about those two IP’s though…some crude DoS tactic perhaps? Or just something innocent but weird? Either way, it sounds like you’re webhost was really nice about it.
You’re making this blogging thing way too easy. Soon I won’t have any excuses left…

Carol, I just e-mailed Leo my theory but will share it here. It might be from sites that broadcast the cams from Leoville on their sites. Maybe Leo needs to put the warning out to not do that. People can just come to Leoville to see the cams anyhow.

I am one of the IPs. I email leo about it. I have a program that saves the webcam images. I was going to put up a site so people could look at the old ones, but I was having problems. I will post the whole email later but I sent it from my home computer. I hope he will let me back on. I feel really bad. I had no idea it would kill the site.

Leo, in my gardening experience, pruning way back tends to cause renewed vigorous growth with many new branches and many more blooms! 😉 I over pruned one flowering bush one year, and thought I had gotten carried away and killed it. But, the next Spring, it was fuller and bigger, and had more flowers than it had in the years before! Pruning can be a good thing in gardening and message boards. Just don’t eat too many prunes, or the results could be… well you know. 😉

Leo, You are amazing…. I have been visiting Leoville for under a year, and I love coming here (as you well know) and I hope Leoville remains robust and healthy (and I know it can withstand a little pruning,) but it is a great site and I hope it keeps on keeping on! Your fans love you, man!
Now, now if only you could turn all this interest and support from your fan base into some profit, then you could rule the world and have the biggest message board in the universe! Bwaaaahaaa haa!
PS: A book by Russel Hoban called “Riddley Walker” deals with the devolution of language, it is a sci-fi type story of the future where people try to rebuild civilization after a nuclear war, it is written in an almost phoenetic lingo. Not exactly light reading, but I enjoyed it a lot.

Philip,
I also have some webcam software which I mainly use during the shows or when Leo has the homecam/iBook running, but neither of those ^^ IP ranges are mine. So the question that comes to mind…
Is it a bug in the software that doesn’t handle the connections properly or disconnect when it should? Or is all webcam software inherently evil and we should stop using it completely?

I think we all are greatfull to Nacio to host leoville..I for one would be favorable to help or donate a big hard drive to keep some or all of leoville message board. but I admit I hardly (if ever) go through the archives.
With the price of hard drives these days it might be the way to go expence wise.
I’ll chip in if anyone else does.

anyone who reading leo’s blog thew email who there????
i am receiving thew my yahoo email account the entire massage in html format and there no way read it with all html tags in the way.
anywhooo. looking for an solution for the best blog service that easy to use.
try xanga.com it the most powerful and flexible service out there. it only took me 1 minute to have it up and running. best of all it free!
here my blog………..http://weblog.tv200.net
as for spellchecking
try….http://www.spellcheck.nethttp://tv200.net

1.5GB total–roughly 1.4GB being your discussion board. is this bandwidth per day, week, or mouth?
i using homestead.com to host my site. i get 3gigs of bandwidth a mouth, this part of the homestead.com personal serive. homestead offer a easy way upgrades for more bandwidth or webspace.
in went to Nacio.com website, i could not find anywhere in their site, of anyway to upgrade to hosting package or any price list of the product line.
leo i would swtich to another hosting service.
powweb offers 100megs and 15gigs of bandwidth for 7.77 a mouthhttp://www.powweb.com

JD, what do you mean like most message board systems? Mosxt of the popular MBS’s I know of host everything onsite, the only ones that I know of that don’t are designed for beginners who don’t get loads of traffic and don’t have loads of messages. On another note…is there a XML RSS feed from Leo’s blog? if not, why not? If so, what’s the address…

Here is a copy of the email I sent Leo:
I am soooooo sorry. I did not mean to kill your website. I have the program called WebCam Watcher (http://www.webcam-watcher.com) which is setup to look at your spy cam pictures (among others) and if there is a new one then it downloads the picture and saves it. I had NO idea it could damage the server. If I knew that I would not have done used the program. I feel really bad. Is there anything I can do to help you? I will stop running that program now that I know there is a problem unless you tell me it is ok. The reason I was running the program is that I wanted to setup a website that people could go to so they could look at the ones they missed. I have not done it because I am still looking for a script the will place all the pictures in a certain directory on to a page so I don’t have to add the pictures every time there is a new one and the page would update itself in real time. I then everyday or so could delete and ones that aren’t funny or has no one in the picture. I noticed that I could not get to your site when I got your blog update email and it said that you where getting consent traffic from 12.xxx.xxx.xxx and I felt so bad. Please let me visit your site. I would understand if you don’t want me to though.
Philip
In the subject of the email I put exactly what my ip is. I like the idea of hosting the images on another page, at least the WebCam pictures. What about a free Yahoo site or something? I would host them my self but my ISP does not allow hosting servers so anything I host can not get a lot of traffic. I am still looking for a script for what I talked about in my email. I have some leads. I REALLY hope Leo let’s me back on. Right now I am getting on the site from school.
Carol,
What program do you use? I sure hope that is not the reason Leoville went down. Maybe just one of us could save the pictures and do something like I am trying and host them for everyone? That should cut down in bandwidth.

Philip,
That’s the same prog I use when I’m on the PC. I use iCammaster for my Mac. So now I wonder if it’s having the same problem with my IP, but not on as great a scale…and if other sites I use it on have the same thing happening? Hmmmmm. I was collecting most of the images for other people too, so it won’t bother me much to kill it.
Leo, is there some way to view what other open sessions may still be lingering so they can be taken care of?

“JD, what do you mean like most message board systems? Mosxt of the popular MBS’s I know of host everything onsite, the only ones that I know of that don’t are designed for beginners who don’t get loads of traffic and don’t have loads of messages.”
UBB, YaBB, phpBB, WWWBoard are amongst some of the more popular sites that have the poster provide the image’s URL rather than upload it to the board’s server.
just trying to be…
JD

Sorry JD, I misunderstood you, I thought you were talking about scripts that automatically uploaded content to a third party server. I know what you’re talking about now though, and I have to say that I’m not a fan of the board Leo uses, I like phpBB myself…

Leo, that’s hilarious! One of the biggest problems I had early on in my programming career was getting nasty errors when referencing HTTP_REFERER. I’d make a call to HTTP_REFERRER and it would choke! Hours upon hours I struggled, even going back to some of my beginning books. It was spelled “referer” in the books, but my eyes couldn’t see it. I finally caught on and assumed I had been spelling it incorrectly for years! Then I read your blog… 🙂

Leo, in the library world we have to weed. This is a tough job. Removing books from the collection and dicarding them it like a knife in the heart of a librarian but it must be done. Books become outdated and well used books become ratty and fall apart. And, much like the board, libraries run out of space. We are limited to the space our building provides. Weeding, as painful as it is, is a good an positive thing. It makes the collection look more attractive, gives overlooked books a chance to be found, and frees up valuable shelf space. The first time I weeded our collection I had nightmares. Now it’s a natural process as I’m sure trimming the board will become. I’m sure that all the regulars love Leoville so much that whatever you have to do to kept it up and running is fine. I don’t know what I would do if Leoville disappeared. Besides memories, both the computer kind and the human kind, are always there for storing all the great past happenings and images.

Come on. Don’t you guys know about programmers in the “old days” when this stuff was created? It is obviously not referrer… it is refer-er, one who uses refer (marijuana).
By the way, Leo, there are lots of things in the Unix world that are standards but misspelled, like the command to create a new file. They made it CREAT just to confuse everyone I think.
Regards,
Jim

It’s time to move from Nacio.
AAAHosting.net offers a corporate package — their biggest one. For $30/month you get 1 GB of space and unlimited bandwidth, yes, UNLIMITED bandwidth for your use. This way, you could set the prune time back to 180 days. It supports Perl 5, PHP4, SSI, all the goodies. I host there on a smaller account and it is wonderful. You can make 50 Sub-accounts, letting you host small sites as long as you do not charge for them in any way (policy).
Consider moving, eh?