You have no idea—or maybe you do—how much email I’ve been getting about how RSS can and should help with the problem of email.

The growing consensus is that email is broken. It’s been broken by spammers and viruses. (And stupid-but-scriptable clients.)

I don’t know if RSS is the answer or part of an answer or not.

When it comes to newsletters, I think it’s obvious that RSS is a great alternative. (Just ask Chris Pirillo.)

But what about mailing lists? Mailing lists are many-to-many. I love mailing lists; I subscribe to quite a few.

One possible solution is to have a weblog that people sign up to be able to post to. You can post but you can’t edit other people’s posts. You could comment on other people’s posts, though. And it would all be available via RSS.

This could be pretty nice, actually, assuming some good software for reading and writing. The important thing is that it be made as easy as joining a mailing list and reading and sending email.

So that leaves one-to-one and one-to-few email.

I’ve learned that some people are creating private RSS feeds to solve this problem. Here’s how it works: say you and I correspond often. I’d create an RSS feed just for you. It would be password-protected and encrypted, of course. Instead of sending you email, I’d add a post to the feed, and it would show up in your newsreader.

And vice versa—you’d create a feed for me.

I find this idea rather difficult, though. If you correspond with lots of people, it means creating lots of feeds.

This could be turned around, I suppose. Each person could have a feed that anyone else could add to. Then, instead of knowing someone’s email address, you’d have the address of their feed. (Or, more precisely, the URL of some posting interface. You couldn’t read their feed, you could just add to it.)

That might be more do-able.

But then, if it took off, wouldn’t a stupid-but-scriptable client come along to break the whole thing? I’m not sure how RSS itself can prevent that. A stupid-but-scriptable client would still attract viruses and security problems.

Anyway, I’m not proposing anything, just thinking about it, since I know that lots of you are thinking about this too.

We’re starting to think about what new features will appear in NetNewsWire 1.1. There are lots of good ideas—some of which appear on the feature ideas page.

One of the things we’re thinking about is embedded browsing. The idea is pretty simple: click on a link in NetNewsWire, and the page opens right there rather than in your web browser.

It would be optional, of course, and probably turned off by default.

There has been a really excellent debate about this on the testing list. People mostly seem to be against it. Which is fine by me—I’d rather work on things like synching and persistence and so on than on this particular feature.

However, some people are in favor of this feature. And it may be that this is just one of the standard features of RSS newsreaders.

Screen shots

Check out the Windows newsreader named Beaver—click on the Main Window screen shot (in the left-side column). (It gives me a 403 if I link to it directly.)

And here’s a screen shot of a quick demo I did for NetNewsWire. Note that there would have to be some more UI (a back button and so on), but otherwise the screen shots are very similar.

Should we do this feature?

Beaver isn’t the only newsreader with this feature. A case can be made that it’s a checklist feature, that we pretty much have to do it.

But, aside from that, and more importantly—do you want this feature? If so, do you want it before synching, persistence, flagged news items, Atom support, more scriptability, Rendezvous, etc. etc.?

To help out the discussion a little bit, I did a special build that has embedded browsing. It doesn’t have the full UI needed, but if you click on a link it opens in place. (I used this build to make the screen shot above.) This is not meant for everyday use, but if you’d like to try it, you can download NetNewsWire 1.1d1. This is for thrill-seekers only, it’s meant as a quick demo only.

PS My testing was with Apache running on the same machine as NetNewsWire. This new build worked fine—but then, it worked fine before, too. So I think the bug is when you have a proxy and your access to the outside is restricted (you can’t do DNS lookups, for instance).

As O’Reilly Network managing editor, Derrick works on MacDevCenter, which has consistently published great articles for Mac developers. The Mac OS X Innovators contest rewards creativity in OS X development.

Watson was one of the first OS X apps to show you can have a product that works with the web but isn’t a web browser—a powerful demonstration that richer interfaces are possible, and that people would like them.

For the most part, we try to accommodate what our users want and have a hard time saying no to features. We support a number of protocols and formats because we feel it is important to err on the side of mass support. From day one of Movable Type to day one of TypePad, we have provided not just an import mechanism but also an export mechanism. We never wanted to hold content hostage in order to guarantee tool lock-in.

I recognized this immediately as the same philosophy Sheila and I have at Ranchero. And I think maybe it’s a necessary condition for the success of any software company.

Anyway, speaking of Six Apart... TypePad was added as a publishing tool choice in NetNewsWire 1.0.4b3. I was pleased by how easy this was to do: I just added it to the popup menu and told NetNewsWire that it works exactly like Movable Type. Piece of cake.

One of the new features in NetNewsWire seems like a little thing, but I’m rather proud of it. Here’s the story...

NetNewsWire is by design a very keyboard-friendly app. There are lots of keyboard shortcuts that aren’t reflected in the menus because they’re not command-key shortcuts. They’re things like using the space bar to go to the next unread message and typing + to expand an item in the Combined View.

The question I had was, how can I make it easier for people to find out about these shortcuts?

I looked around at some other apps and noticed that it was pretty common to have a Keyboard Shortcuts item in the Help menu. So I decided to do that.

But then the question was, how should the keyboard shortcuts be displayed in the window?

I liked OmniOutliner’s approach—it opens an outline. Makes sense, right? The app is an outliner, so it uses an outline to display the list of shortcuts.

An outline wouldn’t be right for NetNewsWire, though. I could do an outline (using the Notepad code), but it didn’t feel right for this. Other options (plain text, a table view, etc.) sounded boring.

Then I thought of Web Kit, and I had an a-ha moment: I could use a very basic window that just displays an HTML page. All it took then was to create the window and write the HTML page itself. Piece of cake.

I’m going to be writing about new NetNewsWire features for the next few days... Here’s another one: HTML differences.

It’s off by default. You turn it on via the General prefs panel, where it asks if you want to highlight differences for updated items.

Deleted text is struck through and red; new text is green and underlined.

This is another one of those features that I didn’t think I’d like, but NetNewsWire users asked me for it. Aaron Swartz, in particular, wanted this feature—and he said we could use his script in NetNewsWire. (So if you look inside NetNewsWire’s bundle, in the Resources directory you’ll find htmlDiff.py, Aaron’s Python script for showing HTML differences.)

So, if you like this feature, be sure to thank Aaron.

Now, as I said, I didn’t think that I personally would like this feature—but it turns out that I totally like it.

That leads to a little advice for app developers—your users are smart. The list of features I didn’t think I’d like, but that I did anyway because people asked for, is pretty much the list of cool features in NetNewsWire. (Groups, for one thing. The Combined View is another big one. And so on.)

The new version of NetNewsWire supports gzip compression—which sounds like a really boring thing.

And it is. Unless you’re looking for ways to save bandwidth.

It works like this: when downloading a feed, NetNewsWire tells the server that it supports compression. If the server also supports compression, it returns a compressed version of the feed. This means less actual bytes are transferred, which saves bandwidth.

This is such a good idea that Ted Leung has been keeping track of which aggregators support compression. (I’ll have to email him about NetNewsWire.)

But here’s the thing—it looks like hardly any servers support compression. Here’s a screen shot of my bandwidth stats window. Note the new gzips column, and note how there are only four (out of 99) feeds that have returned compressed data. (Even my own feeds don’t support compression yet.)

So—I wish I had a cool pitch of some kind, something to say besides, “Hey, server admin, please support compression.”

But I don’t, so I just say hey, server admin, please support compression.

The change notes go into detail about what’s new and different. Some highlights: it uses Web Kit now to display HTML, it’s faster, you can highlight differences in updated items, gzip compression is supported, and there’s a new Delete Read Items command.

Some people have said that the issue with not well-formed feeds and aggregators is a kind of prisoner’s dilemma.

In other words, if all aggregator developers pledged to reject not well-formed feeds, then that would be incentive for people to fix their feeds.

The reasoning goes like this: aggregator developers are handling not well-formed feeds as part of competing with other aggregators. If they all pledged not to do this, then they’d all be equal on this score.

But this isn’t correct. I don’t have NetNewsWire (try to) handle not well-formed feeds for reasons of competition. I do it because people already using NetNewsWire want to read these feeds. And I do it because future users will want to read these feeds too.

It has everything to do with wanting to produce quality software. I suspect the same is true for other aggregator developers.

By the way, I do often email people when they have bugs in their feeds. And I’m sure other aggregator developers do too. But we still need to do the best we can with feeds with bugs.

It’s not our job to be syndication cops.

Our software should support standards, but it should also try to work with feeds with bugs, because that’s in the best interests of our users.

Me, I would absolutely love it if I, as an aggregator developer, could require well-formedness.

In other words, if a feed isn’t well-formed, then NetNewsWire would not parse it and display it.

The thing is, that doesn’t work now for RSS—but not because of anything special to RSS, it’s because feed generators don’t always produce well-formed XML. There’s no reason to expect PEAW feed generators would be any different. (Both RSS and PEAW require well-formedness. No difference there.)

The single most common cause of non-well-formedness that I see is unencoded ampersands. They appear in a feed as & rather than as &amp;. This is most often in <title>s.

In my experience this most often afflicts larger publications, not weblogs using Movable Type or Radio or whatever. My guess is it’s because these larger publications have their own in-house systems. Those systems don’t get tested the way weblog systems get tested. A weblog system will have many thousands of users, but an in-house system has just one user (the publication). (I mean user in the sense of publisher.)

Another thing about these larger publications is that their feeds are often very popular. So when one is non-well-formed, I get a ton of bug reports about it until they fix it.

Actually, that used to happen, but NetNewsWire has gotten progressively better about dealing with the ampersand problem, so I don’t get so many bug reports.

According to Tim aggregators should consider it a fatal error and not process these feeds. If I agreed, NetNewsWire users would pay the price.

Tim writes: “Granted that the RSS legacy necessarily required the use of liberal parsers, but hey, that was then, we have better tools now.”

Setting aside the note about “the RSS legacy,” which isn’t relevant to the issue of well-formedness, I want to note that though we indeed “have better tools now” they aren’t evenly distributed.

And, ironically and interestingly, John Q. NewToBlogging has better feed-generating tools than many of the large publications.

The thing is, the mail reader aggregators are not very much like mail readers. They are smart about what they’re displaying. On the surface they look like email apps, but it’s not a simple substitution, news items for email.

I’ll use NetNewsWire as an example, not because it’s unique in this regard but because I know it well.

Many aggregators have a concept of groups. (They don’t all work exactly the same, of course.) In NetNewsWire, if you select a group, you see the unread news of everything contained in the group. Here’s a screen shot showing my Macintosh group this morning.

So—is this weblog-style or mail-reader-style?

NetNewsWire, like some other aggregators, also lets you just see all the unread headlines regardless of group. Here’s a screen shot. Here’s another screen shot which shows a more email-like view, but still has items from multiple places in the same view.

NetNewsWire is by no means alone with features like this.

A good question might be—well, why not just use a browser-based interface like Radio UserLand?

But a GUI app gives you a richer, targeted interface. A browser is generic: it knows it’s displaying HTML, but it doesn’t know it’s displaying syndicated news items. A GUI newsreader knows it’s displaying syndicated news items, and so the menu commands and toolbar buttons and shortcuts and everything are all about what it’s displaying. This allows you to keep up with more news with less effort.

It comes down to personal preference, though. Browser-based interfaces have their advantages too.

The thing to remember is that the distinction between aggregator types isn’t between mail-reader style and weblog-style, it’s between GUI apps and browser-based apps.

Dave writes: “People who are just using mail-reader style aggregators are really missing something.”

I’ve seen a few people complain about TypePad pricing. It ranges from $4.95 to $14.95 per month.

Listen: those prices are low.

One thing maybe not everyone realizes is that things on the Web have costs. Software costs money to develop and maintain. Servers have to be bought and administered; bandwidth has to be paid for; backups have to be made. Someone has to do the bookkeeping. Someone has to do customer support.

Running a service like TypePad is a long way away from doing some little fun thing for free. That it costs, even at the high end, under 15 bucks a month is utterly fantastic. To the Six Apart folks I say: good job.