Author: Andrew StaceyFormat: MarkdownI've hacked the feeds for this forum a little so that there are now "all comments" feeds.
It's not perfect yet: I get the right thing in Live Bookmarks, but in my feed reader (RNews) then I run into problems because the comments' titles are those of the discussion, so lots of comments have the same title. My feed reader collapses these to the _first_ comment (I wouldn't mind so much if it collapsed them to the _last_ comment!).
There are two possible solutions to this. Either make the feed entry titles unique (perhaps adding the comment number to it) or offering a reversed feed (_in addition_ to the current one) so that when the feed reader collapses entries with the same title then the most recent comment is the one that survives.
Neither is completely simple so I thought I'd canvas opinion first. Also, given the plethora of feed readers out there then the fact that it seems to work in Live Bookmarks and almost works in RNews doesn't mean that it will work in everything so please try it and see if it works for you.

I've hacked the feeds for this forum a little so that there are now "all comments" feeds.

It's not perfect yet: I get the right thing in Live Bookmarks, but in my feed reader (RNews) then I run into problems because the comments' titles are those of the discussion, so lots of comments have the same title. My feed reader collapses these to the first comment (I wouldn't mind so much if it collapsed them to the last comment!).

There are two possible solutions to this. Either make the feed entry titles unique (perhaps adding the comment number to it) or offering a reversed feed (in addition to the current one) so that when the feed reader collapses entries with the same title then the most recent comment is the one that survives.

Neither is completely simple so I thought I'd canvas opinion first. Also, given the plethora of feed readers out there then the fact that it seems to work in Live Bookmarks and almost works in RNews doesn't mean that it will work in everything so please try it and see if it works for you.

Author: Andrew StaceyFormat: MarkdownOkay, so making the names unique isn't all that hard. But there's some choice as to how to do it. The most obvious, comment number within discussion, is tricky since that doesn't get stored with the comment (so one would have to shlep through the lot counting how many were in that discussion before the current one). A considerable amount of data is stored with a comment. The ones that seem reasonable to use are as follows (I'm including the variable name for reference):
1. Global number of comment [CommentID]
2. Discussion this comment is in - either by name [Discussion] or number [DiscussionID]
3. Category containing the discussion - name [Category] or number [CategoryID]
4. Various details about the author: user id [AuthUserID], name [AuthUsername],
5. The same for the editor: user id [EditUserID], name [EditUsername]
6. Date comment was created [DateCreated]
7. Date comment was edited [DateEdited]
(People with high enough status can edit other people's posts. Currently only the administrator can do that.)
Obviously not all of them result in something unique; for pure uniqueness then CommentID is the obvious choice. But given that it is actually easy to modify the title, I thought I may as well see if there is any _useful_ information that could be added. For example, something like the 'Cafe:
_Andrew Stacey comments on RSS/Atom Feeds (238)_

Okay, so making the names unique isn't all that hard. But there's some choice as to how to do it. The most obvious, comment number within discussion, is tricky since that doesn't get stored with the comment (so one would have to shlep through the lot counting how many were in that discussion before the current one). A considerable amount of data is stored with a comment. The ones that seem reasonable to use are as follows (I'm including the variable name for reference):

Global number of comment [CommentID]

Discussion this comment is in - either by name [Discussion] or number [DiscussionID]

Category containing the discussion - name [Category] or number [CategoryID]

Various details about the author: user id [AuthUserID], name [AuthUsername],

The same for the editor: user id [EditUserID], name [EditUsername]

Date comment was created [DateCreated]

Date comment was edited [DateEdited]

(People with high enough status can edit other people's posts. Currently only the administrator can do that.)

Obviously not all of them result in something unique; for pure uniqueness then CommentID is the obvious choice. But given that it is actually easy to modify the title, I thought I may as well see if there is any useful information that could be added. For example, something like the 'Cafe:

Author: Mike ShulmanFormat: MarkdownThis is wonderful, thank you!! I see you added the global number to make them unique. It displays fine in Google Reader now, sorted correctly, and also says "by Andrew Stacey" (or whoever posted the comment) underneath the title. I don't think I would ask for any other information than the discussion the comment was posted in and the person who posted it.

This is wonderful, thank you!! I see you added the global number to make them unique. It displays fine in Google Reader now, sorted correctly, and also says "by Andrew Stacey" (or whoever posted the comment) underneath the title. I don't think I would ask for any other information than the discussion the comment was posted in and the person who posted it.

Author: Andrew StaceyFormat: MarkdownAnnoyingly, although I get the right order and so on, I just get '239 redirects' so I think I will put a little more information in the title. If it's extremely annoying for everyone else then maybe I'll think again.
What do you see for the 'cafe comments, by the way? Is that annoying?

Annoyingly, although I get the right order and so on, I just get '239 redirects' so I think I will put a little more information in the title. If it's extremely annoying for everyone else then maybe I'll think again.

Author: Andrew StaceyFormat: MarkdownOkay, if it's not annoying then I'll change it to put the author in the title. I wonder if there's some way to remove the necessity for making the title unique. Presumably some other field can be used.
However, it's now usable and seems not to be annoying anyone (including me) so I'll get on with more important things.

Okay, if it's not annoying then I'll change it to put the author in the title. I wonder if there's some way to remove the necessity for making the title unique. Presumably some other field can be used.

However, it's now usable and seems not to be annoying anyone (including me) so I'll get on with more important things.

Author: EricFormat: TextThe feeds are great. Thanks!
Is there any way to control the frequency of updates? Currently, I get all comments in one shot periodically throughout the day, e.g. I just now got 12 comments through the RSS feed. Is it possible to send the feed closer to real time? Just curious. With people on multiple continents, being notified when someone is online might help speed communication.

The feeds are great. Thanks!

Is there any way to control the frequency of updates? Currently, I get all comments in one shot periodically throughout the day, e.g. I just now got 12 comments through the RSS feed. Is it possible to send the feed closer to real time? Just curious. With people on multiple continents, being notified when someone is online might help speed communication.

Author: Andrew StaceyFormat: MarkdownI don't really know how these things work, but there's two factors involved: how often your feed reader actually asks for the feed and how often the site generates a new feed. I think that the site generates a new feed whenever it is asked for one. Whenever my feed reader updates then I get all the new comments. Plus, I haven't seen any caching going on here. That might mean that your feed reader is only asking for updates at selected intervals (this is **good** behaviour, by the way). You may be able to override that, or at least to manually ask for an update. Depends on the feed reader that you are using. I don't know much about different readers, though.

I don't really know how these things work, but there's two factors involved: how often your feed reader actually asks for the feed and how often the site generates a new feed. I think that the site generates a new feed whenever it is asked for one. Whenever my feed reader updates then I get all the new comments. Plus, I haven't seen any caching going on here. That might mean that your feed reader is only asking for updates at selected intervals (this is good behaviour, by the way). You may be able to override that, or at least to manually ask for an update. Depends on the feed reader that you are using. I don't know much about different readers, though.

Author: EricFormat: TextHi Andrew,
I should have mentioned I use Google reader. There is definitely a difference between the nCafe RSS feed and the nForum RSS feed. With the nCafe, I receive notices almost in real time (or a small delay) for each individual comment whereas with the nForum, hours can go by before being notified and when I am notified of new content on the nForum, it usually comes in bursts, e.g. 6-10 comments feeding through at the same time.

Hi Andrew,

I should have mentioned I use Google reader. There is definitely a difference between the nCafe RSS feed and the nForum RSS feed. With the nCafe, I receive notices almost in real time (or a small delay) for each individual comment whereas with the nForum, hours can go by before being notified and when I am notified of new content on the nForum, it usually comes in bursts, e.g. 6-10 comments feeding through at the same time.

Author: Andrew StaceyFormat: MarkdownI don't know anything about google reader (and I refuse to find out: I read through the Google ToS and decided that I would definitely not accept it!). Presumably there's some setting for how often it polls for new articles. Is that a global setting, or do you set it for individual feeds?
I don't know how MT (aka the cafe) generates its feeds. They may be static and regenerated every time someone posts something, or they may be dynamic and regenerated for every request. Here they are dynamic. I don't know what effect that has, though. But whenever the request is _made_ then you should get a list of all comments up to that point in time so it feels as though your reader isn't requesting the feeds as often as you like.
I'm not an expert, but I think that the way that RSS works is that the reader has to take the initiative to go and get the information from the feed. Information about updates aren't sent by the server to the client automatically (that would be a ridiculous workload), rather the reader has to go to the server and ask "Are there any updates?". Some readers ensure that no matter how often you click the "refresh feed" button, it only goes and actually asks every, say, half an hour. That's regarded as "playing nicely".

I don't know anything about google reader (and I refuse to find out: I read through the Google ToS and decided that I would definitely not accept it!). Presumably there's some setting for how often it polls for new articles. Is that a global setting, or do you set it for individual feeds?

I don't know how MT (aka the cafe) generates its feeds. They may be static and regenerated every time someone posts something, or they may be dynamic and regenerated for every request. Here they are dynamic. I don't know what effect that has, though. But whenever the request is made then you should get a list of all comments up to that point in time so it feels as though your reader isn't requesting the feeds as often as you like.

I'm not an expert, but I think that the way that RSS works is that the reader has to take the initiative to go and get the information from the feed. Information about updates aren't sent by the server to the client automatically (that would be a ridiculous workload), rather the reader has to go to the server and ask "Are there any updates?". Some readers ensure that no matter how often you click the "refresh feed" button, it only goes and actually asks every, say, half an hour. That's regarded as "playing nicely".

Author: Andrew StaceyFormat: MarkdownFor example, I just tried bookmarking the feed in Firefox using 'Live Bookmarks'. The above comment hasn't appeared there yet but when I go directly to the feed then I do see that comment. This suggests that Firefox hasn't asked for an update yet of the feed. There's a "reload live bookmark" option and when I click on that then I do get the above comment. I don't know how long it would take to do that automatically, though.

For example, I just tried bookmarking the feed in Firefox using 'Live Bookmarks'. The above comment hasn't appeared there yet but when I go directly to the feed then I do see that comment. This suggests that Firefox hasn't asked for an update yet of the feed. There's a "reload live bookmark" option and when I click on that then I do get the above comment. I don't know how long it would take to do that automatically, though.

Author: Andrew StaceyFormat: MarkdownAaaagggghhhh.
I've been trying to implement a feature for the feeds that Mike suggested, namely the ability to have an "all but" feed. The idea being, I guess, that one can split the forum into "latest changes" and "everything but latest changes" and not have to keep updating feeds whenever someone reorganises the categories.
And it sort of works, except that it doesn't. Rather, it works when I'm in firefox and using Live Bookmarks, but it doesn't work when my feed reader does it.
It's not quite at the "point and click" stage yet, but here's how to do it if anyone feels like testing it and seeing if it works for them (TM):
1. Go to the "search" area.
2. Click on "Advanced".
3. In either the "Topics" or "Comments" section, select some categories (ctrl+click to select several).
4. Click "search".
5. This should take you back to the "simple search" form except that your search has been converted to a "simple search" format where the categories that you'd selected are now listed with the keyword "cats:"
6. Change the "cats:" to "ncats:" and click "search" again. That should negate the category selection.
7. On the left-hand side are two links labelled 'RSS' and 'ATOM'. When I click on one of those, I get a feed in firefox that looks right (note that the order of items in the feed and in the search are different). But if I cut-and-paste that URL into my feed reader, I get something completely different.
So if anyone feels like testing this to see what they get, I'd be very grateful. Also, it's possible that I've mucked up the ordinary feeds so if something weird's happening with your ordinary feed then let me know.
If testing, please test both "Topics" and "Comments" searches. And look in particular for the following:
1. Do the negated searches work (i.e. nothing to do with feeds)?
2. Does the RSS/ATOM feed look right _in your browser_? Note that you must click the 'search' button after doing the 'cat' -> 'ncat' transformation otherwise the links don't update.
3. Does the RSS/ATOM feed look right _in your feed reader_?
Thanks.

Aaaagggghhhh.

I've been trying to implement a feature for the feeds that Mike suggested, namely the ability to have an "all but" feed. The idea being, I guess, that one can split the forum into "latest changes" and "everything but latest changes" and not have to keep updating feeds whenever someone reorganises the categories.

And it sort of works, except that it doesn't. Rather, it works when I'm in firefox and using Live Bookmarks, but it doesn't work when my feed reader does it.

It's not quite at the "point and click" stage yet, but here's how to do it if anyone feels like testing it and seeing if it works for them (TM):

Go to the "search" area.

Click on "Advanced".

In either the "Topics" or "Comments" section, select some categories (ctrl+click to select several).

Click "search".

This should take you back to the "simple search" form except that your search has been converted to a "simple search" format where the categories that you'd selected are now listed with the keyword "cats:"

Change the "cats:" to "ncats:" and click "search" again. That should negate the category selection.

On the left-hand side are two links labelled 'RSS' and 'ATOM'. When I click on one of those, I get a feed in firefox that looks right (note that the order of items in the feed and in the search are different). But if I cut-and-paste that URL into my feed reader, I get something completely different.

So if anyone feels like testing this to see what they get, I'd be very grateful. Also, it's possible that I've mucked up the ordinary feeds so if something weird's happening with your ordinary feed then let me know.

If testing, please test both "Topics" and "Comments" searches. And look in particular for the following:

Do the negated searches work (i.e. nothing to do with feeds)?

Does the RSS/ATOM feed look right in your browser? Note that you must click the 'search' button after doing the 'cat' -> 'ncat' transformation otherwise the links don't update.

Author: Mike ShulmanFormat: MarkdownAs far as I can tell,
1. The negated searches work.
1. The feed looks right in my browser.
1. The feeds for Comments seem okay in my feed reader (Thunderbird), although for some reason it's only downloading the first entry in each so far. I haven't been able to get it to recognize the feeds for Topics yet.

As far as I can tell,

The negated searches work.

The feed looks right in my browser.

The feeds for Comments seem okay in my feed reader (Thunderbird), although for some reason it's only downloading the first entry in each so far. I haven't been able to get it to recognize the feeds for Topics yet.

Author: Mike ShulmanFormat: MarkdownHmm, the comment that I just wrote is not showing up in the list my feed reader, though it does show up in the feed as viewed in the browser.
Edit: oh, wait, now it's there.

Hmm, the comment that I just wrote is not showing up in the list my feed reader, though it does show up in the feed as viewed in the browser.

Author: Andrew StaceyFormat: MarkdownRight, I think I've figured out the deep down reason that this wasn't working. The higher level reason is to do with the number of times a URL gets encoded and decoded and the fact that these operations are not idempotent. It seems that in a feed reader everything gets encoded one more time than when it goes direct from a browser. The reason that this was important is because of a little bug in the search engine. The "simple search" way to search for a category is to use the keyword syntax: "cats:Latest Changes;". This _ought_ to work if the semi-colon is missing but what actually happened was that the word "atest Changes" got left behind, so that the search engine thought that you were looking for posts in the Latest Changes category that matched "atest Changes". I've fixed that bug.
However, there still seems to be some art in crafting the URLs correctly and I doubt that the automatically generated URLs on the search page are correct. So long as these don't get mangled by the Markdown interpreter, the following work in my feed reader:
* Latest Changes Discussion list: http://www.math.ntnu.no/~stacey/Vanilla/nForum/search.php?PostBackAction=Search&Type=Topics&Page=1&Feed=ATOM&Keywords=cats:Latest%20Changes;
* Latest Changes Comment list: http://www.math.ntnu.no/~stacey/Vanilla/nForum/search.php?PostBackAction=Search&Type=Comments&Page=1&Feed=ATOM&Keywords=cats:Latest%20Changes;
* Everything _but_ Latest Changes Discussion list: http://www.math.ntnu.no/~stacey/Vanilla/nForum/search.php?PostBackAction=Search&Type=Topics&Page=1&Feed=ATOM&Keywords=ncats:Latest%20Changes;
* Everything _but_ Latest Changes Comment list: http://www.math.ntnu.no/~stacey/Vanilla/nForum/search.php?PostBackAction=Search&Type=Comments&Page=1&Feed=ATOM&Keywords=ncats:Latest%20Changes;
In experimenting, I found that I often ended up with stale feeds; i.e. I'd made a change to the URL and the new feed was getting mangled up with a cached version of the old feed.
If you want to add new categories to the exclude (or include) list, the format is that they should be comma-separated.
Interested parties please test. If it works, I'll try to figure out how to make sure that the URLs are correct on the search form page.

Right, I think I've figured out the deep down reason that this wasn't working. The higher level reason is to do with the number of times a URL gets encoded and decoded and the fact that these operations are not idempotent. It seems that in a feed reader everything gets encoded one more time than when it goes direct from a browser. The reason that this was important is because of a little bug in the search engine. The "simple search" way to search for a category is to use the keyword syntax: "cats:Latest Changes;". This ought to work if the semi-colon is missing but what actually happened was that the word "atest Changes" got left behind, so that the search engine thought that you were looking for posts in the Latest Changes category that matched "atest Changes". I've fixed that bug.

However, there still seems to be some art in crafting the URLs correctly and I doubt that the automatically generated URLs on the search page are correct. So long as these don't get mangled by the Markdown interpreter, the following work in my feed reader:

Author: Mike ShulmanFormat: MarkdownThis has been working fine (thanks!!), but today Toby and Andrew's comments on [this discussion](http://www.math.ntnu.no/~stacey/Vanilla/nForum/comments.php?DiscussionID=379&page=1) didn't show up in the "Everything but Latest Changes Comment list" in my feed reader. Strangely, *my* reply did show up. I do see them all in the feed in my browser, so maybe it was an error in Thunderbird.

This has been working fine (thanks!!), but today Toby and Andrew's comments on this discussion didn't show up in the "Everything but Latest Changes Comment list" in my feed reader. Strangely, my reply did show up. I do see them all in the feed in my browser, so maybe it was an error in Thunderbird.

1 to 17 of 17

Add your comments

Please log in or leave your comment as a "guest post". If commenting as a "guest", please include your name in the message as a courtesy. Note: only certain categories allow guest posts.UsernamePassword

To produce a hyperlink to an nLab entry, simply put double square brackets around its name, e.g. [[category]]. To use (La)TeX mathematics in your post, make sure Markdown+Itex is selected below and put your mathematics between dollar signs as usual. Only a subset of the usual TeX math commands are accepted: see here for a list.