On Tue, Jul 7, 2009 at 4:15 PM, Rick Moen<rick at linuxmafia.com> wrote:
> Quoting Tony Godshall (tony at of.net):
>>> Oh, OK. So then parsing HTML is not needed. Very good.
>> Again, I'm not 100% certain I understand what you mean when you make
> that remark, so I'll review.
>> 1. The CABAL page is (still) using a flat HTML table for its calendar
> events. Its HTML source is the canonical upstream data. Absent some
> rewrite of the page, the only way you're going to generate iCalendar
> from it is to parse HTML.
>> 2. The BALE page _used_ to have the same sort of HTML table. In fact,
> I seem to recall creating the CABAL page's calendar data originally by
> copying and pasting HTML snippets out of the original incarnation of
> BALE.
>> Some years ago, to make BALE easier to maintain and (arguably) more
> consistently accurate, Deirdre coded an ingenious PHP/MySQL replacement
> for the primordial BALE setup, thus using index.php rather than
> index.html, and running a monthly Python cronjob to create "events"
> table entries for an additional upcoming month as specified in separate
> table "eventtemplate".
>> It is not necessary to "parse HTML" to convert the HTML output data
> created by BALE's index.php into iCalendar format. The less-roundabout
> alternative is to query it out of MySQL, much the way index.php does.
>> I could swear I stressed that BALE is PHP/MySQL-based right when you
> started talking about this matter, specifically to avert any such
> confusion -- though I could be misremembering.
Or perhaps I was overwhelmed by detail and confused between the two
calendars. Just because you were clear on something doesn't mean it
made it properly into my mind. I suppose I could go back and re-read
the messages in light of my new understandings of your methodology.
>> > BALE _used_ to be flat HTML. Then, we templatised and automated it.
>> > These days, I have to use my choice of MySQL tool, if I have to edit
>> > either an event template, or edit an event already generated from a
>> > template or entered by me manually.
>> >
>> > Having to chase down events already generated by template is
>> > particularly tedious. For example, Smaug in Santa Cruz, which meets
>> > every Monday night, has just decided to change locations, which means I
>> > need to chase down and edit three months of events rows for them, in
>> > addition to editing the template to fix future months not yet generated.
>> >
>>> Going vcalendar-based would help you there since repetitions can
>> have expiration dates. You just replicate the event, change one's
>> expiration, the other's start, and adjust the new one's details.
>> Again, I'm well aware of that feature of the vCalendar / iCalendar
> specs.
OK. I'm sorry I overcommunicated there. Sometimes I do that.
It helps me think. I don't mean to demean. You probably understand
them much better than I do. Since this entire conversation is CC'd
from the beginning to conspire@ it makes sense to err on the side
of clarity even if it may seem pedantic to you- perhaps others can
assist or learn. I'd probably assume more of a word to the wise
kind of attitude if it were private and we hadn't alread a bit of
miscomm.
> One of the advantages of Deirdre's design is that its treatment of
> recurring events is, in general, smarter than those of the RFC 2445 and
> related specs. There's a clever bit she has to indicate an event being
> on "5th or last" days, which was one of the bizarre variations that one
> of the groups threw at us (Bay Area NeXT Group, now deceased).
Oh, good to know. I don't know the details and assumed from your
frustration at tracking down generated repetitions that it wasnt.
Perhaps you and she should propose such a feature to the next
rev of the spec. Or write an RFC.
> She didn't implement beginning and end dates for templated events -- for
> the simple reason that it wasn't part of the static design that she was
> replacing -- but the great thing about the database design is that such
> things can be added easily, just by adding two more columns to the
> eventtemplate table, and then putting appropriate "if" statements in the
> Python script.
Oh very cool
> And I actually could, on reflection, avoid having to hunt down and
> _edit_ already-generated events, by instead editing "eventtemplate",
> then hunting down and _deleting_ the erroneous event rows, then
> re-running newmonth.py: My recollection is that newmonth.py is designed
> to fill in missing events for the months currently shown, but not
> overwrite them if they exist in the "events" table.
>> I hadn't really worked out that logic, until this moment. Also, I've
> been favouring editing generated events rather than deleting and
> regenerating them in order to not have the events missing while I'm in
> the middle of that work.
I often find when I explain something to someone else that it debugs
my thought process and the issue I was explaining becomes solved.
> [Holidays:]
>>> I think I saw a tool in Linux that has those. Shouldn't have to do them
>> manually.
>> I sort of like "doing" them, to a degree. It allows me to express what
> I think is worth expressing, in the way I consider worthwhile.
>> The selection of which holidays are worth including, and how to describe
> them, is a matter of my personal expression.
>> I'm a writer. Writing and publishing is what writers do. Because I'm a
> writer, and as a reflection of who I am, BALE includes a "Summer Solstice
> / Midsummer Day / Midsommardagen" entry in June, because it pleases me
> to include the Norwegian / Old Norse spelling.
Well, now I'm going to actually have to pay attention to the events
I'd skimmed past in the past.
> It includes "Luciadagen (St. Lucia's Day), Sweden" because I have a
> Swedish mother-in-law. It includes "Mohandas K. Gandhi's birthday
> (1869)" and "Juneteenth" because... well, just because I want them there.
>> It includes "Spring Equinox / Ostara / Eostar / Eostre" because I wanted
> to remind people of the Saxon, Old High German, and Old English
> antecedents of Easter (without hitting them over the head with it).
I understand now.
> And BALE omits a whole lot of other allegedly noteworthy occasions
> because it reflects my taste rather than that of, say, Hallmark (or "a
> tool in Linux that has those").
Yes.
> In short, it is not a bug that I am an originator of information that is
> written in my personal style and reflecting my personal tastes and
> preferences. To the contrary: It's a design goal -- a consideration
> that applies to the selection, wording, and presentation of all the
> entries: recurring events, holidays, one-time technical events,
> everything. The whole megillah. It's what I care to list, omitting
> what I wish to omit, written and presented the way I like.
>> People sometimes act astonished and aggrieved when they hear that BALE
> omits something they care about, or includes something they don't.
> They tell me it's a "community resource", which sort of misses the
> point: It's _my_ resource, my work (with Deirdre's code), written by
> me, on my server, running in my house on bandwidth and electricy I write
> chequees for. I'm happy if they use and like it, but ultimately it's an
> expression of me, by me, for my purposes. Including the holidays.
Absolutely.
Maybe if it had a clever name...
>> Nah, the way I'd do it is switch the "master" to something easier than
>> HTML and switch the HTML to generated. But it turns out you already
>> have a better master than the HTML, so it makes sense to generate
>> both the HTML and the iCalendar from the SQL, I guess.
>> Yes, that's what I was thinking. However, the way your Python script
> parses HTML would be neat for, say, the CABAL calendar -- if we can fix
> the year-end and all-day-event problems.
So you don't want to switch the CABAL calendar to something else?
Are there events in CABAL that are not in BALE? I guess I was assuming
it was a subset and you would get the CABAL events by updating the
BALE list and filtering them out. I even wrote you that assumption and
you didnt contradict it.
>> Yes, I understand now. You are testing using iCalendar to give people a
>> dynamically- generated html of the icalendar stuff.
>> Well, sort of. (Where you say "using iCalendar", above, I gather that
> you meant to say "using PHP iCalendar".)
Yes, of course. Sorry about that. To the peanut gallers: There are
actually three different iCalendars under discussion: iCalendar, the
standard. PHP iCalendar, the specific PHP program. And the
Python library (in a fork of this thread)
> PHP iCalendar (0.9.4) was already there -- has been there for many
> years. All I did was plunk the bale.ics file generated by your script
> into its data directory in order to see how PHP iCalendar presented it.
> At first, the answer was "not at all", because your script omitted the
> mandatory vCalendar header and footer -- and then "rather badly" because
> your script wrote DESCRIPTION where SUMMARY is apparently what was
> needed. And then, it turned out that the data needed some rearrangement
> -- and all of those things were appropriate only after I loaded the file
> in PHP iCalendar. (Please note that I'm not complaining in any way!
> I'm just recapping.)
>> Basically, it was bog-standard QA: Given that the script allegedly
> generated an iCal file, the necessary next step was to verify that at
> least one iCal-parsing tool could read it and do useful things with it.
> As I always say in a sysadmin context, "The task's not done until the
> results are tested and verified." And PHP iCalendar was the tool I had
> handy for that purpose.
>>>> I'd do it statically with make file.
>> Well, of course. I plunked a copy into PHP iCalendar because it was
> there. Didn't I _say_ that PHP iCalendar 0.9.4's been sitting around
> since forever? [/me checks.] Yes, I did.
Oh, and your complaining about how long it took to come up and that
you'd stick with the old version was supposed to tell me you *weren't*
considering it for production use?
> You know, we'll shorten the cycle time on these exchanges if you don't
> make odd assumptions.
Heh. Likewise.
>> Like you found, iCalendar processing is rather CPU-intensive.
>> PHP iCalendar 0.9.4 is perfectly tolerable, FWIW. 2.3.1 appears to be
> ghastly.
Well, yes, perhaps it has to parse the more the complicated
options of a higher rev of the standard. Or maybe it's prettier.
But what I don't see is why you care so much if it takes a
second or five if it's only done statically after an edit.
Premature optimization IMHO.
Tony