State needs to be saved so that we can see a thread of actions taken. Many
people might make assertions about some bit of content. As the author takes
them into account, the content will change. However, we need to keep track of
the state of the content so that as it changes, the author is not receiving
all of the messages if the messages are no longer relevant (messages either
from the tool to prevent it from checking content that has been checked and
o.k.'ed by the author or messages from other people annotating the content,
i.e. raising issues).

We continued discussion of hash. Decided that ETag wasn't useful.The idea
of hash doesn't seem to have gone anywhere. Instead we talked about threads of
assertions and referring between testsubjects so that subsequent test subjects
would update earlier ones.

We concluded that we need EARL threading although it wasn't clear if it was
something that needed to be added to the language or something that would be
taken care of by tools. We decided to think about this and discuss more at
Monday's meeting.

A couple of things to consider

Issues w/adding to the language (and long strings of assertions)

having a thread relationship between the different "snapshots" or events
is not a burden on the server, only on the UI where the composer has to
add the link.

If only point to what comes before (and not worry about what comes next)
end up with an arbitrarily long string of assertions. When does it stop
being relevant? Either forever or it expires at some point.

assertions kept forever, bug tracker throws resolved ones away.

arbitrarily long string of assertions, -- sort of like bank statements
that refer to the previous statement. so long as the statement is useful
on its own, you can dump old statements after n years. so we want the
assertions to be relevent unless there is a later snapshot or event in the
thread?

Or an assertion that a previous assertion is resolved or obsoleted (and
can be hidden or removed).

do we want a "supercedes" property or would it be derived from earlier
reports?

Independent of the language and up to the tool?

assert "this is what I evaluated, you can use this to check it's the
same thing you're evaluating"

EARL needs to be discrete enough to say that accessibility is maintained
across variations; language negotiation etc.

Headers and timestamp not sufficient. we add in the ability to compare
what the server gave us rather than what it said t was giving us in its
headers.

Diffing inputs not efficient and inputs aren't always what expect.

Arguments against adding state to the language

It's up to the query

The thread is only relevant to the tool using them, therefore don't need
to use threads between tools and therefore not part of the EARL
language.

Multiple test subjects (a possible way around state in EARL? yet too
excessive?)

If have 20 images w/out alt-text, need 20 test subjects. excessive?

not convinced that necessary to keep state info in EARL.

multiple test subjects do almost everything as state can be generally
inferred just from that

Public logs of #er

Discussion of what was needed and why. WC and Charles Munat will discuss
the UI. Jim and Nick might do some hacking to help. WC will come back with
more info. AG interested in the tool and perhaps will also help with UI.

Next meeting

Monday (Jim's bday :) at the regularly scheduled time.

Detailed minutes

Here is the IRC log. I moved some bits around, separated out some side
conversations, etc to make it easier to read.

AG Start with a log tape model. IETF timestamp standard allows arbitrarily
fine decimal fraction of seconds. If you are unsure, keep a copy of what you
got and evaluated or at least an MD5 to detect if another is dfferent.

NK earl:fails --> earl:passes after earl:action

wendy wonders how annotea deal with this?

JibberJim believes they just ignore it and let them get orphaned...

WC on monday we talked about hash. w/out resolution.

wendy double checks minutes...

NK I can make two earl statements (before:fails, after:passes), but how to
express the relationship?

NK we made a hash of it?

JL I think you're right Nick, it's unreasonable to just remove user
comments because an author has corrected them, and it needs some feedback
mechanism to let users/evaluators know.

SBP Remember also that hash is not necessarily secure. We may well end up
just asserting that hashes are foolproof in context, though

WC we also discussed ETag, although the downside was not all servers serve
them. Also, what if the content hasn't been published yet?

JL A Hash only covers about 95% of urls in clj FAQ 5% others change despite
the content being basically static.

WC other suggestions from monday:

Use a message-id type construct

Use any daml:UnambiguousProperty

JL ETag was only about 70% and was pretty well duplicated by Last-Modified
so I don't think it's well enough supported.

JL (we could define them all of course, and let the tools pick the best in
context?)

NK I don't think we have to be dogmatic about this

NK Leave it up to tools how to generate unique IDs

WC except that one of the reasons for creating EARL was to pass info
between tools. if each generating own ids, then we can't do that can we?

JL I think we need a way of interchanging "this is what I evaluated, you
can use this to check it's the same thing you're evaluating"

NK Of course we can! Email message-IDs are created with a wide range of
algorithms

NK Jim, yes

NK URL+timestamp

JL That is insufficient with current URLS, due to adverts, small
permanently updated portions, the habit of putting on "todays date" on every
page etc.

AG URL + timestamp does not cover server instability (sniffing)

NK Erm - I assert it does

JL And you say I looked at this 2001-12-10, I have no way of knowing if
it's what I looked at on the same date, unless the server is telling the truth
- they don't.

SBP Yeah... EARL needs to be discrete enough to say that accessibility is
maintained across variations; language negotiation etc.

AG Question is 'why do you care'? If the point is to discern the reason for
a difference in outcome, you basically want to be able to diff the two inputs
anyway.

NK - if we express content negotiation

NK URL + context

NK Context = timestamp + HTTP headers

WC Al - we don't always want to diff the inputs.

WC That's the point.

SBP N.B. Timestamp should/may be in the HTTP headers

NK Not reliable

JL I don't see headers as sufficient.

NK why not?

JL Experience of trying to identify non changing pages.

SBP "Date: Thu, 06 Dec 2001 00:21:15 GMT" - HEAD http://www.w3.org/

AG I agree that for efficiency one doesn't want to always have to diff the
inputs. But all the things that are based on "what I asked for" instead of
"what I got" are unsafe. You need to know if the inputs to the two evaluations
were different. The thorough mode involves canonicalization as in XML
Signatoure.

JL We don't try and work around any, we add in the ability to compare what
the server gave us rather than what it said t was giving us in its
headers.

WC eric - do you track state in annotea?

NK I can make two earl statements (before:fails, after:passes), but how to
express the relationship?

ericP considers...

EP state tracking, annotea has the ability to remember triples associated
with a document.

AG I really don't understand the question; the before:after pair is defined
to express the relationship, no?

EP it also has the ability to flush and re-parse the "document".

NK Al, I just invented them for conciseness of my question

SBP to express the relationship, you simply state that the test subject is
not considered to have changed according to the processor; and that *can* be
either of a purely processor oriented thing, or it can be something more
grounded such as a hash

EP if you wish to compare before and after states, i recommend hanging the
state predicates off of two objects, representing before and after.

NK and if before/after iterates over several changes?

EP beforeId --hadWheelChairRamps-> no

EP beforeId --hadPresentation-> presentation1

AG eric, how do you express the common ancestor of the two
object-states?

EP presentation1 --tooMuchFlash-> yes.

JibberJim wonders if we need anything special for a before and after, can
we not have a second assertion saying the resource passes whatever it failed
before, and the tools would then know it had been fixed?

EP and a similar object called afterId, for example.

WC earl might "cascade" similarly to annotations.

WC we generate a bit of earl on 1st pass. it fails.

EP common ancestor, i would assume they would both annotate some
resource

NK jim, no, that loses tracking info

WC generate more earl after a while. says it passes.

EP beforeId --annotates-> http://www.w3.org/

EP afterId --annotates-> http://www.w3.org/

JL Tracking info is there because you have a different testsubject (with a
later date etc.)

WC a thread of earl. each bit associated w/a hash to identify "state" of
resource.

EP beforeId and afterId may be much more useful if they are urls created
from a timestamp

WC problem is, I have been assuming that the only reason for multiple bits
of EARL to be associated w/a resource would be diff authors. However, we are
now talking about multiple bits of earl from a variety of people and times.
just a hole in my thinking. not sure if others had that as well. had not
considered an earl "thread" before. but seems reasonable.

NK wendy, you only fill in the holes when you start building real-life
apps

NK I've recently started that

WC :)

JibberJim thinks we never should've started implementing things :-)

SBP True. We recognized this right at the start, but you just can't tell
until you implement

WC but another person makes an assertion at time y about only img
id="z"

EP arbitrarily long string of assertions, -- sort of like bank statements
that refer to the previous statement. so long as the statement is useful on
its own, you can dump old statements after n years.

NK years??

TB How big is "n"?

EP so we want the assertions to be relevent unless there is a later
snapshot or event in the thread?

JL The threading is only relevant to a particular tool/use of the EARL, it
doesn't need to be inherent in all versions so expiry would be tool
dependant

WC it will depend on the state of the resource when the assertion was
made.

NK earl:[Updates|Supersedes] ?

TB Or an assertion that a previous assertion is resolved or obsoleted (and
can be hidden or removed).

NK jim, agreed

JL e.g if I was using it as a bug tracking format I'd want them forever,
but if I was using them for deciding whether to visit a URL only the current
would be relevant.

AG whether you want assertions that have not been superceded or all
assertions since King Tut is in the query, not the EARL.

EP it is possible that the thread relationship says Updates|Supersedes

JL I can't see exchange of histories being something that is too common a
usage, but I might be blinkered.

NK wendy, no, I want threading in EARL, but let expiry/archiving be
tool-dependent

JL gets AL, but you still need to define the relationship between two
subjects somehow the query isn't sufficient.

EP changing banks - you lose the thread, but there isn't really one that
crosses statements unless you go down to the threaded transaction level.

AG You exchange histories to narrow the range of time within which an
exceptional event happened.

EP thesis: the cost of threading is only in how much you grow to depend on
it.

EP plus a bit in UI establishing the thread when posting the later
states.

AG I am thinking that 'supercedes' is a derived property, not settable
except by computation. Yes it is necessary to determine when the subject of a
later assertion has a supercedes-ready relationship to athe subject of an
earlier assertion.

WC it's close to 1 a.m. for several of you. does anyone want to take an
action to do more research on threading? write a proposal for how to deal with
state changes?

WC or something else?

WC i get the sense that we're either all thinking deeply or are
stunned?

NK I'm happy @1am

AG I'm numb at 8: p.m. But then, I'm numb at 8: a.m.

JL I'm thinking it might be a good idea to postpone discussion to let us
flesh out ideas and to move on to other areas?

JL It's currently easier to work with Nicks XML output than EARL
output.

NK Agreed. XML is operational; earl was experimental

JL Until we're combining EARL, all single tools can be created more simply
from the XML, XML tools are ubiquitous RDF isn't.

JL So I work with the XML...

SBP Ooh, I might be able to use the list extension property (daml:nil), and
the little built-in that I wrote: math:memberCount

SBP Huh... it kinda works if I put a ":Null" in the list, but it depends
upon how you want to collate results. I think that results should be first
class objects too, and they may even need context information thrown in with
them

JL Was there ever a consensus on pointing to a fragment of a page in EARL,
was it just multiple testsubjects with XPointer - that seems very
verbose...

SBP Jim: Did you come across Len's work on the BNF-to-XML conversion? *
JibberJim is trying to think what the EARL would like for the input to
http://valet.webthing.com/xslt/jim.html

NK As for jim's wysiwyg validator, I'd *love* to implement it as an
alternative view in Page Valet

JL Then Iron out the bugs Nick, and go for it :-)

NK Round tuits !!!!!

NK Time - or lack thereof !!!!!

JL Indeed!

JL Don't we need a new testSubject for each XPOINTER currently - so say if
20 images don't have ALT defined to point to each of the 20 elements to say
that, I'd need 20 test subjects? this seems very excessive.

AG Is there a daml:eachOf?

SBP daml:oneOf

NK damn:allOf?

NK [sorry]

SBP you can iterate through lists to probably achieve the :eachOf effect
that you're looking for

SBP So, the only two important issues that I am aware of are state, and
multiple test subjects

NK I thought the discussion of state useful. Looked like it could lead
somewhere

JL I'm not convinced it's necessary within EARL, but have no real
objections to it.

JL (multiple test subjects do almost everything as state can be generally
inferred just from that.)

SBP I guess some people will find it more useful than others. Anyone can
extend what we have; the question is how much we should do, and how much we
should test the genericity of the architecture :-)

WC jim - Chris and Josh created a whole bunch of test files. we want wcag
wg members to go through the test cases on various platforms and collect a
mass of data. Then we want to combine the data. it would be cool to save as
much of this in EARL as possible. Then combine results in a report like:

8 of 10 people agree this page is accessible

or 1 of 10 people think this alt-text is appropriate

etc.

SBP *sigh* statistics

AG Why the sigh?

NK nowt wrong with stats - as such

SBP Just difficult for me to grok at the moment

WC well, but also:

accesskey works on platforms x,y and browsers a,b

tabindex is broken on ....

WC then, we have a table of:

markup elements

platforms

browsers

assistive technologies

JL So it's general purpose reports about all of WCAG's work really then
:-)

NK So if Jim or I or someone hacked up a database and a form or two ...

WC ...then i would be one of the happiest folks alive!

NK ... in principle at least

WC i actually want to do the xslt of the form input.

WC or at least help.

WC i've been dying to get involved in an imp, but all this other wcag stuff
is keeping me busy. :\

JL So you're thinking form -> earl -> DB

NK ug.. YM you want to do the UI?

JL knows he won't be doing any UI...

WC charles munat has agreed to help as well.

AG XForms. Do model first.

NK That's great, neither Jim nor I care much for UI

WC he's got a 3 week break coming up.

WC i think he'll do UI.

JL The input is the XML testfiles?

WC yes.

JL the output is EARL reports containing result of test, and browser etc.
evaluated on, are the tests dependant on the browser?

AG what? The input is what the user concluded on evaluating the
testfiles?

WC xml test case info gets generated into html for user to look at, as well
as fill in part of the form the user will use to comment on the test.

JL It's definately UI and test cases (which we've got) rather than the
technical programming parts that are time here.

WC sbp - what's "the obivous"

SBP CWM

WC al - if the server were to select tests, it would have to know: which
browser, platform, and AT the person using since we'll have to test

WC each case for each config.

WC (ideally)

NK that's easy - we tell it

WC yes, just another part of the process.

WC making sure it gets included in the scenario.

AG users have registered default profile there; most test subjects only do
one. Others can flop personal for different parameter profiles

WC Process

step 1: collect info from user about their set up

step 2: send them a test

step 3: collect feedback

WC databases needed:

user profiles

test cases

test results

JL The first part of speccing this is to get the UI person to spec out the
UI and the routing, the implementation is hours.

WC the question is...who is the UI person? :) I'm suspecting that will be
me and Charles Munat, and possibly Al Gilman...

WC since he seems interested. :)

NK yep, could do that in one session if an ugly hack was allowed

AG Jim: that's entirely a matter of problem-solving style; others would say
the first thing is to describe the relation you want to winde up with in the
DB

WC Regardless - let me think about this some more and get back to you.

WC unless - there are more thoughts now... :)

WC i haven't any.

NK Al, I think that's too straightforward to make an issue of

AG Now I need the extra-verbal cues. That's too clever for me to decode.
I'm not trying to make issues, I'm trying to get GL the infrastructure they
need to review facts.

JL Nick - do you know about the RDF triple database for storing EARL?

NK OK, what I mean is, it would take me longer to spec up the database than
to implement the whole thing, but the implementation would be an ugly hack

JL (as used by Annotea currently.)

NK jim, sweet f*** all

JL more than me then...

JL it should really be in that. (how do you go from triple to xml-rdf
even)

SBP serialize it!

JL Yes...

NK sbp - did you write a proggie for that?

AG But the problem is that we want to retain a cross relation on (user,
test, result) triples under conditions where there are further detailed
properties of users, tests, and results that are not stable.

SBP If you want to be clever, you store your NS prefixes in a dictionary.
IF not, you just hack off the last char and use that as a namespace

SBP I wrote an XML RDF serialization function, yes

SBP er... use the remainder as a namespace, I meant

JL It's all stored in EARL Al, we could have a single EARL for each report,
and then it's just the EARL combining we need to do for that, I don't see the
problem you're describing,.

AG Wendy, I don't know if I am any use on the UI or not, but don't fail to
ask.

AG Jim, that's OK. The devil is in the ontology that lets us roll up the
EARL that has different keywords all through it. Just like our question about
when does a new test supercede an old test?

JL One problem on the dev work is me and Nick will have very different
approaches, and it's definately one of us do it I'd've thought, we'd get in
the way more than help....