SA: not seeing anythng about exclusion. The case is about different portlets and what we can say about the entire portal. This may need to be addressed in 3.1 or nearby. Disagree with excluding parts - that is a different discussion

14:10:46 [vivienne]

EV: will take out the comment about excluding. Definition is a bit unclear. We will add a clarification in 5c on aggregation.

14:10:57 [vivienne]

SA: define the performance score first in 5c

14:11:02 [shadi]

zakim, mute me

14:11:02 [Zakim]

Shadi should now be muted

14:11:04 [richard]

q+

14:11:32 [vivienne]

EV: add a note about website parts to be explained in 3.1

14:12:00 [ericvelleman]

q?

14:12:02 [vivienne]

EV: clarify the performance scores in 5c and then how to aggregate

14:12:07 [Detlev]

ack me

14:12:12 [Zakim]

-richard

14:12:46 [richard]

Sorry lost phone

14:13:39 [Zakim]

+??P45

14:13:53 [vivienne]

Detlev: the issue is that you can single out a part of the site like a library for evaluation, but you can't use the overall university evaluation and exclude library. Regarding the performance score, we can't make any decision on that until we decide on the score function. Putting in a note now could be confusing about putting in information about score when there isn't anything there yet.

14:13:53 [vivienne]

We can't say anything about aggregation methods yet.

14:14:11 [shadi]

zalom, ??p45 is richard

14:14:16 [shadi]

ack richard

14:14:26 [Detlev]

Zakim, mute me

14:14:26 [Zakim]

Detlev should now be muted

14:14:30 [shadi]

q+

14:14:40 [vivienne]

Richard: Are we talking about aggregate eg Level A or Level AA, we can't have bits being one and bits being another. There is no inbetween, so not sure what we mean about aggregating

14:15:02 [Aurelien]

maybe we can ask him some more details on his comment

14:15:32 [ericvelleman]

q?

14:15:34 [vivienne]

EV: Francois' comment is about 3 sections, which is making it confusing. You cannot aggregate A, AA, AAA, but we'll come back to that in 5c. We could say if part of a website conform to A and others to AAA, you can't say the website is AAA. You would say the website is A with parts at AAA

14:15:34 [shadi]

ack me

14:16:36 [Zakim]

+Mike_Elledge

14:16:52 [ericvelleman]

q?

14:17:20 [vivienne]

SA: agree with Detlev. We need to have the algorithm for scoring in 5C that we provide optional, in addition to the Level A, AA, AAA conformance claim. Before that we can't talk about aggregation till we have pieces. What belongs in 3.1 is the note as Detlev suggests. As to aggregating n the levels basis, we can do that easily by talking about the least common denominator - if one is A and the

14:17:20 [vivienne]

others AA, then the overall accessibility is single A - could add a note that parts are better. But the aggregated value is only A.

14:17:27 [Detlev]

agree with Shadi

14:17:28 [Mike_Elledge]

Mike_Elledge has joined #eval

14:17:34 [richard]

I agree with Shadi

14:17:36 [shadi]

zakim, mute me

14:17:36 [Zakim]

Shadi should now be muted

14:17:38 [vivienne]

SA: 5C is required to be looked at separately

14:18:01 [shadi]

q+

14:18:08 [shadi]

ack me

14:18:09 [Detlev]

fine

14:18:16 [vivienne]

EV: in 3.1 we don't have anything about the conformance aggregation - propose then to just clarify the definition of website parts and to make a note to continue discussion on the performance scores later. Is that okay with everyone?

14:18:35 [vivienne]

SA: let's talk about clarification of the website parts separately

14:19:33 [vivienne]

SA: propose to add all of this to section 5

14:20:27 [vivienne]

Sa: we can say that aggregation on a performance value is the least common denominator. With regard to aggregating the performance score we need a note saying we need to address this later. Not sure where the 2 pieces belong - more in section 5

14:20:37 [vivienne]

EV: 5C says option to provide a performance score

14:21:45 [vivienne]

EV: need to decide what you want to score when you then want to re-aggregate - hasn't been described yet

14:21:49 [ericvelleman]

q?

14:22:58 [vivienne]

SA: 1 defines a portal - maybe a note saying that 1.a - if you already know the accessibilit of a website part, then you don't need to re-evaluate that, you can consider that outcome along with your entire evaluation

14:24:04 [vivienne]

SA: section 5: need 1 step that talks about the conformance claim itself - 5.8 provide documention for each step and part of that is to provide the actual outcome and in there will be the aggregation on a conformance status.

14:24:14 [richard]

The conformance claim for the whole site can only claim the lowest score of any and all parts

14:24:18 [vivienne]

SA: there will then be more stuff in 5c

14:24:36 [agarrison]

It should be noted that you would only be able to aggregate evaluations which are done within a certain time period

14:24:55 [ericvelleman]

q?

14:24:56 [Tim]

Tim has joined #eval

14:25:44 [vivienne]

I agree with what Shadi said, along with Alistair's comment - that's good

14:25:56 [Aurelien]

+q

14:25:57 [shadi]

zakim, mute me

14:25:58 [Zakim]

Shadi should now be muted

14:26:08 [shadi]

ack Aurelien

14:26:42 [vivienne]

Aurelien: agree with Shadi.

14:26:53 [vivienne]

Sorry, Aurelien, I couldn't understand that - could you clarify

14:27:02 [vivienne]

ack me

14:28:00 [Detlev]

q+

14:28:07 [Detlev]

ack me

14:28:09 [vivienne]

EV: if you have looked at website parts that you do not need to re-evaluate them as long as they are within a certain time frame and in step 5 add a short proposed text about how to aggregate the parts. Start putting in info about the lowest common denominator eg A

14:28:15 [vivienne]

zakim, mute me

14:28:15 [Zakim]

vivienne should now be muted

14:28:39 [Aurelien]

yes that's it shadi

14:28:54 [shadi]

q+

14:28:55 [vivienne]

Detlev: lowest common denominator - if you have a site which has 15 portlets and you evaluated 14 and they were all AA and 1 was single A, if you aggregated it, that would be single A. Is that meaningful?

14:29:23 [shadi]

ack me

14:29:24 [vivienne]

Detlev: you need the detail so you need to work on the one that wasn't AA

14:29:30 [richard]

q+

14:29:32 [Detlev]

Zakim, mute me

14:29:32 [Zakim]

Detlev should now be muted

14:30:29 [Kathy_]

q+

14:30:45 [Detlev]

q+

14:30:52 [shadi]

zakim, mute me

14:30:52 [Zakim]

Shadi should now be muted

14:30:53 [vivienne]

SA: agree with Detlev - use cases are difficult - same as when we evaluate a website and it 'nearly meets AA on all occasions' but the conformance reallity is that it is A. The additional documentation allows you to add more information - but we only have the 4 discrete values of conformance defined by WCAG. The report gives more information, but the performance score provides additional

14:30:54 [vivienne]

information so that they can clean up the last one

14:31:01 [shadi]

ack richard

14:31:46 [vivienne]

Richard: you need to be really clear about who the performance claim is aimed at - if it's the user then we go with the WCAG - lowest common. If you're looking at the developer's point of view, but it's different - not a conformance claim but a report on the progress towards compliance. Aimed at the owner/developer and not the user.

14:31:47 [Kathy_]

ack me

14:32:26 [shadi]

+1 to Kathy

14:32:26 [vivienne]

Kathy: agree with Richard. Regarding not having to review a portlet that we'd done in the past within a certain time - but need to add a restriction on whether there was a major change - need to be careful on wording

14:32:33 [vivienne]

+1 also

14:32:34 [ericvelleman]

q?

14:32:37 [Detlev]

ack me

14:32:37 [Kathy_]

zakim, mute me

14:32:38 [Zakim]

Kathy_ should now be muted

14:32:50 [Mike_Elledge]

Mike_Elledge has joined #eval

14:33:34 [vivienne]

Detlev: benchmark might be - not aggregated score where you try to put the performance on a level - if conformance has not been reached on that level, then how many of the success criteria have been met - eg say 3 failed and the rest passed. You could do that for any scope

14:33:48 [ericvelleman]

q?

14:33:50 [vivienne]

q+

14:34:07 [vivienne]

ack me

14:34:31 [vivienne]

EV: report on more than the AA or AAA. This is feasible - maybe add something in section 5

14:34:33 [Detlev]

Zakim, mute me

14:34:33 [Zakim]

Detlev should now be muted

14:34:46 [shadi]

q+

14:35:31 [vivienne]

zakim, mute me

14:35:31 [Zakim]

vivienne should now be muted

14:35:56 [vivienne]

VC: it is good to add information about what has been met for each of the levels - can be encouraging

14:35:57 [agarrison]

q+

14:35:59 [shadi]

ack me

14:37:00 [Zakim]

+Tim_Boland

14:37:11 [vivienne]

SA: Canadian Treasury Board's aggregation was circulated. This was simple and quite effective, and there is an on-going counter. In addition to the methodology we have resources to develop an evaluator interface. Similar to "How to Meet..." the idea is to develop this interface.

14:37:23 [vivienne]

SA: we could make this more clear in our document

14:37:32 [shadi]

zakim, mute me

14:37:32 [Zakim]

Shadi should now be muted

14:37:36 [Kathy]

Kathy has joined #eval

14:37:37 [ericvelleman]

q?

14:37:38 [shadi]

ack agarrison

14:37:48 [Kathy]

zakim, Kathy_ is Kathy

14:37:48 [Zakim]

+Kathy; got it

14:38:17 [vivienne]

Alistair: not convinced that we should be looking at AAA, when we are asked to look at AA. If people ask for AA, then I stick with AA and not look for AAA, this is too big an increase in time required.

14:38:47 [vivienne]

Alistair: you have to add sign language for AAA, that is too expensive for most

14:38:50 [shadi]

ack me

14:38:59 [vivienne]

EV: currently marked as optional?

14:39:12 [vivienne]

Alistair: don't want to encourage people to expect AAA when we put that in

14:39:14 [shadi]

zakim, mute me

14:39:14 [Zakim]

Shadi should now be muted

14:39:41 [vivienne]

EV: in 3.1c there is a sentence about that it can be useful to do AAA - need to clarify

14:40:15 [richard]

zakim, mute me

14:40:15 [Zakim]

sorry, richard, I do not know which phone connection belongs to you

14:40:21 [ericvelleman]

q?

14:40:26 [Aurelien]

no

14:42:10 [vivienne]

EV: what do we want to do with the remark by Francois - wants the scores better defined. We need more discussion about that. I can write a new proposal on the mailing list and see if we can agree on a sentence to add for the scores n section 5.c

14:42:38 [shadi]

ack me

14:42:39 [vivienne]

EV: all agree that I make a clarification on website parts?

14:43:56 [vivienne]

SA: will write on some actions

14:44:15 [shadi]

[Suggested actions: (1) provide a section that is more clear about "determining conformance outcome" (Level A, AA, AAA); (2) add a note in there about aggregating such conformance outcomes from individual website parts; (3) add an editor note to section 5.c about aggregation of individual website parts (for future, after we agree on the actual algorithms); (4) add pointers to these concepts early in the document (eg. step 1.a)]

14:46:10 [ericvelleman]

+1

14:46:14 [vivienne]

Thanks, Shadi. Makes my job easier.

14:46:15 [Detlev]

fine

14:46:15 [Aurelien]

+1

14:46:17 [vivienne]

+1 from me

14:46:18 [MartijnHoutepen]

+1

14:46:19 [Kathy]

+1

14:46:38 [shadi]

zakim, mute me

14:46:38 [Zakim]

Shadi should now be muted

14:47:38 [shadi]

q+

14:47:42 [vivienne]

Ev: ID 20 - other audiences -

14:47:44 [Kathy]

zakim, mute me

14:47:44 [Zakim]

Kathy was already muted, Kathy

14:47:48 [shadi]

ack me

14:48:18 [Detlev]

q+

14:48:22 [vivienne]

SA: 1.3 says that it is expected that the reader is aware of the resources

14:48:45 [vivienne]

EV: could add it just about 1.4 - a short and concise pointing to the business case

14:49:23 [vivienne]

SA: seems to be out of scope of the document - should assume they already want to do an evaluation - this is the guidance on how to do it. Maybe reference this later in the document.

14:49:23 [Detlev]

ack me

14:49:25 [shadi]

zakim, mute me

14:49:26 [Zakim]

Shadi should now be muted

14:50:07 [vivienne]

Detlev: Peter was getting at the need for business to progress and making that comparable at different stages. We can re-write it later and upgrade it later when we know what we can offer for the business side.

14:50:28 [vivienne]

EV: it is in the scoring - for comparison

14:50:44 [vivienne]

Detlev: if you do the evaluations 3 times, then you have a score you can compare it yourself

SA: Peter is talking about the progress - not only does it meet/not meet, but how is the support for WCAG improving over time. This is the performance score section that is still to be developed. We need to finalise that first and talk about how we can motivate and inform developers even if they have minimal support for WCAG 2. It will become later what we can say at the front. Need a resolution

14:53:11 [vivienne]

to come back to it later after 5C has been developed.

14:54:09 [vivienne]

EV: ID:21 Add definition of "ancillary functionality" we agreed that as a concept it might be useful. But just hold the thought and not currently add it to the document.

14:54:12 [ericvelleman]

q?

14:54:18 [Detlev]

Fine, can be added later

14:54:27 [vivienne]

SA: mark the comment as 'closed' - first one!

14:54:38 [agarrison]

I don't know whether allowing people to say they meet 59 out of the 61 checkpoints is encouraging them - or allow a status quo to develop. If you only talk about A or AA that might be more encouraging...

SA: this could be a longer discussion. There is a bit of a miss-match. Partial conformance does not mean excluding a part from the evalution. Part of the comment seems to say 'exclude the ability to remove part of a website' . It is more about how you write a conformance statement, but it does assume you have done the evaluation. We'll need more in section 5 about the documentation. Removing

14:58:39 [vivienne]

part of a website would not allow you to make a statement of partial conformance because you don't know what that missing piece does or does not meet.

EV: it is a clarification that you can't leave something out - partial conformance is meant in a different way

15:01:01 [vivienne]

EV: will see if we can speed it up by doing part of the discussion on the mailing list and put comments and resolutions in that mailing list. All please put your comments down for discussion. We can go through these in the next call.

15:01:03 [Tim]

how does partial conformance apply to websites instead of webpages as WCAG defines?

15:01:07 [shadi]

Topic: F2F Meeting

15:02:22 [agarrison]

CSUN would be better - given the choice

15:02:57 [vivienne]

SA: news about Lyon. Survey - we have 13 likely attending, 13 can't likely attend. Some might be able to travel from the US, but also another maybe a CSUN in the US and we would have the opposite figures there. Would still recommend that 13 people would be a good turnout for a sub-group meeting - a good idea still. We could do a lot of ironing out over the course of 2 days. Are there any

15:02:57 [vivienne]

objections?

15:03:03 [vivienne]

I'm still going to see if I can get there.

15:03:18 [vivienne]

Sa: we will still have some dial-in discussion

15:03:31 [vivienne]

SA: any objections to this sub-meeting?

15:03:57 [vivienne]

EV: no objections noted - will try to organise it

15:04:08 [vivienne]

EV: other issues?

15:04:14 [agarrison]

q+

15:04:20 [ericvelleman]

q?

15:04:50 [vivienne]

Alistair: re techniques being optional? Can we put this forward at a later date?

15:05:33 [vivienne]

Ev: added ID 47 which was a missing comment in the disposition. So I'll add another one for Alistair's comment.