<AWK> +AWK

<scribe> Scribe: Rachael

AWK: Adding WILCO to Jan 10th as scribe. Volunteers for future dates are welcome

<david-macdonald> I have to drop off for a moment for a client

AWK: We will be taking next week off. It will be the only week we will be off.

<alastairc> @david-macdonald please mute, you probably don't want us over-hearing!

Requirements for SCs (https://www.w3.org/WAI/GL/wiki/WCAG_2.1_Success_Criteria)

<AWK> https://www.w3.org/WAI/GL/wiki/WCAG_2.1_Success_Criteria

<david-macdonald> oops

<david-macdonald> back

<david-macdonald> you can unmute me

AWK: Idea behind this document is looking at what are the requirements for this success criteria. In the list, we have some shalls and some shoulds in this list. We have seen these before. Should we walk through these one at a time or do people have particular issues?
... I'm going to go through them. Any questions on "Address a situation where a user with a disability will be disproportionately disadvantaged (as compared to a user without a disability) if the criteria is not met." ?

Lisa: I want to clarify that we had added that this is provided as guidance. They are not set in stone (at least this is how it was explained to me). Especially when we get to the structure. They are less acceptance criteria and more guidance.
... You can't retroactively insert acceptance criteria.

AWK: I'll respond from my perspective. We did set these up as guidance for the task forces but we are going to need actual requirements for what makes it into 2.1. This guidance may well be looser than what we ultimately have or we may say that this is it. That is why we need to talk about these.

Lisa: We have to be clear that what we are starting now is a consensus that these are changing from Guidance to criteria for what makes the cut. We need to be clear that this is the conversation.
... I suggest we should test against these things and where things are weak, we should tag whether it is a strong enough issue to reject the item. Things should be tagged as lacking but not necessarily rejected.
... For instance, do we want to tag it and rewrite it to make it more backwards compatible. These should be issues to tag, not acceptance criteria.

<Zakim> alastairc, you wanted to ask about new vs editing

AWK: We would not be making this decision not just on the list

Joshue: This is a good set that fits into the 2.1 framework. If there is something that is worrying people about them, this is the time to bring it up. My quesiton is whether we are allowing editing of current success criteria. There was a previous decision where we had asked these be written in an additive way. Are we at the point of that conversation?

AWK: That is essentially agenda item 2.

Joshue: Some points imply that we will be able to edit exisitng criteria.

AWK: We were leaning that way at the time, but that is the next agenda.

Joshue108: We did work on making these as tight as possible at the time we wrote them. Is there something that you have a particular issue with at this point?

Lisa: We had a couple of calls at the time this was written where I had significant reservations about the template. We added the sentences that these were guidance only to address my concerns.

<JF> s/JF: We did work on making these as tight as possible/ Joshue: We did work on making these as tight as possible

Joshue: What is the issue with the criteria that is there?

Lisa: David's comment that only 2 of the SC met these criteria concerns me.

Joshue: That was useful work but does not represent the group consensus or agreement.

Lisa: I need to review again but the testability is a concern.

<david-macdonald> Here's what Lisa is talking about http://tinyurl.com/jmo9st4

AWK: Can we go through in order and figure out which are the issues?

<Zakim> JF, you wanted to confirm clarity on what we mean by "Acceptance Criteria" (versus Success Criteria and Guidelines)

Joshue: In terms of testability, anything we write will need to fall into the well worn path of what WCAG has done in the past.

JF: In Lisbon, we discussed the possibility of introducing a new guideline under perceivable or robust, it may address some of the more abstract/fluid concepts you want to ensure are included. I think we need to keep the possibility open of creating new guidelines.

<AWK> "acceptance criteria" is being used as a term that speaks to the requirements for an SC to be incorporated into WCAG 2.1

<Ryladog> +1 to new GLs if needed

<bruce_bailey> From abstract: WCAG 2.0 success criteria are written as testable statements that are not technology-specific

gowerm: Thank you for sharing this with the team. Based on the review I was conducting of SC, this fits in well. This is a useful tool to use when reviewing items. Just because someone points out that it doesn't meet one of these criteria, doesnt mean it won't make it in. Just that it needs to be beefed up.

<Joshue108> That sounds ok with me Lisa

lisa: That ties into the idea of tagging. I feel more comfortable if we don't accept based on these but tag work.

<Joshue108> I hadn't heard your tagging idea before - but whatever works is fine with me!

David: There are no no's in this thing. I only put yeses in. This is my own exercise but if anyone wants to look over my shoulder and contribute, let me know. These aren't new. They are from WCAG 2. If we want to change this for 2.1, its a pretty major move.

John: I don't think we should for backward compatibility.

<bruce_bailey> please don't characterize these requirements/expectations for SC as quote acceptance criteria unquote

David: They aren't new. They come from the original editors. I wrote a blog article after I spoke with them. There may be a great reason to stray from these but if not, we should stay in the existing guidelines.

<bruce_bailey> I am pretty sure “acceptance criteria” may be a term of art for wcag 2.1 going through the TR process

DavidL Better to hammer it out now.

<lisaseeman> 8 is problematic

AWK: These came from WCAG 2.0 timeframe initially but I take a bit of exception of these only coming from there. We talked about these for weeks and modified some things. This is the work of this current group, now. There are some that Greg doesn't completely agree with. We need to discuss them and agree to them. I think we have substantial agreement to them but there are still some concerns,

<JF> +1 to AWK ref: stone tablets

<lisaseeman> and we need to clarify testable

AWK: Lets look through them and see if we can quickly identify where additional discussion is needed.

<david-macdonald> Here's the acceptance Criteria according to Gregg and Loretta http://www.davidmacd.com/blog/what-are-WCAG-success-criteria.html

<Zakim> bruce_bailey, you wanted to ask what 3 means

Wayne: I think this is a pretty good list. What I notice is that as we look at them one by one they bring out some exceptions. We are going to run into some barriers as we get into these individually. I think using these as general guidance works. Disability categories were missed in WCAG 2.0 and I wonder if the criteria used led to that.

<JF> From WCAG: Guidelines - Under the principles are guidelines. The 12 guidelines provide the basic goals that authors should work toward in order to make content more accessible to users with different disabilities. The guidelines are not testable, but provide the framework and overall objectives to help authors understand the success criteria and better implement the techniques. Success Criteria - For each guideline, testable success criteria are provided to allow WC

<bruce_bailey> Describe the specific condition required to meet the criteria, not the method to address the criteria.

Bruce: I don't see anything controversial on the list at all. They seem straightforward to what SC need to do. The one exception is #3.

AWK: Lets talk about that at #3. #1 Address a situation where a user with a disability will be disproportionately disadvantaged (as compared to a user without a disability) if the criteria is not met. Does anyone have any concerns with that?

<Joshue108> good catch

Mike: Is there a definition of "disproportionately disadvantaged"?

AWK: No we do not. Currently there is some measure of subjectivity in this.

<bruce_bailey> example with 2.0 of (1) is 3.3.1 Error Identification: If an input error is automatically detected, the item that is in error is identified and the error is described to the user in text. (Level A)

Joshue108: Good catch. Do we want to worksmith around these now? Would this be simpler if we dropped disporportionately?

<lisaseeman> +1 to josh 's idea

<bruce_bailey> sorry, 3.3.3 is better example: 3.3.3 Error Suggestion: If an input error is automatically detected and suggestions for correction are known, then the suggestions are provided to the user, unless it would jeopardize the security or purpose of the content. (Level AA)

<lisaseeman> makybe change the word to significalty more disadvanteged

<Glenda> +1 to leaving disporportionately stays (otherwise it is a usability problem for all humans)

<bruce_bailey> Ex: 3.3.3 is just good design

<Joshue108> I actually don't think we should have any adverb in there

AWK: Disporportionately indicates that it causes users with disabilities with difficulty.

<bruce_bailey> +1 to what AKW is saying

JF: Again, as we are walking through this document. We have two terms we are using. Per WCAG, Guidelines are not testable while Criteria are testable.

<bruce_bailey> 3.3.3 is an example of something more important to keyboard or AT user moreso than a mouse user

JF: This is a group of guidelines.

AWK: I think that what you are saying is in line with what I am thinking. We are not suggesting we add this to WCAG. We are working on our internal process.

<Ryladog> This is our internal test process

<kirkwood> why not “significantly”?

JF: I look at "These requirements are provided as guidance to the WCAG Working Group..." This is simply guidance and I don't want to get caught up in wordsmithing.

Joshue108: We do need to get these things right. I don't think using an adverb like disproportionately is that helpful. I'm not going to loose sleep but we could remove this or widdle it down.

<gowerm> +1

<lisaseeman> LV just like coga- it bugs everyone but some people get completely stuck

<david-macdonald> I don't think we should change that word "Dispropostionate" it comes from WCAG 2

Wayne: Disproportionate applies a great deal in the low vision case. One example I can think of is that if you have to read something with scrolling its 1000 scrolls. When you have to do 10 to 20 times the amount of work as a user with standard technology, then it is disproportionate.

AWK: I agree we should now wordsmith this right now but we can discuss further. #2 Be testable through automated or manual processes.

<gowerm> add "repeatable"?

<gowerm> or "reproducible"?

AWK: This is based on WCAG definition. The expectation is that multiple people can test it and get repeatable results.

<Ryladog> I recall 8 out of 10

Lisa: First thing,we need to clarify and have consensus on what is testable and put it in this document. In WCAG 2.0, we said it was consensus. It wasn't 9/10 experts, we couldn't agree on that. We settled on the majority of experts will agree.
... That is as far as we got on 2.0. We shouldn't be more exclusionary here.
... It isn't just about running a test. You might have to be an expert or read the documentation. It is 1) expert opinion and 2) consensus but not unanimous.
... In the template, we didn't say that in a testable section that we give a formal test.
... we were asked to provide ideas on how to test it. Then the working group needs to look at it. They are ideas but need to be flushed out.
... We may not have a complete criteria. We should add this here.

<bruce_bailey> from understanding: A Success Criterion is a testable statement that will be either true or false when applied to specific Web content.

Mike: I'm wondering if there is value in adding the words repeatable or reproducable in these?

AWK: I think Lisa and Mike's points about repeatable testing and about needing clarity in what testable means is valuable. Its a good point to keep in mind.

JF: With all due respect, test criteria must be testable if we want to be backward compatible. \
... I don't see how we stray from the testable statement. Maybe if something can't be testable, we should make it a guideline.

<bruce_bailey> More from Understanding: All WCAG 2.0 Success Criteria are written as testable criteria for objectively determining if content satisfies the Success Criteria. While some of the testing can be automated using software evaluation programs, others require human testers for part or all of the test...

<bruce_bailey> The content should be tested by those who understand how people with different types of disabilities use the Web. It is recommended that users with disabilities be include

Lisa: I'm not saying it shouldnt' be testable. I'm saying it should be as testable as WCAG 2.0 but not more testable than WCAG 2.0

<bruce_bailey> https://www.w3.org/TR/UNDERSTANDING-WCAG20/intro.html

JF: What makes good alt text is similar. At some point the human brain needs to be involved. I agree both mechanical and human testing is needed. At the end of they day, the criteria needs to be a question with a yes or no answer.

Lisa: That is not WCAG 2.0

JF: WCAG 2.0 has to have a true/false answer.

<kirkwood> +1 to Katie

<bruce_bailey> The bit about “A Success Criterion is a testable statement that will be either true or false when applied to specific Web content” is the second sentence in the second paragraph (Abstract) of Understanding.

Katie: Maybe we need to go through and identify success criteria and which we think are good criteria but we don't have a clear way to test. Then we should ask for feedback from the review process on how to test them. We need to be clear as part of the first public working draft that we want to draw on the expertise of those who are not on the working group. Also, I remember an 8/10.

<jeanne> +1 to FPWD doesn't have to be perfect.

Wayne: I thought we assumed we were getting something imperfect from the groups and the SC managers are responsible for getting them through.
... I was looking at the COGA ones, and unsurprisingly the COGA criteria need a lot of human testing.

<JF> @Lisa: From https://www.w3.org/TR/WCAG20/#intro - "WCAG 2.0 builds on WCAG 1.0 [WCAG10] and is designed to apply broadly to different Web technologies now and in the future, and to be testable with a combination of automated testing and human evaluation." (Normative)

<david-macdonald> I think our stakeholdes and the accessibility world is waiting for something reasonably good after 9 years.

<JF> @Lisa: from https://www.w3.org/TR/UNDERSTANDING-WCAG20/#abstract - "WCAG 2.0 establishes a set of Success Criteria to define conformance to the WCAG 2.0 Guidelines. A Success Criterion is a testable statement that will be either true or false when applied to specific Web content. " (Non-Normative)

AWK: I agree that the FPWD does not need to be perfect, but I do not think we should add all the criteria without using some guidelines to winnow it down. It will overwhelm commenters if we put in the 70 success criteria that we have. I think we need to have our own sense of how we are measuring it and then commenters can tell us what they think of that.

Katie: I don't think those are mutually exclusive.

AWK: I don't think its what you meant, but it could be interpretted as we don't worry about testability.

Katie: I think an At Risk Section makes sense to take advantage of expertise not here.

Bruce: I disagree that #2 is debatable. I am looking back and that the success criteria be testable is not up for debate.

<JF> +1

Bruce: Its a statement of fact.

<bruce_bailey> the 8/10 experts agree was never adopted in final I do not think

JF: I posted in the posts from WCAG 2.0 that describe testable.

Lisa: That wasn't the internal agreement.

AWK: We have more to do to make sure we are all on the same page with this.

<lisaseeman> I can help andrew make a draft if you want, based on wcag 2.0

<bruce_bailey> the article with cite to 8/10 experts predated wcag 2.0

<bruce_bailey> http://alistapart.com/article/testability is june 2007

AWK: What qualifies as a text equivilent. There is some matter of subjectivity to the evaluation of a text alternative and whether it is equivilent. The ultimate success criteria can be a true/false answer.
... Lets move on to see if there are any other hot button issues:
... 3. Describe the specific condition required to meet the criteria, not the method to address the criteria.

This is talking about "There needs to be a text alternative available" vs. "There needs to be an alt tag" It needs to talk about what needs to be done not how to do it.

<david-macdonald> We dropped 8 out of 10 for a "strong correlation"

Mike: If you look at things, the testing happens at the technique level not the success criteria level. You can't easily talk about whether something is testable until you are at the technique level. Same thing with the specific conditions.

Lisa: That is a really interesting point. There are enough techniques that it is clear that you have met one or not.

<JF> +1 - Techniques are not complete, and so linking testability to techniques is limiting IMHO

<jon_avila> Agree with David

David: In term of the success criteria being testable, we do have techniques. The testability of the techniques is independent of the testability of the SC. If you test the techniques and pass it and if it is sufficient, then the SC passes.

AWK: Where there is testability. If you imagine we had a success criteria that said "Your web content must be interesting." it is untestable. We have no way to tell if content is interesting to everybody.

We need to make sure we are talking about more specific information to ensure we have a title or page. That is more the level we are thinking of.

JF: Techniques are not complete.

Just becuase you haven't passed all the techniques doesn't mean you've failed the criteria.

Katie: There should be at least one technique completed as part of this situation.

<alastairc> It needs to at least be possible to create a technique / test for FPWD, even if we don't have one specified yet.

Katie: Either it is able to be explained and tested broadly or it has a specific technique that shows how to test for it. Again, it can still be a point that falls in the At Risk Section.

Lisa: I think a lot of people are speaking in abstract. When you try to write the SC which we did and then we're told to move it to techniques. Then you've got something that is hard to test because the testable part is in techniques.

<bruce_bailey> we have SC that include lists

<bruce_bailey> SC 1.1.1 is a list

Lisa: Its a balance between avoiding a long list of items and ensuring it is specific enough to be testable.

Numbering & updating SC (https://www.w3.org/WAI/GL/wiki/WCAG_2.1_SC_Numbering)

AWK: I would like ot move discussion of this to the list so we can talk about it there. Please engage there and lets work to common agreement. I think that will help us as we go through the process of reviewing the success criteria.
... If you are an SC manager and wondering where to put it, this is that conversation. https://www.w3.org/WAI/GL/wiki/WCAG_2.1_SC_Numbering

David: I hadn't thought about the tools. I was thinking about the legislation example. New language would be backwards compatible and new items could be added. I'd be interesting in hearing what Wilco or someone else working on the tools thinks.

gowerm: I'm thinking about this from the pt of view of someone testing and reporting. Stick to current numbering scheme and add to it. If we edit exisitng criteria, those reporting will need to create an alternate numbering for all criteria that were modified.

I would need to test the old 2.1 and the new 2.1 so would need a 2.1a and 2.1b etc. The less we change, the easier it will be for folks who have to test against both 2.0 and 2.1

<jon_avila> Agree with Mike

Proposal 2 is close but it may be worth noting what existing numbers have been modified.

Alaister: I think Model 2 is good and I would default to additive. Where there is significant overlap, we may consider changes on a case by case basis. 1.4.4 is at the top of my list for conversation. It should be completely backwards compatible but I think it could be cleaner for new people coming to it if it is a replacement instead of an addition.

<Glenda> +1 to “do not change any existing WCAG 2.0 SC”

Katie: I thought we had this clearly defined that we were not going to change any existing criteria. I Thought we'd agreed that we were not modifying anything existing but not going to make changes. Am I wrong about that and we did not make that decision?

AWK: We had generally decided we would not make changes to exisitng SC but that we would revisit it.

<gowerm> +1

Katie: I think we should leave what's there and add new cases to it so they don't have to change it.

<JF> https://www.w3.org/WAI/GL/wiki/WCAG_2.1_SC_Numbering

JF: We did a lot of research on this back in the fall. We had feedback from government agency that if we were going to make changes, we should introduce a 1.1.3a for changes/additions. This was based on the legislative model. I agree that we do not want to rewrite exisitng criteria but we do need to expand some (color contrast for example).

<gowerm> +1

<Zakim> bruce_bailey, you wanted to thank AWK for listing out the permutations, i like model 2, but maybe 1.3.1 would be an exception

<kirkwood> +1 to alpha

Bruce: Thank you for listing out the choices. They are helpful. I like the model 2 the most. I don't want to see the numbers changing. The one exception I would make is 1.3.1 which seems more of a kitchen sink. I'd like to collect all the data table items into one piece of 1.3.1

Lisa: Couple of issues. There is a contradiction between the backward compatibility. If you are extending the requirement you have to add that to the existing one. We may need to rewrite. You have a problem also when you want to make something a different level. Move something from AAA to AA.

<alastairc> +1 that requirement 8 (Avoid creating a requirement for something that is already required) conflicts with the additive model.

Wayne: We need a very clear reference back to what it said.

<AWK> "avoid" is different from "never do"

<Ryladog> As long as it falls under the relevant GL it should b OK

Whether we use a or not it needs to be clear what is appended.

<gowerm> Changing levelling should be fine

AWK: This feels a bit easier than the previous topic. It feels like we've got general agreement that we do not want to modify criteria in line though we may want to change the leveling of it. We need to think about how we deal with that. We will also discuss this one on the list so more people can weigh in. We are done for today. We will meet again 3 January. Have a good holiday season.

trackbot, end meeting

<laura> bye all.

This is scribe.perl Revision: 1.148  of Date: 2016/10/11 12:55:14  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: RRSAgent_Text_Format (score 1.00)

Succeeded: s/JF/Joshue108/
FAILED: s/JF: We did work on making these as tight as possible/ Joshue: We did work on making these as tight as possible/
Succeeded: s/Lida/Lisa/
Succeeded: s/Lisa's point/Lisa and Mike's points about repeatable testing and/
Succeeded: s/Kaite/Katie/
Found Scribe: Rachael
Inferring ScribeNick: Rachael
Default Present: AWK, Rachael, Laura, MikeGower, Kirkwood, David-macdonald, alastairc, Bruce_Bailey, Joshue108, jeanne, Lisa, Seeman, Jf, steverep, MoeKraft, Glenda, Katie_Haritos-Shea

WARNING: Replacing previous Present list. (Old list: (no, one), Rachael, Laura, MikeGower, Kirkwood, David-macdonald, alastairc)
Use 'Present+ ... ' if you meant to add people without replacing the list,
such as: <dbooth> Present+ AWK, Rachael, Laura, MikeGower, Kirkwood, David-macdonald, alastairc

Present: AWK Rachael Laura MikeGower Kirkwood David-macdonald alastairc Bruce_Bailey Joshue108 jeanne Lisa Seeman Jf steverep MoeKraft Glenda Katie_Haritos-Shea
Regrets: Makoto ShawnL Greg_Lowney Srinivasu Wilco
Found Date: 20 Dec 2016
Guessing minutes URL: http://www.w3.org/2016/12/20-wai-wcag-minutes.html
People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.