<Chuck> https://docs.google.com/spreadsheets/d/1iCJfyMtcsSq7GHmwnc4aTNguadRfGDa0H8FBZMaJpcQ/edit#gid=1825600441

<Chuck> https://docs.google.com/spreadsheets/d/1iCJfyMtcsSq7GHmwnc4aTNguadRfGDa0H8FBZMaJpcQ/edit#gid=633158340

<Chuck> https://docs.google.com/drawings/d/1nRhQ2DWCGe8imhZgV8972IWSRSwCtmKaOnZOB6qm3gk/edit

<mbgower> scribe: mbgower

Alastairc: Are there any new members or topics?

Announcements

Bruce: Is it preferable to +1 on the list?

Alastairc: CFC responses are better in public

New members and topics

WCAG 2: Candidate Recommendation Check In (5 minutes)

WCAG 3: Discuss Test Subgroup proposal for new test type terminology

Alastair: This was talked through in the Friday Silver meeting
... I'm hoping Wilco can provide a summary. My understanding is there a few things we're hoping to get progress or agreement on .

Wilco: I was asked to keep this short. There is a recording of the presentation.

<Rachael> Slides at: https://docs.google.com/presentation/d/1KhuTdTemRjjJIFViftKYPXc1nKhcJ4LqrQum0qOJLBI/edit#slide=id.p

Wilco: We've come up with proposals for new test type terms.
... We are proposing computational testing instead of unconditional
... You can often test with tools. It's the type of test that doesn't need expertise. A qualitative test does require judgement.
... an image having an alt or text meeting contrast is computational
... 'Does A describe B' is qualitative.
... For "Conventional" testing, we came up with 2 types: adaptive testings and extensible testing

Adaptive testing changes based on user needs.

Extensible testing comes from the idea that WCAG doesn't need to prescribe everything.

What of instead of prescribing use this method or that method, say 'use any of these'?

Wilco: If you came up with a better method, you could extend it.
... For 'procedural' we are leaning towards 'protocol' but still considering
... Test type: computational and qualitative

<Zakim> bruce_bailey, you wanted to ask if five categories independent from each other?

Wilco: Requirement types: prescriptive, adaptive, extensible, procedural
... We are not sure if all these requirement types will all have computational and qualitative.

Bruce: Great to see so much thinking going into this.

<Zakim> alastairc, you wanted to ask about 'adaptive', and whether we would start with the 1st two categories as exploratory, and the others as placeholder?

Alastair: I have not had a chance to watch the video yet. Adaptive does seem like a potentially misleading term.
... My suggestion is to start off with the test types, we are familiar with. We know how to handle those. The requirement types, as you have them presented, need to have stuff created for them to establish as categoires.

Gregg: Thanks for doing this work. Terminology and naming is critical to peoples' understanding.
... 'Computation' is brilliant.

<bruce_bailey> +1 to AC comment that "adaptive" label seems like it will be conflated with other a11y concepts

Gregg: 'Qualitative' has a particular meaning in science.

<jon_avila> The term computation could imply computer based test - automatics

<Rachael> +1 to concerns about adaptive and qualitative.

Gregg: Doing qualitative research ranges from opinion (what I'd like to be versus what it is, etc). There can be rank ordering.
... It sounds like you are saying 'this is not objective'
... We may want to think of a different term, or divide into 2 categories.
... Conditional and Unconditional... You can have conditional computation.
... I'd get rid of these from the test types
... Adaptive sounds active. It sounds like the test changes. WHen you are testing, you have no idea who the user is.
... I don't know that we would change the test.
... Extensible.... I think that's the one where you don't say what you're going to use.
... You can't say 'high contrast using what tool you choose' because someone could choose a low contrast formula

<AWK> +AWK

Gregg: Procedural.... I was wondering if we're actually testing the procedure or just looking for evidence or someone affirming they did a protocol, then the only thing you can test is the affirmation that they did it. So maybe we should call it an 'affirmation' test

<Zakim> Rachael, you wanted to ask about requirement types

Rachael: Can you explain the though that went into the grouping?

<alastairc> Suggest "evaluations"

Wilco: We haven't thought too much about this group except that they aren't test types.
... Maybe they should be thought of as methods.

<Rachael> +1 to the idea of types of methods.

Wilco: THey are ways to judge if an outcome has been met.
... We haven't gotten this far.

Alastair: We may have to rename methods we have elsewhere.

Gregg: Could you explain prescriptive again?

<Zakim> GreggVan, you wanted to ask you to explain "prescriptive" again

Wilco: It's new in the sense of the term, but WCAG is full of them. You need to have a certain contrast.
... Adaptive could cover the audience for a particular page. I'm writing this for children, so I follow different guidance.

Gregg: I'm not sure you can restrict a requirement to a single group.

<jon_avila> Examples might be scientist such as a journal article.

Wilco: This doesn't need to exclude someone. But someone may need high contrast and others may struggle with that and need lower contrast.

Gregg: So are we going to determine who will be looking at the pages?

Shadi: I got on queue for another question, but one of the examples that Raine brought forward is which language you're writing a page for.
... Wilco, maybe this is more on the process side... Since we're trying to put in a PR by Tuesday but still having discussions on terminology, we are planning to link back to a reference
... I hope that will be sufficient?

<Zakim> Rachael, you wanted to talk to goal for today

Rachael: I think that's an excellent idea. Bring up the questions and put them in the PR.
... The goal for the conversation today is to pick some terms we can live with for the next few months or at least TPAC.
... That doesn't mean we can't change it. BUt let's say the same thing for the next conversations.

Chuck: I've appreciated the conversation. I'm okay moving forward as long as comments are captured.

<sarahhorton> scribe: sarahhorton

<Zakim> alastairc, you wanted to ask about adaptive crossing over with accessibility supported?

alastairc: Adaptive, seems to cross over with accessibility supported, check how page reacts with adaptations, would that be built into other guidelines?
... extension of other tests? Confused about adaptive
... examples welcome!

<Zakim> GreggVan, you wanted to ask if types are exclusive. Can something be prescriptive and adaptive? (like the example you gave of if you have low and high low is at least

GreggVan: Are requirement types exclusive? Sort into categories or tags?
... also request, post to list current terms, definitions, examples to help understand
... people can propose terms in response
... concepts are key, post them, would be useful

<shadi> https://docs.google.com/document/d/1crGP5tfsVj_CTgWsImrLqPHsNvrIH9m5bBVmW7Vt8W4/edit#

Wilco: Adaptive, come up with examples, high contrast/low contrast mode, writing in English, write in English plain language, guidance different in other language (e.g., Hebrew)
... pull request will go into survey
... currently thinking of them as categories rather than tags

<Zakim> mbgower, you wanted to say totally find with Test types, but wonder about requirements/methods. Do we need to resolve that part today?

alastairc: Catch up on Friday meeting, linked above

mbgower: Likes idea of defining requirement types, also concern with adaptive

<Zakim> janina, you wanted to suggest adaptive might be UA

mbgower: thanks for work, very useful

janina: Agree with adaptive, think of source/page side that gets transmitted, bring in user agent, will have new ways to adapt

<bruce_bailey> antonym for prescriptive.. need to think about that

janina: CSS that overwrite author-provided CSS, building iterative, e.g., less contrast, even less, share with all my devices
... conversation happening at TPAC, if fruitful, adaptive makes sense

<alastairc> Poll: Can you tolerate the Test Type terms for the near future, "Computational" and "Qualitative"

alastairc: Happy with test types, less happy with requirement types

<mbgower> +1

<Ben_Tillyer> +1

<Chuck> +1

<joweismantel> +1

<JakeAbma> +1

<maryjom> +1

<Rachael> +1 to computational, 0 to qualitative

<bruce_bailey> +1 to computational and qualitative

shadi: Requirement types, put together quickly, capturing ideas, are overlaps, needs cleanup

<janina> +1 to terms for moving forward for now

shadi: finding something to represent ideas to use as handles

<jon_avila> +0 this is very short notice to make a decision that is going to be published

<GreggVan> +1 to computational (but computational/logical migh be better) 0 to qualitative if you remove the conditional unconditional subtags

<Rachael> Jon, this won't be published. The PR is later. This is just for our internal use for cconsistent conversation

<mbgower> how about "alternatives"?

shadi: yes, lots of questions, but can we go ahead with descriptions and examples, as something to work with

alastairc: Requirement types might take more familiarization, more people on call, need to catch up
... are there 4, thought there were 2

Wilco: Added prescriptive, tests with only 1 way to do it
... extensible came out of convention, two types of conventional tests

<Zakim> bruce_bailey, you wanted to ask if we can keep Convention and/or Using Testing in the mix?

bruce_bailey: Face to face, need terms that are clear and intuitive so people know what they're talking about

<Zakim> GreggVan, you wanted to say " computational/logical?" and are conditional unconditional being removed (old terms)?

GreggVan: Computational, logical, thinking of logical in computational?

Wilco: Think so

GreggVan: Conditional, unconditional, were calling them that before, not subtags, noting same as used before
... TPAC, should do best effort to get terminology, adaptive with terms behind it, otherwise argument about term rather than concept
... candidate names, open to better candidate names, but focus on concept

<alastairc> q/

<alastairc> q/

shadi: Do have list for every item, other suggestions, had to pick one for now

<Rachael> Would it be helpful to move back a slide?

<bruce_bailey> for face-to-face might just have Type 1, Type 2, Type 3, etc.

GreggVan: On doc, stuff that's crossed out, good to clean up for review

shadi: Slide is easier, with table

<Zakim> mbgower, you wanted to say do we need to settle on requirement terminology?

mbgower: Sense that figuring out requirements fits into method, idea that two test types, compute or qual
... protocol, check reporting of a thing, others more about reporting

<jon_avila> Computation and qualitative are much more clear to me than the previous terms - so glad to see that progress.

Wilco: Procedural, check report but can't watch do procedure
... others could be either test type

<jon_avila> I don't think people will make their procedures public.

Juanita: Not affirmation, more that that, read procedure, test, judge outcome

<bruce_bailey> My previous experience w/ face-to-face was not arguing over labels -- but people having different idea of what was meant as work was done -- and not realizing that.

<Zakim> alastairc, you wanted to ask whether the requirement types go "under" guidelines?

<mbgower> Thanks for the clarification, Jaunita. So there is a potential for a 3rd test type.

alastairc: Test types under guideline, method, test; requirements under guidelines, others not under guidelines

Wilco: Don't know yet, not explored fully

Rachael: Had divided protocols into procedural under outcome and protocols in conformance section
... initial idea of 4 tests, subgroup says no 2 types but 4 ways of looking at them
... for TPAC, which do we want to use?

alastairc: So many clashing terms, next poll
... low on levels of maturity, not much content in requirement types

Wilco: Wait until pull request is ready
... good feedback, refine, and PR for survey

alastairc: Other suggestions?

GreggVan: Will share comments with list

WCAG 3: Review WCAG 3 architecture and Vocabulary

<Rachael> https://docs.google.com/presentation/d/1BmDMiiGai1-jScOn_KUozGyEd_Kmmnd0tgj1yG5B2yg/edit#slide=id.p

Rachael: Thanks to test group, needed shared conversation about terminology
... where are we as group, what do we have, shared understanding, topics we won't revisit at TPAC but rather build on
... concepts discussed and settled enough for conversation
... call out anything that doesn't have shared understanding

<Rachael> https://www.w3.org/WAI/GL/wiki/Meetings/TPAC_2022#AG_WG_Meeting_at_TPAC_2022

Rachael: hope for TPAC, subgroups doing pre-conversations, information gathering
... put together decisions to explore with larger group later in week
... [walks through slides]
... recommend reading through drafts, supporting docs before TPAC
... Guidelines, outcomes (testable criteria), methods (detailed info on meeting outcome)
... [reviews slides]
... [reviews captions example]
... Methods include intro, background, tests, glossary
... how-tos are associated with guideline level
... architecture hasn't shifted, has been stable

GreggVan: Helpful, looks like Guidelines, SC (looks like outcome), Techniques (Methods)
... what about procedures, protocols, past work fits nicely, but how to get things not outcome oriented, testable, into model
... looks like same as previous model, procedural can't be an outcome
... user testing doesn't mean there was a change, how to handle not testable, not outcome

Rachael: Potentially methods, different methods would have different ways of being written, different ways of meeting

GreggVan: Suggest Criteria rather than Outcome, outcome only one type of thing, want to include other things
... think about name, doing different things

<Zakim> mbgower, you wanted to say I still find the actual implementation a little disorienting

<AndySomers> "Goal" instead of "outcome"

mbgower: Hard to work on something, continues to find disorienting, don't think only one, having problems, not sure terminology, implementation
... designers doing research on implementation, have had same design, want to tweak to improve to assist people getting through content
... coming around to outcomes, methods, might need to revisit with test terminology

<Zakim> alastairc, you wanted to say I think I've worked out why I find the structure odd (not a criticism)

mbgower: issue is with implementation

<Zakim> MichaelC, you wanted to say separate suggestions from update

alastairc: Don't think need to change anything, realize have pages that look like what we have now, how to and method that look like this

<jon_avila> I agree Alastair - when I was viewing a method I was not clear which outcome and guideline it was in.

MichaelC: Focus, update presentation, suggestions can be part of TPAC, download information then think about it

Rachael: Revisit, confusion points, okay to continue as basis
... research request, make more usable, language simple where possible, summaries, filter and sort by tags
... how-to for beginners, project managers, non-tech users
... examples with links
... design principles, requirements, things guidelines should do not not requirements to test against
... [reads slide]
... [reads requirements slide]
... will be discussing the requirements and decisions

mbgower: Motivation is prescriptive, have we agreed on scoring?

Rachael: Don't see it prescribes scoring, but we do need to revisit language

<AWK> I was going to say that whether people find a scoring system motivating is hard to evaluate as a metric

<AWK> "Scaled scoring"

mbgower: Other ways to motivate, beyond scoring

Rachael: Makes sense, capture and circle back for meeting, can add as issue to review

<bruce_bailey> +1 to what i heard MG say about scoring is one example of how to motivate

AWK: Decide about whether motivation or scoring, hard to measure whether people are motivated

<AndySomers> "incentive" instead of "motivation" ??

Rachael: Other concerns to capture for TPAC

<Jennie> Motivation could also include additions like the how-tos

GreggVan: Make sure terminology is consistent and don't use same word to mean different things (outcomes) and use words that are known
... people are grappling with terminology

alastairc: Yes, need consistency for published version, but may have to juggle in between
... messy, confusing, define things internally

jon_avila: Scope, Guidelines, is that for normative or informative?

Rachael: Haven't fully discussed, no final decisions
... [reads testing scope slide]
... terms using at TPAC
... [reads types of tests slide]
... will be refining through survey, capture in slides
... [reads conformance levels]
... have discussed, but haven't determine what makes level

<Zakim> mbgower, you wanted to say I think the conformance section should contain some info on reporting, and should bronze, silver, gold still be listed?

Rachael: suggestions, haven't made decisions, can capture considerations

mbgower: Mention reporting, to investigate, thought we had agreed not to talk about levels

<AWK> +1 to no color-specific level names :)

Rachael: Removed definitions, didn't remove concept
... where would you capture reporting

<AndySomers> Levels as concept are very important, particularly for complex subjects such as text contrast

mbgower: Possible approaches, or maybe different slide, guidance on ways that reporting can be considered
... not in 2x, implied with conformance levels
... Protocols infers reporting

<Zakim> GreggVan, you wanted to say +1 to keeping bronze silver gold I think that is our best path to something different than WCAG 2.x and still be adopted

GreggVan: Keep levels, best path to something different that could be adopted

Rachael: [reads conformance approaches slide]
... explore the approaches, proposals
... different ideas that have come about
... share ideas, send email to chairs

<Zakim> bruce_bailey, you wanted to ask about program maturity reporting

bruce_bailey: Maturity? Is this here?

Rachael: Not a solution to conformance, but open to discussion

<alastairc> I think maturity models would build on conformance, not be part of it.

janina: Not conformance approach, way of tracking, covers more than web content, expecting public review next Tues

<Jaunita_george> We might want to consider evaluation of core user flows (making a purchase, finding information, etc.) as a method of conformance --can the user complete critical paths or use the site as intended?

Rachael: Maturity levels for working drafts, placeholder, exploratory, etc, using and trying out
... slides go into concepts, definitions, process
... review and see how we're managing maturity in document

<Jennie> Suggestion: add a link to the wiki page to this slide.

<bruce_bailey> scribe: bruce_bailey

Rachael: Any questions or comments for now?
... What else do we need for TPAC?

List of questions/decision we need to make at TPAC

<Rachael> Decisions to make at TPAC https://docs.google.com/spreadsheets/d/1_yqsQyGe4fJ6y1XaU14Cuhupm8HZqykhPTGBpgugzNo/edit#gid=0

Rachael: link is for spreadsheet list of questions for TPAC in order to start making conformance decisions.
... Q14 is what MikeG asked about for example.

<Rachael> What is our tolerance for variability of test result?

<Rachael> What types of test will we accept?

How much interrater reliability? 2.x is 8/10 SME

<Rachael> What approach will we use to measure?

<Rachael> What approach will we use to measure?

<Rachael> How will we include ATAG?

<Rachael> How will we include UAAG?

scribe: points , percentage , severness , per earlier conversation

<Rachael> How will we ensure equity?

<Rachael> Should WCAG 3 allow for tolerance?

scribe: We equity subgroup working on that question.

<Rachael> What does "good enough" conformance?

<Rachael> Should WCAG 3 allow exceptions?

scribe: Can we accept less than 100% ?

<Rachael> Should WCAG 3 support issue severity?

<Rachael> What scopes will we test? (maybe already answered)

<Rachael> Will we include accessibility supported?

<Rachael> Should WCAG 3 include reporting requirements?

scribe: exceptions question came out of third-party discussions -- might be out of scope -- but we need conversations

<Rachael> How do we make it easier for people with disabilities to learn about the accessibility of web sites and web apps? (maybe more writing process than conformance)

<Rachael> If metadata outside the content itself is needed to validate a test, how is that reported?

scribe: ease of contribution tied to requirements
... these questions are recurrent themes from different conversations.

Rachael: What other questions / discussion points can people suggest?

GreggV: WRT to making it easier for PWD to learn, what is context?

Rachael: How do we make it easier for people to use what we produce?

janina: i didn't hear about work about conformance-options task force discussions -- how to acknowledge where soluntions are not yet known...
... where do be make sugestions for regulators to scope or qualify or implement ?

Rachael reiterates.

jon_avila: what is current state of functional needs and categories ? Ties to accessibility support , but how do we fall back to FPC ?

Rachael: 1st agenda item for TPAC is with APA on functional needs.
... Please continue to think about and make suggestions to chairs or on list.

Jennie: Is this a good opportunity to explain other W3C documents? When i was starting, it was very confusing between informative and normative.
... It ties into the complications with structure. That a reader might veer off and not be able to get back to main doc to know which is normative or not...
... people get lost on W3C website.

<kirkwood> +1 to Jennie

Rachael: Hearing no more questions, next week we plan for presentations and conversation on PR and sub groups.
... Again this is hard problem, thank you all for listening and contributing.

This is scribe.perl Revision VERSION of 2020-12-31
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/Face to face, need terms that are clear/Face to face, need terms that are clear and intuitive/
Succeeded: s/againist/against/
Default Present: alastairc, Rachael, Chuck, Jaunita_george, shadi, JakeAbma, ToddL, Ben_Tillyer, jon_avila, janina, bruce_bailey, Francis_Storr, mbgower, ThompsonS, MichaelC, Nicaise, Wilco, GreggVan, kirkwood, Laura_Carlson, julierawe, Glenda, Makoto, sarahhorton, maryjom, Detlev, Jennie, Jen_G, AWK, Katie_Haritos-Shea, joweismantel, AndySomers
Present: alastairc, Rachael, Chuck, Jaunita_george, shadi, JakeAbma, ToddL, Ben_Tillyer, jon_avila, janina, bruce_bailey, Francis_Storr, mbgower, ThompsonS, MichaelC, Nicaise, Wilco, GreggVan, kirkwood, Laura_Carlson, julierawe, Glenda, Makoto, sarahhorton, maryjom, Detlev, Jennie, Jen_G, Katie_Haritos-Shea, joweismantel, AndySomers
Found Scribe: mbgower
Inferring ScribeNick: mbgower
Found Scribe: sarahhorton
Inferring ScribeNick: sarahhorton
Found Scribe: bruce_bailey
Inferring ScribeNick: bruce_bailey
Scribes: mbgower, sarahhorton, bruce_bailey
ScribeNicks: mbgower, sarahhorton, bruce_bailey

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)