documentreview/index.html at c9d53105f63eab79af257c3eb4c85bf965b7aaf8 · w3c/documentreview · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
<!DOCTYPE html>
<html dir="ltr" lang="en-US">

<head>
<meta charset="utf-8">
<title>Document Review</title>
<script src="toc.js"> </script>
<link rel="stylesheet" href="https://www.w3.org/Guide/assets/main.css">
<link rel="stylesheet" href="style.css" />
</head>

<body>
<h1>How to do Wide Review</h1>
<p>Getting <i class="term">early</i> and <i class="term">wide</i> review of a document is very important, yet in practice it can be challenging.  This document provides some best practices for getting document review; it does not define explicit mandatory steps.</p>
<p>This page is linked to from <a rel="nofollow" class="external text" href="https://www.w3.org/Guide/">The Guide</a>. See also the <a href="https://www.w3.org/Consortium/Process/#wide-review">Wide Review</a> section in the W3C Process document.</p>
<p><strong>Feedback on this document is welcome, preferably by <a href="https://github.com/w3c/documentreview/issues">raising an issue</a> or a pull request.</strong></p>
<div id="toc">
<h2 class="notoc">Contents</h2>
<div id="toc_placeholder"></div>
</div>


<section id="who_to_ask_for_review">
<h2>Who to ask for review?</h2>

<p>Much of this document focuses on how and when to conduct horizontal reviews, but they are only a subset of a full wide review, which must also include other stakeholders including Web developers, technology providers and implementers not active in the Working Group, external groups and standards organizations working on related areas, etc. Here is a list of suggestions:</p>
<ul>
<li> Groups listed in the WG's charter, especially those who manage dependencies.</li>
<li> Groups jointly responsible for a particular document (if any)  (duh!).</li>
<li> All horizontal groups (listed below).  If it appears that one of those is not relevant (and is not listed in your charter), talk to them informally to confirm that.</li>
<li> Other groups, at your discretion, even if not in the WG charter, including other W3C groups, external organizations and SSOs, implementers, application developers, etc.</li>
<li>The general public. Consider using blog posts, social media, or other ways of alerting the public to your document and requesting feedback.</li>
</ul>
<p>Use <a rel="nofollow" class="external text" href="mailto:public-review-announce@w3.org">public-review-announce@w3.org</a> for general announcements regarding wide and horizontal reviews. <em>However, note that sending an email to this list alone is not sufficient to trigger horizontal reviews. You will still need to formally request review from the Horizontal Groups, as described below.</em></p>
</section>


<section id="how_to_get_horizontal_review">
<h2>How to get horizontal review</h2>
<p>When you have published a First Public Working Draft, you should work through available "self-review" materials and request review of your results in at least the following areas. <em>Long enough</em> before you request a transition to CR, you should do the same again, identifying substantive specification changes since the first review. </p><p>The meaning of "Long enough" depends on how many changes there are, how clearly you have explained them, and how much discussion is needed to resolve issues. Pointing to 14 concise points for a small spec means a short time if they are simple fixes, pointing to 900 diffs from commits and hoping people understand them in a 300 page spec means it will take a <strong>long</strong> time to get review, and potentially a long time to also discuss and agree on how to solve the issues. If you have effectively identified issues for review during development and received feedback on them, the review time will probably be shorter. Horizontal review groups sometimes get bogged down; planning in advance is useful.</p>
<dl>
<dt>Accessibility</dt>
<dd>
  <span class="step">Work through <a rel="nofollow" class="external text" href="https://w3c.github.io/apa/fast/checklist.html">this questionnaire</a></span> then
  <span class="step"><a rel="nofollow" class="external text" href="https://github.com/w3c/a11y-request/issues/new/choose">request a review via GitHub</a> from <a rel="nofollow" class="external text" href="https://www.w3.org/WAI/APA/">APA</a></span>
<details>
<summary>Show useful links</summary>
<ul><li> groups
<ul><li> <a rel="nofollow" class="external text" href="https://www.w3.org/WAI/APA/">Accessible Platform Architectures Working Group</a>; <a rel="nofollow" class="external text" href="https://lists.w3.org/Archives/Public/public-apa/">public-apa</a>; <a rel="nofollow" class="external text" href="https://www.w3.org/WAI/APA/wiki/Spec_Review">APA Spec Review tracking</a></li>
<li> <a rel="nofollow" class="external text" href="https://www.w3.org/WAI/about/groups/waiig/">Web Accessibility Interest Group</a>; <a rel="nofollow" class="external text" href="https://lists.w3.org/Archives/Public/w3c-wai-ig/">w3c-wai-ig</a></li></ul></li>
<li> links
<ul><li> <a rel="nofollow" class="external text" href="https://w3c.github.io/apa/fast/checklist.html">Framework for Accessibility in the Specification of Technologies Checklist; APA WG</a></li>
<li> <a rel="nofollow" class="external text" href="https://www.w3.org/TR/media-accessibility-reqs/">Media Accessibility User Requirements; PFWG</a></li></ul></li></ul>
</details></dd>


<dt>Architecture</dt>
<dd>
  <span class="step">Ask the <a rel="nofollow" class="external text" href="https://www.w3.org/2001/tag/">TAG</a> for review; see <a rel="nofollow" class="external text" href="https://tag.w3.org/workmode/">how to work with the TAG</a></span>.
  <span class="step">If you are developing javascript APIs you may also want to ask <a rel="nofollow" class="external text" href="mailto:public-script-coord@w3.org">public-script-coord@w3.org</a>, a technical discussion list shared by W3C and ECMA's TC 39</span>.
<details>
<summary>Show useful links</summary>
<ul><li> groups
<ul><li> <a rel="nofollow" class="external text" href="https://www.w3.org/2001/tag/">Technical Architecture Group (TAG)</a>; <a rel="nofollow" class="external text" href="https://lists.w3.org/Archives/Public/www-tag/">www-tag</a></li></ul></li>
<li> links
<ul><li> <a rel="nofollow" class="external text" href="https://www.w3.org/2001/tag/doc/webcomponents-design-guidelines/">Guidelines for creating web platform compatible components</a></li>
<li> <a rel="nofollow" class="external text" href="https://www.w3.org/2001/tag/doc/promises-guide">Writing Promise-Using Specifications</a></li>
<li> <a rel="nofollow" class="external text" href="https://w3ctag.github.io/design-principles/">Client-side API Design Principles</a></li>
<li> <a rel="nofollow" class="external text" href="https://www.w3.org/2001/tag/doc/capability-urls/">Good Practices for Capability URLs</a></li>
<li> <a rel="nofollow" class="external text" href="https://github.com/w3ctag/design-reviews">TAG Repository for reviews</a></li></ul></li></ul>
</details>
</dd>


<dt>Internationalisation</dt>
<dd>
  <span class="step">Read the <a rel="nofollow" class="external text" href="https://www.w3.org/International/review-request">Request a review</a> page</span>, then
  <span class="step">work through the <a rel="nofollow" class="external text" href="https://w3c.github.io/i18n-drafts/techniques/shortchecklist">Short Checklist</a></span>, then
  <span class="step"><a rel="nofollow" class="external text" href="https://github.com/w3c/i18n-request/issues/new/choose">request a review via GitHub</a></span>.
<details>
<summary>Show useful links</summary>
<ul><li> groups
<ul><li><a rel="nofollow" class="external text" href="https://www.w3.org/International/">Internationalization Working Group</a>; <a rel="nofollow" class="external text" href="https://lists.w3.org/Archives/Public/www-international/">www-international</a> Reviews by Internationalization generally also involve the <a rel="nofollow" class="external text" href="https://www.w3.org/International/ig/">Interest Group</a>, but are arranged through the WG.</li></ul></li>
<li> links
<ul><li><a rel="nofollow" class="external text" href="https://www.w3.org/International/review-request">Request a review</a></li>
<li><a rel="nofollow" class="external text" href="https://w3c.github.io/i18n-drafts/techniques/shortchecklist">Self-Review Questionnaire</a>.</li>
<li><a rel="nofollow" class="external text" href="https://www.w3.org/International/techniques/developing-specs">Detailed, expanding-collapsing checklist</a>, generated from <a rel="nofollow" class="external text" href="https://w3c.github.io/bp-i18n-specdev/"><cite>Internationalization Best Practices for Spec Developers</cite></a></li>
<li> <a rel="nofollow" class="external text" href="https://www.w3.org/International/reviews/projReview.html">A brief overview of the review process</a> (with pictures)</li>
<li> The <a rel="nofollow" class="external text" href="https://github.com/w3c/i18n-request/projects/1">Review Radar</a> shows the status of open reviews.</li></ul></li></ul>
</details>
</dd>


<dt>Privacy</dt>
<dd>
  <span class="step">Write a "Privacy Considerations" section for your document, taking into account the <a rel="nofollow" class="external text" href="https://w3ctag.github.io/security-questionnaire/">Self-Review Questionnaire: Security and Privacy</a>, <a rel="nofollow" class="external text" href="https://w3c.github.io/fingerprinting-guidance/">Mitigating Browser Fingerprinting in Web Specifications</a>, and <a rel="nofollow" class="external text" href="https://tools.ietf.org/html/rfc6973">RFC6973</a></span> then
  <span class="step"><a rel="nofollow" class="external text" href="https://github.com/w3cping/privacy-request/issues/new/choose">request a review via GitHub</a> from the <a rel="nofollow" class="external text" href="https://www.w3.org/Privacy/">Privacy Interest Group</a></span>.
<details>
<summary>Show useful links</summary>
<ul><li>groups
<ul><li> <a rel="nofollow" class="external text" href="https://www.w3.org/Privacy/">Privacy Interest Group</a>; <a rel="nofollow" class="external text" href="https://lists.w3.org/Archives/Public/public-privacy/">public-privacy</a></li></ul></li>
<li> links
<ul><li> <a rel="nofollow" class="external text" href="https://w3ctag.github.io/security-questionnaire/"><cite>Self-Review Questionnaire: Security and Privacy</cite>, published by the TAG and PING</a></li>
<li> <a rel="nofollow" class="external text" href="https://w3c.github.io/fingerprinting-guidance/">Mitigating Browser Fingerprinting in Web Specifications</a></li>
  <li> <a rel="nofollow" class="external text" href="https://tools.ietf.org/html/rfc6973">Privacy Considerations for Internet Protocols (RFC6973)</a>, particularly <a rel="nofollow" class="external text" href="https://tools.ietf.org/html/rfc6973#section-7">Section 7</a></li>
<li> <a rel="nofollow" class="external text" href="https://tools.ietf.org/html/rfc3552">Guidelines for Writing RFC Text on Security Considerations (RFC3552)</a>, particularly <a rel="nofollow" class="external text" href="https://tools.ietf.org/html/rfc3552#section-5">Section 5</a></li>

</ul></li></ul>
</details>
</dd>


<dt>Security</dt>
<dd>
  <span class="step">Write a "Security Considerations" section for your document, taking into account the <a rel="nofollow" class="external text" href="https://w3ctag.github.io/security-questionnaire/">Self-Review Questionnaire: Security and Privacy</a></span>, then
  <span class="step"><a rel="nofollow" class="external text" href="https://github.com/w3c/security-request/issues/new/choose">request a review via GitHub</a></span>

<details>
<summary>Show useful links</summary>
<ul><li> groups
<ul><li> None</li></ul></li>
<li> links
<ul><li> <a rel="nofollow" class="external text" href="https://w3ctag.github.io/security-questionnaire/"><cite>Self-Review Questionnaire: Security and Privacy</cite>, published by the TAG and PING</a></li>
<li> <a rel="nofollow" class="external text" href="https://w3c.github.io/fingerprinting-guidance/">Mitigating Browser Fingerprinting in Web Specifications</a></li>
<li> <a rel="nofollow" class="external text" href="https://tools.ietf.org/html/rfc3552">Guidelines for Writing RFC Text on Security Considerations (RFC3552)</a>, particularly <a rel="nofollow" class="external text" href="https://tools.ietf.org/html/rfc3552#section-5">Section 5</a></li></ul></li></ul>
</details>
</dd>
</dl>

<details id="githubissue" hidden>
  <summary>Generate a meta-issue to track horizontal review steps in a GitHub repository</summary>
  <p>You may find it useful to create an issue in the GitHub repository of your spec to track your progress. Add the name of your GitHub repository to the field below and hit the "Create GitHub issue" button. This opens the "new issue" form in your repository, and pre-fills the body with horizontal review steps as a list of checkboxes.</p>
  <p>
    <label for="repository">GitHub repository where issue should be created (format: <code>owner/repo</code>):</label>
    <br/><input type="text" name="repository" id="repository" placeholder="Repo name, e.g. w3c/documentreview">
    <br/><button>Create GitHub issue</button>
  </p>
  <p>Note: You will be able to edit the issue's title and body before it gets created.</p>
</details>

<p>You should familiarize yourself with the rest of this document. This section is just a quick reminder for when you are in the middle of doing the work.</p>
<p>Recognize that horizontal review groups may be resource limited and may only be able to do one review or may have difficulty scheduling your review quickly.  Give them as much time as you can, consistent with asking for review while it is still reasonable to change the technology to accommodate the issues they find.</p>
</section>


<section id="when_should_review_be_requested">
<h2>When should review be requested?</h2>
<ul>
<li>For most documents, after a <em>First Public Working Draft</em> is published</li>
<li> Process-2019 requires wide review before a document is published at CR (Candidate Recommendation)</li>
<li> When a document is both reasonably stable and still flexible enough to allow changes where issues are identified</li>
<li> When new features are added after a document has already gotten wide review (perhaps requesting a review limited to the new features)</li>
</ul>
</section>


<section id="working_with_horizontal_review_labels">
<h2><a href="#working_with_horizontal_review_labels">Working with Horizontal Review labels</a></h2>

<p>Applying these labels doesn't replace the need to schedule a review of your spec at an appropriate time. (See <a rel="nofollow" class="external text" href="https://w3c.github.io/documentreview/#how_to_get_horizontal_review">How to Get Horizontal Review</a> above.) Horizontal groups participants can find <a rel="nofollow" class="external text" href="https://w3c.github.io/horizontal-issue-tracker/HOWTO">detailed process information here</a>.
</p>

<section id="day-to-day_use_of_labels">
<h3>Day-to-day use of labels</h3>
<p>Apply the <span style="font-weight: bold; border:1px solid gray; border-radius:.5em; padding:2px 5px;">*-tracker</span> label in your own repository to draw a horizontal review group’s attention to an issue in one of your own repositories. Horizontal review groups may also apply the label if they are interested in tracking a particular issue.  Tooling will automatically notify the horizontal group that you attached the label.
</p><p>If you want some specific advice from the horizontal group, describe that request in the issue thread.
</p><p>Horizontal review groups may apply the <span style="font-weight: bold; border:1px solid gray; border-radius:.5em; padding:2px 5px;">*-needs-resolution</span> label to issues they expect to be resolved before the specification moves to a new maturity level. Working Groups must not remove or add this label (not even when you close your issue).
</p><p>If the horizontal group believes that an issue with a <i class="term">*-tracker</i> label needs to be resolved before a transition, they may apply a <i class="term">*-needs-resolution</i> label to the issue. Automatic tooling will later remove the <i class="term">*-tracker</i> label.</p>
 <p>If you close an issue with a <i class="term">*-tracker</i> or <i class="term">*-needs-resolution</i> label attached, do not remove the label.  Keeping the label maintains the tracking if the issue is reopened, but also provides potentially useful information about what was tracked. (Closed issues in your repository have no effect on tools that check for unresolved issues.)</p>
 <p>At the end of a review, and before attempting to transition the spec, you should clarify that a resolution is described for all of the issues with a <i class="term">*-needs-resolution</i> label, and check that the horizontal group is aware of those resolutions. You don't have to do this for issues with the <i class="term">*-tracker</i> label. (The horizontal group was only tracking those issues, not expecting a particular resolution.)</p>
 <p>Note that the label may be applied by setting it directly on the issue if you have proper rights, or by appending it preceded with a PLUS sign (+) in the issue description, +<i class="term">*-tracker</i> or +<i class="term">*-needs-resolution</i>.</p>
</section>


<section id="What_happens_to_unresolved_issues_marked_needs-resolution">
<h3>What happens to unresolved issues marked *-needs-resolution?</h3>
<p>As lead technical architect, the W3C Director is tasked (among many things) to assess consensus within W3C for architectural issues and to decide on the outcome of <a rel="nofollow" class="external text" href="https://www.w3.org/2019/Process-20190301/#FormalObjection">Formal Objections</a>. When a horizontal issue gets flagged as *-needs-resolution and a Group chooses to request a new Maturity level despite the lack of consensus with the horizontal group, it is the task of the Director to assess the issue and the outcome of the request. A horizontal group MAY choose to elevate an horizontal issue as a Formal Objection to elevate further the importance of an issue per the W3C Process.
</p><p>In the case where an horizontal issue hasn’t been addressed and the document was allowed to move forward, it is recommended that the issue remains open in the horizontal group repository (it MAY get closed in the specification repository unless the Director requests otherwise). Some issues may take years to get resolved, but that doesn’t mean those should be forgotten.</p>
</section>
</section>


<section id="horizontal_review_boards">
<h2><a href="#horizontal_review_boards">Horizontal review boards</a></h2>
<p>The horizontal groups maintain repositories containing issues that track those raised in the WG repos. Key information about the current state of those tracker repositories is reflected in a <a rel="nofollow" class="external text" href="https://w3c.github.io/horizontal-issue-tracker/index">set of tracker boards</a>, one per horizontal function. The board points to WG issues and the corresponding tracker issues, and groups issues by specification.
</p><p>Horizontal groups participants can find <a rel="nofollow" class="external text" href="https://w3c.github.io/horizontal-issue-tracker/HOWTO">detailed process information here</a>.</p>
</section>


<section id="faq">
<h2><a href="#faq">FAQ</a></h2>

<dl>
<dt>Q: Is <i class="term">security</i> and/or <i class="term">privacy</i> review mandatory before a <i class="term">First Public Working Draft</i> is published?</dt>
<dd>
<ul>
 <li>A: It is <strong>strongly encouraged</strong> but not required.  Resolving security and privacy issues can require significant changes in specs, so it helps to identify them early.</li>
</ul>
</dd>

<dt>Q: Does a group need to prove its documents have had wide review before proceeding to CR?</dt>
<dd>
<ul>
<li>A: Yes, since Process-2019.</li>
</ul>
</dd>

<dt>Q: How does a group prove its documents have had wide review before proceeding to CR?</dt>
<dd><ul><li>A: Requests to the horizontal review groups will appear in Github, as above. </li>
<li>A: Keep a Disposition of Comments (DoC) Document showing comments received and a summary of their resolution.</li></ul></dd>

<dt>Q: Is there such a thing as too many reviews?</dt>
<dd><ul><li>A: Not really</li></ul></dd>

<dt>Q: Is it possible to make too many requests for review?</dt>
<dd><ul><li> A: Yes.  To help the review groups, only request updated reviews when substantive changes have been made, and clearly identify the changes since the last review.</li>
<li> A: This is also the reason that the Process clearly suggests there should be (TR) Working Drafts published when there are "significant changes that would benefit from review beyond the Working Group", rather than every day or only twice in the life of a spec…</li>
<li> A: TR Working Drafts are also useful for reviews since they provide a dated snapshot which can be recovered when the review comments are being discussed. Trying to discuss review comments against a document which has changed out of all recognition can be a frustrating and inefficient experience.</li></ul></dd>
</dl>
</section>


<section id="terminology_and_abbreviations">
<h2><a href="#terminology_and_abbreviations">Terminology  and abbreviations</a></h2>
<ul><li> <strong>pre-CR</strong> - This is a version of a Working Draft that is created to get wide review.
<ul><li> note that this is a bad way to get review. In general, features should be reviewed as they are developed. Waiting for a "Last Call" for most review means that when reviews suggest changes it is far harder to make them, due to a commonly observed and logical reluctance to break deployed systems or content. <a href="https://www.w3.org/wiki/User:Charles" title="User:Charles">Charles McCathie Nevile</a> 11:18, 13 October 2014 (UTC)</li></ul></li></ul>
<ul><li> <strong>requesting group</strong> - a group that is requesting a review </li></ul>
<p>Abbreviations:
</p>
<ul><li> BP = Best Practices</li>
<li> CR = Candidate Recommendation</li>
<li> RfC = Request for Comments (aka Review Request)</li>
<li> <a rel="nofollow" class="external text" href="https://www.w3.org/TR/">TR</a> = Technical Report, i.e. a formal W3C publication.</li>
<li> WD = Working Draft</li></ul>
</section>


<section id="enhancement_requests">
<h2><a href="#enhancement_requests">Enhancement Requests</a></h2>
<ul><li> See the <a rel="nofollow" class="external text" href="https://www.w3.org/wiki/Dashboard#Document_Review_Dashboard">Document Review Dashboard</a> document for information about creating a dashboard type service to facilitate document reviews.</li></ul>
</section>


<div id="versionInfo"><small>Last updated 18 Jun 2021. See the <a href="https://github.com/w3c/documentreview/commits/gh-pages/index.html">commit history</a>.</small></div>

<script>
createtoc(3)
</script>
<script type="text/javascript" src="createGitHubIssue.js"></script>
</body>
</html>