From nobody Fri Aug 21 09:34:03 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 040F83A0CEA for ; Fri, 21 Aug 2020 09:34:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.099 X-Spam-Level: X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=akamai.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5nQkEulmWaq8 for ; Fri, 21 Aug 2020 09:34:00 -0700 (PDT) Received: from mx0b-00190b01.pphosted.com (mx0b-00190b01.pphosted.com [IPv6:2620:100:9005:57f::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 05E193A0CE7 for ; Fri, 21 Aug 2020 09:33:49 -0700 (PDT) Received: from pps.filterd (m0050096.ppops.net [127.0.0.1]) by m0050096.ppops.net-00190b01. (8.16.0.42/8.16.0.42) with SMTP id 07LGXemQ007337 for ; Fri, 21 Aug 2020 17:33:49 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=from : to : subject : date : message-id : content-type : mime-version; s=jan2016.eng; bh=1f8JEvDqmDXcOLGiM/sr5Lr5Dj+PfoYpKyDrABM8Sgk=; b=PQPOURZDe/g/eyfT+UsjkRB9L7WTdowO8q7B8ysh484MaEsRthexcFKrwHtfcnYU06PR PP/45m499hukRLrlRCALRFGqFoukFrcTs+kRktm3aIclFV/g7ntbijxe5mbsQLRkQep5 fA8BPSpojTsocTbQ6AEQXaDYUrkQ4vkTDekmWKvyA9kPkDpnjaqk/3mtxg92wRwbKzAt 3u0lKQ61P2/BNcd6BQoP0h748yc+97sQ5MD0SbusWfmVblv+pjCs9MDFwqsZHosJM4/p YQ1hDXbQrvs8VPXWnFWgpyxri7UjDI3H8rLTsUwRD2Sdj7AxXKDblb5kIj+6g2bo/3+j tQ== Received: from prod-mail-ppoint5 (prod-mail-ppoint5.akamai.com [184.51.33.60] (may be forged)) by m0050096.ppops.net-00190b01. with ESMTP id 3304hvyf07-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 21 Aug 2020 17:33:49 +0100 Received: from pps.filterd (prod-mail-ppoint5.akamai.com [127.0.0.1]) by prod-mail-ppoint5.akamai.com (8.16.0.42/8.16.0.42) with SMTP id 07LG1xAe024209 for ; Fri, 21 Aug 2020 09:33:48 -0700 Received: from email.msg.corp.akamai.com ([172.27.123.32]) by prod-mail-ppoint5.akamai.com with ESMTP id 32xdpb7472-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 21 Aug 2020 09:33:48 -0700 Received: from USMA1EX-DAG1MB3.msg.corp.akamai.com (172.27.123.103) by usma1ex-dag1mb2.msg.corp.akamai.com (172.27.123.102) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 21 Aug 2020 12:33:47 -0400 Received: from USMA1EX-DAG1MB3.msg.corp.akamai.com ([172.27.123.103]) by usma1ex-dag1mb3.msg.corp.akamai.com ([172.27.123.103]) with mapi id 15.00.1497.006; Fri, 21 Aug 2020 12:33:47 -0400 From: "Salz, Rich" To: "tools-arch@ietf.org" Thread-Topic: next steps? Thread-Index: AQHWd9jXthLlZKE5jkWSYe0BoAujIQ== Date: Fri, 21 Aug 2020 16:33:47 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/16.39.20071300 x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [172.19.115.167] Content-Type: multipart/alternative; boundary="_000_A04F660B2B924436A708F6703E2DFDBFakamaicom_" MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-08-21_08:2020-08-21, 2020-08-21 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 suspectscore=0 malwarescore=0 mlxscore=0 mlxlogscore=881 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008210147 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-08-21_08:2020-08-21, 2020-08-21 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 phishscore=0 impostorscore=0 lowpriorityscore=0 priorityscore=1501 suspectscore=0 clxscore=1011 mlxlogscore=814 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008210156 Archived-At: Subject: [Tools-arch] next steps? X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2020 16:34:02 -0000 --_000_A04F660B2B924436A708F6703E2DFDBFakamaicom_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 U28gZG8gd2UgdGhpbmsgd2XigJlyZSB3YWl0aW5nIGZvciB0aGUgc3VydmV5IEpheSBtZW50aW9u ZWQ/ICBTaG91bGQgd2UgY29uc2lkZXIgY3JhZnRpbmcgc29tZSDigJx1c2VyIHN0b3JpZXPigJ0g dG8gc2VlIHdobyBvdXIgdG9vbHMgc2hvdWxkIGJlIGFkZHJlc3Npbmc/ICBEbyBmb2xrcyBldmVu IHJlbWVtYmVyIChhKSB0aGF0IHRoZXnigJlyZSBvbiB0aGlzIGxpc3Q7IGFuZCAoYikgdGhlIHN1 cnZleSBKYXkgdGFsa2VkIGFib3V0Pw0KDQo= --_000_A04F660B2B924436A708F6703E2DFDBFakamaicom_ Content-Type: text/html; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6bz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2ZmaWNlIiB4 bWxuczp3PSJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiB4bWxuczptPSJo dHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL29mZmljZS8yMDA0LzEyL29tbWwiIHhtbG5zPSJo dHRwOi8vd3d3LnczLm9yZy9UUi9SRUMtaHRtbDQwIj4NCjxoZWFkPg0KPG1ldGEgaHR0cC1lcXVp dj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPg0KPG1l dGEgbmFtZT0iR2VuZXJhdG9yIiBjb250ZW50PSJNaWNyb3NvZnQgV29yZCAxNSAoZmlsdGVyZWQg bWVkaXVtKSI+DQo8c3R5bGU+PCEtLQ0KLyogRm9udCBEZWZpbml0aW9ucyAqLw0KQGZvbnQtZmFj ZQ0KCXtmb250LWZhbWlseToiQ2FtYnJpYSBNYXRoIjsNCglwYW5vc2UtMToyIDQgNSAzIDUgNCA2 IDMgMiA0O30NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6Q2FsaWJyaTsNCglwYW5vc2UtMToy IDE1IDUgMiAyIDIgNCAzIDIgNDt9DQovKiBTdHlsZSBEZWZpbml0aW9ucyAqLw0KcC5Nc29Ob3Jt YWwsIGxpLk1zb05vcm1hbCwgZGl2Lk1zb05vcm1hbA0KCXttYXJnaW46MGluOw0KCWZvbnQtc2l6 ZToxMi4wcHQ7DQoJZm9udC1mYW1pbHk6IkNhbGlicmkiLHNhbnMtc2VyaWY7fQ0Kc3Bhbi5FbWFp bFN0eWxlMTcNCgl7bXNvLXN0eWxlLXR5cGU6cGVyc29uYWwtY29tcG9zZTsNCglmb250LWZhbWls eToiQ2FsaWJyaSIsc2Fucy1zZXJpZjsNCgljb2xvcjp3aW5kb3d0ZXh0O30NCi5Nc29DaHBEZWZh dWx0DQoJe21zby1zdHlsZS10eXBlOmV4cG9ydC1vbmx5Ow0KCWZvbnQtc2l6ZToxMi4wcHQ7DQoJ Zm9udC1mYW1pbHk6IkNhbGlicmkiLHNhbnMtc2VyaWY7fQ0KQHBhZ2UgV29yZFNlY3Rpb24xDQoJ e3NpemU6OC41aW4gMTEuMGluOw0KCW1hcmdpbjoxLjBpbiAxLjBpbiAxLjBpbiAxLjBpbjt9DQpk aXYuV29yZFNlY3Rpb24xDQoJe3BhZ2U6V29yZFNlY3Rpb24xO30NCi0tPjwvc3R5bGU+DQo8L2hl YWQ+DQo8Ym9keSBsYW5nPSJFTi1VUyIgbGluaz0iIzA1NjNDMSIgdmxpbms9IiM5NTRGNzIiPg0K PGRpdiBjbGFzcz0iV29yZFNlY3Rpb24xIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0 eWxlPSJmb250LXNpemU6MTEuMHB0Ij5TbyBkbyB3ZSB0aGluayB3ZeKAmXJlIHdhaXRpbmcgZm9y IHRoZSBzdXJ2ZXkgSmF5IG1lbnRpb25lZD8mbmJzcDsgU2hvdWxkIHdlIGNvbnNpZGVyIGNyYWZ0 aW5nIHNvbWUg4oCcdXNlciBzdG9yaWVz4oCdIHRvIHNlZSB3aG8gb3VyIHRvb2xzIHNob3VsZCBi ZSBhZGRyZXNzaW5nPyZuYnNwOyBEbyBmb2xrcyBldmVuIHJlbWVtYmVyIChhKSB0aGF0IHRoZXni gJlyZSBvbiB0aGlzIGxpc3Q7DQogYW5kIChiKSB0aGUgc3VydmV5IEpheSB0YWxrZWQgYWJvdXQ/ PG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9 ImZvbnQtc2l6ZToxMS4wcHQiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0K PC9ib2R5Pg0KPC9odG1sPg0K --_000_A04F660B2B924436A708F6703E2DFDBFakamaicom_-- From nobody Sun Aug 30 23:09:39 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 818DB3A0F3E for ; Sun, 30 Aug 2020 23:09:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=lowentropy.net header.b=q1U5ho6m; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=ToKUdULO Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e3h3LkhgbCGh for ; Sun, 30 Aug 2020 23:09:34 -0700 (PDT) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 660323A0F3D for ; Sun, 30 Aug 2020 23:09:34 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id A27FD5C018A for ; Mon, 31 Aug 2020 02:09:33 -0400 (EDT) Received: from imap10 ([10.202.2.60]) by compute2.internal (MEProxy); Mon, 31 Aug 2020 02:09:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lowentropy.net; h=mime-version:message-id:date:from:to:subject:content-type; s= fm3; bh=RNrekzrSzgMwsBq5NEtGtH6l+xNDskP9T9VV3wsrQyY=; b=q1U5ho6m wFT5qR8ixW6UAtZhaUR7NcOaT+vYrP2uWC296pdcdhJp48+Mnnd4FYAdKqf061U2 2Z38X+VY4pKM0j17wsdqMGjO/gbqx5dJAyqZsJgCozw2IKPl4YLnqBDdCRuMHm8U 99tc3BnwnR05Ta0iWDm3ZRyLdwt7ZRYU0uOPJvMkGx9fJw/RVWWwBf+hC4GPByxJ SltzxwvTHkVBlietB6OG6t68RQL27v84rGx3yE+Vl1bMBhvkXovukgAFnKgNNwa8 6O6irXE4oX2sMwprztqXWzV4gNqC7opQlRUZxvnvHWzOEqoGQdNZsRGIRgjo9Rqf ERcULR1bFX255A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; bh=RNrekzrSzgMwsBq5NEtGtH6l+xNDs kP9T9VV3wsrQyY=; b=ToKUdULObk1zSay6XJDz5+b5BRbMxx6bFcDt7AMt2KPfY voAUCe7OS/WAFjC4U6RVhhofbURxcxLIrIGCUVjEbd2igbSmQTYF/7Xd8s8JU9na 2I5drEl1rWavUjmaSjyAzAqPpYykCImQLtGZfBynN5clhsgoIFypqH1VUeRKucPO R94UEmuM/u+GpKqxPNvcYH0kIG0rL2H/aM3uZj08ZzSvbDK0jI5niD4hB7g8sVdi H7A3BBT8nqH1tR3ulWqFsehrANHfR8DPp4NFQBQ1wzsBVqiMc0r6czSr80lz51Vo MFkq7/T51tXLsJIxtMHNaYc5FEvwc7hO+0C55skNw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedrudefgedguddtiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepofgfggfkfffhvffutgesthdtre dtreertdenucfhrhhomhepfdforghrthhinhcuvfhhohhmshhonhdfuceomhhtsehlohif vghnthhrohhphidrnhgvtheqnecuggftrfgrthhtvghrnhepvddttedthfekleejtedule etkeefieffveejffekieeiheefgfefffdvgedtudeknecuffhomhgrihhnpeifohhrugdv mhgurdgtohhmpdhgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigvpedtnecurf grrhgrmhepmhgrihhlfhhrohhmpehmtheslhhofigvnhhtrhhophihrdhnvght X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 33E3A20064; Mon, 31 Aug 2020 02:09:33 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-232-g4bdb081-fm-20200825.002-g4bdb081a Mime-Version: 1.0 Message-Id: <5841e863-80b8-42a8-a989-98bd1ad8e837@www.fastmail.com> Date: Mon, 31 Aug 2020 16:09:12 +1000 From: "Martin Thomson" To: tools-arch@ietf.org Content-Type: text/plain Archived-At: Subject: [Tools-arch] Document Production Methods X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Aug 2020 06:09:37 -0000 There has been a bunch of discussion recently on the rfced-future list about the tools that people use to produce documents. It seems like this is one of the major areas in which we could provide some input. Here's my initial suggestion. # Production Methods The point of this document is to examine how the tools we use to produce a document influence the style of collaboration we use. And to examine where there are gaps in the process that might be addressed with better tooling. ## Straight to XML This is how I used to operate. It is an tolerable choice, but not a great one. XML is fiddly to the point of being user-hostile. Add to that the (recent) instability in the format and I would argue that this is not a good format to author documents in. I understand that this is (now) how the RPC operates, so this is an important option to consider. It is possible that, due to our choice of publication format, that this remains important as the single place on which we can focus attention. For specialized tooling, this might be the best format to concentrate on. For instance, tools that validate code fragments could target the XML format and rely on conversion to XML from other formats. This isn't perfect as, for instance, none(?) of the conversion tools preserve mappings to line numbers in source, so finding errors can be frustrating. However, being able to target a single format for a tool can save on development effort. ## Markdown I'm aware of two variants here (kramdown-rfc2629 and mmark), both of which are easy to use and provide native support for all the important parts of a functional document. There are some weak points, but they are quite minor. The overall experience of authoring in markdown is excellent. This is how I currently recommend that people work on documents. You might not always *start* here (see below), but you should probably spend most of your time here. I find that that most documents can start and end here anyway. The main open question I have is what level of support is provided once the collaborative portion of the process ends and the document moves to editing and publication. To my mind, the ideal situation is that the original authoring format is retained and final edits are done to that format. This would allow for working groups to pick up something that is close to the final publication form and produce a revision. This is not the model the RPC currently adopts. Only XML is accepted for editing. If we want to allow for a single set of tools for things like spell checking, this might be the right answer. However, maintaining continuity for editors is important, and editors that want markdown should be provided that option. A converter from XML to markdown might be all that is needed. Having perfect down-conversion is possible, but likely to be unworkable. An imperfect conversion might be OK provided that there is no loss of semantic information, or - worst case - lost semantics could be logged for manual examination. ## Microsoft Word/Google Docs Producing content collaboratively using Word or Docs is pretty damned good. Inline comments and suggestions are two features I rely on a massive amount in my day-to-day work. Both tools do very well at this. However, I don't think that we need a tool that has this level of fast iteration capability. This style of interaction doesn't scale out very well, so I believe that it is only useful during early stages of document production. That is, when there is a small group of people who are collaborating on an initial revision of a document. In that case, the power that these tools provide is of great use. Commentary and interactive editing are hugely useful at this stage of the process. You might also consider these tools for design teams or collaborating on small edits to an existing document. What I have found in the past is that producing a first draft of a document using these tools is great. Once more open collaboration is required, it is better to have a textual artifact in a revision control system (see RFC 8874). Revision control and issue tracking in particular allow you to better control changes and - as the process evolves - give you much better visibility into what has happened and why. Using a text-based format is a precondition of getting support from these tools. For transfer into markdown, there are online tools that can convert a simple Word document (in .doc or .docx format) into markdown. https://word2md.com uses https://github.com/benbalter/word-to-markdown, but there are others. This needs a little massaging to cleanup, which could be mechanized if this happens often, but the changes are usually small. I know of one case where an extension to Google Docs was written to do this conversion. The first draft of RFC 8752 was produced using this method and it worked very well. Change tracking is a feature I've seen used a lot in other bodies that exclusively use Word. I don't think that this feature works especially well relative to the text-based diffs we routinely use. ### Microsoft Word Template Joe Touch maintains a world template that produces text in the paged text form of yore. I think that this results in a terrible process. Even if it were to able to produce XML, it does not produce an artifact that can be collaborated on easily. Once you beyond a small group of collaborators, Word is not as good as textual formats at managing collaboration. Authoring in Word at best follows a collaboration model where you have a single editor (or, if you are especially lucky, a tight editor team) and a group of supplicants who can ask the editor to make changes. I've seen this form of process in other groups (3GPP and OMA in particular) and it's not the sort of standards collaboration I would wish on anyone. There is a strong tendency for editors to become gatekeepers anyway, but this reinforces the privilege and control of that position as every change needs to be touched by an editor. People also need to have accounts with the online service in order to collaborate effectively, which has proven to be a real barrier in practice. ## Other Tools There is a long tail here, obviously. The list of "others" includes nroff, outline.el, and others I am just not aware of. I have no idea how widespread any of these tools are, but unless presented with evidence that suggests lots of use, I don't think we need to entertain these. My suggestion is that those that can produce XML should do so promptly. Those that can only produce text, like the Word template, might find a path to markdown easier. # Recommendation The overriding theme here is how easy it is to collaborate. In the context of the IETF processes, that is to achieve consensus around the content of a single artifact. I would argue that markdown is the best tool we currently have for this. First-class support for markdown in our processes is something we should aim for. Many of the utilities that support markdown will also support XML, and some people will produce XML, so a modest effort to support XML would be appreciated. As textual formats, this allows us to use many of the same processes and tools that are available for source code. This should be familiar to many of the people we hope will contribute. For general tool support, targeting XML alone might be cheaper in terms of addressing the largest number of users. But generic support for text-based formats would be ideal. I am also going to deliberately suggest that no support is offered for other authoring formats. We might respect the choice of individuals to use other formats for authoring, but those people cannot expect others to carry the cost of their choices. That means providing encouragement to use the standard tools. We might provide an on-ramp for people who want to do initial authoring in tools that are better suited to rapid collaboration. But this should concentrate on as few options as possible. # Potential New Tools A tool that converts from XML to markdown would be a great addition to our toolset. This would allow people to pick up XML documents that were produced from a variety of formats and restart collaboration in a format that is most accessible. A tool that converts from text to XML or markdown would be the only other addition we might consider. I don't know if one of these exists already. Such a tool might exist, it's fairly easy to script something. Improving support for conversion from word to markdown would be helpful for those who want to use Microsoft Word or Google Docs. This might also help get people migrate from the Word template as well. From nobody Mon Aug 31 07:57:27 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7D2F73A14EC for ; Mon, 31 Aug 2020 07:57:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.1 X-Spam-Level: X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=akamai.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fCts3PryZ4eG for ; Mon, 31 Aug 2020 07:57:18 -0700 (PDT) Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [IPv6:2620:100:9001:583::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 79B083A14A6 for ; Mon, 31 Aug 2020 07:57:18 -0700 (PDT) Received: from pps.filterd (m0122333.ppops.net [127.0.0.1]) by mx0a-00190b01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 07VEr2tA017342; Mon, 31 Aug 2020 15:57:18 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=from : to : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=jan2016.eng; bh=t80cWjkCnkU5m/FuatOj5SwWUGR4LnQgwwG224wR+Rg=; b=CSQOMA9TmqXa+6tFa1409Y2aJfGknPLf7dOuTntiR+4iCTKnh10mXXDKTSwy9ObaMOHo aBCY5TMIbNjpVgskb0MduZjtDzXIDIm5Akw2ZSk9CzGdGScDWDwT1TI97v6NgTPaLTGu biFAc+DFExXxGTNvD1yJ/VGioNI/QkJwhWvwtye2GuHuNUEP0Mxbs1MmMZUp6VfPP/M+ pVmZJjU4iviOXADyAwhYMHCmPAYq1vCgqCEiQFmU14d8cEvDj2IKtdc00MBmhE7/F5od lHVV86PYbTgRuLkVJ1yu/FV2IF9TnP00dM/V/+QVOCBPo/9zqF38OMSYK0U+UP+8AWHx 5w== Received: from prod-mail-ppoint7 (a72-247-45-33.deploy.static.akamaitechnologies.com [72.247.45.33] (may be forged)) by mx0a-00190b01.pphosted.com with ESMTP id 337e9e03j8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 31 Aug 2020 15:57:18 +0100 Received: from pps.filterd (prod-mail-ppoint7.akamai.com [127.0.0.1]) by prod-mail-ppoint7.akamai.com (8.16.0.42/8.16.0.42) with SMTP id 07VEZWWH006529; Mon, 31 Aug 2020 10:57:16 -0400 Received: from email.msg.corp.akamai.com ([172.27.123.53]) by prod-mail-ppoint7.akamai.com with ESMTP id 337jbxtqg9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 31 Aug 2020 10:57:16 -0400 Received: from USMA1EX-DAG1MB5.msg.corp.akamai.com (172.27.123.105) by usma1ex-dag3mb2.msg.corp.akamai.com (172.27.123.59) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 31 Aug 2020 10:57:15 -0400 Received: from USMA1EX-DAG1MB3.msg.corp.akamai.com (172.27.123.103) by usma1ex-dag1mb5.msg.corp.akamai.com (172.27.123.105) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 31 Aug 2020 10:57:15 -0400 Received: from USMA1EX-DAG1MB3.msg.corp.akamai.com ([172.27.123.103]) by usma1ex-dag1mb3.msg.corp.akamai.com ([172.27.123.103]) with mapi id 15.00.1497.006; Mon, 31 Aug 2020 10:57:15 -0400 From: "Salz, Rich" To: Martin Thomson , "tools-arch@ietf.org" Thread-Topic: [Tools-arch] Document Production Methods Thread-Index: AQHWf11S3bI7Xv0AKUWQV+yWR2wkAalST0sA Date: Mon, 31 Aug 2020 14:57:15 +0000 Message-ID: <5CC37CFF-CAAE-4BA7-B6EC-276CEC98C88F@akamai.com> References: <5841e863-80b8-42a8-a989-98bd1ad8e837@www.fastmail.com> In-Reply-To: <5841e863-80b8-42a8-a989-98bd1ad8e837@www.fastmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/16.40.20081201 x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [172.27.118.139] Content-Type: text/plain; charset="utf-8" Content-ID: <16AFFC36ADF742439564A8AAA1123B79@akamai.com> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-08-31_06:2020-08-31, 2020-08-31 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 adultscore=0 mlxscore=0 bulkscore=0 spamscore=0 mlxlogscore=653 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008310086 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-08-31_07:2020-08-31, 2020-08-31 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 malwarescore=0 priorityscore=1501 bulkscore=0 clxscore=1015 mlxscore=0 lowpriorityscore=0 impostorscore=0 phishscore=0 mlxlogscore=587 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008310088 Archived-At: Subject: Re: [Tools-arch] Document Production Methods X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Aug 2020 14:57:27 -0000 TmljZSBub3RlLiAgV3JpdHRlbiBpbiBtYXJrZG93biA6KQ0KDQpDb2xsYWJvcmF0aW9uIGlzIGlt cG9ydGFudCwgYnV0IGF0IHRoZSBlbmQgb2YgdGhlIGRheSwgaXNuJ3QgImVuZCBwcm9kdWNlIGlz IFhNTCBpbiB0aGUgc3R5bGUgZGVmaW5lZCBpbiB0aGUgc3R5bGUgZGVmaW5lZCIgdGhlIG1haW4g cmVxdWlyZW1lbnQ/DQogDQoNCg== From nobody Mon Aug 31 09:43:46 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7A25B3A1768 for ; Mon, 31 Aug 2020 09:43:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.027 X-Spam-Level: X-Spam-Status: No, score=-3.027 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.948, T_SPF_HELO_PERMERROR=0.01, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nostrum.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BKxo4j4hLfAz for ; Mon, 31 Aug 2020 09:43:42 -0700 (PDT) Received: from nostrum.com (raven-v6.nostrum.com [IPv6:2001:470:d:1130::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 43C8D3A1759 for ; Mon, 31 Aug 2020 09:43:42 -0700 (PDT) Received: from unescapeable.local ([47.186.30.41]) (authenticated bits=0) by nostrum.com (8.16.1/8.15.2) with ESMTPSA id 07VGhdIh039391 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Mon, 31 Aug 2020 11:43:40 -0500 (CDT) (envelope-from rjsparks@nostrum.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=nostrum.com; s=default; t=1598892220; bh=VT1YXD8voqt4MnsB091PHD4r1pqBiuyH9IV0PvfFIg0=; h=Subject:To:References:From:Date:In-Reply-To; b=WBQyDgwiD9e3SDqWkjBJ5nKsQLdqRw1HItTsnqKy38LEeRK6sK1gTGuRMr14ngTo8 LDsKw2Uq1Y4kw5B2qmODquACT8vCEiR2d2s751caw2tZ978RH/LTECw0JLOjiHrTsq G1WDPG890xqsVukgs2icn0IkRZciKJ4dUJAiUUYY= X-Authentication-Warning: raven.nostrum.com: Host [47.186.30.41] claimed to be unescapeable.local To: tools-arch@ietf.org References: <5841e863-80b8-42a8-a989-98bd1ad8e837@www.fastmail.com> From: Robert Sparks Message-ID: Date: Mon, 31 Aug 2020 11:43:38 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <5841e863-80b8-42a8-a989-98bd1ad8e837@www.fastmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US Archived-At: Subject: Re: [Tools-arch] Document Production Methods X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Aug 2020 16:43:45 -0000 On 8/31/20 1:09 AM, Martin Thomson wrote: > > A tool that converts from text to XML or markdown would be the only other addition we might consider. I don't know if one of these exists already. Such a tool might exist, it's fairly easy to script something. https://pypi.org/project/id2xml/ From nobody Mon Aug 31 09:49:42 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 806AA3A176D for ; Mon, 31 Aug 2020 09:49:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R4yGLzCE_Ns2 for ; Mon, 31 Aug 2020 09:49:38 -0700 (PDT) Received: from mail2.augustcellars.com (augustcellars.com [50.45.239.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F64F3A0BC5 for ; Mon, 31 Aug 2020 09:49:37 -0700 (PDT) Received: from Jude (73.180.8.170) by mail2.augustcellars.com (192.168.0.56) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Mon, 31 Aug 2020 09:49:30 -0700 From: Jim Schaad To: 'Martin Thomson' , References: <5841e863-80b8-42a8-a989-98bd1ad8e837@www.fastmail.com> In-Reply-To: <5841e863-80b8-42a8-a989-98bd1ad8e837@www.fastmail.com> Date: Mon, 31 Aug 2020 09:49:27 -0700 Message-ID: <001101d67fb6$b2284f90$1678eeb0$@augustcellars.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQNBBLh3yEgZxAqz+6hgGFjOoHu8rqZ9Y7mw Content-Language: en-us X-Originating-IP: [73.180.8.170] Archived-At: Subject: Re: [Tools-arch] Document Production Methods X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Aug 2020 16:49:41 -0000 Carsten has a tool which goes from XML to markdown. I don't know how much fidelity you have for a round trip process due to the fact that things like line breaks are probably not preserved. Jim -----Original Message----- From: Tools-arch On Behalf Of Martin Thomson Sent: Sunday, August 30, 2020 11:09 PM To: tools-arch@ietf.org Subject: [Tools-arch] Document Production Methods There has been a bunch of discussion recently on the rfced-future list about the tools that people use to produce documents. It seems like this is one of the major areas in which we could provide some input. Here's my initial suggestion. # Production Methods The point of this document is to examine how the tools we use to produce a document influence the style of collaboration we use. And to examine where there are gaps in the process that might be addressed with better tooling. ## Straight to XML This is how I used to operate. It is an tolerable choice, but not a great one. XML is fiddly to the point of being user-hostile. Add to that the (recent) instability in the format and I would argue that this is not a good format to author documents in. I understand that this is (now) how the RPC operates, so this is an important option to consider. It is possible that, due to our choice of publication format, that this remains important as the single place on which we can focus attention. For specialized tooling, this might be the best format to concentrate on. For instance, tools that validate code fragments could target the XML format and rely on conversion to XML from other formats. This isn't perfect as, for instance, none(?) of the conversion tools preserve mappings to line numbers in source, so finding errors can be frustrating. However, being able to target a single format for a tool can save on development effort. ## Markdown I'm aware of two variants here (kramdown-rfc2629 and mmark), both of which are easy to use and provide native support for all the important parts of a functional document. There are some weak points, but they are quite minor. The overall experience of authoring in markdown is excellent. This is how I currently recommend that people work on documents. You might not always *start* here (see below), but you should probably spend most of your time here. I find that that most documents can start and end here anyway. The main open question I have is what level of support is provided once the collaborative portion of the process ends and the document moves to editing and publication. To my mind, the ideal situation is that the original authoring format is retained and final edits are done to that format. This would allow for working groups to pick up something that is close to the final publication form and produce a revision. This is not the model the RPC currently adopts. Only XML is accepted for editing. If we want to allow for a single set of tools for things like spell checking, this might be the right answer. However, maintaining continuity for editors is important, and editors that want markdown should be provided that option. A converter from XML to markdown might be all that is needed. Having perfect down-conversion is possible, but likely to be unworkable. An imperfect conversion might be OK provided that there is no loss of semantic information, or - worst case - lost semantics could be logged for manual examination. ## Microsoft Word/Google Docs Producing content collaboratively using Word or Docs is pretty damned good. Inline comments and suggestions are two features I rely on a massive amount in my day-to-day work. Both tools do very well at this. However, I don't think that we need a tool that has this level of fast iteration capability. This style of interaction doesn't scale out very well, so I believe that it is only useful during early stages of document production. That is, when there is a small group of people who are collaborating on an initial revision of a document. In that case, the power that these tools provide is of great use. Commentary and interactive editing are hugely useful at this stage of the process. You might also consider these tools for design teams or collaborating on small edits to an existing document. What I have found in the past is that producing a first draft of a document using these tools is great. Once more open collaboration is required, it is better to have a textual artifact in a revision control system (see RFC 8874). Revision control and issue tracking in particular allow you to better control changes and - as the process evolves - give you much better visibility into what has happened and why. Using a text-based format is a precondition of getting support from these tools. For transfer into markdown, there are online tools that can convert a simple Word document (in .doc or .docx format) into markdown. https://word2md.com uses https://github.com/benbalter/word-to-markdown, but there are others. This needs a little massaging to cleanup, which could be mechanized if this happens often, but the changes are usually small. I know of one case where an extension to Google Docs was written to do this conversion. The first draft of RFC 8752 was produced using this method and it worked very well. Change tracking is a feature I've seen used a lot in other bodies that exclusively use Word. I don't think that this feature works especially well relative to the text-based diffs we routinely use. ### Microsoft Word Template Joe Touch maintains a world template that produces text in the paged text form of yore. I think that this results in a terrible process. Even if it were to able to produce XML, it does not produce an artifact that can be collaborated on easily. Once you beyond a small group of collaborators, Word is not as good as textual formats at managing collaboration. Authoring in Word at best follows a collaboration model where you have a single editor (or, if you are especially lucky, a tight editor team) and a group of supplicants who can ask the editor to make changes. I've seen this form of process in other groups (3GPP and OMA in particular) and it's not the sort of standards collaboration I would wish on anyone. There is a strong tendency for editors to become gatekeepers anyway, but this reinforces the privilege and control of that position as every change needs to be touched by an editor. People also need to have accounts with the online service in order to collaborate effectively, which has proven to be a real barrier in practice. ## Other Tools There is a long tail here, obviously. The list of "others" includes nroff, outline.el, and others I am just not aware of. I have no idea how widespread any of these tools are, but unless presented with evidence that suggests lots of use, I don't think we need to entertain these. My suggestion is that those that can produce XML should do so promptly. Those that can only produce text, like the Word template, might find a path to markdown easier. # Recommendation The overriding theme here is how easy it is to collaborate. In the context of the IETF processes, that is to achieve consensus around the content of a single artifact. I would argue that markdown is the best tool we currently have for this. First-class support for markdown in our processes is something we should aim for. Many of the utilities that support markdown will also support XML, and some people will produce XML, so a modest effort to support XML would be appreciated. As textual formats, this allows us to use many of the same processes and tools that are available for source code. This should be familiar to many of the people we hope will contribute. For general tool support, targeting XML alone might be cheaper in terms of addressing the largest number of users. But generic support for text-based formats would be ideal. I am also going to deliberately suggest that no support is offered for other authoring formats. We might respect the choice of individuals to use other formats for authoring, but those people cannot expect others to carry the cost of their choices. That means providing encouragement to use the standard tools. We might provide an on-ramp for people who want to do initial authoring in tools that are better suited to rapid collaboration. But this should concentrate on as few options as possible. # Potential New Tools A tool that converts from XML to markdown would be a great addition to our toolset. This would allow people to pick up XML documents that were produced from a variety of formats and restart collaboration in a format that is most accessible. A tool that converts from text to XML or markdown would be the only other addition we might consider. I don't know if one of these exists already. Such a tool might exist, it's fairly easy to script something. Improving support for conversion from word to markdown would be helpful for those who want to use Microsoft Word or Google Docs. This might also help get people migrate from the Word template as well. -- Tools-arch mailing list Tools-arch@ietf.org https://www.ietf.org/mailman/listinfo/tools-arch From nobody Mon Aug 31 20:52:36 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 504F53A09F6 for ; Mon, 31 Aug 2020 20:52:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aM_jXiOCxq0B; Mon, 31 Aug 2020 20:52:33 -0700 (PDT) Received: from jays-mbp.localdomain (unknown [158.140.230.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPSA id 332B13A0A63; Mon, 31 Aug 2020 20:52:32 -0700 (PDT) From: Jay Daley Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_75C8E57E-4847-4FC8-A9C0-40253664DB13" Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Date: Tue, 1 Sep 2020 15:52:30 +1200 In-Reply-To: <5841e863-80b8-42a8-a989-98bd1ad8e837@www.fastmail.com> Cc: tools-arch@ietf.org To: Martin Thomson References: <5841e863-80b8-42a8-a989-98bd1ad8e837@www.fastmail.com> X-Mailer: Apple Mail (2.3608.80.23.2.2) Archived-At: Subject: Re: [Tools-arch] Document Production Methods X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Sep 2020 03:52:35 -0000 --Apple-Mail=_75C8E57E-4847-4FC8-A9C0-40253664DB13 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 If I take what you=E2=80=99ve written and rearrange it in a different = way, you are describing three different stages of document authoring = each of which has a different use case and each use case would be better = supported by a different format. # Stage one - initial authoring This is the initial production of the document either by one person or a = small group, which only requires basic revision tracking and comments. = As you say, that is better suited to Google Docs or something similar = for fast, easy collaboration. # Stage two - group text revision This the laborious process of multiple contributors working with changes = across the document. Again, as you say, that works best with something = as close to plain text as possible, such as markdown as the markup only = gets in the way and XML is so much markup. # Stage three - pre-publication preparation This consists mainly of filling out boilerplate sections (contributors = etc), adding semantic annotation and layout control, with text changes = focused on clarity/language etc not substance. The moment we get into = semantic annotation and layout control we need complex markup such as = XML. The boilerplate stuff and text cleanup could be done in stage = two-point-five before the XML comes into it. Automated format translation from one stage to the next higher stage is = easy provided conventions are followed in both stages one and two, which = is easy enough with templates. =20 Translating a document back a stage (your XML to markdown convertor) is = where more thought is required both in terms of what processes this will = be used for as well as the technical side. For example, is this = intended to be a one-way translation so that any semantic and layout = markup is lost, or is it meant to be a reversible process where the = markdown is "extracted" from the XML, then edited and the "reinserted" = into the XML? A simple XML to markdown convertor is not hard [1], while = the two way is more complex but still doable. Jay [1] In my current role I don=E2=80=99t do technology, but when I did, = XML to [=E2=80=A6] conversion was something I did = https://github.com/JayDaley/XML-to-JSON-in-XSLT > On 31/08/2020, at 6:09 PM, Martin Thomson wrote: >=20 > There has been a bunch of discussion recently on the rfced-future list = about the tools that people use to produce documents. It seems like = this is one of the major areas in which we could provide some input. = Here's my initial suggestion. >=20 >=20 > # Production Methods >=20 > The point of this document is to examine how the tools we use to = produce a document influence the style of collaboration we use. And to = examine where there are gaps in the process that might be addressed with = better tooling. >=20 > ## Straight to XML >=20 > This is how I used to operate. It is an tolerable choice, but not a = great one. XML is fiddly to the point of being user-hostile. Add to = that the (recent) instability in the format and I would argue that this = is not a good format to author documents in. >=20 > I understand that this is (now) how the RPC operates, so this is an = important option to consider. It is possible that, due to our choice of = publication format, that this remains important as the single place on = which we can focus attention. >=20 > For specialized tooling, this might be the best format to concentrate = on. For instance, tools that validate code fragments could target the = XML format and rely on conversion to XML from other formats. This isn't = perfect as, for instance, none(?) of the conversion tools preserve = mappings to line numbers in source, so finding errors can be = frustrating. However, being able to target a single format for a tool = can save on development effort. >=20 > ## Markdown >=20 > I'm aware of two variants here (kramdown-rfc2629 and mmark), both of = which are easy to use and provide native support for all the important = parts of a functional document. There are some weak points, but they = are quite minor. The overall experience of authoring in markdown is = excellent. >=20 > This is how I currently recommend that people work on documents. You = might not always *start* here (see below), but you should probably spend = most of your time here. I find that that most documents can start and = end here anyway. >=20 > The main open question I have is what level of support is provided = once the collaborative portion of the process ends and the document = moves to editing and publication. To my mind, the ideal situation is = that the original authoring format is retained and final edits are done = to that format. This would allow for working groups to pick up = something that is close to the final publication form and produce a = revision. >=20 > This is not the model the RPC currently adopts. Only XML is accepted = for editing. If we want to allow for a single set of tools for things = like spell checking, this might be the right answer. However, = maintaining continuity for editors is important, and editors that want = markdown should be provided that option. A converter from XML to = markdown might be all that is needed. Having perfect down-conversion is = possible, but likely to be unworkable. An imperfect conversion might be = OK provided that there is no loss of semantic information, or - worst = case - lost semantics could be logged for manual examination. >=20 > ## Microsoft Word/Google Docs >=20 > Producing content collaboratively using Word or Docs is pretty damned = good. Inline comments and suggestions are two features I rely on a = massive amount in my day-to-day work. Both tools do very well at this. >=20 > However, I don't think that we need a tool that has this level of fast = iteration capability. This style of interaction doesn't scale out very = well, so I believe that it is only useful during early stages of = document production. That is, when there is a small group of people who = are collaborating on an initial revision of a document. In that case, = the power that these tools provide is of great use. Commentary and = interactive editing are hugely useful at this stage of the process. =20 >=20 > You might also consider these tools for design teams or collaborating = on small edits to an existing document. >=20 > What I have found in the past is that producing a first draft of a = document using these tools is great. Once more open collaboration is = required, it is better to have a textual artifact in a revision control = system (see RFC 8874). >=20 > Revision control and issue tracking in particular allow you to better = control changes and - as the process evolves - give you much better = visibility into what has happened and why. Using a text-based format is = a precondition of getting support from these tools. >=20 > For transfer into markdown, there are online tools that can convert a = simple Word document (in .doc or .docx format) into markdown. = https://word2md.com uses https://github.com/benbalter/word-to-markdown, = but there are others. This needs a little massaging to cleanup, which = could be mechanized if this happens often, but the changes are usually = small. I know of one case where an extension to Google Docs was written = to do this conversion. The first draft of RFC 8752 was produced using = this method and it worked very well. >=20 > Change tracking is a feature I've seen used a lot in other bodies that = exclusively use Word. I don't think that this feature works especially = well relative to the text-based diffs we routinely use. >=20 > ### Microsoft Word Template >=20 > Joe Touch maintains a world template that produces text in the paged = text form of yore. I think that this results in a terrible process. = Even if it were to able to produce XML, it does not produce an artifact = that can be collaborated on easily. =20 >=20 > Once you beyond a small group of collaborators, Word is not as good as = textual formats at managing collaboration. Authoring in Word at best = follows a collaboration model where you have a single editor (or, if you = are especially lucky, a tight editor team) and a group of supplicants = who can ask the editor to make changes. I've seen this form of process = in other groups (3GPP and OMA in particular) and it's not the sort of = standards collaboration I would wish on anyone. There is a strong = tendency for editors to become gatekeepers anyway, but this reinforces = the privilege and control of that position as every change needs to be = touched by an editor. >=20 > People also need to have accounts with the online service in order to = collaborate effectively, which has proven to be a real barrier in = practice. >=20 > ## Other Tools >=20 > There is a long tail here, obviously. The list of "others" includes = nroff, outline.el, and others I am just not aware of. I have no idea = how widespread any of these tools are, but unless presented with = evidence that suggests lots of use, I don't think we need to entertain = these. >=20 > My suggestion is that those that can produce XML should do so = promptly. Those that can only produce text, like the Word template, = might find a path to markdown easier. >=20 > # Recommendation >=20 > The overriding theme here is how easy it is to collaborate. In the = context of the IETF processes, that is to achieve consensus around the = content of a single artifact. >=20 > I would argue that markdown is the best tool we currently have for = this. First-class support for markdown in our processes is something we = should aim for. Many of the utilities that support markdown will also = support XML, and some people will produce XML, so a modest effort to = support XML would be appreciated. >=20 > As textual formats, this allows us to use many of the same processes = and tools that are available for source code. This should be familiar = to many of the people we hope will contribute. >=20 > For general tool support, targeting XML alone might be cheaper in = terms of addressing the largest number of users. But generic support = for text-based formats would be ideal. >=20 > I am also going to deliberately suggest that no support is offered for = other authoring formats. We might respect the choice of individuals to = use other formats for authoring, but those people cannot expect others = to carry the cost of their choices. That means providing encouragement = to use the standard tools. >=20 > We might provide an on-ramp for people who want to do initial = authoring in tools that are better suited to rapid collaboration. But = this should concentrate on as few options as possible. >=20 > # Potential New Tools >=20 > A tool that converts from XML to markdown would be a great addition to = our toolset. This would allow people to pick up XML documents that were = produced from a variety of formats and restart collaboration in a format = that is most accessible. >=20 > A tool that converts from text to XML or markdown would be the only = other addition we might consider. I don't know if one of these exists = already. Such a tool might exist, it's fairly easy to script something. >=20 > Improving support for conversion from word to markdown would be = helpful for those who want to use Microsoft Word or Google Docs. This = might also help get people migrate from the Word template as well. >=20 > --=20 > Tools-arch mailing list > Tools-arch@ietf.org > https://www.ietf.org/mailman/listinfo/tools-arch >=20 --=20 Jay Daley IETF Executive Director jay@ietf.org --Apple-Mail=_75C8E57E-4847-4FC8-A9C0-40253664DB13 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 If = I take what you=E2=80=99ve written and rearrange it in a different way, = you are describing three different stages of document authoring each of = which has a different use case and each use case would be better = supported by a different format.

# Stage one - initial authoring
This is = the initial production of the document either by one person or a small = group, which only requires basic revision tracking and comments. =  As you say, that is better suited to Google Docs or something = similar for fast, easy collaboration.

# Stage two - group text = revision
This the laborious process of multiple = contributors working with changes across the document.  Again, as = you say, that works best with something as close to plain text as = possible, such as markdown as the markup only gets in the way and XML is = so much markup.

# Stage three - pre-publication preparation
This consists mainly of filling out boilerplate sections = (contributors etc), adding semantic annotation and layout control, with = text changes focused on clarity/language etc not substance.  The = moment we get into semantic annotation and layout control we need = complex markup such as XML.  The boilerplate stuff and text cleanup = could be done in stage two-point-five before the XML comes into = it.


Automated format translation from one = stage to the next higher stage is easy provided conventions are followed = in both stages one and two, which is easy enough with templates. =  

Translating a document back a stage (your XML to markdown = convertor) is where more thought is required both in terms of what = processes this will be used for as well as the technical side.  For = example, is this intended to be a one-way translation so that any = semantic and layout markup is lost, or is it meant to be a reversible = process where the markdown is "extracted" from the XML, then edited and = the "reinserted" into the XML?  A simple XML to markdown convertor = is not hard [1], while the two way is more complex but still = doable.

Jay

[1]  In my current role I don=E2=80=99t do technology, = but when I did, XML to [=E2=80=A6] conversion was something I = did https://github.com/JayDaley/XML-to-JSON-in-XSLT


On = 31/08/2020, at 6:09 PM, Martin Thomson <mt@lowentropy.net> = wrote:

There has been a bunch of discussion recently on the = rfced-future list about the tools that people use to produce documents. =  It seems like this is one of the major areas in which we could = provide some input.  Here's my initial suggestion.


# Production Methods

The point of this document is to examine how the tools we use = to produce a document influence the style of collaboration we use. =  And to examine where there are gaps in the process that might be = addressed with better tooling.

## Straight = to XML

This is how I used to operate. =  It is an tolerable choice, but not a great one.  XML is = fiddly to the point of being user-hostile.  Add to that the = (recent) instability in the format and I would argue that this is not a = good format to author documents in.

I = understand that this is (now) how the RPC operates, so this is an = important option to consider.  It is possible that, due to our = choice of publication format, that this remains important as the single = place on which we can focus attention.

For = specialized tooling, this might be the best format to concentrate on. =  For instance, tools that validate code fragments could target the = XML format and rely on conversion to XML from other formats.  This = isn't perfect as, for instance, none(?) of the conversion tools preserve = mappings to line numbers in source, so finding errors can be = frustrating.  However, being able to target a single format for a = tool can save on development effort.

## = Markdown

I'm aware of two variants here = (kramdown-rfc2629 and mmark), both of which are easy to use and provide = native support for all the important parts of a functional document. =  There are some weak points, but they are quite minor.  The = overall experience of authoring in markdown is excellent.

This is how I currently recommend that people = work on documents.  You might not always *start* here (see below), = but you should probably spend most of your time here.  I find that = that most documents can start and end here anyway.

The main open question I have is what level of support is = provided once the collaborative portion of the process ends and the = document moves to editing and publication.  To my mind, the ideal = situation is that the original authoring format is retained and final = edits are done to that format.  This would allow for working groups = to pick up something that is close to the final publication form and = produce a revision.

This is not the model = the RPC currently adopts.  Only XML is accepted for editing. =  If we want to allow for a single set of tools for things like = spell checking, this might be the right answer.  However, = maintaining continuity for editors is important, and editors that want = markdown should be provided that option.  A converter from XML to = markdown might be all that is needed.  Having perfect = down-conversion is possible, but likely to be unworkable.  An = imperfect conversion might be OK provided that there is no loss of = semantic information, or - worst case - lost semantics could be logged = for manual examination.

## Microsoft = Word/Google Docs

Producing content = collaboratively using Word or Docs is pretty damned good.  Inline = comments and suggestions are two features I rely on a massive amount in = my day-to-day work.  Both tools do very well at this.

However, I don't think that we need a tool = that has this level of fast iteration capability.  This style of = interaction doesn't scale out very well, so I believe that it is only = useful during early stages of document production.  That is, when = there is a small group of people who are collaborating on an initial = revision of a document.  In that case, the power that these tools = provide is of great use.  Commentary and interactive editing are = hugely useful at this stage of the process.  

You might also consider these tools for design teams or = collaborating on small edits to an existing document.

What I have found in the past is that producing a first draft = of a document using these tools is great.  Once more open = collaboration is required, it is better to have a textual artifact in a = revision control system (see RFC 8874).

Revision control and issue tracking in particular allow you = to better control changes and - as the process evolves - give you much = better visibility into what has happened and why.  Using a = text-based format is a precondition of getting support from these = tools.

For transfer into markdown, there = are online tools that can convert a simple Word document (in .doc or = .docx format) into markdown.  https://word2md.com uses https://github.com/benbalter/word-to-markdown, but there = are others.  This needs a little massaging to cleanup, which could = be mechanized if this happens often, but the changes are usually small. =  I know of one case where an extension to Google Docs was written = to do this conversion.  The first draft of RFC 8752 was produced = using this method and it worked very well.

Change tracking is a feature I've seen used a lot in other = bodies that exclusively use Word.  I don't think that this feature = works especially well relative to the text-based diffs we routinely = use.

### Microsoft Word Template

Joe Touch maintains a world template that = produces text in the paged text form of yore.  I think that this = results in a terrible process.  Even if it were to able to produce = XML, it does not produce an artifact that can be collaborated on easily. =  

Once you beyond a small group of = collaborators, Word is not as good as textual formats at managing = collaboration.  Authoring in Word at best follows a collaboration = model where you have a single editor (or, if you are especially lucky, a = tight editor team) and a group of supplicants who can ask the editor to = make changes.  I've seen this form of process in other groups (3GPP = and OMA in particular) and it's not the sort of standards collaboration = I would wish on anyone.  There is a strong tendency for editors to = become gatekeepers anyway, but this reinforces the privilege and control = of that position as every change needs to be touched by an editor.

People also need to have accounts with the = online service in order to collaborate effectively, which has proven to = be a real barrier in practice.

## Other = Tools

There is a long tail here, obviously. =  The list of "others" includes nroff, outline.el, and others I am = just not aware of.  I have no idea how widespread any of these = tools are, but unless presented with evidence that suggests lots of use, = I don't think we need to entertain these.

My = suggestion is that those that can produce XML should do so promptly. =  Those that can only produce text, like the Word template, might = find a path to markdown easier.

# = Recommendation

The overriding theme here is = how easy it is to collaborate.  In the context of the IETF = processes, that is to achieve consensus around the content of a single = artifact.

I would argue that markdown is = the best tool we currently have for this.  First-class support for = markdown in our processes is something we should aim for.  Many of = the utilities that support markdown will also support XML, and some = people will produce XML, so a modest effort to support XML would be = appreciated.

As textual formats, this = allows us to use many of the same processes and tools that are available = for source code.  This should be familiar to many of the people we = hope will contribute.

For general tool = support, targeting XML alone might be cheaper in terms of addressing the = largest number of users.  But generic support for text-based = formats would be ideal.

I am also going to = deliberately suggest that no support is offered for other authoring = formats.  We might respect the choice of individuals to use other = formats for authoring, but those people cannot expect others to carry = the cost of their choices.  That means providing encouragement to = use the standard tools.

We might provide an = on-ramp for people who want to do initial authoring in tools that are = better suited to rapid collaboration.  But this should concentrate = on as few options as possible.

# Potential = New Tools

A tool that converts from XML to = markdown would be a great addition to our toolset.  This would = allow people to pick up XML documents that were produced from a variety = of formats and restart collaboration in a format that is most = accessible.

A tool that converts from text = to XML or markdown would be the only other addition we might consider. =   I don't know if one of these exists already.  Such a = tool might exist, it's fairly easy to script something.

Improving support for conversion from word to markdown would = be helpful for those who want to use Microsoft Word or Google Docs. =  This might also help get people migrate from the Word template as = well.

--
Tools-arch mailing = list
Tools-arch@ietf.org
https://www.ietf.org/mailman/listinfo/tools-arch


-- 
Jay Daley
IETF = Executive Director
jay@ietf.org

= --Apple-Mail=_75C8E57E-4847-4FC8-A9C0-40253664DB13-- From nobody Mon Aug 31 22:21:26 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5C7283A0BE3; Mon, 31 Aug 2020 22:21:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=lowentropy.net header.b=AYups8uc; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=uUW2CrC4 Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zng1-_CLgsMX; Mon, 31 Aug 2020 22:21:22 -0700 (PDT) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0CB6D3A0BDE; Mon, 31 Aug 2020 22:21:22 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 5EF035C0050; Tue, 1 Sep 2020 01:21:21 -0400 (EDT) Received: from imap10 ([10.202.2.60]) by compute2.internal (MEProxy); Tue, 01 Sep 2020 01:21:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lowentropy.net; h=mime-version:message-id:in-reply-to:references:date:from:to :cc:subject:content-type:content-transfer-encoding; s=fm3; bh=ff CB8tnHxzzkSvq/p4JVjXIySbwtw/Ifb7EDukVUmKI=; b=AYups8uca/xcyk5jN9 O9RUnUsbo+os0hHwstjVgGTl9qjjJCD59PpUTftimQvLm3u5/lEyhol70bRZ7Bt7 RWIQGRDkfnSyiL95fl9qL/WchqBnVKSq1j7Mup9BqXEqr1g1kaFEwbZJq/bHlo0S vC8HB4LVX2luuKVZR4FmSQULGsJ/iqNxIa/RmKKAJjs+v1Re0QYN9OrgQnU8uP7Y xScrjxB7Gtsfn5dHT/ZUS6tt558GNXCCHA56Uix3nWSrhQzKI+YbzdLE07sro8DX 7TPMe2vNVn7ythi5Uo0+egvNVB00hYJtE7LNBIiKeKm+y8pRHYODes7+Jpn3wqmd iwrQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=ffCB8tnHxzzkSvq/p4JVjXIySbwtw/Ifb7EDukVUm KI=; b=uUW2CrC4USOMsncs2t7lif5Flndjzjjadt98jodF/bWbXERVJGs4+x5WV AsOdrsp8vGDqPfuK7otCAnMA+cJFADf+3wC4+jJlusG3VtCm9UFp7G5kialxdyFB R0u32zZ0bQkf7hnBK/na2tE8/ymt63XMDti7YgbDpdFyMuO0nF54h8CUy175i+OC 2yrQG9L2FgnxtvwRZ4sQ/izAOhL01OwrzOdxDMzWM9NIIJ5m/HcZ0/+jZ1kGBdli HCuIRAS2KZaXgL497vIgtJ/lpg4Mpc5reUybOCuOugPJRtZIgnRyvNeEuRlu81rG HUE7NSvzHhDUThplZVU7j6zmBQ5aQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedrudefiedgleehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth hqredtreerjeenucfhrhhomhepfdforghrthhinhcuvfhhohhmshhonhdfuceomhhtsehl ohifvghnthhrohhphidrnhgvtheqnecuggftrfgrthhtvghrnhepieetuddtvdevleduie fhvdefheeigffgveeiledvfeeghfeuudejtdehtdelkeeknecuffhomhgrihhnpehgihht hhhusgdrtghomhdpfihorhguvdhmugdrtghomhdpihgvthhfrdhorhhgnecuvehluhhsth gvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhhtsehlohifvghnthhr ohhphidrnhgvth X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 029842015D; Tue, 1 Sep 2020 01:21:21 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-248-gcd102cb-fm-20200901.001-gcd102cb9 Mime-Version: 1.0 Message-Id: <2062f219-5b09-46aa-bf47-ea55f287e142@www.fastmail.com> In-Reply-To: References: <5841e863-80b8-42a8-a989-98bd1ad8e837@www.fastmail.com> Date: Tue, 01 Sep 2020 15:21:00 +1000 From: "Martin Thomson" To: "Jay Daley" Cc: tools-arch@ietf.org Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Tools-arch] Document Production Methods X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Sep 2020 05:21:24 -0000 That's a fair summary. I would say however that you might be overstating the XML thing. Those = who designed the v3 format might believe that it is all needed, but I am= less certain of the value of it all. What is true is that this is what= we have. What I think is important to preserve across conversions is the text. T= he dressings (boilerplate, use of