Obsolete reference to START-TIMERS

From speechsc-bounces@ietf.org Tue Jul 05 03:26:17 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dphov-0005zs-LJ; Tue, 05 Jul 2005 03:26:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dphol-0005y6-3E for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 03:26:07 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA07353 for ; Tue, 5 Jul 2005 03:26:05 -0400 (EDT) Received: from pb-exchcon2.scansoft.com ([199.4.160.64]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DpiFT-0001p7-LC for speechsc@ietf.org; Tue, 05 Jul 2005 03:53:44 -0400 Received: by pb-exchcon2.pb.scansoft.com with Internet Mail Service (5.5.2658.27) id ; Tue, 5 Jul 2005 03:25:47 -0400 Message-ID: From: "Reifenrath, Klaus" To: "'sarvi@cisco.com'" Date: Tue, 5 Jul 2005 03:25:25 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2658.27) X-Spam-Score: 0.7 (/) X-Scan-Signature: 39bd8f8cbb76cae18b7e23f7cf6b2b9f Cc: "'speechsc@ietf.org'" Subject: [Speechsc] Obsolete reference to START-TIMERS X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0386563461==" Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --===============0386563461== Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C58132.B6307D8A" This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C58132.B6307D8A Content-Type: text/plain In section 10.4 "(C) - START-TIMERS" need to be replaced by "(C) - START-INPUT-TIMERS". Klaus ------_=_NextPart_001_01C58132.B6307D8A Content-Type: text/html Content-Transfer-Encoding: quoted-printable Obsolete reference to START-TIMERS

In section 10.4 "(C) - = START-TIMERS" need to be replaced by "(C) - = START-INPUT-TIMERS".

Klaus

------_=_NextPart_001_01C58132.B6307D8A-- --===============0386563461== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc --===============0386563461==-- From speechsc-bounces@ietf.org Tue Jul 05 03:46:59 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dpi8w-0002dX-Tw; Tue, 05 Jul 2005 03:46:58 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dpi8u-0002dQ-7B for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 03:46:57 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA11859 for ; Tue, 5 Jul 2005 03:46:54 -0400 (EDT) Received: from pb-exchcon2.scansoft.com ([199.4.160.64]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DpiZe-0003dD-68 for speechsc@ietf.org; Tue, 05 Jul 2005 04:14:34 -0400 Received: by pb-exchcon2.pb.scansoft.com with Internet Mail Service (5.5.2658.27) id ; Tue, 5 Jul 2005 03:46:47 -0400 Message-ID: From: "Reifenrath, Klaus" To: "'speechsc@ietf.org'" Date: Tue, 5 Jul 2005 03:46:31 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2658.27) X-Spam-Score: 0.9 (/) X-Scan-Signature: f607d15ccc2bc4eaf3ade8ffa8af02a0 Subject: [Speechsc] START-OF-SPEECH in DTMF-only mode X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1328829717==" Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --===============1328829717== Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C58135.79661952" This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C58135.79661952 Content-Type: text/plain The current spec is not clear when START-OF-SPEECH need to be send in the following scenarios: A) The client requested a DTMF Recognizer. Is the START-OF-SPEECH event send to the client also if speech was detected? B) The client requested a Speech Recognizer, but only activated DTMF grammars. Is the START-OF-SPEECH event send to the client also if speech was detected? I think in both cases START-OF-SPEECH should only be send after detecting a DTMF digit (see Figure 12 of VoiceXML 2.0: http://www.w3.org/TR/voicexml20/#dmlATiming ). Klaus ------_=_NextPart_001_01C58135.79661952 Content-Type: text/html Content-Transfer-Encoding: quoted-printable START-OF-SPEECH in DTMF-only mode

The current spec is not clear when = START-OF-SPEECH need to be send in the following scenarios:
A) The client requested a DTMF = Recognizer. Is the START-OF-SPEECH event send to the client also if = speech was detected?

B) The client requested a Speech = Recognizer, but only activated DTMF grammars. Is the START-OF-SPEECH = event send to the client also if speech was detected?

I think in both cases START-OF-SPEECH = should only be send after detecting a DTMF digit (see Figure 12 of VoiceXML 2.0: = http://www.w3.org/TR/voicexml20/#dmlATiming).

Klaus

------_=_NextPart_001_01C58135.79661952-- --===============1328829717== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc --===============1328829717==-- From speechsc-bounces@ietf.org Tue Jul 05 08:49:07 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DpmrK-0000mw-NG; Tue, 05 Jul 2005 08:49:06 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DpmrI-0000mG-LG for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 08:49:05 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA24848 for ; Tue, 5 Jul 2005 08:49:00 -0400 (EDT) Received: from sj-iport-2-in.cisco.com ([171.71.176.71] helo=sj-iport-2.cisco.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DpmuO-0004aj-0j for speechsc@ietf.org; Tue, 05 Jul 2005 08:52:17 -0400 Received: from sj-core-2.cisco.com (171.71.177.254) by sj-iport-2.cisco.com with ESMTP; 05 Jul 2005 05:24:26 -0700 Received: from imail.cisco.com (imail.cisco.com [128.107.200.91]) by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id j65COLod014829; Tue, 5 Jul 2005 05:24:21 -0700 (PDT) Received: from [10.32.245.152] (stealth-10-32-245-152.cisco.com [10.32.245.152]) by imail.cisco.com (8.12.11/8.12.10) with SMTP id j65CNYV1032067; Tue, 5 Jul 2005 05:23:35 -0700 In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v730) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: David R Oran Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode Date: Tue, 5 Jul 2005 08:24:23 -0400 To: Klaus Reifenrath X-Mailer: Apple Mail (2.730) IIM-SIG: v:"1.1"; h:"imail.cisco.com"; d:"cisco.com"; z:"home"; m:"krs"; t:"1120566215.561866"; x:"432200"; a:"rsa-sha1"; b:"nofws:1020"; e:"Iw=="; n:"sQYarK2E51MdcTiUqeif3F7cWdxIfoCiXhdfb9vD5ee/j0jXL15gbFxF2p" "XIweAblu0N6XAgK7k+wrbr7bQDJaCDqOmzqpRUBjIRQAXQ7NzadpmR3pUL6wxaRUtW+c43sl9jC" "50Qg1sXHpPjt8Y+Y16ioyQAQAdSunM4YhevURc="; s:"gHG7lLJoD+JURa73wdD9Kptv7NH6lVON2bXrFnYVMGMRCh92frsfWoqB1pWWzhQB0kzmrGmx" "NJMJQ1yNXBrcJv1kZk56DxdWvBbclPrRRqIDkC8Ou6yI1c/gNd3LBs8C4WtaAJh0pODskqWXqP1" "eLIdLUb6toFKxxivW9Nal1Uk="; c:"From: David R Oran "; c:"Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode"; c:"Date: Tue, 5 Jul 2005 08:24:23 -0400" IIM-VERIFY: s:"y"; v:"y"; r:"60"; h:"imail.cisco.com"; c:"message from imail.cisco.com verified; " X-Spam-Score: 0.0 (/) X-Scan-Signature: e5ba305d0e64821bf3d8bc5d3bb07228 Content-Transfer-Encoding: 7bit Cc: "'speechsc@ietf.org'" X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: > The current spec is not clear when START-OF-SPEECH need to be send > in the following scenarios: > A) The client requested a DTMF Recognizer. Is the START-OF-SPEECH > event send to the client also if speech was detected? I suspect so, since one of the prime purposes is to enable client- mediated barge-in handling. However, if the recognizer is in fact only capable of recognizing DTMF then it may in fact not report anythin unless it's using some primitive thresholding machinery, like a SN threshold. > B) The client requested a Speech Recognizer, but only activated > DTMF grammars. Is the START-OF-SPEECH event send to the client also > if speech was detected? Again, I'd say yes, for the same reason as above. > I think in both cases START-OF-SPEECH should only be send after > detecting a DTMF digit (see Figure 12 of VoiceXML 2.0: http:// > www.w3.org/TR/voicexml20/#dmlATiming). We seem to have reached different conclusions. I'd be interested in why you think my analysis above is wrong. Dave. > Klaus > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Tue Jul 05 09:19:12 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DpnKR-0006hF-PD; Tue, 05 Jul 2005 09:19:11 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DpnK8-0006Z0-PW for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 09:18:53 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA14334 for ; Tue, 5 Jul 2005 09:18:48 -0400 (EDT) Received: from fw01.db01.voxpilot.com ([212.17.54.82] helo=mail.voxpilot.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DpjRo-00052n-GZ for speechsc@ietf.org; Tue, 05 Jul 2005 05:10:33 -0400 Received: from daburkewxp (unknown [10.0.0.203]) by mail.voxpilot.com (Postfix) with ESMTP id 1F58D214042; Tue, 5 Jul 2005 08:42:38 +0000 (GMT) Message-ID: <034b01c5813d$7efbb360$cb00000a@db01.voxpilot.com> From: "Dave Burke" To: "Reifenrath, Klaus" , References: Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode Date: Tue, 5 Jul 2005 09:42:37 +0100 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.9 (/) X-Scan-Signature: 3fbd9b434023f8abfcb1532abaec7a21 Cc: X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============1544096530==" Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org This is a multi-part message in MIME format. --===============1544096530== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0348_01C58145.E09B7C60" This is a multi-part message in MIME format. ------=_NextPart_000_0348_01C58145.E09B7C60 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable START-OF-SPEECH in DTMF-only modeGood catch.=20 There has to be a way to select the input mode to dtmf, speech, or both. = For example, an application in a noisy environment might fall back to = DTMF (but still use its speechrecog resource). If (B) were true, then = the fall back would not work due to bargin from noise! For me, choosing a dtmfrecog implies you only want a DTMF input mode. = Choosing a speechrecog is ambigious - there are now 3 choices of input = mode.=20 Using the mode attribute of the activated grammars to decide the input = mode will work but this is NOT what VoiceXML does. For example, I am not = sure how MRCP would then let me implement "Grammar activation is not = affected by the inputmodes property. For instance, if the inputmodes = property restricts input to just voice, DTMF grammars will still be = activated, but cannot be matched." Sounds like a case of YAMHR (yet another MRCP header required). Dave ----- Original Message -----=20 From: Reifenrath, Klaus=20 To: 'speechsc@ietf.org'=20 Sent: Tuesday, July 05, 2005 8:46 AM Subject: [Speechsc] START-OF-SPEECH in DTMF-only mode The current spec is not clear when START-OF-SPEECH need to be send in = the following scenarios:=20 A) The client requested a DTMF Recognizer. Is the START-OF-SPEECH = event send to the client also if speech was detected?=20 B) The client requested a Speech Recognizer, but only activated DTMF = grammars. Is the START-OF-SPEECH event send to the client also if speech = was detected? I think in both cases START-OF-SPEECH should only be send after = detecting a DTMF digit (see Figure 12 of VoiceXML 2.0: = http://www.w3.org/TR/voicexml20/#dmlATiming). Klaus=20 -------------------------------------------------------------------------= ----- _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc ------=_NextPart_000_0348_01C58145.E09B7C60 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable START-OF-SPEECH in DTMF-only mode

Good catch.

There has to be a way to select the = input mode to=20 dtmf, speech, or both. For example, an application in a noisy = environment might=20 fall back to DTMF (but still use its speechrecog resource). If (B) were = true,=20 then the fall back would not work due to bargin from noise!

For me, choosing a dtmfrecog = implies you only=20 want a DTMF input mode. Choosing a speechrecog is ambigious - there are = now 3=20 choices of input mode.

Using the mode attribute of the = activated=20 grammars to decide the input mode will work but this is NOT what = VoiceXML does.=20 For example, I am not sure how MRCP would then let me implement "Grammar = activation is not affected by the inputmodes property. For instance, if = the=20 inputmodes property restricts input to just voice, DTMF grammars will = still be=20 activated, but cannot be matched."

Sounds like a case of YAMHR (yet = another MRCP=20 header required).

Dave

----- Original Message -----

From:=20 Reifenrath, Klaus =

To: 'speechsc@ietf.org'

Sent: Tuesday, July 05, 2005 = 8:46=20 AM

Subject: [Speechsc] = START-OF-SPEECH in=20 DTMF-only mode

The current spec is not clear when = START-OF-SPEECH=20 need to be send in the following scenarios:
A) The client requested a DTMF Recognizer. Is the = START-OF-SPEECH event=20 send to the client also if speech was detected?

B) The client requested a Speech = Recognizer, but=20 only activated DTMF grammars. Is the START-OF-SPEECH event send to the = client=20 also if speech was detected?

I think in both cases START-OF-SPEECH = should only=20 be send after detecting a DTMF digit (see Figure 12 of VoiceXML 2.0:=20 http://www.w3.org/TR/voicexml20/#dmlATiming).

Klaus

_______________________________________________
Speechsc = mailing=20 = list
Speechsc@ietf.org
https://www1.ietf.org/mailman/listinfo/speec= hsc
------=_NextPart_000_0348_01C58145.E09B7C60-- --===============1544096530== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc --===============1544096530==-- From speechsc-bounces@ietf.org Tue Jul 05 11:40:43 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DppXM-0000VH-CL; Tue, 05 Jul 2005 11:40:40 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DppX9-0000OH-V0 for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 11:40:30 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA11146 for ; Tue, 5 Jul 2005 11:40:25 -0400 (EDT) Received: from letter.nuance.com ([207.107.210.132]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1Dppw2-0001eP-BM for speechsc@ietf.org; Tue, 05 Jul 2005 12:06:11 -0400 Received: from postcard.nuance.com ([10.3.6.20]:16689) by letter.nuance.com with esmtp id 1DppV0-0007sj-Jx; Tue, 05 Jul 2005 08:38:14 -0700 Received: from mtb1exch01.nuance.com ([10.3.2.6]) by postcard.nuance.com with Microsoft SMTPSVC(6.0.3790.0); Tue, 5 Jul 2005 11:38:09 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode Date: Tue, 5 Jul 2005 11:38:07 -0400 Message-ID: <7DE7C4EF3B7C8B4B82955191378290D802ED3CE2@mtb1exch01.nuance.com> Thread-Topic: [Speechsc] START-OF-SPEECH in DTMF-only mode Thread-Index: AcWBYODQKOULysVUQeeQX5iGXSpUGQAFlHkA From: "Pierre Forgues" To: "David R Oran" , "Klaus Reifenrath" X-OriginalArrivalTime: 05 Jul 2005 15:38:09.0412 (UTC) FILETIME=[8BA8E440:01C58177] X-FromHost: postcard.nuance.com [10.3.6.20]:16689 Lines: 57 X-Spam-Score: 0.0 (/) X-Scan-Signature: 5a9a1bd6c2d06a21d748b7d0070ddcb8 Content-Transfer-Encoding: quoted-printable Cc: speechsc@ietf.org X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org The START-OF-SPEECH event should be sent to the client whether the user spoke or pressed DTMF. I agree with Dave Oran's comment that it applies to both scenarios mentioned below. The action to take on the client side is likely going to be the same in both cases, i.e. to stop the prompt. Pierre -----Original Message----- From: speechsc-bounces@ietf.org [mailto:speechsc-bounces@ietf.org] On Behalf Of David R Oran Sent: Tuesday, July 05, 2005 8:24 AM To: Klaus Reifenrath Cc: 'speechsc@ietf.org' Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: > The current spec is not clear when START-OF-SPEECH need to be send =20 > in the following scenarios: > A) The client requested a DTMF Recognizer. Is the START-OF-SPEECH =20 > event send to the client also if speech was detected? I suspect so, since one of the prime purposes is to enable client-=20 mediated barge-in handling. However, if the recognizer is in fact =20 only capable of recognizing DTMF then it may in fact not report =20 anythin unless it's using some primitive thresholding machinery, like =20 a SN threshold. > B) The client requested a Speech Recognizer, but only activated =20 > DTMF grammars. Is the START-OF-SPEECH event send to the client also =20 > if speech was detected? Again, I'd say yes, for the same reason as above. > I think in both cases START-OF-SPEECH should only be send after =20 > detecting a DTMF digit (see Figure 12 of VoiceXML 2.0: http://=20 > www.w3.org/TR/voicexml20/#dmlATiming). We seem to have reached different conclusions. I'd be interested in =20 why you think my analysis above is wrong. Dave. > Klaus > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc =20 =20 _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Tue Jul 05 12:36:59 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DpqPr-0002dz-2k; Tue, 05 Jul 2005 12:36:59 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DpqJx-0008Sc-SQ for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 12:30:54 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA18251 for ; Tue, 5 Jul 2005 12:30:46 -0400 (EDT) Received: from sj-iport-4.cisco.com ([171.68.10.86]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DpqcZ-0005Tk-MZ for speechsc@ietf.org; Tue, 05 Jul 2005 12:50:10 -0400 Received: from sj-core-3.cisco.com (171.68.223.137) by sj-iport-4.cisco.com with ESMTP; 05 Jul 2005 09:22:15 -0700 Received: from vtg-um-e2k6.sj21ad.cisco.com (vtg-um-e2k6.cisco.com [171.70.93.77]) by sj-core-3.cisco.com (8.12.10/8.12.6) with ESMTP id j65GMC6p019439; Tue, 5 Jul 2005 09:22:13 -0700 (PDT) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode Date: Tue, 5 Jul 2005 09:22:12 -0700 Message-ID: <03772D1EC8DE624A863058C75874A75C112692@vtg-um-e2k6.sj21ad.cisco.com> Thread-Topic: [Speechsc] START-OF-SPEECH in DTMF-only mode Thread-Index: AcWBYQywOyk/8ZziSciIVsVduk7X1AAHB1WQ From: "Shanmugham, Saravanan" To: "David R Oran" , "Klaus Reifenrath" X-Spam-Score: 0.0 (/) X-Scan-Signature: f607d15ccc2bc4eaf3ade8ffa8af02a0 Content-Transfer-Encoding: quoted-printable Cc: speechsc@ietf.org X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org I agree with Dave's analysis. The purpose of this event was barge-in.=20 And barge-in should happen for both DTMF and speech. Is there a case where you think it should not behave this way. If soe, please provide a scenario where you think 1. Barge-in should happen for DTMF and not voice or vice-versa. 2. You would benefit from the client knowing what caused the barge-in, DTMF Vs speech. Sarvi -----Original Message----- From: speechsc-bounces@ietf.org=20 [mailto:speechsc-bounces@ietf.org] On Behalf Of David R Oran Sent: Tuesday, July 05, 2005 5:24 AM To: Klaus Reifenrath Cc: 'speechsc@ietf.org' Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode =20 =20 On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: =20 > The current spec is not clear when START-OF-SPEECH need=20 to be send in=20 > the following scenarios: > A) The client requested a DTMF Recognizer. Is the=20 START-OF-SPEECH=20 > event send to the client also if speech was detected? I suspect so, since one of the prime purposes is to enable=20 client- mediated barge-in handling. However, if the=20 recognizer is in fact only capable of recognizing DTMF=20 then it may in fact not report anythin unless it's using=20 some primitive thresholding machinery, like a SN threshold. > B) The client requested a Speech Recognizer, but only=20 activated DTMF=20 > grammars. Is the START-OF-SPEECH event send to the=20 client also if=20 > speech was detected? Again, I'd say yes, for the same reason as above. > I think in both cases START-OF-SPEECH should only be send after=20 > detecting a DTMF digit (see Figure 12 of VoiceXML 2.0: http://=20 > www.w3.org/TR/voicexml20/#dmlATiming). We seem to have reached different conclusions. I'd be=20 interested in why you think my analysis above is wrong. =20 Dave. =20 > Klaus > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > =20 _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc =20 _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Tue Jul 05 13:04:20 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DpqqJ-0003Tu-DK; Tue, 05 Jul 2005 13:04:19 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dpqq8-0003Qm-A2 for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 13:04:08 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA23855 for ; Tue, 5 Jul 2005 13:04:03 -0400 (EDT) Received: from fw01.db01.voxpilot.com ([212.17.54.82] helo=mail.voxpilot.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1Dpr7T-0008J5-1J for speechsc@ietf.org; Tue, 05 Jul 2005 13:22:04 -0400 Received: from daburkewxp (unknown [10.0.0.203]) by mail.voxpilot.com (Postfix) with ESMTP id D4944214042; Tue, 5 Jul 2005 16:54:15 +0000 (GMT) Message-ID: <052f01c58182$2d78f300$cb00000a@db01.voxpilot.com> From: "Dave Burke" To: "Shanmugham, Saravanan" , "David R Oran" , "Klaus Reifenrath" References: <03772D1EC8DE624A863058C75874A75C112692@vtg-um-e2k6.sj21ad.cisco.com> Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode Date: Tue, 5 Jul 2005 17:54:15 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.0 (/) X-Scan-Signature: f66b12316365a3fe519e75911daf28a8 Content-Transfer-Encoding: 7bit Cc: speechsc@ietf.org X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org Inline. Dave ----- Original Message ----- From: "Shanmugham, Saravanan" To: "David R Oran" ; "Klaus Reifenrath" Cc: Sent: Tuesday, July 05, 2005 5:22 PM Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode I agree with Dave's analysis. The purpose of this event was barge-in. And barge-in should happen for both DTMF and speech. Is there a case where you think it should not behave this way. If soe, please provide a scenario where you think 1. Barge-in should happen for DTMF and not voice or vice-versa. DB> You want to do a DTMF recognition only because it is noisy. While waiting for DTMF input, the speechrecog resource (or advanced dtmfrecog) generates a START-OF-SPEECH because it heard some speech. The client does not want to stop prompt playing unless DTMF was heard but it can't tell by the START-OF-SPEECH whether speech or DTMF was heard. Similarly vice versa. 2. You would benefit from the client knowing what caused the barge-in, DTMF Vs speech. DB> See previous comment. And previous e-mail: either add an inputmodes header (taking value speech, dtmf, both) to the RECOGNIZE request or add a header to the START-OF-SPEECH event indicating DTMF or speech. Sarvi -----Original Message----- From: speechsc-bounces@ietf.org [mailto:speechsc-bounces@ietf.org] On Behalf Of David R Oran Sent: Tuesday, July 05, 2005 5:24 AM To: Klaus Reifenrath Cc: 'speechsc@ietf.org' Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: > The current spec is not clear when START-OF-SPEECH need to be send in > the following scenarios: > A) The client requested a DTMF Recognizer. Is the START-OF-SPEECH > event send to the client also if speech was detected? I suspect so, since one of the prime purposes is to enable client- mediated barge-in handling. However, if the recognizer is in fact only capable of recognizing DTMF then it may in fact not report anythin unless it's using some primitive thresholding machinery, like a SN threshold. > B) The client requested a Speech Recognizer, but only activated DTMF > grammars. Is the START-OF-SPEECH event send to the client also if > speech was detected? Again, I'd say yes, for the same reason as above. > I think in both cases START-OF-SPEECH should only be send after > detecting a DTMF digit (see Figure 12 of VoiceXML 2.0: http:// > www.w3.org/TR/voicexml20/#dmlATiming). We seem to have reached different conclusions. I'd be interested in why you think my analysis above is wrong. Dave. > Klaus > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Tue Jul 05 13:20:05 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dpr5Y-0006nH-QM; Tue, 05 Jul 2005 13:20:04 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dpr5U-0006hO-5z for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 13:20:01 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA25491 for ; Tue, 5 Jul 2005 13:19:56 -0400 (EDT) Received: from sj-iport-5.cisco.com ([171.68.10.87]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DprUe-0001i8-4S for speechsc@ietf.org; Tue, 05 Jul 2005 13:46:00 -0400 Received: from sj-core-3.cisco.com (171.68.223.137) by sj-iport-5.cisco.com with ESMTP; 05 Jul 2005 10:18:07 -0700 X-IronPort-AV: i="3.93,261,1115017200"; d="scan'208"; a="196414407:sNHT53070004" Received: from vtg-um-e2k6.sj21ad.cisco.com (vtg-um-e2k6.cisco.com [171.70.93.77]) by sj-core-3.cisco.com (8.12.10/8.12.6) with ESMTP id j65HI46p013563; Tue, 5 Jul 2005 10:18:04 -0700 (PDT) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode Date: Tue, 5 Jul 2005 10:18:03 -0700 Message-ID: <03772D1EC8DE624A863058C75874A75C1126AD@vtg-um-e2k6.sj21ad.cisco.com> Thread-Topic: [Speechsc] START-OF-SPEECH in DTMF-only mode Thread-Index: AcWBgjwBvP5/iTu5R72fvf7+5Qgx4wAAH1JA From: "Shanmugham, Saravanan" To: "Dave Burke" , "David R Oran" , "Klaus Reifenrath" X-Spam-Score: 0.0 (/) X-Scan-Signature: 36b1f8810cb91289d885dc8ab4fc8172 Content-Transfer-Encoding: quoted-printable Cc: speechsc@ietf.org X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org Barge-in does not necessarily happen only through the START-OF-SPEECH event. Note that if both the ASR and TTS resources are on the same box they could optimize internally for better performance. So I think what you are suggesting that is that the recognizer should be sensitive to what input is being chosen, when deciding when to generate the START-OF-SPEECH event. I was going to say, this could be based on the grammar and whether it contains tokens other than digits or not. But I guess even if they contained only digits we could still either speak it or press it and the recognizer should be able to deal with both. In which case the problem you are raising becomes less of a barge-in issue. It more about restricting the recognizer to do recognition in a speech-only or dtmf-only or both modes and whether we want to allow it.=20 If we want to DTMF-only, we could always allocate a DTMF-only recognizer and thus not waste precious speech resources for that channel. If we want to do both a regular recognizer resource can do both. The question then becomes, is there a case where you may want to do speech only recognition or not. Is there a use case for this restriction. The noisy channel argument does not apply here I am guessing.=20 To address the barge-in problem we could just add some text to clarify that a dtmf-recog resource generates the barge-in event only for DTMF digits while the speech-recog resource generates the barge-in event for both speech or DTMF. Sarvi -----Original Message----- From: Dave Burke [mailto:david.burke@voxpilot.com]=20 Sent: Tuesday, July 05, 2005 9:54 AM To: Shanmugham, Saravanan; David R Oran; Klaus Reifenrath Cc: speechsc@ietf.org Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode =20 Inline. =20 Dave =20 ----- Original Message ----- From: "Shanmugham, Saravanan" To: "David R Oran" ; "Klaus Reifenrath"=20 Cc: Sent: Tuesday, July 05, 2005 5:22 PM Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode =20 =20 I agree with Dave's analysis. The purpose of this event=20 was barge-in. And barge-in should happen for both DTMF and speech. =20 Is there a case where you think it should not behave this=20 way. If soe, please provide a scenario where you think 1. Barge-in should happen for DTMF and not voice or vice-versa. =20 DB> You want to do a DTMF recognition only because it is=20 noisy. While=20 waiting for DTMF input, the speechrecog resource (or=20 advanced dtmfrecog)=20 generates a START-OF-SPEECH because it heard some speech.=20 The client does=20 not want to stop prompt playing unless DTMF was heard but=20 it can't tell by=20 the START-OF-SPEECH whether speech or DTMF was heard.=20 Similarly vice versa. =20 2. You would benefit from the client knowing what caused the barge-in, DTMF Vs speech. =20 DB> See previous comment. And previous e-mail: either add=20 an inputmodes=20 header (taking value speech, dtmf, both) to the RECOGNIZE=20 request or add a=20 header to the START-OF-SPEECH event indicating DTMF or speech. =20 Sarvi =20 -----Original Message----- From: speechsc-bounces@ietf.org [mailto:speechsc-bounces@ietf.org] On Behalf Of David R Oran Sent: Tuesday, July 05, 2005 5:24 AM To: Klaus Reifenrath Cc: 'speechsc@ietf.org' Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode =20 =20 On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: =20 > The current spec is not clear when START-OF-SPEECH need to be send in > the following scenarios: > A) The client requested a DTMF Recognizer. Is the START-OF-SPEECH > event send to the client also if speech was detected? I suspect so, since one of the prime purposes is to enable client- mediated barge-in handling. However, if the recognizer is in fact only capable of recognizing DTMF then it may in fact not report anythin unless it's using some primitive thresholding machinery, like a SN threshold. > B) The client requested a Speech Recognizer, but only activated DTMF > grammars. Is the START-OF-SPEECH event send to the client also if > speech was detected? Again, I'd say yes, for the same reason as above. > I think in both cases START-OF-SPEECH should only=20 be send after > detecting a DTMF digit (see Figure 12 of VoiceXML=20 2.0: http:// > www.w3.org/TR/voicexml20/#dmlATiming). We seem to have reached different conclusions. I'd be interested in why you think my analysis above is wrong. =20 Dave. =20 > Klaus > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > =20 _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc =20 =20 _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc =20 _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Tue Jul 05 14:08:52 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dprqm-0005q8-Pc; Tue, 05 Jul 2005 14:08:52 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dprqi-0005ow-KE for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 14:08:51 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01499 for ; Tue, 5 Jul 2005 14:08:47 -0400 (EDT) Received: from sj-iport-2-in.cisco.com ([171.71.176.71] helo=sj-iport-2.cisco.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DpsHL-0008Mo-8u for speechsc@ietf.org; Tue, 05 Jul 2005 14:36:21 -0400 Received: from sj-core-2.cisco.com (171.71.177.254) by sj-iport-2.cisco.com with ESMTP; 05 Jul 2005 11:08:26 -0700 Received: from imail.cisco.com (imail.cisco.com [128.107.200.91]) by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id j65I8Lod010674; Tue, 5 Jul 2005 11:08:21 -0700 (PDT) Received: from [10.32.245.152] (stealth-10-32-245-152.cisco.com [10.32.245.152]) by imail.cisco.com (8.12.11/8.12.10) with SMTP id j65I7XvD002261; Tue, 5 Jul 2005 11:07:33 -0700 In-Reply-To: <052f01c58182$2d78f300$cb00000a@db01.voxpilot.com> References: <03772D1EC8DE624A863058C75874A75C112692@vtg-um-e2k6.sj21ad.cisco.com> <052f01c58182$2d78f300$cb00000a@db01.voxpilot.com> Mime-Version: 1.0 (Apple Message framework v730) X-Priority: 3 Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: David R Oran Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode Date: Tue, 5 Jul 2005 14:08:22 -0400 To: Dave Burke X-Mailer: Apple Mail (2.730) IIM-SIG: v:"1.1"; h:"imail.cisco.com"; d:"cisco.com"; z:"home"; m:"krs"; t:"1120586854.625253"; x:"432200"; a:"rsa-sha1"; b:"nofws:4040"; e:"Iw=="; n:"sQYarK2E51MdcTiUqeif3F7cWdxIfoCiXhdfb9vD5ee/j0jXL15gbFxF2p" "XIweAblu0N6XAgK7k+wrbr7bQDJaCDqOmzqpRUBjIRQAXQ7NzadpmR3pUL6wxaRUtW+c43sl9jC" "50Qg1sXHpPjt8Y+Y16ioyQAQAdSunM4YhevURc="; s:"AbFyUerOK3WlIVi9M59G81DqcQAzyS2gdPqEYeWd+Y5y3uUpIXbwrCAwClmHZ6pNnDY8pN7O" "Zi++oNDwMwcsEQ6rX/EJiSSJjCLjlRmJgI+eJblXiM8ZDXMPv+33pI3Wfvbl5f0nsahfIM2+Tq+" "OTAaQFYNGHz95lLfOANrVJnE="; c:"From: David R Oran "; c:"Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode"; c:"Date: Tue, 5 Jul 2005 14:08:22 -0400" IIM-VERIFY: s:"y"; v:"y"; r:"60"; h:"imail.cisco.com"; c:"message from imail.cisco.com verified; " X-Spam-Score: 0.0 (/) X-Scan-Signature: b2809b6f39decc6de467dcf252f42af1 Content-Transfer-Encoding: 7bit Cc: speechsc@ietf.org, "Shanmugham, Saravanan" , Klaus Reifenrath X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org On Jul 5, 2005, at 12:54 PM, Dave Burke wrote: > Inline. > > Dave > > ----- Original Message ----- From: "Shanmugham, Saravanan" > > To: "David R Oran" ; "Klaus Reifenrath" > > Cc: > Sent: Tuesday, July 05, 2005 5:22 PM > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > I agree with Dave's analysis. The purpose of this event was barge-in. > And barge-in should happen for both DTMF and speech. > > Is there a case where you think it should not behave this way. If soe, > please provide a scenario where you think > 1. Barge-in should happen for DTMF and not voice or vice-versa. > > DB> You want to do a DTMF recognition only because it is noisy. > While waiting for DTMF input, the speechrecog resource (or advanced > dtmfrecog) generates a START-OF-SPEECH because it heard some > speech. The client does not want to stop prompt playing unless DTMF > was heard but it can't tell by the START-OF-SPEECH whether speech > or DTMF was heard. Similarly vice versa. > It's an interesting design question what part of the policy resides at the client and what at the server, and who makes the "final decision" about whether what was heard was relevant to the control channel. Right now we (IMO) have a weird partitioning in many cases where the client basically says "do what I mean", but there are no constraints of what the server actually does, and no normalized basis for the client to figure out what to set various magic numbers to (e.g. sensitivity). In this case the only thing the client needs to decide is whether to kill the prompt because the server thinks something that would interfere with the feedback ear/mouth/finger control happened. What this says to me is that it isn't necessarily a good idea for the client to have more knobs to control the server (especially if those knows are just more value/policy input ungrounded in any physics/ acoustics). On the other hand, having the server tell the client more about what it thinks is going on is probably valuable. So, Coming to the point after this long rambling introduction, I think it would in fact be useful for the START-Of-SPEECH event to indicate some extra information, for example: a) I got something enough above the noise floor to qualify for exceeding the "Sensisitvity" parameter you sent in on the request but I really can't tell what it is (could be a hippopatmus fart, or a siren in the background, or captain crunch trying to whistle DTMF). b) I think I'm hearing speech c) I think I'm hearing DTMF > 2. You would benefit from the client knowing what caused the > barge-in, DTMF Vs speech. > > DB> See previous comment. And previous e-mail: either add an > inputmodes header (taking value speech, dtmf, both) to the > RECOGNIZE request or add a header to the START-OF-SPEECH event > indicating DTMF or speech. > I'm leaning in your direction on this latter point - as should be evident from what I wrote above. > Sarvi > > -----Original Message----- > From: speechsc-bounces@ietf.org > [mailto:speechsc-bounces@ietf.org] On Behalf Of David R Oran > Sent: Tuesday, July 05, 2005 5:24 AM > To: Klaus Reifenrath > Cc: 'speechsc@ietf.org' > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: > > > The current spec is not clear when START-OF-SPEECH need > to be send in > > the following scenarios: > > A) The client requested a DTMF Recognizer. Is the > START-OF-SPEECH > > event send to the client also if speech was detected? > I suspect so, since one of the prime purposes is to enable > client- mediated barge-in handling. However, if the > recognizer is in fact only capable of recognizing DTMF > then it may in fact not report anythin unless it's using > some primitive thresholding machinery, like a SN threshold. > > B) The client requested a Speech Recognizer, but only > activated DTMF > > grammars. Is the START-OF-SPEECH event send to the > client also if > > speech was detected? > Again, I'd say yes, for the same reason as above. > > I think in both cases START-OF-SPEECH should only be send after > > detecting a DTMF digit (see Figure 12 of VoiceXML 2.0: http:// > > www.w3.org/TR/voicexml20/#dmlATiming). > We seem to have reached different conclusions. I'd be > interested in why you think my analysis above is wrong. > > Dave. > > > Klaus > > > > _______________________________________________ > > Speechsc mailing list > > Speechsc@ietf.org > > https://www1.ietf.org/mailman/listinfo/speechsc > > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Tue Jul 05 16:04:27 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dptec-0004Mz-Se; Tue, 05 Jul 2005 16:04:26 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dptea-0004Ky-Cd for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 16:04:25 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA16366 for ; Tue, 5 Jul 2005 16:04:18 -0400 (EDT) Received: from sj-iport-5.cisco.com ([171.68.10.87]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1Dptta-0006ky-VS for speechsc@ietf.org; Tue, 05 Jul 2005 16:19:55 -0400 Received: from sj-core-3.cisco.com (171.68.223.137) by sj-iport-5.cisco.com with ESMTP; 05 Jul 2005 12:52:01 -0700 X-IronPort-AV: i="3.93,262,1115017200"; d="scan'208"; a="196455833:sNHT31745464" Received: from vtg-um-e2k6.sj21ad.cisco.com (vtg-um-e2k6.cisco.com [171.70.93.77]) by sj-core-3.cisco.com (8.12.10/8.12.6) with ESMTP id j65Jpw6p023568; Tue, 5 Jul 2005 12:51:58 -0700 (PDT) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode Date: Tue, 5 Jul 2005 12:51:57 -0700 Message-ID: <03772D1EC8DE624A863058C75874A75C1126EE@vtg-um-e2k6.sj21ad.cisco.com> Thread-Topic: [Speechsc] START-OF-SPEECH in DTMF-only mode Thread-Index: AcWBjIp7vAuEXNbLT1iALkA9BtfHuwADPUKQ From: "Shanmugham, Saravanan" To: "David R Oran" , "Dave Burke" X-Spam-Score: 0.0 (/) X-Scan-Signature: 0e9ebc0cbd700a87c0637ad0e2c91610 Content-Transfer-Encoding: quoted-printable Cc: speechsc@ietf.org, Klaus Reifenrath X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org inline. -----Original Message----- From: David R Oran [mailto:oran@cisco.com]=20 Sent: Tuesday, July 05, 2005 11:08 AM To: Dave Burke Cc: Shanmugham, Saravanan; Klaus Reifenrath; speechsc@ietf.org Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode =20 =20 On Jul 5, 2005, at 12:54 PM, Dave Burke wrote: =20 > Inline. > > Dave > > ----- Original Message ----- From: "Shanmugham, Saravanan" =20 > > To: "David R Oran" ; "Klaus Reifenrath" =20 > > Cc: > Sent: Tuesday, July 05, 2005 5:22 PM > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > I agree with Dave's analysis. The purpose of this event=20 was barge-in. > And barge-in should happen for both DTMF and speech. > > Is there a case where you think it should not behave=20 this way. If soe,=20 > please provide a scenario where you think > 1. Barge-in should happen for DTMF and not voice or=20 vice-versa. > > DB> You want to do a DTMF recognition only because it is noisy. =20 > While waiting for DTMF input, the speechrecog resource=20 (or advanced > dtmfrecog) generates a START-OF-SPEECH because it heard=20 some speech.=20 > The client does not want to stop prompt playing unless=20 DTMF was heard=20 > but it can't tell by the START-OF-SPEECH whether speech=20 or DTMF was=20 > heard. Similarly vice versa. > It's an interesting design question what part of the=20 policy resides at the client and what at the server, and=20 who makes the "final decision" about whether what was=20 heard was relevant to the control channel. Right now we=20 (IMO) have a weird partitioning in many cases where the=20 client basically says "do what I mean", but there are no=20 constraints of what the server actually does, and no=20 normalized basis for the client to figure out what to set=20 various magic numbers to (e.g. sensitivity). =20 In this case the only thing the client needs to decide is=20 whether to kill the prompt because the server thinks=20 something that would interfere with the feedback=20 ear/mouth/finger control happened. What this says to me is=20 that it isn't necessarily a good idea for the client to=20 have more knobs to control the server (especially if those=20 knows are just more value/policy input ungrounded in any=20 physics/ acoustics). On the other hand, having the server=20 tell the client more about what it thinks is going on is=20 probably valuable. =20 So, Coming to the point after this long rambling=20 introduction, I think it would in fact be useful for the=20 START-Of-SPEECH event to indicate some extra information,=20 for example: a) I got something enough above the noise floor to qualify=20 for exceeding the "Sensisitvity" parameter you sent in on=20 the request but I really can't tell what it is (could be a=20 hippopatmus fart, or a siren in the background, or captain=20 crunch trying to whistle DTMF). b) I think I'm hearing speech c) I think I'm hearing DTMF Though I agree with your former part of your response. I am not sure I agree with your proposed solution.=20 The way I see this problem is that, it is more of what constitues a barge-in event. This boils down to whether it is speech, DTMF or both. This is inturn boils down to what type of recognizer resource we are using, dtmf-recog, speech-recog, and speech-only-recog(we don't have this and I don't think we should add it, but think of this as a place holder that explains the concept). A client knowing what type of barge-in happenned, does not impact the barge-in operation itself as it may be too late(for the optimized barge-in case). It may have other use cases, and if we can identify them, I don't mind adding support for the START-OF-SPEECH event to say what type of barge-in happenned. But that itself does not solve the original problem raised. Refer to my previous response. The solution lies in defining what what is a barge-in event. That boils down to what type of recognition is happenning, dtmf-only, speech-dtmf or speech-only. We do not support speech-only as a resource today, the question is do we need a header to force it.=20 Sarvi =20 =20 =20 > 2. You would benefit from the client knowing what caused the=20 > barge-in, DTMF Vs speech. > > DB> See previous comment. And previous e-mail: either add an > inputmodes header (taking value speech, dtmf, both) to=20 the RECOGNIZE=20 > request or add a header to the START-OF-SPEECH event=20 indicating DTMF=20 > or speech. > I'm leaning in your direction on this latter point - as=20 should be evident from what I wrote above. =20 > Sarvi > > -----Original Message----- > From: speechsc-bounces@ietf.org > [mailto:speechsc-bounces@ietf.org] On Behalf Of David R Oran > Sent: Tuesday, July 05, 2005 5:24 AM > To: Klaus Reifenrath > Cc: 'speechsc@ietf.org' > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: > > > The current spec is not clear when START-OF-SPEECH need > to be send in > > the following scenarios: > > A) The client requested a DTMF Recognizer. Is the > START-OF-SPEECH > > event send to the client also if speech was detected? > I suspect so, since one of the prime purposes is to enable > client- mediated barge-in handling. However, if the > recognizer is in fact only capable of recognizing DTMF > then it may in fact not report anythin unless it's using > some primitive thresholding machinery, like a SN threshold. > > B) The client requested a Speech Recognizer, but only > activated DTMF > > grammars. Is the START-OF-SPEECH event send to the > client also if > > speech was detected? > Again, I'd say yes, for the same reason as above. > > I think in both cases START-OF-SPEECH should only=20 be send after > > detecting a DTMF digit (see Figure 12 of VoiceXML=20 2.0: http:// > > www.w3.org/TR/voicexml20/#dmlATiming). > We seem to have reached different conclusions. I'd be > interested in why you think my analysis above is wrong. > > Dave. > > > Klaus > > > > _______________________________________________ > > Speechsc mailing list > > Speechsc@ietf.org > > https://www1.ietf.org/mailman/listinfo/speechsc > > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > =20 _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Tue Jul 05 16:27:50 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dpu1G-0005h8-2B; Tue, 05 Jul 2005 16:27:50 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dptzr-0003fI-TP for speechsc@megatron.ietf.org; Tue, 05 Jul 2005 16:26:24 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA19100 for ; Tue, 5 Jul 2005 16:21:39 -0400 (EDT) Received: from letter.nuance.com ([207.107.210.132]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DpuDU-0002zx-O4 for speechsc@ietf.org; Tue, 05 Jul 2005 16:40:30 -0400 Received: from postcard.nuance.com ([10.3.6.20]:20768) by letter.nuance.com with esmtp id 1DptmT-0003Xm-PO; Tue, 05 Jul 2005 13:12:33 -0700 Received: from mtb1exch01.nuance.com ([10.3.2.6]) by postcard.nuance.com with Microsoft SMTPSVC(6.0.3790.0); Tue, 5 Jul 2005 16:12:27 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode Date: Tue, 5 Jul 2005 16:12:27 -0400 Message-ID: <7DE7C4EF3B7C8B4B82955191378290D802ED3D53@mtb1exch01.nuance.com> Thread-Topic: [Speechsc] START-OF-SPEECH in DTMF-only mode Thread-Index: AcWBjIp7vAuEXNbLT1iALkA9BtfHuwADPUKQAAEATEA= From: "Pierre Forgues" To: "Shanmugham, Saravanan" , "David R Oran" , "Dave Burke" X-OriginalArrivalTime: 05 Jul 2005 20:12:27.0852 (UTC) FILETIME=[DDA788C0:01C5819D] X-FromHost: postcard.nuance.com [10.3.6.20]:20768 Lines: 201 X-Spam-Score: 0.0 (/) X-Scan-Signature: 311e798ce51dbeacf5cdfcc8e9fda21b Content-Transfer-Encoding: quoted-printable Cc: speechsc@ietf.org, Klaus Reifenrath X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org The client will be told in the recognition results exactly what triggered the START-OF-SPEECH event (dtmf or speech). At the time the SOS event is reported, the knowledge of what triggered the event is not needed to make a decision to stop a prompt (nor does the server even know at that time what triggered it). Pierre -----Original Message----- From: speechsc-bounces@ietf.org [mailto:speechsc-bounces@ietf.org] On Behalf Of Shanmugham, Saravanan Sent: Tuesday, July 05, 2005 3:52 PM To: David R Oran; Dave Burke Cc: speechsc@ietf.org; Klaus Reifenrath Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode inline. -----Original Message----- From: David R Oran [mailto:oran@cisco.com]=20 Sent: Tuesday, July 05, 2005 11:08 AM To: Dave Burke Cc: Shanmugham, Saravanan; Klaus Reifenrath; speechsc@ietf.org Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode =20 =20 On Jul 5, 2005, at 12:54 PM, Dave Burke wrote: =20 > Inline. > > Dave > > ----- Original Message ----- From: "Shanmugham, Saravanan" =20 > > To: "David R Oran" ; "Klaus Reifenrath" =20 > > Cc: > Sent: Tuesday, July 05, 2005 5:22 PM > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > I agree with Dave's analysis. The purpose of this event=20 was barge-in. > And barge-in should happen for both DTMF and speech. > > Is there a case where you think it should not behave=20 this way. If soe,=20 > please provide a scenario where you think > 1. Barge-in should happen for DTMF and not voice or=20 vice-versa. > > DB> You want to do a DTMF recognition only because it is noisy. =20 > While waiting for DTMF input, the speechrecog resource=20 (or advanced > dtmfrecog) generates a START-OF-SPEECH because it heard=20 some speech.=20 > The client does not want to stop prompt playing unless=20 DTMF was heard=20 > but it can't tell by the START-OF-SPEECH whether speech=20 or DTMF was=20 > heard. Similarly vice versa. > It's an interesting design question what part of the=20 policy resides at the client and what at the server, and=20 who makes the "final decision" about whether what was=20 heard was relevant to the control channel. Right now we=20 (IMO) have a weird partitioning in many cases where the=20 client basically says "do what I mean", but there are no=20 constraints of what the server actually does, and no=20 normalized basis for the client to figure out what to set=20 various magic numbers to (e.g. sensitivity). =20 In this case the only thing the client needs to decide is=20 whether to kill the prompt because the server thinks=20 something that would interfere with the feedback=20 ear/mouth/finger control happened. What this says to me is=20 that it isn't necessarily a good idea for the client to=20 have more knobs to control the server (especially if those=20 knows are just more value/policy input ungrounded in any=20 physics/ acoustics). On the other hand, having the server=20 tell the client more about what it thinks is going on is=20 probably valuable. =20 So, Coming to the point after this long rambling=20 introduction, I think it would in fact be useful for the=20 START-Of-SPEECH event to indicate some extra information,=20 for example: a) I got something enough above the noise floor to qualify=20 for exceeding the "Sensisitvity" parameter you sent in on=20 the request but I really can't tell what it is (could be a=20 hippopatmus fart, or a siren in the background, or captain=20 crunch trying to whistle DTMF). b) I think I'm hearing speech c) I think I'm hearing DTMF Though I agree with your former part of your response. I am not sure I agree with your proposed solution.=20 The way I see this problem is that, it is more of what constitues a barge-in event. This boils down to whether it is speech, DTMF or both. This is inturn boils down to what type of recognizer resource we are using, dtmf-recog, speech-recog, and speech-only-recog(we don't have this and I don't think we should add it, but think of this as a place holder that explains the concept). A client knowing what type of barge-in happenned, does not impact the barge-in operation itself as it may be too late(for the optimized barge-in case). It may have other use cases, and if we can identify them, I don't mind adding support for the START-OF-SPEECH event to say what type of barge-in happenned. But that itself does not solve the original problem raised. Refer to my previous response. The solution lies in defining what what is a barge-in event. That boils down to what type of recognition is happenning, dtmf-only, speech-dtmf or speech-only. We do not support speech-only as a resource today, the question is do we need a header to force it.=20 Sarvi =20 =20 =20 > 2. You would benefit from the client knowing what caused the=20 > barge-in, DTMF Vs speech. > > DB> See previous comment. And previous e-mail: either add an > inputmodes header (taking value speech, dtmf, both) to=20 the RECOGNIZE=20 > request or add a header to the START-OF-SPEECH event=20 indicating DTMF=20 > or speech. > I'm leaning in your direction on this latter point - as=20 should be evident from what I wrote above. =20 > Sarvi > > -----Original Message----- > From: speechsc-bounces@ietf.org > [mailto:speechsc-bounces@ietf.org] On Behalf Of David R Oran > Sent: Tuesday, July 05, 2005 5:24 AM > To: Klaus Reifenrath > Cc: 'speechsc@ietf.org' > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: > > > The current spec is not clear when START-OF-SPEECH need > to be send in > > the following scenarios: > > A) The client requested a DTMF Recognizer. Is the > START-OF-SPEECH > > event send to the client also if speech was detected? > I suspect so, since one of the prime purposes is to enable > client- mediated barge-in handling. However, if the > recognizer is in fact only capable of recognizing DTMF > then it may in fact not report anythin unless it's using > some primitive thresholding machinery, like a SN threshold. > > B) The client requested a Speech Recognizer, but only > activated DTMF > > grammars. Is the START-OF-SPEECH event send to the > client also if > > speech was detected? > Again, I'd say yes, for the same reason as above. > > I think in both cases START-OF-SPEECH should only=20 be send after > > detecting a DTMF digit (see Figure 12 of VoiceXML=20 2.0: http:// > > www.w3.org/TR/voicexml20/#dmlATiming). > We seem to have reached different conclusions. I'd be > interested in why you think my analysis above is wrong. > > Dave. > > > Klaus > > > > _______________________________________________ > > Speechsc mailing list > > Speechsc@ietf.org > > https://www1.ietf.org/mailman/listinfo/speechsc > > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > =20 _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc =20 =20 _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Wed Jul 06 05:32:17 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dq6GP-0005OW-EI; Wed, 06 Jul 2005 05:32:17 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dq6GN-0005Lg-2q for speechsc@megatron.ietf.org; Wed, 06 Jul 2005 05:32:15 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA04570 for ; Wed, 6 Jul 2005 05:32:12 -0400 (EDT) Received: from fw01.db01.voxpilot.com ([212.17.54.82] helo=mail.voxpilot.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1Dq6hJ-00054t-CF for speechsc@ietf.org; Wed, 06 Jul 2005 06:00:06 -0400 Received: from daburkewxp (unknown [10.0.0.203]) by mail.voxpilot.com (Postfix) with ESMTP id 6E784214041; Wed, 6 Jul 2005 09:31:51 +0000 (GMT) Message-ID: <073a01c5820d$8a2c4530$cb00000a@db01.voxpilot.com> From: "Dave Burke" To: "Shanmugham, Saravanan" , "David R Oran" References: <03772D1EC8DE624A863058C75874A75C1126EE@vtg-um-e2k6.sj21ad.cisco.com> Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Date: Wed, 6 Jul 2005 10:31:51 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.0 (/) X-Scan-Signature: 223e3c753032a50d5dc4443c921c3fcd Content-Transfer-Encoding: 7bit Cc: speechsc@ietf.org, Klaus Reifenrath X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org + Attempting to summarise: 1. START-OF-SPEECH is useful for the client to know when to stop playing prompts the in non-optimised case 2. START-OF-SPEECH is useful for the client to calculate the bargin time (e.g. VoiceXML 2.1 ) 3. In the optimised case, a bargin automatically stops prompt playing (assuming prompts barginable) 4. Because of the previous point, the question of what input type caused bargin is different and less important to what input type(s) the recogniser is listening for 5. Currently, MRCPv2 has no way of indicating what input type(s) a recogniser is listening for + Why implement 5? i. Noisy case: Need DTMF-only recognition (and may only have a speechrecog) ii. Flexibility: Want speech-only recognition (because a second recogniser is doing hotword on DTMF) + How to implement 5? a. Implicitly: - dtmfrecog: always DTMF-only recognition - speechrecog: depends on active grammar type > if a dtmf grammar is active then DTMF input is "on" > if a speech grammar is active then speech input is "on" b. Explicitly: - Add inputmodes header to RECOGNIZE Option a is David's "do what I mean case"; option b is the extra dial for the client. It is worth noting that VoiceXML uses option b. This allows one to activate both speech grammars and DTMF grammars (and therefore be informed of any errors in the grammars at activation time) but independently turn on whichever input mode you like e.g. perhaps start with "both" then change to "dtmf". Dave Sarvi makes a good point that adding the reason why the START-OF-SPEECH occurred does not fix the optimised bargin case. dtmfrecog - listens for DTMF only speechrecog - listens for DTMF only, or speech only, or speech & DTMF ----- Original Message ----- From: "Shanmugham, Saravanan" To: "David R Oran" ; "Dave Burke" Cc: ; "Klaus Reifenrath" Sent: Tuesday, July 05, 2005 8:51 PM Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode inline. -----Original Message----- From: David R Oran [mailto:oran@cisco.com] Sent: Tuesday, July 05, 2005 11:08 AM To: Dave Burke Cc: Shanmugham, Saravanan; Klaus Reifenrath; speechsc@ietf.org Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode On Jul 5, 2005, at 12:54 PM, Dave Burke wrote: > Inline. > > Dave > > ----- Original Message ----- From: "Shanmugham, Saravanan" > > To: "David R Oran" ; "Klaus Reifenrath" > > Cc: > Sent: Tuesday, July 05, 2005 5:22 PM > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > I agree with Dave's analysis. The purpose of this event was barge-in. > And barge-in should happen for both DTMF and speech. > > Is there a case where you think it should not behave this way. If soe, > please provide a scenario where you think > 1. Barge-in should happen for DTMF and not voice or vice-versa. > > DB> You want to do a DTMF recognition only because it is noisy. > While waiting for DTMF input, the speechrecog resource (or advanced > dtmfrecog) generates a START-OF-SPEECH because it heard some speech. > The client does not want to stop prompt playing unless DTMF was heard > but it can't tell by the START-OF-SPEECH whether speech or DTMF was > heard. Similarly vice versa. > It's an interesting design question what part of the policy resides at the client and what at the server, and who makes the "final decision" about whether what was heard was relevant to the control channel. Right now we (IMO) have a weird partitioning in many cases where the client basically says "do what I mean", but there are no constraints of what the server actually does, and no normalized basis for the client to figure out what to set various magic numbers to (e.g. sensitivity). In this case the only thing the client needs to decide is whether to kill the prompt because the server thinks something that would interfere with the feedback ear/mouth/finger control happened. What this says to me is that it isn't necessarily a good idea for the client to have more knobs to control the server (especially if those knows are just more value/policy input ungrounded in any physics/ acoustics). On the other hand, having the server tell the client more about what it thinks is going on is probably valuable. So, Coming to the point after this long rambling introduction, I think it would in fact be useful for the START-Of-SPEECH event to indicate some extra information, for example: a) I got something enough above the noise floor to qualify for exceeding the "Sensisitvity" parameter you sent in on the request but I really can't tell what it is (could be a hippopatmus fart, or a siren in the background, or captain crunch trying to whistle DTMF). b) I think I'm hearing speech c) I think I'm hearing DTMF Though I agree with your former part of your response. I am not sure I agree with your proposed solution. The way I see this problem is that, it is more of what constitues a barge-in event. This boils down to whether it is speech, DTMF or both. This is inturn boils down to what type of recognizer resource we are using, dtmf-recog, speech-recog, and speech-only-recog(we don't have this and I don't think we should add it, but think of this as a place holder that explains the concept). A client knowing what type of barge-in happenned, does not impact the barge-in operation itself as it may be too late(for the optimized barge-in case). It may have other use cases, and if we can identify them, I don't mind adding support for the START-OF-SPEECH event to say what type of barge-in happenned. But that itself does not solve the original problem raised. Refer to my previous response. The solution lies in defining what what is a barge-in event. That boils down to what type of recognition is happenning, dtmf-only, speech-dtmf or speech-only. We do not support speech-only as a resource today, the question is do we need a header to force it. Sarvi > 2. You would benefit from the client knowing what caused the > barge-in, DTMF Vs speech. > > DB> See previous comment. And previous e-mail: either add an > inputmodes header (taking value speech, dtmf, both) to the RECOGNIZE > request or add a header to the START-OF-SPEECH event indicating DTMF > or speech. > I'm leaning in your direction on this latter point - as should be evident from what I wrote above. > Sarvi > > -----Original Message----- > From: speechsc-bounces@ietf.org > [mailto:speechsc-bounces@ietf.org] On Behalf Of David R Oran > Sent: Tuesday, July 05, 2005 5:24 AM > To: Klaus Reifenrath > Cc: 'speechsc@ietf.org' > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: > > > The current spec is not clear when START-OF-SPEECH need > to be send in > > the following scenarios: > > A) The client requested a DTMF Recognizer. Is the > START-OF-SPEECH > > event send to the client also if speech was detected? > I suspect so, since one of the prime purposes is to enable > client- mediated barge-in handling. However, if the > recognizer is in fact only capable of recognizing DTMF > then it may in fact not report anythin unless it's using > some primitive thresholding machinery, like a SN threshold. > > B) The client requested a Speech Recognizer, but only > activated DTMF > > grammars. Is the START-OF-SPEECH event send to the > client also if > > speech was detected? > Again, I'd say yes, for the same reason as above. > > I think in both cases START-OF-SPEECH should only be send after > > detecting a DTMF digit (see Figure 12 of VoiceXML 2.0: http:// > > www.w3.org/TR/voicexml20/#dmlATiming). > We seem to have reached different conclusions. I'd be > interested in why you think my analysis above is wrong. > > Dave. > > > Klaus > > > > _______________________________________________ > > Speechsc mailing list > > Speechsc@ietf.org > > https://www1.ietf.org/mailman/listinfo/speechsc > > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Wed Jul 06 08:27:07 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dq8zb-0004W4-PZ; Wed, 06 Jul 2005 08:27:07 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1Dq8xa-0002Y2-8L for speechsc@megatron.ietf.org; Wed, 06 Jul 2005 08:25:02 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA11206 for ; Wed, 6 Jul 2005 08:23:35 -0400 (EDT) Received: from sj-iport-3-in.cisco.com ([171.71.176.72] helo=sj-iport-3.cisco.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1Dq9A3-0001QT-H2 for speechsc@ietf.org; Wed, 06 Jul 2005 08:37:57 -0400 Received: from sj-core-2.cisco.com (171.71.177.254) by sj-iport-3.cisco.com with ESMTP; 06 Jul 2005 05:09:53 -0700 X-IronPort-AV: i="3.93,264,1115017200"; d="scan'208"; a="291190975:sNHT107149376" Received: from imail.cisco.com (imail.cisco.com [128.107.200.91]) by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id j66C9lod018559; Wed, 6 Jul 2005 05:09:47 -0700 (PDT) Received: from [10.32.245.152] (stealth-10-32-245-152.cisco.com [10.32.245.152]) by imail.cisco.com (8.12.11/8.12.10) with SMTP id j66C78hi007631; Wed, 6 Jul 2005 05:07:45 -0700 In-Reply-To: <073a01c5820d$8a2c4530$cb00000a@db01.voxpilot.com> References: <03772D1EC8DE624A863058C75874A75C1126EE@vtg-um-e2k6.sj21ad.cisco.com> <073a01c5820d$8a2c4530$cb00000a@db01.voxpilot.com> Mime-Version: 1.0 (Apple Message framework v730) X-Priority: 3 Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <1CC816CD-E142-421F-BB6C-BBE8572A41FD@cisco.com> Content-Transfer-Encoding: 7bit From: David R Oran Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Date: Wed, 6 Jul 2005 08:07:37 -0400 To: "Dave Burke" X-Mailer: Apple Mail (2.730) IIM-SIG: v:"1.1"; h:"imail.cisco.com"; d:"cisco.com"; z:"home"; m:"krs"; t:"1120651738.615169"; x:"432200"; a:"rsa-sha1"; b:"nofws:11557"; e:"Iw=="; n:"sQYarK2E51MdcTiUqeif3F7cWdxIfoCiXhdfb9vD5ee/j0jXL15gbFxF2p" "XIweAblu0N6XAgK7k+wrbr7bQDJaCDqOmzqpRUBjIRQAXQ7NzadpmR3pUL6wxaRUtW+c43sl9jC" "50Qg1sXHpPjt8Y+Y16ioyQAQAdSunM4YhevURc="; s:"BIygREnmXUZTlAsdioB4L61G7FyL2Q+F2FS74SxRWGqKo9e0KFUQE6HH224m7I5crsVyL7JR" "ZldzF3PxpQKPCMimv6bDlrt66tb6/AiNbTn7BC9jyP9O0Z6uzzcWus9xvzzbecBCnBGIx/uFac6" "xhseQjTOmj6yVDPCdkiULrk0="; c:"From: David R Oran "; c:"Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary" "(?)"; c:"Date: Wed, 6 Jul 2005 08:07:37 -0400" IIM-VERIFY: s:"y"; v:"y"; r:"60"; h:"imail.cisco.com"; c:"message from imail.cisco.com verified; " X-Spam-Score: 0.0 (/) X-Scan-Signature: e95407604bef3289cd27cb4f3b3a35b4 Content-Transfer-Encoding: 7bit Cc: speechsc@ietf.org, "Shanmugham, Saravanan" , Klaus Reifenrath X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org On Jul 6, 2005, at 5:31 AM, Dave Burke wrote: > + Attempting to summarise: > > 1. START-OF-SPEECH is useful for the client to know when to stop > playing prompts the in non-optimised case > 2. START-OF-SPEECH is useful for the client to calculate the bargin > time (e.g. VoiceXML 2.1 ) > 3. In the optimised case, a bargin automatically stops prompt > playing (assuming prompts barginable) > 4. Because of the previous point, the question of what input type > caused bargin is different and less important to what input type(s) > the recogniser is listening for I'm not sure I follow point 4. Could you elaborate? > 5. Currently, MRCPv2 has no way of indicating what input type(s) a > recogniser is listening for > Do you mean exactly this, or do you mean "for the client to indicate to the resource what input types it should look for"? > + Why implement 5? > > i. Noisy case: Need DTMF-only recognition (and may only have a > speechrecog) I'm having difficulty following the logic of why noise would necessarily trigger START-OF-SPEECH if you were listening for speech but not DTMF. I suppose you can use a more forgiving discriminator if all you need to tell is if you're getting DTMF, but I've had a number of real-world cases where wind noise was detected as DTMF, and there's always the ambiguity when you have Captain Crunch on the phone. In either case in the non-optimized case it's the client who gets to decide whether an event should be interpreted as barge-in or not, so it seems an aesthetic protocol design decision whether the client tells the server ahead of time what circumstances to generate the START-OF-SPEECH event for, or whether the event gets generated and the client decides based on what's in the event whether it should be treated as barge- in. I suppose one could make the argument that because the spec implies that the event can only be generated once per request that if a DTMF/ speech capable recognizer first hears enough noise to think it's hearing speech and later hears DTMF, the client will declare barge-in when the event comes and not when he DTMF actually gets heard. If that's deemed a problem, we can still handle that in the design where the server just reports what it's hearing by allowing multiple events to be generated during a single request. Between the approach just outlined above, and an approach where the client provides a filter for whether to generate the event or not, I have a mild preference (based on aesthetics rather than some hard engineering tradeoff) for the approach where the server just reports what it's hearing. Having had some useful exchanges on this topic, it also is becoming apparent to me that this event is poorly named, and we should consider renaming it to "INTERESTING-INPUT-HEARD" or something akin to that, because as others have pointed out, a DTMF-only recognizer will never detect "start of speech". Another consideration to fold into the design choice is extensibility. Bear with me through a little gedankenexperiment. Suppose we want to define a new recognizer type, which I'll call the "name that tune" recognizer. The client plays music to the server and the server recognizes musical notes. The grammar is a standard musical notation, augmented with a semantic interpretation that transforms the notes into the title of the tune and provides that as an answer. First, there's no speech involved (or is there...hang on a minute). Second, in order to accommodate the "name that tune" recognizer, we'd have to extend both the client and the server to undetstand a directive as to whether to recognize music or now, inaddition to what the server already knows what to do based on the grammar. If you follow my logic above, whether or not we do that, we have to extend "start-of-speech" to say "I'm hearing music". So far fairly straightforward, but let me now throw in the pathological twist. Suppose what I feed to a combined music/speech recognizer is a work in sprechstimme (spoken music), like the "Geographical Fugue" (aside: this is a wonderful piece of music I highly recommend to anyone interested in small ensemble singing). In this case, the tune could be named by either doing speech or music recognition. Why is there any need for the client to constrain the server as to which it tries to do when it's already told the server what it wants through the grammar? A few other comments below > ii. Flexibility: Want speech-only recognition (because a second > recogniser is doing hotword on DTMF) > I don't see how flexibility is affected by this deisgn choice. If that's what you want, feed the speech-only recognizer a grammar without any DTMF rules. > + How to implement 5? > > a. Implicitly: > - dtmfrecog: always DTMF-only recognition > - speechrecog: depends on active grammar type > > if a dtmf grammar is active then DTMF input is "on" > > if a speech grammar is active then speech input is "on" > > b. Explicitly: > - Add inputmodes header to RECOGNIZE > > Option a is David's "do what I mean case"; option b is the extra > dial for the client. > Actually, that's not the point I was making with "do what I mean", but it's not essential to the discussion so let's move on. > It is worth noting that VoiceXML uses option b. This allows one to > activate both speech grammars and DTMF grammars (and therefore be > informed of any errors in the grammars at activation time) but > independently turn on whichever input mode you like e.g. perhaps > start with "both" then change to "dtmf". > I'm not sure the VXML precedent is relevant here, because the application behind VXML is working a different part of the problem - how to traverse a TUI dialog based on different parts of the input space. In fact, I suspect that the VXML: choice was conditioned more by limitations at the time it was specified than an underlying good design choice. Clearly having to specify this in VXML make the job of handling a TUI with nodes like "Say or press 5" harder rather than easier. Summing up, while I don't feel strongly one way or the other, I have a preference for handling this as follows: a) Rename "START-OF-SPEECH" to "INTERESTING-INPUT-RECEIVED" or something equivalent. b) Include a parameter in the event saying what was interesting about the input you received, with a registry of values which includes: - signal above noise floor - speech - dtmf - (possibly) music c) allow the event to be generated multiple times during a request Note that all of the above I'm saying with my technical hat on and my chair hat off. Putting my chair hat on for a moment, we really need to get this spec to last call, so at some point Eric or I is going to declare rough consensus so we can move on. Dave Oran. > Dave > > > > > > > > > > > > > Sarvi makes a good point that adding the reason why the START-OF- > SPEECH occurred does not fix the optimised bargin case. > > dtmfrecog - listens for DTMF only > speechrecog - listens for DTMF only, or speech only, or speech & DTMF > > > > > > > > > ----- Original Message ----- From: "Shanmugham, Saravanan" > > To: "David R Oran" ; "Dave Burke" > > Cc: ; "Klaus Reifenrath" > > Sent: Tuesday, July 05, 2005 8:51 PM > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > inline. > > -----Original Message----- > From: David R Oran [mailto:oran@cisco.com] > Sent: Tuesday, July 05, 2005 11:08 AM > To: Dave Burke > Cc: Shanmugham, Saravanan; Klaus Reifenrath; speechsc@ietf.org > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > On Jul 5, 2005, at 12:54 PM, Dave Burke wrote: > > > Inline. > > > > Dave > > > > ----- Original Message ----- From: "Shanmugham, Saravanan" > > > > To: "David R Oran" ; "Klaus Reifenrath" > > > > Cc: > > Sent: Tuesday, July 05, 2005 5:22 PM > > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > > > > I agree with Dave's analysis. The purpose of this event > was barge-in. > > And barge-in should happen for both DTMF and speech. > > > > Is there a case where you think it should not behave > this way. If soe, > > please provide a scenario where you think > > 1. Barge-in should happen for DTMF and not voice or > vice-versa. > > > > DB> You want to do a DTMF recognition only because it is noisy. > > While waiting for DTMF input, the speechrecog resource > (or advanced > > dtmfrecog) generates a START-OF-SPEECH because it heard > some speech. > > The client does not want to stop prompt playing unless > DTMF was heard > > but it can't tell by the START-OF-SPEECH whether speech > or DTMF was > > heard. Similarly vice versa. > > > It's an interesting design question what part of the > policy resides at the client and what at the server, and > who makes the "final decision" about whether what was > heard was relevant to the control channel. Right now we > (IMO) have a weird partitioning in many cases where the > client basically says "do what I mean", but there are no > constraints of what the server actually does, and no > normalized basis for the client to figure out what to set > various magic numbers to (e.g. sensitivity). > > In this case the only thing the client needs to decide is > whether to kill the prompt because the server thinks > something that would interfere with the feedback > ear/mouth/finger control happened. What this says to me is > that it isn't necessarily a good idea for the client to > have more knobs to control the server (especially if those > knows are just more value/policy input ungrounded in any > physics/ acoustics). On the other hand, having the server > tell the client more about what it thinks is going on is > probably valuable. > > So, Coming to the point after this long rambling > introduction, I think it would in fact be useful for the > START-Of-SPEECH event to indicate some extra information, > for example: > a) I got something enough above the noise floor to qualify > for exceeding the "Sensisitvity" parameter you sent in on > the request but I really can't tell what it is (could be a > hippopatmus fart, or a siren in the background, or captain > crunch trying to whistle DTMF). > b) I think I'm hearing speech > c) I think I'm hearing DTMF > > Though I agree with your former part of your response. I am not sure I > agree with your proposed solution. > The way I see this problem is that, it is more of what constitues a > barge-in event. This boils down to whether it is speech, DTMF or both. > This is inturn boils down to what type of recognizer resource we are > using, dtmf-recog, speech-recog, and speech-only-recog(we don't have > this and I don't think we should add it, but think of this as a place > holder that explains the concept). > > A client knowing what type of barge-in happenned, does not impact the > barge-in operation itself as it may be too late(for the optimized > barge-in case). It may have other use cases, and if we can identify > them, I don't mind adding support for the START-OF-SPEECH event to say > what type of barge-in happenned. But that itself does not solve the > original problem raised. Refer to my previous response. > > The solution lies in defining what what is a barge-in event. That > boils > down to what type of recognition is happenning, dtmf-only, speech-dtmf > or speech-only. We do not support speech-only as a resource today, the > question is do we need a header to force it. > > Sarvi > > > > 2. You would benefit from the client knowing what caused the > > barge-in, DTMF Vs speech. > > > > DB> See previous comment. And previous e-mail: either add an > > inputmodes header (taking value speech, dtmf, both) to > the RECOGNIZE > > request or add a header to the START-OF-SPEECH event > indicating DTMF > > or speech. > > > I'm leaning in your direction on this latter point - as > should be evident from what I wrote above. > > > Sarvi > > > > -----Original Message----- > > From: speechsc-bounces@ietf.org > > [mailto:speechsc-bounces@ietf.org] On Behalf Of David R Oran > > Sent: Tuesday, July 05, 2005 5:24 AM > > To: Klaus Reifenrath > > Cc: 'speechsc@ietf.org' > > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode > > > > > > On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: > > > > > The current spec is not clear when START-OF-SPEECH need > > to be send in > > > the following scenarios: > > > A) The client requested a DTMF Recognizer. Is the > > START-OF-SPEECH > > > event send to the client also if speech was detected? > > I suspect so, since one of the prime purposes is to enable > > client- mediated barge-in handling. However, if the > > recognizer is in fact only capable of recognizing DTMF > > then it may in fact not report anythin unless it's using > > some primitive thresholding machinery, like a SN threshold. > > > B) The client requested a Speech Recognizer, but only > > activated DTMF > > > grammars. Is the START-OF-SPEECH event send to the > > client also if > > > speech was detected? > > Again, I'd say yes, for the same reason as above. > > > I think in both cases START-OF-SPEECH should only > be send after > > > detecting a DTMF digit (see Figure 12 of VoiceXML > 2.0: http:// > > > www.w3.org/TR/voicexml20/#dmlATiming). > > We seem to have reached different conclusions. I'd be > > interested in why you think my analysis above is wrong. > > > > Dave. > > > > > Klaus > > > > > > _______________________________________________ > > > Speechsc mailing list > > > Speechsc@ietf.org > > > https://www1.ietf.org/mailman/listinfo/speechsc > > > > > > > _______________________________________________ > > Speechsc mailing list > > Speechsc@ietf.org > > https://www1.ietf.org/mailman/listinfo/speechsc > > > > > > _______________________________________________ > > Speechsc mailing list > > Speechsc@ietf.org > > https://www1.ietf.org/mailman/listinfo/speechsc > > > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Wed Jul 06 10:39:38 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqB3q-0000WR-9J; Wed, 06 Jul 2005 10:39:38 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqB3n-0000Rf-6N for speechsc@megatron.ietf.org; Wed, 06 Jul 2005 10:39:37 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA04579 for ; Wed, 6 Jul 2005 10:39:28 -0400 (EDT) Received: from fw01.db01.voxpilot.com ([212.17.54.82] helo=mail.voxpilot.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DqBKr-0001UD-Qb for speechsc@ietf.org; Wed, 06 Jul 2005 10:57:17 -0400 Received: from daburkewxp (unknown [10.0.0.203]) by mail.voxpilot.com (Postfix) with ESMTP id B408E2140F0; Wed, 6 Jul 2005 14:29:12 +0000 (GMT) Message-ID: <087701c58237$147262a0$cb00000a@db01.voxpilot.com> From: "Dave Burke" To: "David R Oran" References: <03772D1EC8DE624A863058C75874A75C1126EE@vtg-um-e2k6.sj21ad.cisco.com><073a01c5820d$8a2c4530$cb00000a@db01.voxpilot.com> <1CC816CD-E142-421F-BB6C-BBE8572A41FD@cisco.com> Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Date: Wed, 6 Jul 2005 15:29:12 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.0 (/) X-Scan-Signature: 14278aea5bdd1edf35ec09ffb7b61f9d Content-Transfer-Encoding: 7bit Cc: speechsc@ietf.org, "Shanmugham, Saravanan" , Klaus Reifenrath X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org Inline. Dave ----- Original Message ----- From: "David R Oran" To: "Dave Burke" Cc: ; "Shanmugham, Saravanan" ; "Klaus Reifenrath" Sent: Wednesday, July 06, 2005 1:07 PM Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) > > On Jul 6, 2005, at 5:31 AM, Dave Burke wrote: > >> + Attempting to summarise: >> >> 1. START-OF-SPEECH is useful for the client to know when to stop playing >> prompts the in non-optimised case >> 2. START-OF-SPEECH is useful for the client to calculate the bargin time >> (e.g. VoiceXML 2.1 ) >> 3. In the optimised case, a bargin automatically stops prompt playing >> (assuming prompts barginable) >> 4. Because of the previous point, the question of what input type caused >> bargin is different and less important to what input type(s) the >> recogniser is listening for > I'm not sure I follow point 4. Could you elaborate? DB> Adding a parameter to the START-OF-SPEECH event would certainly allow the client to ignore (i.e. let prompts continue playing) the event if the event type is not of interest (e.g. the client would ignore speech start events when it is interested only in DTMF start events). This _only_ works for the non-optimised case, however. For the optimised case, assuming START-OF-SPEECH coincides with the bargin signal to the speechsynth, prompts will stop playing for inputs that the client might not be interested in (e.g. a speech input will stop prompts playing even if the client is only interested in DTMF). > >> 5. Currently, MRCPv2 has no way of indicating what input type(s) a >> recogniser is listening for >> > Do you mean exactly this, or do you mean "for the client to indicate to > the resource what input types it should look for"? DB> Yes exactly - apologies for not being clear. > >> + Why implement 5? >> >> i. Noisy case: Need DTMF-only recognition (and may only have a >> speechrecog) > I'm having difficulty following the logic of why noise would necessarily > trigger START-OF-SPEECH if you were listening for speech but not DTMF. I > suppose you can use a more forgiving discriminator if all you need to > tell is if you're getting DTMF, but I've had a number of real-world cases > where wind noise was detected as DTMF, and there's always the ambiguity > when you have Captain Crunch on the phone. In either case in the > non-optimized case it's the client who gets to decide whether an event > should be interpreted as barge-in or not, so it seems an aesthetic > protocol design decision whether the client tells the server ahead of > time what circumstances to generate the START-OF-SPEECH event for, or > whether the event gets generated and the client decides based on what's > in the event whether it should be treated as barge- in. > > I suppose one could make the argument that because the spec implies that > the event can only be generated once per request that if a DTMF/ speech > capable recognizer first hears enough noise to think it's hearing speech > and later hears DTMF, the client will declare barge-in when the event > comes and not when he DTMF actually gets heard. > > If that's deemed a problem, we can still handle that in the design where > the server just reports what it's hearing by allowing multiple events to > be generated during a single request. > > Between the approach just outlined above, and an approach where the > client provides a filter for whether to generate the event or not, I have > a mild preference (based on aesthetics rather than some hard engineering > tradeoff) for the approach where the server just reports what it's > hearing. > > > Having had some useful exchanges on this topic, it also is becoming > apparent to me that this event is poorly named, and we should consider > renaming it to "INTERESTING-INPUT-HEARD" or something akin to that, > because as others have pointed out, a DTMF-only recognizer will never > detect "start of speech". > > Another consideration to fold into the design choice is extensibility. > Bear with me through a little gedankenexperiment. > > Suppose we want to define a new recognizer type, which I'll call the > "name that tune" recognizer. The client plays music to the server and the > server recognizes musical notes. The grammar is a standard musical > notation, augmented with a semantic interpretation that transforms the > notes into the title of the tune and provides that as an answer. > > First, there's no speech involved (or is there...hang on a minute). > Second, in order to accommodate the "name that tune" recognizer, we'd > have to extend both the client and the server to undetstand a directive > as to whether to recognize music or now, inaddition to what the server > already knows what to do based on the grammar. If you follow my logic > above, whether or not we do that, we have to extend "start-of-speech" to > say "I'm hearing music". So far fairly straightforward, but let me now > throw in the pathological twist. > > Suppose what I feed to a combined music/speech recognizer is a work in > sprechstimme (spoken music), like the "Geographical Fugue" (aside: this > is a wonderful piece of music I highly recommend to anyone interested in > small ensemble singing). In this case, the tune could be named by either > doing speech or music recognition. Why is there any need for the client > to constrain the server as to which it tries to do when it's already told > the server what it wants through the grammar? > > A few other comments below > >> ii. Flexibility: Want speech-only recognition (because a second >> recogniser is doing hotword on DTMF) >> > I don't see how flexibility is affected by this deisgn choice. If that's > what you want, feed the speech-only recognizer a grammar without any DTMF > rules. > >> + How to implement 5? >> >> a. Implicitly: >> - dtmfrecog: always DTMF-only recognition >> - speechrecog: depends on active grammar type >> > if a dtmf grammar is active then DTMF input is "on" >> > if a speech grammar is active then speech input is "on" >> >> b. Explicitly: >> - Add inputmodes header to RECOGNIZE >> >> Option a is David's "do what I mean case"; option b is the extra dial >> for the client. >> > Actually, that's not the point I was making with "do what I mean", but > it's not essential to the discussion so let's move on. > >> It is worth noting that VoiceXML uses option b. This allows one to >> activate both speech grammars and DTMF grammars (and therefore be >> informed of any errors in the grammars at activation time) but >> independently turn on whichever input mode you like e.g. perhaps start >> with "both" then change to "dtmf". >> > I'm not sure the VXML precedent is relevant here, because the application > behind VXML is working a different part of the problem - how to traverse > a TUI dialog based on different parts of the input space. In fact, I > suspect that the VXML: choice was conditioned more by limitations at the > time it was specified than an underlying good design choice. Clearly > having to specify this in VXML make the job of handling a TUI with nodes > like "Say or press 5" harder rather than easier. DB> The VoiceXML edge-case is pretty weird so it's not a major concern. My main concern is that the client can indicate, somehow, what the input modes are. > > Summing up, while I don't feel strongly one way or the other, I have a > preference for handling this as follows: > > a) Rename "START-OF-SPEECH" to "INTERESTING-INPUT-RECEIVED" or something > equivalent. > b) Include a parameter in the event saying what was interesting about the > input you received, with a registry of values which includes: > - signal above noise floor > - speech > - dtmf > - (possibly) music > c) allow the event to be generated multiple times during a request > DB> I like these suggestions (START-OF-INPUT?). However, I don't see how the problem of the optimised case is not solved by them. I think the optimised case is fine if we have the following rules: 1. START-OF-SPEECH (and optimised bargin) is only generated for the input type that is being listened for 2. A speechrecog listens for DTMF if DTMF grammars are active, speech if speech grammars are active, or speech and DTMF if both grammar types are active. > Note that all of the above I'm saying with my technical hat on and my > chair hat off. > Putting my chair hat on for a moment, we really need to get this spec to > last call, so at some point Eric or I is going to declare rough consensus > so we can move on. DB> Agreed! > > Dave Oran. >> Dave >> >> >> >> >> >> >> >> >> >> >> >> >> Sarvi makes a good point that adding the reason why the START-OF- SPEECH >> occurred does not fix the optimised bargin case. >> >> dtmfrecog - listens for DTMF only >> speechrecog - listens for DTMF only, or speech only, or speech & DTMF >> >> >> >> >> >> >> >> >> ----- Original Message ----- From: "Shanmugham, Saravanan" >> >> To: "David R Oran" ; "Dave Burke" >> >> Cc: ; "Klaus Reifenrath" >> >> Sent: Tuesday, July 05, 2005 8:51 PM >> Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode >> >> >> inline. >> >> -----Original Message----- >> From: David R Oran [mailto:oran@cisco.com] >> Sent: Tuesday, July 05, 2005 11:08 AM >> To: Dave Burke >> Cc: Shanmugham, Saravanan; Klaus Reifenrath; speechsc@ietf.org >> Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode >> >> >> On Jul 5, 2005, at 12:54 PM, Dave Burke wrote: >> >> > Inline. >> > >> > Dave >> > >> > ----- Original Message ----- From: "Shanmugham, Saravanan" >> > >> > To: "David R Oran" ; "Klaus Reifenrath" >> > >> > Cc: >> > Sent: Tuesday, July 05, 2005 5:22 PM >> > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode >> > >> > >> > I agree with Dave's analysis. The purpose of this event >> was barge-in. >> > And barge-in should happen for both DTMF and speech. >> > >> > Is there a case where you think it should not behave >> this way. If soe, >> > please provide a scenario where you think >> > 1. Barge-in should happen for DTMF and not voice or >> vice-versa. >> > >> > DB> You want to do a DTMF recognition only because it is noisy. >> > While waiting for DTMF input, the speechrecog resource >> (or advanced >> > dtmfrecog) generates a START-OF-SPEECH because it heard >> some speech. >> > The client does not want to stop prompt playing unless >> DTMF was heard >> > but it can't tell by the START-OF-SPEECH whether speech >> or DTMF was >> > heard. Similarly vice versa. >> > >> It's an interesting design question what part of the >> policy resides at the client and what at the server, and >> who makes the "final decision" about whether what was >> heard was relevant to the control channel. Right now we >> (IMO) have a weird partitioning in many cases where the >> client basically says "do what I mean", but there are no >> constraints of what the server actually does, and no >> normalized basis for the client to figure out what to set >> various magic numbers to (e.g. sensitivity). >> >> In this case the only thing the client needs to decide is >> whether to kill the prompt because the server thinks >> something that would interfere with the feedback >> ear/mouth/finger control happened. What this says to me is >> that it isn't necessarily a good idea for the client to >> have more knobs to control the server (especially if those >> knows are just more value/policy input ungrounded in any >> physics/ acoustics). On the other hand, having the server >> tell the client more about what it thinks is going on is >> probably valuable. >> >> So, Coming to the point after this long rambling >> introduction, I think it would in fact be useful for the >> START-Of-SPEECH event to indicate some extra information, >> for example: >> a) I got something enough above the noise floor to qualify >> for exceeding the "Sensisitvity" parameter you sent in on >> the request but I really can't tell what it is (could be a >> hippopatmus fart, or a siren in the background, or captain >> crunch trying to whistle DTMF). >> b) I think I'm hearing speech >> c) I think I'm hearing DTMF >> >> Though I agree with your former part of your response. I am not sure I >> agree with your proposed solution. >> The way I see this problem is that, it is more of what constitues a >> barge-in event. This boils down to whether it is speech, DTMF or both. >> This is inturn boils down to what type of recognizer resource we are >> using, dtmf-recog, speech-recog, and speech-only-recog(we don't have >> this and I don't think we should add it, but think of this as a place >> holder that explains the concept). >> >> A client knowing what type of barge-in happenned, does not impact the >> barge-in operation itself as it may be too late(for the optimized >> barge-in case). It may have other use cases, and if we can identify >> them, I don't mind adding support for the START-OF-SPEECH event to say >> what type of barge-in happenned. But that itself does not solve the >> original problem raised. Refer to my previous response. >> >> The solution lies in defining what what is a barge-in event. That boils >> down to what type of recognition is happenning, dtmf-only, speech-dtmf >> or speech-only. We do not support speech-only as a resource today, the >> question is do we need a header to force it. >> >> Sarvi >> >> >> > 2. You would benefit from the client knowing what caused the >> > barge-in, DTMF Vs speech. >> > >> > DB> See previous comment. And previous e-mail: either add an >> > inputmodes header (taking value speech, dtmf, both) to >> the RECOGNIZE >> > request or add a header to the START-OF-SPEECH event >> indicating DTMF >> > or speech. >> > >> I'm leaning in your direction on this latter point - as >> should be evident from what I wrote above. >> >> > Sarvi >> > >> > -----Original Message----- >> > From: speechsc-bounces@ietf.org >> > [mailto:speechsc-bounces@ietf.org] On Behalf Of David R Oran >> > Sent: Tuesday, July 05, 2005 5:24 AM >> > To: Klaus Reifenrath >> > Cc: 'speechsc@ietf.org' >> > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode >> > >> > >> > On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: >> > >> > > The current spec is not clear when START-OF-SPEECH need >> > to be send in >> > > the following scenarios: >> > > A) The client requested a DTMF Recognizer. Is the >> > START-OF-SPEECH >> > > event send to the client also if speech was detected? >> > I suspect so, since one of the prime purposes is to enable >> > client- mediated barge-in handling. However, if the >> > recognizer is in fact only capable of recognizing DTMF >> > then it may in fact not report anythin unless it's using >> > some primitive thresholding machinery, like a SN threshold. >> > > B) The client requested a Speech Recognizer, but only >> > activated DTMF >> > > grammars. Is the START-OF-SPEECH event send to the >> > client also if >> > > speech was detected? >> > Again, I'd say yes, for the same reason as above. >> > > I think in both cases START-OF-SPEECH should only >> be send after >> > > detecting a DTMF digit (see Figure 12 of VoiceXML >> 2.0: http:// >> > > www.w3.org/TR/voicexml20/#dmlATiming). >> > We seem to have reached different conclusions. I'd be >> > interested in why you think my analysis above is wrong. >> > >> > Dave. >> > >> > > Klaus >> > > >> > > _______________________________________________ >> > > Speechsc mailing list >> > > Speechsc@ietf.org >> > > https://www1.ietf.org/mailman/listinfo/speechsc >> > > >> > >> > _______________________________________________ >> > Speechsc mailing list >> > Speechsc@ietf.org >> > https://www1.ietf.org/mailman/listinfo/speechsc >> > >> > >> > _______________________________________________ >> > Speechsc mailing list >> > Speechsc@ietf.org >> > https://www1.ietf.org/mailman/listinfo/speechsc >> > >> >> >> _______________________________________________ >> Speechsc mailing list >> Speechsc@ietf.org >> https://www1.ietf.org/mailman/listinfo/speechsc >> > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Wed Jul 06 10:55:32 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqBJE-0004RF-4y; Wed, 06 Jul 2005 10:55:32 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqBJB-0004Nt-EX for speechsc@megatron.ietf.org; Wed, 06 Jul 2005 10:55:30 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA06077 for ; Wed, 6 Jul 2005 10:55:22 -0400 (EDT) Received: from sj-iport-1-in.cisco.com ([171.71.176.70] helo=sj-iport-1.cisco.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DqBaX-0005Ao-TR for speechsc@ietf.org; Wed, 06 Jul 2005 11:13:26 -0400 Received: from sj-core-5.cisco.com (171.71.177.238) by sj-iport-1.cisco.com with ESMTP; 06 Jul 2005 07:45:23 -0700 X-IronPort-AV: i="3.93,265,1115017200"; d="scan'208"; a="647275400:sNHT35807128" Received: from imail.cisco.com (imail.cisco.com [128.107.200.91]) by sj-core-5.cisco.com (8.12.10/8.12.6) with ESMTP id j66EjHVX008081; Wed, 6 Jul 2005 07:45:17 -0700 (PDT) Received: from [10.32.245.152] (stealth-10-32-245-152.cisco.com [10.32.245.152]) by imail.cisco.com (8.12.11/8.12.10) with SMTP id j66EiFFR008438; Wed, 6 Jul 2005 07:44:18 -0700 In-Reply-To: <087701c58237$147262a0$cb00000a@db01.voxpilot.com> References: <03772D1EC8DE624A863058C75874A75C1126EE@vtg-um-e2k6.sj21ad.cisco.com><073a01c5820d$8a2c4530$cb00000a@db01.voxpilot.com> <1CC816CD-E142-421F-BB6C-BBE8572A41FD@cisco.com> <087701c58237$147262a0$cb00000a@db01.voxpilot.com> Mime-Version: 1.0 (Apple Message framework v730) X-Priority: 3 Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <11CA998B-CA62-494E-9868-9DC2EEF2DB04@cisco.com> Content-Transfer-Encoding: 7bit From: David R Oran Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Date: Wed, 6 Jul 2005 10:45:06 -0400 To: "Dave Burke" X-Mailer: Apple Mail (2.730) IIM-SIG: v:"1.1"; h:"imail.cisco.com"; d:"cisco.com"; z:"home"; m:"krs"; t:"1120661063.176080"; x:"432200"; a:"rsa-sha1"; b:"nofws:14378"; e:"Iw=="; n:"sQYarK2E51MdcTiUqeif3F7cWdxIfoCiXhdfb9vD5ee/j0jXL15gbFxF2p" "XIweAblu0N6XAgK7k+wrbr7bQDJaCDqOmzqpRUBjIRQAXQ7NzadpmR3pUL6wxaRUtW+c43sl9jC" "50Qg1sXHpPjt8Y+Y16ioyQAQAdSunM4YhevURc="; s:"PGO8a8D3UPb5IC/w63SFaZ1tM/Y8O1fPN40XgZkniOMDr/qQtFYmfj5bGJbXYkMuyUYQmztp" "v9mB4I/rX7mZozwQY6wukWFkdevEEJdZHN2oZtSCZ9A4k33RbkPhtjCqwy8LodyhNBqonK3/7qP" "RdUi/12RZPGrfNHNpZ702b34="; c:"From: David R Oran "; c:"Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary" "(?)"; c:"Date: Wed, 6 Jul 2005 10:45:06 -0400" IIM-VERIFY: s:"y"; v:"y"; r:"60"; h:"imail.cisco.com"; c:"message from imail.cisco.com verified; " X-Spam-Score: 0.0 (/) X-Scan-Signature: d2bb8c7db26f2d12109e0cd8e454db52 Content-Transfer-Encoding: 7bit Cc: speechsc@ietf.org, "Shanmugham, Saravanan" , Klaus Reifenrath X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org I think we're getting close. I though about snipping out some pieces to cut down the text, but I realized the context is still needed. See inline. On Jul 6, 2005, at 10:29 AM, Dave Burke wrote: > Inline. > > Dave > > ----- Original Message ----- From: "David R Oran" > To: "Dave Burke" > Cc: ; "Shanmugham, Saravanan" ; > "Klaus Reifenrath" > Sent: Wednesday, July 06, 2005 1:07 PM > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary > (?) > > > >> >> On Jul 6, 2005, at 5:31 AM, Dave Burke wrote: >> >> >>> + Attempting to summarise: >>> >>> 1. START-OF-SPEECH is useful for the client to know when to stop >>> playing prompts the in non-optimised case >>> 2. START-OF-SPEECH is useful for the client to calculate the >>> bargin time (e.g. VoiceXML 2.1 ) >>> 3. In the optimised case, a bargin automatically stops prompt >>> playing (assuming prompts barginable) >>> 4. Because of the previous point, the question of what input >>> type caused bargin is different and less important to what input >>> type(s) the recogniser is listening for >>> >> I'm not sure I follow point 4. Could you elaborate? >> > > DB> Adding a parameter to the START-OF-SPEECH event would certainly > allow the client to ignore (i.e. let prompts continue playing) the > event if the event type is not of interest (e.g. the client would > ignore speech start events when it is interested only in DTMF > start events). This _only_ works for the non-optimised case, > however. For the optimised case, assuming START-OF-SPEECH coincides > with the bargin signal to the speechsynth, prompts will stop > playing for inputs that the client might not be interested in (e.g. > a speech input will stop prompts playing even if the client is only > interested in DTMF). > OK, now I get it. There's a need for the client to both handle the non-optimized case itself, and influence or at least have a clue what the server is going to do in the optimized case. > >> >> >>> 5. Currently, MRCPv2 has no way of indicating what input type(s) >>> a recogniser is listening for >>> >>> >> Do you mean exactly this, or do you mean "for the client to >> indicate to the resource what input types it should look for"? >> > > DB> Yes exactly - apologies for not being clear. > > >> >> >>> + Why implement 5? >>> >>> i. Noisy case: Need DTMF-only recognition (and may only have a >>> speechrecog) >>> >> I'm having difficulty following the logic of why noise would >> necessarily trigger START-OF-SPEECH if you were listening for >> speech but not DTMF. I suppose you can use a more forgiving >> discriminator if all you need to tell is if you're getting DTMF, >> but I've had a number of real-world cases where wind noise was >> detected as DTMF, and there's always the ambiguity when you have >> Captain Crunch on the phone. In either case in the non-optimized >> case it's the client who gets to decide whether an event should >> be interpreted as barge-in or not, so it seems an aesthetic >> protocol design decision whether the client tells the server >> ahead of time what circumstances to generate the START-OF-SPEECH >> event for, or whether the event gets generated and the client >> decides based on what's in the event whether it should be treated >> as barge- in. >> >> I suppose one could make the argument that because the spec >> implies that the event can only be generated once per request >> that if a DTMF/ speech capable recognizer first hears enough noise >> to think it's hearing speech and later hears DTMF, the client >> will declare barge-in when the event comes and not when he DTMF >> actually gets heard. >> >> If that's deemed a problem, we can still handle that in the >> design where the server just reports what it's hearing by >> allowing multiple events to be generated during a single request. >> >> Between the approach just outlined above, and an approach where >> the client provides a filter for whether to generate the event or >> not, I have a mild preference (based on aesthetics rather than >> some hard engineering tradeoff) for the approach where the server >> just reports what it's hearing. >> >> >> Having had some useful exchanges on this topic, it also is >> becoming apparent to me that this event is poorly named, and we >> should consider renaming it to "INTERESTING-INPUT-HEARD" or >> something akin to that, because as others have pointed out, a >> DTMF-only recognizer will never detect "start of speech". >> >> Another consideration to fold into the design choice is >> extensibility. Bear with me through a little gedankenexperiment. >> >> Suppose we want to define a new recognizer type, which I'll call >> the "name that tune" recognizer. The client plays music to the >> server and the server recognizes musical notes. The grammar is a >> standard musical notation, augmented with a semantic >> interpretation that transforms the notes into the title of the >> tune and provides that as an answer. >> >> First, there's no speech involved (or is there...hang on a >> minute). Second, in order to accommodate the "name that tune" >> recognizer, we'd have to extend both the client and the server to >> undetstand a directive as to whether to recognize music or now, >> inaddition to what the server already knows what to do based on >> the grammar. If you follow my logic above, whether or not we do >> that, we have to extend "start-of-speech" to say "I'm hearing >> music". So far fairly straightforward, but let me now throw in >> the pathological twist. >> >> Suppose what I feed to a combined music/speech recognizer is a >> work in sprechstimme (spoken music), like the "Geographical >> Fugue" (aside: this is a wonderful piece of music I highly >> recommend to anyone interested in small ensemble singing). In >> this case, the tune could be named by either doing speech or >> music recognition. Why is there any need for the client to >> constrain the server as to which it tries to do when it's already >> told the server what it wants through the grammar? >> >> A few other comments below >> >> >>> ii. Flexibility: Want speech-only recognition (because a second >>> recogniser is doing hotword on DTMF) >>> >>> >> I don't see how flexibility is affected by this deisgn choice. If >> that's what you want, feed the speech-only recognizer a grammar >> without any DTMF rules. >> >> >>> + How to implement 5? >>> >>> a. Implicitly: >>> - dtmfrecog: always DTMF-only recognition >>> - speechrecog: depends on active grammar type >>> > if a dtmf grammar is active then DTMF input is "on" >>> > if a speech grammar is active then speech input is "on" >>> >>> b. Explicitly: >>> - Add inputmodes header to RECOGNIZE >>> >>> Option a is David's "do what I mean case"; option b is the extra >>> dial for the client. >>> >>> >> Actually, that's not the point I was making with "do what I >> mean", but it's not essential to the discussion so let's move on. >> >> >>> It is worth noting that VoiceXML uses option b. This allows one >>> to activate both speech grammars and DTMF grammars (and therefore >>> be informed of any errors in the grammars at activation time) but >>> independently turn on whichever input mode you like e.g. perhaps >>> start with "both" then change to "dtmf". >>> >>> >> I'm not sure the VXML precedent is relevant here, because the >> application behind VXML is working a different part of the problem >> - how to traverse a TUI dialog based on different parts of the >> input space. In fact, I suspect that the VXML: choice was >> conditioned more by limitations at the time it was specified than >> an underlying good design choice. Clearly having to specify this >> in VXML make the job of handling a TUI with nodes like "Say or >> press 5" harder rather than easier. >> > > DB> The VoiceXML edge-case is pretty weird so it's not a major > concern. My main concern is that the client can indicate, somehow, > what the input modes are. > > >> >> Summing up, while I don't feel strongly one way or the other, I >> have a preference for handling this as follows: >> >> a) Rename "START-OF-SPEECH" to "INTERESTING-INPUT-RECEIVED" or >> something equivalent. >> b) Include a parameter in the event saying what was interesting >> about the input you received, with a registry of values which >> includes: >> - signal above noise floor >> - speech >> - dtmf >> - (possibly) music >> c) allow the event to be generated multiple times during a request >> >> > > DB> I like these suggestions (START-OF-INPUT?). However, I don't > see how the problem of the optimised case is not solved by them. I > think the optimised case is fine if we have the following rules: > Yes, I hadn't thought through the optimized case as thoroughly as you. Your suggested method name is fine by me as well > 1. START-OF-SPEECH (and optimised bargin) is only generated for the > input type that is being listened for > 2. A speechrecog listens for DTMF if DTMF grammars are active, > speech if speech grammars are active, or speech and DTMF if both > grammar types are active. > Works for me. > >> Note that all of the above I'm saying with my technical hat on and >> my chair hat off. >> Putting my chair hat on for a moment, we really need to get this >> spec to last call, so at some point Eric or I is going to declare >> rough consensus so we can move on. >> > > DB> Agreed! > > >> >> Dave Oran. >> >>> Dave >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Sarvi makes a good point that adding the reason why the START-OF- >>> SPEECH occurred does not fix the optimised bargin case. >>> >>> dtmfrecog - listens for DTMF only >>> speechrecog - listens for DTMF only, or speech only, or speech & >>> DTMF >>> >>> >>> >>> >>> >>> >>> >>> >>> ----- Original Message ----- From: "Shanmugham, Saravanan" >>> >>> To: "David R Oran" ; "Dave Burke" >>> >>> Cc: ; "Klaus Reifenrath" >>> >>> Sent: Tuesday, July 05, 2005 8:51 PM >>> Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode >>> >>> >>> inline. >>> >>> -----Original Message----- >>> From: David R Oran [mailto:oran@cisco.com] >>> Sent: Tuesday, July 05, 2005 11:08 AM >>> To: Dave Burke >>> Cc: Shanmugham, Saravanan; Klaus Reifenrath; speechsc@ietf.org >>> Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode >>> >>> >>> On Jul 5, 2005, at 12:54 PM, Dave Burke wrote: >>> >>> > Inline. >>> > >>> > Dave >>> > >>> > ----- Original Message ----- From: "Shanmugham, Saravanan" >>> > >>> > To: "David R Oran" ; "Klaus Reifenrath" >>> > >>> > Cc: >>> > Sent: Tuesday, July 05, 2005 5:22 PM >>> > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode >>> > >>> > >>> > I agree with Dave's analysis. The purpose of this event >>> was barge-in. >>> > And barge-in should happen for both DTMF and speech. >>> > >>> > Is there a case where you think it should not behave >>> this way. If soe, >>> > please provide a scenario where you think >>> > 1. Barge-in should happen for DTMF and not voice or >>> vice-versa. >>> > >>> > DB> You want to do a DTMF recognition only because it is >>> noisy. >>> > While waiting for DTMF input, the speechrecog resource >>> (or advanced >>> > dtmfrecog) generates a START-OF-SPEECH because it heard >>> some speech. >>> > The client does not want to stop prompt playing unless >>> DTMF was heard >>> > but it can't tell by the START-OF-SPEECH whether speech >>> or DTMF was >>> > heard. Similarly vice versa. >>> > >>> It's an interesting design question what part of the >>> policy resides at the client and what at the server, and >>> who makes the "final decision" about whether what was >>> heard was relevant to the control channel. Right now we >>> (IMO) have a weird partitioning in many cases where the >>> client basically says "do what I mean", but there are no >>> constraints of what the server actually does, and no >>> normalized basis for the client to figure out what to set >>> various magic numbers to (e.g. sensitivity). >>> >>> In this case the only thing the client needs to decide is >>> whether to kill the prompt because the server thinks >>> something that would interfere with the feedback >>> ear/mouth/finger control happened. What this says to me is >>> that it isn't necessarily a good idea for the client to >>> have more knobs to control the server (especially if those >>> knows are just more value/policy input ungrounded in any >>> physics/ acoustics). On the other hand, having the server >>> tell the client more about what it thinks is going on is >>> probably valuable. >>> >>> So, Coming to the point after this long rambling >>> introduction, I think it would in fact be useful for the >>> START-Of-SPEECH event to indicate some extra information, >>> for example: >>> a) I got something enough above the noise floor to qualify >>> for exceeding the "Sensisitvity" parameter you sent in on >>> the request but I really can't tell what it is (could be a >>> hippopatmus fart, or a siren in the background, or captain >>> crunch trying to whistle DTMF). >>> b) I think I'm hearing speech >>> c) I think I'm hearing DTMF >>> >>> Though I agree with your former part of your response. I am not >>> sure I >>> agree with your proposed solution. >>> The way I see this problem is that, it is more of what constitues a >>> barge-in event. This boils down to whether it is speech, DTMF or >>> both. >>> This is inturn boils down to what type of recognizer resource we are >>> using, dtmf-recog, speech-recog, and speech-only-recog(we don't have >>> this and I don't think we should add it, but think of this as a >>> place >>> holder that explains the concept). >>> >>> A client knowing what type of barge-in happenned, does not impact >>> the >>> barge-in operation itself as it may be too late(for the optimized >>> barge-in case). It may have other use cases, and if we can identify >>> them, I don't mind adding support for the START-OF-SPEECH event >>> to say >>> what type of barge-in happenned. But that itself does not solve the >>> original problem raised. Refer to my previous response. >>> >>> The solution lies in defining what what is a barge-in event. >>> That boils >>> down to what type of recognition is happenning, dtmf-only, speech- >>> dtmf >>> or speech-only. We do not support speech-only as a resource >>> today, the >>> question is do we need a header to force it. >>> >>> Sarvi >>> >>> >>> > 2. You would benefit from the client knowing what caused >>> the >>> > barge-in, DTMF Vs speech. >>> > >>> > DB> See previous comment. And previous e-mail: either add an >>> > inputmodes header (taking value speech, dtmf, both) to >>> the RECOGNIZE >>> > request or add a header to the START-OF-SPEECH event >>> indicating DTMF >>> > or speech. >>> > >>> I'm leaning in your direction on this latter point - as >>> should be evident from what I wrote above. >>> >>> > Sarvi >>> > >>> > -----Original Message----- >>> > From: speechsc-bounces@ietf.org >>> > [mailto:speechsc-bounces@ietf.org] On Behalf Of David R >>> Oran >>> > Sent: Tuesday, July 05, 2005 5:24 AM >>> > To: Klaus Reifenrath >>> > Cc: 'speechsc@ietf.org' >>> > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode >>> > >>> > >>> > On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: >>> > >>> > > The current spec is not clear when START-OF-SPEECH need >>> > to be send in >>> > > the following scenarios: >>> > > A) The client requested a DTMF Recognizer. Is the >>> > START-OF-SPEECH >>> > > event send to the client also if speech was detected? >>> > I suspect so, since one of the prime purposes is to enable >>> > client- mediated barge-in handling. However, if the >>> > recognizer is in fact only capable of recognizing DTMF >>> > then it may in fact not report anythin unless it's using >>> > some primitive thresholding machinery, like a SN >>> threshold. >>> > > B) The client requested a Speech Recognizer, but only >>> > activated DTMF >>> > > grammars. Is the START-OF-SPEECH event send to the >>> > client also if >>> > > speech was detected? >>> > Again, I'd say yes, for the same reason as above. >>> > > I think in both cases START-OF-SPEECH should only >>> be send after >>> > > detecting a DTMF digit (see Figure 12 of VoiceXML >>> 2.0: http:// >>> > > www.w3.org/TR/voicexml20/#dmlATiming). >>> > We seem to have reached different conclusions. I'd be >>> > interested in why you think my analysis above is wrong. >>> > >>> > Dave. >>> > >>> > > Klaus >>> > > >>> > > _______________________________________________ >>> > > Speechsc mailing list >>> > > Speechsc@ietf.org >>> > > https://www1.ietf.org/mailman/listinfo/speechsc >>> > > >>> > >>> > _______________________________________________ >>> > Speechsc mailing list >>> > Speechsc@ietf.org >>> > https://www1.ietf.org/mailman/listinfo/speechsc >>> > >>> > >>> > _______________________________________________ >>> > Speechsc mailing list >>> > Speechsc@ietf.org >>> > https://www1.ietf.org/mailman/listinfo/speechsc >>> > >>> >>> >>> _______________________________________________ >>> Speechsc mailing list >>> Speechsc@ietf.org >>> https://www1.ietf.org/mailman/listinfo/speechsc >>> >>> >> >> _______________________________________________ >> Speechsc mailing list >> Speechsc@ietf.org >> https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Wed Jul 06 11:27:31 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqBoB-00042u-Bg; Wed, 06 Jul 2005 11:27:31 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqBny-0003vk-EG for speechsc@megatron.ietf.org; Wed, 06 Jul 2005 11:27:18 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA09596 for ; Wed, 6 Jul 2005 11:27:13 -0400 (EDT) Received: from e31.co.us.ibm.com ([32.97.110.129]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DqCEU-0006y6-AP for speechsc@ietf.org; Wed, 06 Jul 2005 11:54:44 -0400 Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11]) by e31.co.us.ibm.com (8.12.10/8.12.9) with ESMTP id j66FQb8D014394 for ; Wed, 6 Jul 2005 11:26:38 -0400 Received: from d03av04.boulder.ibm.com (d03av04.boulder.ibm.com [9.17.195.170]) by westrelay02.boulder.ibm.com (8.12.10/NCO/VER6.6) with ESMTP id j66FQboO233824 for ; Wed, 6 Jul 2005 09:26:37 -0600 Received: from d03av04.boulder.ibm.com (loopback [127.0.0.1]) by d03av04.boulder.ibm.com (8.12.11/8.13.3) with ESMTP id j66FQb1V014191 for ; Wed, 6 Jul 2005 09:26:37 -0600 Received: from d03nm119.boulder.ibm.com (d03nm119.boulder.ibm.com [9.17.195.145]) by d03av04.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id j66FQb40014182 for ; Wed, 6 Jul 2005 09:26:37 -0600 To: speechsc@ietf.org MIME-Version: 1.0 X-Mailer: Lotus Notes Release 6.0.2CF1 June 9, 2003 Message-ID: From: Brett Gavagni Date: Wed, 6 Jul 2005 11:26:35 -0400 X-MIMETrack: Serialize by Router on D03NM119/03/M/IBM(Release 6.5.4|March 27, 2005) at 07/06/2005 09:26:37, Serialize complete at 07/06/2005 09:26:37 Content-Type: text/plain; charset="US-ASCII" X-Spam-Score: 0.0 (/) X-Scan-Signature: 21c69d3cfc2dd19218717dbe1d974352 Subject: [Speechsc] Speed-Vs-Accuracy inconsistency with MRCPv2 and VoiceXML 2.0 X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org Hi, Currently there exists some inconsistent interpretation of the values for Speed-Vs-Accuracy in the MRCPv2 specification. The current MRCPv2 draft states the following: Speed Vs Accuracy Depending on the implementation and capability of the recognizer resource it may be tunable towards Performance or Accuracy. Higher accuracy may mean more processing and higher CPU utilization, meaning less calls per server and vice versa. This header is a float value between 0.0 and 1.0 and allows this field to be tuned by the speed-vs-accuracy header. This header field MAY occur in RECOGNIZE, SET-PARAMS or GET-PARAMS. A higher value for this field means higher speed. The default value for this field is platform specific. speed-vs-accuracy = "Speed-Vs-Accuracy" ":" FLOAT CRLF The VoiceXML 2.0 specification states the following: http://www.w3.org/TR/voicexml20/#dml6.3.2 6.3.2 Generic Speech Recognizer Properties speedvsaccuracy A hint specifying the desired balance between speed vs. accuracy. A value of 0.0 means fastest recognition. A value of 1.0 means best accuracy. The value is a Real Number Designation (see Section 6.5). The default is value 0.5. The VoiceXML spec, states that a 1.0 value means best accuracy, and the MRCPv2 draft states that the higher the value means higher speed. Was it intentional to have the MRCPv2 draft convey an inconsistent interpretation of the values in comparison to VoiceXML 2.0? Thanks, Brett Gavagni WebSphere Voice Server Development http://www-306.ibm.com/software/pervasive/voice_server/ (561) 862-2097 T/L (975) gavagni@us.ibm.com _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Wed Jul 06 11:46:01 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqC4M-0007YJ-9e; Wed, 06 Jul 2005 11:44:14 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqC4I-0007Rw-Rz for speechsc@megatron.ietf.org; Wed, 06 Jul 2005 11:44:13 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA12356 for ; Wed, 6 Jul 2005 11:44:07 -0400 (EDT) Received: from sj-iport-1-in.cisco.com ([171.71.176.70] helo=sj-iport-1.cisco.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1DqCOM-0000kN-DY for speechsc@ietf.org; Wed, 06 Jul 2005 12:04:54 -0400 Received: from sj-core-1.cisco.com (171.71.177.237) by sj-iport-1.cisco.com with ESMTP; 06 Jul 2005 08:36:51 -0700 X-IronPort-AV: i="3.93,265,1115017200"; d="scan'208"; a="647288291:sNHT29836508" Received: from imail.cisco.com (imail.cisco.com [128.107.200.91]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id j66FaovM004025; Wed, 6 Jul 2005 08:36:50 -0700 (PDT) Received: from [10.32.245.152] (stealth-10-32-245-152.cisco.com [10.32.245.152]) by imail.cisco.com (8.12.11/8.12.10) with SMTP id j66FZsrE008859; Wed, 6 Jul 2005 08:35:55 -0700 In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v730) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: David R Oran Subject: Re: [Speechsc] Speed-Vs-Accuracy inconsistency with MRCPv2 and VoiceXML 2.0 Date: Wed, 6 Jul 2005 11:36:47 -0400 To: Brett Gavagni X-Mailer: Apple Mail (2.730) IIM-SIG: v:"1.1"; h:"imail.cisco.com"; d:"cisco.com"; z:"home"; m:"krs"; t:"1120664155.684038"; x:"432200"; a:"rsa-sha1"; b:"nofws:1644"; e:"Iw=="; n:"sQYarK2E51MdcTiUqeif3F7cWdxIfoCiXhdfb9vD5ee/j0jXL15gbFxF2p" "XIweAblu0N6XAgK7k+wrbr7bQDJaCDqOmzqpRUBjIRQAXQ7NzadpmR3pUL6wxaRUtW+c43sl9jC" "50Qg1sXHpPjt8Y+Y16ioyQAQAdSunM4YhevURc="; s:"n9LsV68C6cw9bPgat9f8O1GRoQA9g6bcCKUsHLmHVFEtMsREwwvT23oHqR2bNSLtkrvMz+WY" "4/evPdzL2+BuXCXAwwQKGQ7Gwrure5SYfaMOY8FnBUDZT8PujmC347HEzzXA2CMrPRy35pwVGyk" "+fuFux96SDNmJUoE+UR25aDs="; c:"From: David R Oran "; c:"Subject: Re: [Speechsc] Speed-Vs-Accuracy inconsistency with MRCPv2 " "and VoiceXML 2.0"; c:"Date: Wed, 6 Jul 2005 11:36:47 -0400" IIM-VERIFY: s:"y"; v:"y"; r:"60"; h:"imail.cisco.com"; c:"message from imail.cisco.com verified; " X-Spam-Score: 0.0 (/) X-Scan-Signature: 02ec665d00de228c50c93ed6b5e4fc1a Content-Transfer-Encoding: 7bit Cc: speechsc@ietf.org X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org On Jul 6, 2005, at 11:26 AM, Brett Gavagni wrote: > Hi, > > Currently there exists some inconsistent interpretation of the > values for > Speed-Vs-Accuracy in the MRCPv2 specification. > > The current MRCPv2 draft states the following: > Speed Vs Accuracy > > Depending on the implementation and capability of the recognizer > resource it may be tunable towards Performance or Accuracy. Higher > accuracy may mean more processing and higher CPU utilization, > meaning less calls per server and vice versa. This header is a > float > value between 0.0 and 1.0 and allows this field to be tuned by the > speed-vs-accuracy header. This header field MAY occur in RECOGNIZE, > SET-PARAMS or GET-PARAMS. A higher value for this field means > higher > speed. The default value for this field is platform specific. > > speed-vs-accuracy = "Speed-Vs-Accuracy" ":" FLOAT CRLF > > The VoiceXML 2.0 specification states the following: > http://www.w3.org/TR/voicexml20/#dml6.3.2 > 6.3.2 Generic Speech Recognizer Properties > > speedvsaccuracy > A hint specifying the desired balance between speed vs. accuracy. > A value of 0.0 means fastest recognition. A value of 1.0 means best > accuracy. > The value is a Real Number Designation (see Section 6.5). The > default is > value 0.5. > > The VoiceXML spec, states that a 1.0 value means best accuracy, and > the > MRCPv2 draft states that the higher the value means higher speed. > > Was it intentional to have the MRCPv2 draft convey an inconsistent > interpretation of the values in comparison to VoiceXML 2.0? > I don't think so. I have no objection to reversing it. Anyone else want to chime in? > Thanks, > > Brett Gavagni > WebSphere Voice Server Development > http://www-306.ibm.com/software/pervasive/voice_server/ > (561) 862-2097 T/L (975) > gavagni@us.ibm.com > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Thu Jul 07 08:40:26 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqVg2-0000R0-Fp; Thu, 07 Jul 2005 08:40:26 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqVg0-0000NT-Gr for speechsc@megatron.ietf.org; Thu, 07 Jul 2005 08:40:24 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA01572 for ; Thu, 7 Jul 2005 08:40:22 -0400 (EDT) Received: from salvelinus.brooktrout.com ([204.176.205.6]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1DqW7B-00040z-DD for speechsc@ietf.org; Thu, 07 Jul 2005 09:08:30 -0400 Received: from nhmail2.needham.brooktrout.com (nhmail2.brooktrout.com [204.176.205.242]) by salvelinus.brooktrout.com (8.12.5/8.12.5) with ESMTP id j67CbAhM016425 for ; Thu, 7 Jul 2005 08:37:10 -0400 (EDT) Received: by nhmail2.brooktrout.com with Internet Mail Service (5.5.2653.19) id ; Thu, 7 Jul 2005 08:32:09 -0400 Message-ID: From: Eric Burger To: speechsc@ietf.org Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Date: Thu, 7 Jul 2005 08:32:06 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Scan-Signature: 9b157e6e8a3799aef911c0bc37fc93a6 X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org Can we declare consensus? > -----Original Message----- > From: speechsc-bounces@ietf.org [mailto:speechsc-bounces@ietf.org] > Sent: Wednesday, July 06, 2005 10:45 AM > To: Dave Burke > Cc: speechsc@ietf.org; Shanmugham, Saravanan; Klaus Reifenrath > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> > summary(?) > > I think we're getting close. I though about snipping out some > pieces to cut down the text, but I realized the context is > still needed. See inline. > > On Jul 6, 2005, at 10:29 AM, Dave Burke wrote: > > > Inline. > > > > Dave > > > > ----- Original Message ----- From: "David R Oran" > > To: "Dave Burke" > > Cc: ; "Shanmugham, Saravanan" ; > > "Klaus Reifenrath" > > Sent: Wednesday, July 06, 2005 1:07 PM > > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary > > (?) > > > > > > > >> > >> On Jul 6, 2005, at 5:31 AM, Dave Burke wrote: > >> > >> > >>> + Attempting to summarise: > >>> > >>> 1. START-OF-SPEECH is useful for the client to know when > to stop > >>> playing prompts the in non-optimised case > >>> 2. START-OF-SPEECH is useful for the client to calculate the > >>> bargin time (e.g. VoiceXML 2.1 ) > >>> 3. In the optimised case, a bargin automatically stops prompt > >>> playing (assuming prompts barginable) > >>> 4. Because of the previous point, the question of what input > >>> type caused bargin is different and less important to > what input > >>> type(s) the recogniser is listening for > >>> > >> I'm not sure I follow point 4. Could you elaborate? > >> > > > > DB> Adding a parameter to the START-OF-SPEECH event would > certainly > > allow the client to ignore (i.e. let prompts continue playing) the > > event if the event type is not of interest (e.g. the client would > > ignore speech start events when it is interested only in DTMF > > start events). This _only_ works for the non-optimised case, > > however. For the optimised case, assuming START-OF-SPEECH > coincides > > with the bargin signal to the speechsynth, prompts will stop > > playing for inputs that the client might not be interested > in (e.g. > > a speech input will stop prompts playing even if the client > is only > > interested in DTMF). > > > OK, now I get it. There's a need for the client to both handle the > non-optimized case itself, and influence or at least have a > clue what > the server is going to do in the optimized case. > > > > >> > >> > >>> 5. Currently, MRCPv2 has no way of indicating what input type(s) > >>> a recogniser is listening for > >>> > >>> > >> Do you mean exactly this, or do you mean "for the client to > >> indicate to the resource what input types it should look for"? > >> > > > > DB> Yes exactly - apologies for not being clear. > > > > > >> > >> > >>> + Why implement 5? > >>> > >>> i. Noisy case: Need DTMF-only recognition (and may only have a > >>> speechrecog) > >>> > >> I'm having difficulty following the logic of why noise would > >> necessarily trigger START-OF-SPEECH if you were listening for > >> speech but not DTMF. I suppose you can use a more forgiving > >> discriminator if all you need to tell is if you're getting DTMF, > >> but I've had a number of real-world cases where wind noise was > >> detected as DTMF, and there's always the ambiguity when you have > >> Captain Crunch on the phone. In either case in the non-optimized > >> case it's the client who gets to decide whether an event should > >> be interpreted as barge-in or not, so it seems an aesthetic > >> protocol design decision whether the client tells the server > >> ahead of time what circumstances to generate the START-OF-SPEECH > >> event for, or whether the event gets generated and the client > >> decides based on what's in the event whether it should be > treated > >> as barge- in. > >> > >> I suppose one could make the argument that because the spec > >> implies that the event can only be generated once per request > >> that if a DTMF/ speech capable recognizer first hears > enough noise > >> to think it's hearing speech and later hears DTMF, the client > >> will declare barge-in when the event comes and not when he DTMF > >> actually gets heard. > >> > >> If that's deemed a problem, we can still handle that in the > >> design where the server just reports what it's hearing by > >> allowing multiple events to be generated during a single request. > >> > >> Between the approach just outlined above, and an approach where > >> the client provides a filter for whether to generate the event or > >> not, I have a mild preference (based on aesthetics rather than > >> some hard engineering tradeoff) for the approach where > the server > >> just reports what it's hearing. > >> > >> > >> Having had some useful exchanges on this topic, it also is > >> becoming apparent to me that this event is poorly named, and we > >> should consider renaming it to "INTERESTING-INPUT-HEARD" or > >> something akin to that, because as others have pointed out, a > >> DTMF-only recognizer will never detect "start of speech". > >> > >> Another consideration to fold into the design choice is > >> extensibility. Bear with me through a little gedankenexperiment. > >> > >> Suppose we want to define a new recognizer type, which I'll call > >> the "name that tune" recognizer. The client plays music to the > >> server and the server recognizes musical notes. The grammar is a > >> standard musical notation, augmented with a semantic > >> interpretation that transforms the notes into the title of the > >> tune and provides that as an answer. > >> > >> First, there's no speech involved (or is there...hang on a > >> minute). Second, in order to accommodate the "name that tune" > >> recognizer, we'd have to extend both the client and the server to > >> undetstand a directive as to whether to recognize music or now, > >> inaddition to what the server already knows what to do based on > >> the grammar. If you follow my logic above, whether or not we do > >> that, we have to extend "start-of-speech" to say "I'm hearing > >> music". So far fairly straightforward, but let me now throw in > >> the pathological twist. > >> > >> Suppose what I feed to a combined music/speech recognizer is a > >> work in sprechstimme (spoken music), like the "Geographical > >> Fugue" (aside: this is a wonderful piece of music I highly > >> recommend to anyone interested in small ensemble singing). In > >> this case, the tune could be named by either doing speech or > >> music recognition. Why is there any need for the client to > >> constrain the server as to which it tries to do when it's > already > >> told the server what it wants through the grammar? > >> > >> A few other comments below > >> > >> > >>> ii. Flexibility: Want speech-only recognition (because a second > >>> recogniser is doing hotword on DTMF) > >>> > >>> > >> I don't see how flexibility is affected by this deisgn > choice. If > >> that's what you want, feed the speech-only recognizer a grammar > >> without any DTMF rules. > >> > >> > >>> + How to implement 5? > >>> > >>> a. Implicitly: > >>> - dtmfrecog: always DTMF-only recognition > >>> - speechrecog: depends on active grammar type > >>> > if a dtmf grammar is active then DTMF input is "on" > >>> > if a speech grammar is active then speech input is "on" > >>> > >>> b. Explicitly: > >>> - Add inputmodes header to RECOGNIZE > >>> > >>> Option a is David's "do what I mean case"; option b is > the extra > >>> dial for the client. > >>> > >>> > >> Actually, that's not the point I was making with "do what I > >> mean", but it's not essential to the discussion so let's move on. > >> > >> > >>> It is worth noting that VoiceXML uses option b. This allows one > >>> to activate both speech grammars and DTMF grammars (and > therefore > >>> be informed of any errors in the grammars at activation > time) but > >>> independently turn on whichever input mode you like e.g. > perhaps > >>> start with "both" then change to "dtmf". > >>> > >>> > >> I'm not sure the VXML precedent is relevant here, because the > >> application behind VXML is working a different part of the > problem > >> - how to traverse a TUI dialog based on different parts of the > >> input space. In fact, I suspect that the VXML: choice was > >> conditioned more by limitations at the time it was > specified than > >> an underlying good design choice. Clearly having to specify this > >> in VXML make the job of handling a TUI with nodes like "Say or > >> press 5" harder rather than easier. > >> > > > > DB> The VoiceXML edge-case is pretty weird so it's not a major > > concern. My main concern is that the client can indicate, somehow, > > what the input modes are. > > > > > >> > >> Summing up, while I don't feel strongly one way or the other, I > >> have a preference for handling this as follows: > >> > >> a) Rename "START-OF-SPEECH" to "INTERESTING-INPUT-RECEIVED" or > >> something equivalent. > >> b) Include a parameter in the event saying what was interesting > >> about the input you received, with a registry of values which > >> includes: > >> - signal above noise floor > >> - speech > >> - dtmf > >> - (possibly) music > >> c) allow the event to be generated multiple times during a request > >> > >> > > > > DB> I like these suggestions (START-OF-INPUT?). However, I don't > > see how the problem of the optimised case is not solved by them. I > > think the optimised case is fine if we have the following rules: > > > Yes, I hadn't thought through the optimized case as thoroughly as > you. Your suggested method name is fine by me as well > > > 1. START-OF-SPEECH (and optimised bargin) is only generated > for the > > input type that is being listened for > > 2. A speechrecog listens for DTMF if DTMF grammars are active, > > speech if speech grammars are active, or speech and DTMF if both > > grammar types are active. > > > Works for me. > > > > >> Note that all of the above I'm saying with my technical > hat on and > >> my chair hat off. > >> Putting my chair hat on for a moment, we really need to get this > >> spec to last call, so at some point Eric or I is going to > declare > >> rough consensus so we can move on. > >> > > > > DB> Agreed! > > > > > >> > >> Dave Oran. > >> > >>> Dave > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> Sarvi makes a good point that adding the reason why the > START-OF- > >>> SPEECH occurred does not fix the optimised bargin case. > >>> > >>> dtmfrecog - listens for DTMF only > >>> speechrecog - listens for DTMF only, or speech only, or speech & > >>> DTMF > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> ----- Original Message ----- From: "Shanmugham, Saravanan" > >>> > >>> To: "David R Oran" ; "Dave Burke" > >>> > >>> Cc: ; "Klaus Reifenrath" > >>> > >>> Sent: Tuesday, July 05, 2005 8:51 PM > >>> Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode > >>> > >>> > >>> inline. > >>> > >>> -----Original Message----- > >>> From: David R Oran [mailto:oran@cisco.com] > >>> Sent: Tuesday, July 05, 2005 11:08 AM > >>> To: Dave Burke > >>> Cc: Shanmugham, Saravanan; Klaus Reifenrath; speechsc@ietf.org > >>> Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode > >>> > >>> > >>> On Jul 5, 2005, at 12:54 PM, Dave Burke wrote: > >>> > >>> > Inline. > >>> > > >>> > Dave > >>> > > >>> > ----- Original Message ----- From: "Shanmugham, Saravanan" > >>> > > >>> > To: "David R Oran" ; "Klaus Reifenrath" > >>> > > >>> > Cc: > >>> > Sent: Tuesday, July 05, 2005 5:22 PM > >>> > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode > >>> > > >>> > > >>> > I agree with Dave's analysis. The purpose of this event > >>> was barge-in. > >>> > And barge-in should happen for both DTMF and speech. > >>> > > >>> > Is there a case where you think it should not behave > >>> this way. If soe, > >>> > please provide a scenario where you think > >>> > 1. Barge-in should happen for DTMF and not voice or > >>> vice-versa. > >>> > > >>> > DB> You want to do a DTMF recognition only because it is > >>> noisy. > >>> > While waiting for DTMF input, the speechrecog resource > >>> (or advanced > >>> > dtmfrecog) generates a START-OF-SPEECH because it heard > >>> some speech. > >>> > The client does not want to stop prompt playing unless > >>> DTMF was heard > >>> > but it can't tell by the START-OF-SPEECH whether speech > >>> or DTMF was > >>> > heard. Similarly vice versa. > >>> > > >>> It's an interesting design question what part of the > >>> policy resides at the client and what at the server, and > >>> who makes the "final decision" about whether what was > >>> heard was relevant to the control channel. Right now we > >>> (IMO) have a weird partitioning in many cases where the > >>> client basically says "do what I mean", but there are no > >>> constraints of what the server actually does, and no > >>> normalized basis for the client to figure out what to set > >>> various magic numbers to (e.g. sensitivity). > >>> > >>> In this case the only thing the client needs to decide is > >>> whether to kill the prompt because the server thinks > >>> something that would interfere with the feedback > >>> ear/mouth/finger control happened. What this says to me is > >>> that it isn't necessarily a good idea for the client to > >>> have more knobs to control the server (especially if those > >>> knows are just more value/policy input ungrounded in any > >>> physics/ acoustics). On the other hand, having the server > >>> tell the client more about what it thinks is going on is > >>> probably valuable. > >>> > >>> So, Coming to the point after this long rambling > >>> introduction, I think it would in fact be useful for the > >>> START-Of-SPEECH event to indicate some extra information, > >>> for example: > >>> a) I got something enough above the noise floor to qualify > >>> for exceeding the "Sensisitvity" parameter you sent in on > >>> the request but I really can't tell what it is (could be a > >>> hippopatmus fart, or a siren in the background, or captain > >>> crunch trying to whistle DTMF). > >>> b) I think I'm hearing speech > >>> c) I think I'm hearing DTMF > >>> > >>> Though I agree with your former part of your response. I am not > >>> sure I > >>> agree with your proposed solution. > >>> The way I see this problem is that, it is more of what > constitues a > >>> barge-in event. This boils down to whether it is speech, DTMF or > >>> both. > >>> This is inturn boils down to what type of recognizer > resource we are > >>> using, dtmf-recog, speech-recog, and speech-only-recog(we > don't have > >>> this and I don't think we should add it, but think of this as a > >>> place > >>> holder that explains the concept). > >>> > >>> A client knowing what type of barge-in happenned, does > not impact > >>> the > >>> barge-in operation itself as it may be too late(for the optimized > >>> barge-in case). It may have other use cases, and if we > can identify > >>> them, I don't mind adding support for the START-OF-SPEECH event > >>> to say > >>> what type of barge-in happenned. But that itself does not > solve the > >>> original problem raised. Refer to my previous response. > >>> > >>> The solution lies in defining what what is a barge-in event. > >>> That boils > >>> down to what type of recognition is happenning, > dtmf-only, speech- > >>> dtmf > >>> or speech-only. We do not support speech-only as a resource > >>> today, the > >>> question is do we need a header to force it. > >>> > >>> Sarvi > >>> > >>> > >>> > 2. You would benefit from the client knowing > what caused > >>> the > >>> > barge-in, DTMF Vs speech. > >>> > > >>> > DB> See previous comment. And previous e-mail: either add an > >>> > inputmodes header (taking value speech, dtmf, both) to > >>> the RECOGNIZE > >>> > request or add a header to the START-OF-SPEECH event > >>> indicating DTMF > >>> > or speech. > >>> > > >>> I'm leaning in your direction on this latter point - as > >>> should be evident from what I wrote above. > >>> > >>> > Sarvi > >>> > > >>> > -----Original Message----- > >>> > From: speechsc-bounces@ietf.org > >>> > [mailto:speechsc-bounces@ietf.org] On Behalf Of > David R > >>> Oran > >>> > Sent: Tuesday, July 05, 2005 5:24 AM > >>> > To: Klaus Reifenrath > >>> > Cc: 'speechsc@ietf.org' > >>> > Subject: Re: [Speechsc] START-OF-SPEECH in > DTMF-only mode > >>> > > >>> > > >>> > On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: > >>> > > >>> > > The current spec is not clear when > START-OF-SPEECH need > >>> > to be send in > >>> > > the following scenarios: > >>> > > A) The client requested a DTMF Recognizer. Is the > >>> > START-OF-SPEECH > >>> > > event send to the client also if speech was detected? > >>> > I suspect so, since one of the prime purposes > is to enable > >>> > client- mediated barge-in handling. However, if the > >>> > recognizer is in fact only capable of recognizing DTMF > >>> > then it may in fact not report anythin unless it's using > >>> > some primitive thresholding machinery, like a SN > >>> threshold. > >>> > > B) The client requested a Speech Recognizer, but only > >>> > activated DTMF > >>> > > grammars. Is the START-OF-SPEECH event send to the > >>> > client also if > >>> > > speech was detected? > >>> > Again, I'd say yes, for the same reason as above. > >>> > > I think in both cases START-OF-SPEECH should only > >>> be send after > >>> > > detecting a DTMF digit (see Figure 12 of VoiceXML > >>> 2.0: http:// > >>> > > www.w3.org/TR/voicexml20/#dmlATiming). > >>> > We seem to have reached different conclusions. I'd be > >>> > interested in why you think my analysis above is wrong. > >>> > > >>> > Dave. > >>> > > >>> > > Klaus > >>> > > > >>> > > _______________________________________________ > >>> > > Speechsc mailing list > >>> > > Speechsc@ietf.org > >>> > > https://www1.ietf.org/mailman/listinfo/speechsc > >>> > > > >>> > > >>> > _______________________________________________ > >>> > Speechsc mailing list > >>> > Speechsc@ietf.org > >>> > https://www1.ietf.org/mailman/listinfo/speechsc > >>> > > >>> > > >>> > _______________________________________________ > >>> > Speechsc mailing list > >>> > Speechsc@ietf.org > >>> > https://www1.ietf.org/mailman/listinfo/speechsc > >>> > > >>> > >>> > >>> _______________________________________________ > >>> Speechsc mailing list > >>> Speechsc@ietf.org > >>> https://www1.ietf.org/mailman/listinfo/speechsc > >>> > >>> > >> > >> _______________________________________________ > >> Speechsc mailing list > >> Speechsc@ietf.org > >> https://www1.ietf.org/mailman/listinfo/speechsc > > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Thu Jul 07 08:40:28 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqVg4-0000Ss-Q2; Thu, 07 Jul 2005 08:40:28 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqVg4-0000Sn-1p for speechsc@megatron.ietf.org; Thu, 07 Jul 2005 08:40:28 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA01575 for ; Thu, 7 Jul 2005 08:40:26 -0400 (EDT) Received: from salvelinus.brooktrout.com ([204.176.205.6]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1DqW7F-00040z-NQ for speechsc@ietf.org; Thu, 07 Jul 2005 09:08:34 -0400 Received: from nhmail2.needham.brooktrout.com (nhmail2.brooktrout.com [204.176.205.242]) by salvelinus.brooktrout.com (8.12.5/8.12.5) with ESMTP id j67CcKhM016459 for ; Thu, 7 Jul 2005 08:38:20 -0400 (EDT) Received: by nhmail2.brooktrout.com with Internet Mail Service (5.5.2653.19) id ; Thu, 7 Jul 2005 08:33:19 -0400 Message-ID: From: Eric Burger To: speechsc@ietf.org Subject: RE: [Speechsc] Speed-Vs-Accuracy inconsistency with MRCPv2 andVoi ceXML 2.0 Date: Thu, 7 Jul 2005 08:33:12 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Scan-Signature: b280b4db656c3ca28dd62e5e0b03daa8 X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org Does silence here mean, "OK with me" or "I haven't bothered to read the mail"? > -----Original Message----- > From: speechsc-bounces@ietf.org [mailto:speechsc-bounces@ietf.org] > Sent: Wednesday, July 06, 2005 11:37 AM > To: Brett Gavagni > Cc: speechsc@ietf.org > Subject: Re: [Speechsc] Speed-Vs-Accuracy inconsistency with > MRCPv2 andVoiceXML 2.0 > > > On Jul 6, 2005, at 11:26 AM, Brett Gavagni wrote: > > > Hi, > > > > Currently there exists some inconsistent interpretation of > the values > > for Speed-Vs-Accuracy in the MRCPv2 specification. > > > > The current MRCPv2 draft states the following: > > Speed Vs Accuracy > > > > Depending on the implementation and capability of the recognizer > > resource it may be tunable towards Performance or > Accuracy. Higher > > accuracy may mean more processing and higher CPU utilization, > > meaning less calls per server and vice versa. This header is a > > float > > value between 0.0 and 1.0 and allows this field to be > tuned by the > > speed-vs-accuracy header. This header field MAY occur in > RECOGNIZE, > > SET-PARAMS or GET-PARAMS. A higher value for this field means > > higher > > speed. The default value for this field is platform specific. > > > > speed-vs-accuracy = "Speed-Vs-Accuracy" ":" FLOAT CRLF > > > > The VoiceXML 2.0 specification states the following: > > http://www.w3.org/TR/voicexml20/#dml6.3.2 > > 6.3.2 Generic Speech Recognizer Properties > > > > speedvsaccuracy > > A hint specifying the desired balance between speed vs. accuracy. > > A value of 0.0 means fastest recognition. A value of 1.0 means best > > accuracy. > > The value is a Real Number Designation (see Section 6.5). > The default > > is value 0.5. > > > > The VoiceXML spec, states that a 1.0 value means best accuracy, and > > the > > MRCPv2 draft states that the higher the value means higher speed. > > > > Was it intentional to have the MRCPv2 draft convey an inconsistent > > interpretation of the values in comparison to VoiceXML 2.0? > > > I don't think so. I have no objection to reversing it. Anyone > else want to chime in? > > > Thanks, > > > > Brett Gavagni > > WebSphere Voice Server Development > > http://www-306.ibm.com/software/pervasive/voice_server/ > > (561) 862-2097 T/L (975) > > gavagni@us.ibm.com > > > > > > _______________________________________________ > > Speechsc mailing list > > Speechsc@ietf.org > > https://www1.ietf.org/mailman/listinfo/speechsc > > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Thu Jul 07 09:04:39 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqW3T-0001CL-Gw; Thu, 07 Jul 2005 09:04:39 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqW3N-00017H-Li for speechsc@megatron.ietf.org; Thu, 07 Jul 2005 09:04:35 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA03567 for ; Thu, 7 Jul 2005 09:04:31 -0400 (EDT) Received: from pb-exchcon2.scansoft.com ([199.4.160.64]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1DqWUU-0006FV-DZ for speechsc@ietf.org; Thu, 07 Jul 2005 09:32:39 -0400 Received: by pb-exchcon2.pb.scansoft.com with Internet Mail Service (5.5.2658.27) id ; Thu, 7 Jul 2005 09:04:08 -0400 Message-ID: From: "Reifenrath, Klaus" To: "'Eric Burger'" , speechsc@ietf.org Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Date: Thu, 7 Jul 2005 09:03:56 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2658.27) Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Scan-Signature: d67762704726a1bed57e7f4595960d34 Cc: X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org I) START-XYZ for the new event is misleading if we allow the event to be generated multiple times. Do we really need to introduce a new (optional) event? Why should a DTMF only resource do speech detection? II) 1. START-OF-SPEECH (and optimised bargin) is only generated for the input type that is being listened for 2. A speechrecog listens for DTMF if DTMF grammars are active, speech if speech grammars are active, or speech and DTMF if both grammar types are active. I personally like this, BUT doesn't it conflict with VoiceXML 2.0 section 4.1.5.1 (bargein type speech): "The prompt will be stopped as soon as speech or DTMF input is detected. The prompt is stopped irrespective of whether or not the input matches a grammar and irrespective of which grammars are active." Klaus -----Original Message----- From: Eric Burger [mailto:eburger@brooktrout.com] Sent: Donnerstag, 7. Juli 2005 14:32 To: speechsc@ietf.org Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Can we declare consensus? > -----Original Message----- > From: speechsc-bounces@ietf.org [mailto:speechsc-bounces@ietf.org] > Sent: Wednesday, July 06, 2005 10:45 AM > To: Dave Burke > Cc: speechsc@ietf.org; Shanmugham, Saravanan; Klaus Reifenrath > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> > summary(?) > > I think we're getting close. I though about snipping out some pieces > to cut down the text, but I realized the context is still needed. See > inline. > > On Jul 6, 2005, at 10:29 AM, Dave Burke wrote: > > > Inline. > > > > Dave > > > > ----- Original Message ----- From: "David R Oran" > > To: "Dave Burke" > > Cc: ; "Shanmugham, Saravanan" ; > > "Klaus Reifenrath" > > Sent: Wednesday, July 06, 2005 1:07 PM > > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary > > (?) > > > > > > > >> > >> On Jul 6, 2005, at 5:31 AM, Dave Burke wrote: > >> > >> > >>> + Attempting to summarise: > >>> > >>> 1. START-OF-SPEECH is useful for the client to know when > to stop > >>> playing prompts the in non-optimised case 2. START-OF-SPEECH is > >>> useful for the client to calculate the bargin time (e.g. VoiceXML > >>> 2.1 ) > >>> 3. In the optimised case, a bargin automatically stops prompt > >>> playing (assuming prompts barginable) 4. Because of the previous > >>> point, the question of what input type caused bargin is different > >>> and less important to > what input > >>> type(s) the recogniser is listening for > >>> > >> I'm not sure I follow point 4. Could you elaborate? > >> > > > > DB> Adding a parameter to the START-OF-SPEECH event would > certainly > > allow the client to ignore (i.e. let prompts continue playing) the > > event if the event type is not of interest (e.g. the client would > > ignore speech start events when it is interested only in DTMF start > > events). This _only_ works for the non-optimised case, however. For > > the optimised case, assuming START-OF-SPEECH > coincides > > with the bargin signal to the speechsynth, prompts will stop playing > > for inputs that the client might not be interested > in (e.g. > > a speech input will stop prompts playing even if the client > is only > > interested in DTMF). > > > OK, now I get it. There's a need for the client to both handle the > non-optimized case itself, and influence or at least have a clue what > the server is going to do in the optimized case. > > > > >> > >> > >>> 5. Currently, MRCPv2 has no way of indicating what input type(s) a > >>> recogniser is listening for > >>> > >>> > >> Do you mean exactly this, or do you mean "for the client to > >> indicate to the resource what input types it should look for"? > >> > > > > DB> Yes exactly - apologies for not being clear. > > > > > >> > >> > >>> + Why implement 5? > >>> > >>> i. Noisy case: Need DTMF-only recognition (and may only have a > >>> speechrecog) > >>> > >> I'm having difficulty following the logic of why noise would > >> necessarily trigger START-OF-SPEECH if you were listening for > >> speech but not DTMF. I suppose you can use a more forgiving > >> discriminator if all you need to tell is if you're getting DTMF, > >> but I've had a number of real-world cases where wind noise was > >> detected as DTMF, and there's always the ambiguity when you have > >> Captain Crunch on the phone. In either case in the non-optimized > >> case it's the client who gets to decide whether an event should be > >> interpreted as barge-in or not, so it seems an aesthetic protocol > >> design decision whether the client tells the server ahead of time > >> what circumstances to generate the START-OF-SPEECH event for, or > >> whether the event gets generated and the client decides based on > >> what's in the event whether it should be > treated > >> as barge- in. > >> > >> I suppose one could make the argument that because the spec implies > >> that the event can only be generated once per request that if a > >> DTMF/ speech capable recognizer first hears > enough noise > >> to think it's hearing speech and later hears DTMF, the client will > >> declare barge-in when the event comes and not when he DTMF > >> actually gets heard. > >> > >> If that's deemed a problem, we can still handle that in the design > >> where the server just reports what it's hearing by allowing > >> multiple events to be generated during a single request. > >> > >> Between the approach just outlined above, and an approach where the > >> client provides a filter for whether to generate the event or not, > >> I have a mild preference (based on aesthetics rather than some > >> hard engineering tradeoff) for the approach where > the server > >> just reports what it's hearing. > >> > >> > >> Having had some useful exchanges on this topic, it also is becoming > >> apparent to me that this event is poorly named, and we should > >> consider renaming it to "INTERESTING-INPUT-HEARD" or something akin > >> to that, because as others have pointed out, a DTMF-only recognizer > >> will never detect "start of speech". > >> > >> Another consideration to fold into the design choice is > >> extensibility. Bear with me through a little gedankenexperiment. > >> > >> Suppose we want to define a new recognizer type, which I'll call > >> the "name that tune" recognizer. The client plays music to the > >> server and the server recognizes musical notes. The grammar is a > >> standard musical notation, augmented with a semantic > >> interpretation that transforms the notes into the title of the > >> tune and provides that as an answer. > >> > >> First, there's no speech involved (or is there...hang on a minute). > >> Second, in order to accommodate the "name that tune" > >> recognizer, we'd have to extend both the client and the server to > >> undetstand a directive as to whether to recognize music or now, > >> inaddition to what the server already knows what to do based on > >> the grammar. If you follow my logic above, whether or not we do > >> that, we have to extend "start-of-speech" to say "I'm hearing > >> music". So far fairly straightforward, but let me now throw in the > >> pathological twist. > >> > >> Suppose what I feed to a combined music/speech recognizer is a > >> work in sprechstimme (spoken music), like the "Geographical Fugue" > >> (aside: this is a wonderful piece of music I highly recommend to > >> anyone interested in small ensemble singing). In this case, the > >> tune could be named by either doing speech or music recognition. > >> Why is there any need for the client to constrain the server as to > >> which it tries to do when it's > already > >> told the server what it wants through the grammar? > >> > >> A few other comments below > >> > >> > >>> ii. Flexibility: Want speech-only recognition (because a second > >>> recogniser is doing hotword on DTMF) > >>> > >>> > >> I don't see how flexibility is affected by this deisgn > choice. If > >> that's what you want, feed the speech-only recognizer a grammar > >> without any DTMF rules. > >> > >> > >>> + How to implement 5? > >>> > >>> a. Implicitly: > >>> - dtmfrecog: always DTMF-only recognition > >>> - speechrecog: depends on active grammar type > >>> > if a dtmf grammar is active then DTMF input is "on" > >>> > if a speech grammar is active then speech input is "on" > >>> > >>> b. Explicitly: > >>> - Add inputmodes header to RECOGNIZE > >>> > >>> Option a is David's "do what I mean case"; option b is > the extra > >>> dial for the client. > >>> > >>> > >> Actually, that's not the point I was making with "do what I mean", > >> but it's not essential to the discussion so let's move on. > >> > >> > >>> It is worth noting that VoiceXML uses option b. This allows one to > >>> activate both speech grammars and DTMF grammars (and > therefore > >>> be informed of any errors in the grammars at activation > time) but > >>> independently turn on whichever input mode you like e.g. > perhaps > >>> start with "both" then change to "dtmf". > >>> > >>> > >> I'm not sure the VXML precedent is relevant here, because the > >> application behind VXML is working a different part of the > problem > >> - how to traverse a TUI dialog based on different parts of the > >> input space. In fact, I suspect that the VXML: choice was > >> conditioned more by limitations at the time it was > specified than > >> an underlying good design choice. Clearly having to specify this > >> in VXML make the job of handling a TUI with nodes like "Say or > >> press 5" harder rather than easier. > >> > > > > DB> The VoiceXML edge-case is pretty weird so it's not a major > > concern. My main concern is that the client can indicate, somehow, > > what the input modes are. > > > > > >> > >> Summing up, while I don't feel strongly one way or the other, I > >> have a preference for handling this as follows: > >> > >> a) Rename "START-OF-SPEECH" to "INTERESTING-INPUT-RECEIVED" or > >> something equivalent. > >> b) Include a parameter in the event saying what was interesting > >> about the input you received, with a registry of values which > >> includes: > >> - signal above noise floor > >> - speech > >> - dtmf > >> - (possibly) music > >> c) allow the event to be generated multiple times during a request > >> > >> > > > > DB> I like these suggestions (START-OF-INPUT?). However, I don't > > see how the problem of the optimised case is not solved by them. I > > think the optimised case is fine if we have the following rules: > > > Yes, I hadn't thought through the optimized case as thoroughly as you. > Your suggested method name is fine by me as well > > > 1. START-OF-SPEECH (and optimised bargin) is only generated > for the > > input type that is being listened for 2. A speechrecog listens for > > DTMF if DTMF grammars are active, speech if speech grammars are > > active, or speech and DTMF if both grammar types are active. > > > Works for me. > > > > >> Note that all of the above I'm saying with my technical > hat on and > >> my chair hat off. > >> Putting my chair hat on for a moment, we really need to get this > >> spec to last call, so at some point Eric or I is going to > declare > >> rough consensus so we can move on. > >> > > > > DB> Agreed! > > > > > >> > >> Dave Oran. > >> > >>> Dave > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> Sarvi makes a good point that adding the reason why the > START-OF- > >>> SPEECH occurred does not fix the optimised bargin case. > >>> > >>> dtmfrecog - listens for DTMF only > >>> speechrecog - listens for DTMF only, or speech only, or speech & > >>> DTMF > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> ----- Original Message ----- From: "Shanmugham, Saravanan" > >>> > >>> To: "David R Oran" ; "Dave Burke" > >>> > >>> Cc: ; "Klaus Reifenrath" > >>> > >>> Sent: Tuesday, July 05, 2005 8:51 PM > >>> Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode > >>> > >>> > >>> inline. > >>> > >>> -----Original Message----- > >>> From: David R Oran [mailto:oran@cisco.com] > >>> Sent: Tuesday, July 05, 2005 11:08 AM > >>> To: Dave Burke > >>> Cc: Shanmugham, Saravanan; Klaus Reifenrath; speechsc@ietf.org > >>> Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode > >>> > >>> > >>> On Jul 5, 2005, at 12:54 PM, Dave Burke wrote: > >>> > >>> > Inline. > >>> > > >>> > Dave > >>> > > >>> > ----- Original Message ----- From: "Shanmugham, Saravanan" > >>> > > >>> > To: "David R Oran" ; "Klaus Reifenrath" > >>> > > >>> > Cc: > >>> > Sent: Tuesday, July 05, 2005 5:22 PM > >>> > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode > >>> > > >>> > > >>> > I agree with Dave's analysis. The purpose of this event > >>> was barge-in. > >>> > And barge-in should happen for both DTMF and speech. > >>> > > >>> > Is there a case where you think it should not behave > >>> this way. If soe, > >>> > please provide a scenario where you think > >>> > 1. Barge-in should happen for DTMF and not voice or > >>> vice-versa. > >>> > > >>> > DB> You want to do a DTMF recognition only because it is > >>> noisy. > >>> > While waiting for DTMF input, the speechrecog resource > >>> (or advanced > >>> > dtmfrecog) generates a START-OF-SPEECH because it heard > >>> some speech. > >>> > The client does not want to stop prompt playing unless > >>> DTMF was heard > >>> > but it can't tell by the START-OF-SPEECH whether speech > >>> or DTMF was > >>> > heard. Similarly vice versa. > >>> > > >>> It's an interesting design question what part of the > >>> policy resides at the client and what at the server, and > >>> who makes the "final decision" about whether what was > >>> heard was relevant to the control channel. Right now we > >>> (IMO) have a weird partitioning in many cases where the > >>> client basically says "do what I mean", but there are no > >>> constraints of what the server actually does, and no > >>> normalized basis for the client to figure out what to set > >>> various magic numbers to (e.g. sensitivity). > >>> > >>> In this case the only thing the client needs to decide is > >>> whether to kill the prompt because the server thinks > >>> something that would interfere with the feedback > >>> ear/mouth/finger control happened. What this says to me is > >>> that it isn't necessarily a good idea for the client to > >>> have more knobs to control the server (especially if those > >>> knows are just more value/policy input ungrounded in any > >>> physics/ acoustics). On the other hand, having the server > >>> tell the client more about what it thinks is going on is > >>> probably valuable. > >>> > >>> So, Coming to the point after this long rambling > >>> introduction, I think it would in fact be useful for the > >>> START-Of-SPEECH event to indicate some extra information, > >>> for example: > >>> a) I got something enough above the noise floor to qualify > >>> for exceeding the "Sensisitvity" parameter you sent in on > >>> the request but I really can't tell what it is (could be a > >>> hippopatmus fart, or a siren in the background, or captain > >>> crunch trying to whistle DTMF). > >>> b) I think I'm hearing speech > >>> c) I think I'm hearing DTMF > >>> > >>> Though I agree with your former part of your response. I am not > >>> sure I agree with your proposed solution. > >>> The way I see this problem is that, it is more of what > constitues a > >>> barge-in event. This boils down to whether it is speech, DTMF or > >>> both. > >>> This is inturn boils down to what type of recognizer > resource we are > >>> using, dtmf-recog, speech-recog, and speech-only-recog(we > don't have > >>> this and I don't think we should add it, but think of this as a > >>> place holder that explains the concept). > >>> > >>> A client knowing what type of barge-in happenned, does > not impact > >>> the > >>> barge-in operation itself as it may be too late(for the optimized > >>> barge-in case). It may have other use cases, and if we > can identify > >>> them, I don't mind adding support for the START-OF-SPEECH event to > >>> say what type of barge-in happenned. But that itself does not > solve the > >>> original problem raised. Refer to my previous response. > >>> > >>> The solution lies in defining what what is a barge-in event. > >>> That boils > >>> down to what type of recognition is happenning, > dtmf-only, speech- > >>> dtmf > >>> or speech-only. We do not support speech-only as a resource today, > >>> the question is do we need a header to force it. > >>> > >>> Sarvi > >>> > >>> > >>> > 2. You would benefit from the client knowing > what caused > >>> the > >>> > barge-in, DTMF Vs speech. > >>> > > >>> > DB> See previous comment. And previous e-mail: either add an > >>> > inputmodes header (taking value speech, dtmf, both) to > >>> the RECOGNIZE > >>> > request or add a header to the START-OF-SPEECH event > >>> indicating DTMF > >>> > or speech. > >>> > > >>> I'm leaning in your direction on this latter point - as > >>> should be evident from what I wrote above. > >>> > >>> > Sarvi > >>> > > >>> > -----Original Message----- > >>> > From: speechsc-bounces@ietf.org > >>> > [mailto:speechsc-bounces@ietf.org] On Behalf Of > David R > >>> Oran > >>> > Sent: Tuesday, July 05, 2005 5:24 AM > >>> > To: Klaus Reifenrath > >>> > Cc: 'speechsc@ietf.org' > >>> > Subject: Re: [Speechsc] START-OF-SPEECH in > DTMF-only mode > >>> > > >>> > > >>> > On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: > >>> > > >>> > > The current spec is not clear when > START-OF-SPEECH need > >>> > to be send in > >>> > > the following scenarios: > >>> > > A) The client requested a DTMF Recognizer. Is the > >>> > START-OF-SPEECH > >>> > > event send to the client also if speech was detected? > >>> > I suspect so, since one of the prime purposes > is to enable > >>> > client- mediated barge-in handling. However, if the > >>> > recognizer is in fact only capable of recognizing DTMF > >>> > then it may in fact not report anythin unless it's using > >>> > some primitive thresholding machinery, like a SN > >>> threshold. > >>> > > B) The client requested a Speech Recognizer, but only > >>> > activated DTMF > >>> > > grammars. Is the START-OF-SPEECH event send to the > >>> > client also if > >>> > > speech was detected? > >>> > Again, I'd say yes, for the same reason as above. > >>> > > I think in both cases START-OF-SPEECH should only > >>> be send after > >>> > > detecting a DTMF digit (see Figure 12 of VoiceXML > >>> 2.0: http:// > >>> > > www.w3.org/TR/voicexml20/#dmlATiming). > >>> > We seem to have reached different conclusions. I'd be > >>> > interested in why you think my analysis above is wrong. > >>> > > >>> > Dave. > >>> > > >>> > > Klaus > >>> > > > >>> > > _______________________________________________ > >>> > > Speechsc mailing list > >>> > > Speechsc@ietf.org > >>> > > https://www1.ietf.org/mailman/listinfo/speechsc > >>> > > > >>> > > >>> > _______________________________________________ > >>> > Speechsc mailing list > >>> > Speechsc@ietf.org > >>> > https://www1.ietf.org/mailman/listinfo/speechsc > >>> > > >>> > > >>> > _______________________________________________ > >>> > Speechsc mailing list > >>> > Speechsc@ietf.org > >>> > https://www1.ietf.org/mailman/listinfo/speechsc > >>> > > >>> > >>> > >>> _______________________________________________ > >>> Speechsc mailing list > >>> Speechsc@ietf.org > >>> https://www1.ietf.org/mailman/listinfo/speechsc > >>> > >>> > >> > >> _______________________________________________ > >> Speechsc mailing list > >> Speechsc@ietf.org > >> https://www1.ietf.org/mailman/listinfo/speechsc > > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Thu Jul 07 09:15:15 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqWDj-0004M1-7J; Thu, 07 Jul 2005 09:15:15 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqWDh-0004IV-GY for speechsc@megatron.ietf.org; Thu, 07 Jul 2005 09:15:13 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA04839 for ; Thu, 7 Jul 2005 09:15:11 -0400 (EDT) Received: from fw01.db01.voxpilot.com ([212.17.54.82] helo=mail.voxpilot.com) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1DqWer-0006wJ-K1 for speechsc@ietf.org; Thu, 07 Jul 2005 09:43:19 -0400 Received: from daburkewxp (unknown [10.0.0.203]) by mail.voxpilot.com (Postfix) with ESMTP id 09924214041; Thu, 7 Jul 2005 13:14:49 +0000 (GMT) Message-ID: <00c901c582f5$da41f040$cb00000a@db01.voxpilot.com> From: "Dave Burke" To: "Eric Burger" , References: Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Date: Thu, 7 Jul 2005 14:14:48 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 X-Spam-Score: 0.0 (/) X-Scan-Signature: 07d4bcb4600b627a0786c2557bc62e06 Content-Transfer-Encoding: 7bit Cc: X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org I consent! ----- Original Message ----- From: "Eric Burger" To: Sent: Thursday, July 07, 2005 1:32 PM Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) > Can we declare consensus? > >> -----Original Message----- >> From: speechsc-bounces@ietf.org [mailto:speechsc-bounces@ietf.org] >> Sent: Wednesday, July 06, 2005 10:45 AM >> To: Dave Burke >> Cc: speechsc@ietf.org; Shanmugham, Saravanan; Klaus Reifenrath >> Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> >> summary(?) >> >> I think we're getting close. I though about snipping out some >> pieces to cut down the text, but I realized the context is >> still needed. See inline. >> >> On Jul 6, 2005, at 10:29 AM, Dave Burke wrote: >> >> > Inline. >> > >> > Dave >> > >> > ----- Original Message ----- From: "David R Oran" >> > To: "Dave Burke" >> > Cc: ; "Shanmugham, Saravanan" ; >> > "Klaus Reifenrath" >> > Sent: Wednesday, July 06, 2005 1:07 PM >> > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary >> > (?) >> > >> > >> > >> >> >> >> On Jul 6, 2005, at 5:31 AM, Dave Burke wrote: >> >> >> >> >> >>> + Attempting to summarise: >> >>> >> >>> 1. START-OF-SPEECH is useful for the client to know when >> to stop >> >>> playing prompts the in non-optimised case >> >>> 2. START-OF-SPEECH is useful for the client to calculate the >> >>> bargin time (e.g. VoiceXML 2.1 ) >> >>> 3. In the optimised case, a bargin automatically stops prompt >> >>> playing (assuming prompts barginable) >> >>> 4. Because of the previous point, the question of what input >> >>> type caused bargin is different and less important to >> what input >> >>> type(s) the recogniser is listening for >> >>> >> >> I'm not sure I follow point 4. Could you elaborate? >> >> >> > >> > DB> Adding a parameter to the START-OF-SPEECH event would >> certainly >> > allow the client to ignore (i.e. let prompts continue playing) the >> > event if the event type is not of interest (e.g. the client would >> > ignore speech start events when it is interested only in DTMF >> > start events). This _only_ works for the non-optimised case, >> > however. For the optimised case, assuming START-OF-SPEECH >> coincides >> > with the bargin signal to the speechsynth, prompts will stop >> > playing for inputs that the client might not be interested >> in (e.g. >> > a speech input will stop prompts playing even if the client >> is only >> > interested in DTMF). >> > >> OK, now I get it. There's a need for the client to both handle the >> non-optimized case itself, and influence or at least have a >> clue what >> the server is going to do in the optimized case. >> >> > >> >> >> >> >> >>> 5. Currently, MRCPv2 has no way of indicating what input type(s) >> >>> a recogniser is listening for >> >>> >> >>> >> >> Do you mean exactly this, or do you mean "for the client to >> >> indicate to the resource what input types it should look for"? >> >> >> > >> > DB> Yes exactly - apologies for not being clear. >> > >> > >> >> >> >> >> >>> + Why implement 5? >> >>> >> >>> i. Noisy case: Need DTMF-only recognition (and may only have a >> >>> speechrecog) >> >>> >> >> I'm having difficulty following the logic of why noise would >> >> necessarily trigger START-OF-SPEECH if you were listening for >> >> speech but not DTMF. I suppose you can use a more forgiving >> >> discriminator if all you need to tell is if you're getting DTMF, >> >> but I've had a number of real-world cases where wind noise was >> >> detected as DTMF, and there's always the ambiguity when you have >> >> Captain Crunch on the phone. In either case in the non-optimized >> >> case it's the client who gets to decide whether an event should >> >> be interpreted as barge-in or not, so it seems an aesthetic >> >> protocol design decision whether the client tells the server >> >> ahead of time what circumstances to generate the START-OF-SPEECH >> >> event for, or whether the event gets generated and the client >> >> decides based on what's in the event whether it should be >> treated >> >> as barge- in. >> >> >> >> I suppose one could make the argument that because the spec >> >> implies that the event can only be generated once per request >> >> that if a DTMF/ speech capable recognizer first hears >> enough noise >> >> to think it's hearing speech and later hears DTMF, the client >> >> will declare barge-in when the event comes and not when he DTMF >> >> actually gets heard. >> >> >> >> If that's deemed a problem, we can still handle that in the >> >> design where the server just reports what it's hearing by >> >> allowing multiple events to be generated during a single request. >> >> >> >> Between the approach just outlined above, and an approach where >> >> the client provides a filter for whether to generate the event or >> >> not, I have a mild preference (based on aesthetics rather than >> >> some hard engineering tradeoff) for the approach where >> the server >> >> just reports what it's hearing. >> >> >> >> >> >> Having had some useful exchanges on this topic, it also is >> >> becoming apparent to me that this event is poorly named, and we >> >> should consider renaming it to "INTERESTING-INPUT-HEARD" or >> >> something akin to that, because as others have pointed out, a >> >> DTMF-only recognizer will never detect "start of speech". >> >> >> >> Another consideration to fold into the design choice is >> >> extensibility. Bear with me through a little gedankenexperiment. >> >> >> >> Suppose we want to define a new recognizer type, which I'll call >> >> the "name that tune" recognizer. The client plays music to the >> >> server and the server recognizes musical notes. The grammar is a >> >> standard musical notation, augmented with a semantic >> >> interpretation that transforms the notes into the title of the >> >> tune and provides that as an answer. >> >> >> >> First, there's no speech involved (or is there...hang on a >> >> minute). Second, in order to accommodate the "name that tune" >> >> recognizer, we'd have to extend both the client and the server to >> >> undetstand a directive as to whether to recognize music or now, >> >> inaddition to what the server already knows what to do based on >> >> the grammar. If you follow my logic above, whether or not we do >> >> that, we have to extend "start-of-speech" to say "I'm hearing >> >> music". So far fairly straightforward, but let me now throw in >> >> the pathological twist. >> >> >> >> Suppose what I feed to a combined music/speech recognizer is a >> >> work in sprechstimme (spoken music), like the "Geographical >> >> Fugue" (aside: this is a wonderful piece of music I highly >> >> recommend to anyone interested in small ensemble singing). In >> >> this case, the tune could be named by either doing speech or >> >> music recognition. Why is there any need for the client to >> >> constrain the server as to which it tries to do when it's >> already >> >> told the server what it wants through the grammar? >> >> >> >> A few other comments below >> >> >> >> >> >>> ii. Flexibility: Want speech-only recognition (because a second >> >>> recogniser is doing hotword on DTMF) >> >>> >> >>> >> >> I don't see how flexibility is affected by this deisgn >> choice. If >> >> that's what you want, feed the speech-only recognizer a grammar >> >> without any DTMF rules. >> >> >> >> >> >>> + How to implement 5? >> >>> >> >>> a. Implicitly: >> >>> - dtmfrecog: always DTMF-only recognition >> >>> - speechrecog: depends on active grammar type >> >>> > if a dtmf grammar is active then DTMF input is "on" >> >>> > if a speech grammar is active then speech input is "on" >> >>> >> >>> b. Explicitly: >> >>> - Add inputmodes header to RECOGNIZE >> >>> >> >>> Option a is David's "do what I mean case"; option b is >> the extra >> >>> dial for the client. >> >>> >> >>> >> >> Actually, that's not the point I was making with "do what I >> >> mean", but it's not essential to the discussion so let's move on. >> >> >> >> >> >>> It is worth noting that VoiceXML uses option b. This allows one >> >>> to activate both speech grammars and DTMF grammars (and >> therefore >> >>> be informed of any errors in the grammars at activation >> time) but >> >>> independently turn on whichever input mode you like e.g. >> perhaps >> >>> start with "both" then change to "dtmf". >> >>> >> >>> >> >> I'm not sure the VXML precedent is relevant here, because the >> >> application behind VXML is working a different part of the >> problem >> >> - how to traverse a TUI dialog based on different parts of the >> >> input space. In fact, I suspect that the VXML: choice was >> >> conditioned more by limitations at the time it was >> specified than >> >> an underlying good design choice. Clearly having to specify this >> >> in VXML make the job of handling a TUI with nodes like "Say or >> >> press 5" harder rather than easier. >> >> >> > >> > DB> The VoiceXML edge-case is pretty weird so it's not a major >> > concern. My main concern is that the client can indicate, somehow, >> > what the input modes are. >> > >> > >> >> >> >> Summing up, while I don't feel strongly one way or the other, I >> >> have a preference for handling this as follows: >> >> >> >> a) Rename "START-OF-SPEECH" to "INTERESTING-INPUT-RECEIVED" or >> >> something equivalent. >> >> b) Include a parameter in the event saying what was interesting >> >> about the input you received, with a registry of values which >> >> includes: >> >> - signal above noise floor >> >> - speech >> >> - dtmf >> >> - (possibly) music >> >> c) allow the event to be generated multiple times during a request >> >> >> >> >> > >> > DB> I like these suggestions (START-OF-INPUT?). However, I don't >> > see how the problem of the optimised case is not solved by them. I >> > think the optimised case is fine if we have the following rules: >> > >> Yes, I hadn't thought through the optimized case as thoroughly as >> you. Your suggested method name is fine by me as well >> >> > 1. START-OF-SPEECH (and optimised bargin) is only generated >> for the >> > input type that is being listened for >> > 2. A speechrecog listens for DTMF if DTMF grammars are active, >> > speech if speech grammars are active, or speech and DTMF if both >> > grammar types are active. >> > >> Works for me. >> >> > >> >> Note that all of the above I'm saying with my technical >> hat on and >> >> my chair hat off. >> >> Putting my chair hat on for a moment, we really need to get this >> >> spec to last call, so at some point Eric or I is going to >> declare >> >> rough consensus so we can move on. >> >> >> > >> > DB> Agreed! >> > >> > >> >> >> >> Dave Oran. >> >> >> >>> Dave >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> Sarvi makes a good point that adding the reason why the >> START-OF- >> >>> SPEECH occurred does not fix the optimised bargin case. >> >>> >> >>> dtmfrecog - listens for DTMF only >> >>> speechrecog - listens for DTMF only, or speech only, or speech & >> >>> DTMF >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> ----- Original Message ----- From: "Shanmugham, Saravanan" >> >>> >> >>> To: "David R Oran" ; "Dave Burke" >> >>> >> >>> Cc: ; "Klaus Reifenrath" >> >>> >> >>> Sent: Tuesday, July 05, 2005 8:51 PM >> >>> Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode >> >>> >> >>> >> >>> inline. >> >>> >> >>> -----Original Message----- >> >>> From: David R Oran [mailto:oran@cisco.com] >> >>> Sent: Tuesday, July 05, 2005 11:08 AM >> >>> To: Dave Burke >> >>> Cc: Shanmugham, Saravanan; Klaus Reifenrath; speechsc@ietf.org >> >>> Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode >> >>> >> >>> >> >>> On Jul 5, 2005, at 12:54 PM, Dave Burke wrote: >> >>> >> >>> > Inline. >> >>> > >> >>> > Dave >> >>> > >> >>> > ----- Original Message ----- From: "Shanmugham, Saravanan" >> >>> > >> >>> > To: "David R Oran" ; "Klaus Reifenrath" >> >>> > >> >>> > Cc: >> >>> > Sent: Tuesday, July 05, 2005 5:22 PM >> >>> > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode >> >>> > >> >>> > >> >>> > I agree with Dave's analysis. The purpose of this event >> >>> was barge-in. >> >>> > And barge-in should happen for both DTMF and speech. >> >>> > >> >>> > Is there a case where you think it should not behave >> >>> this way. If soe, >> >>> > please provide a scenario where you think >> >>> > 1. Barge-in should happen for DTMF and not voice or >> >>> vice-versa. >> >>> > >> >>> > DB> You want to do a DTMF recognition only because it is >> >>> noisy. >> >>> > While waiting for DTMF input, the speechrecog resource >> >>> (or advanced >> >>> > dtmfrecog) generates a START-OF-SPEECH because it heard >> >>> some speech. >> >>> > The client does not want to stop prompt playing unless >> >>> DTMF was heard >> >>> > but it can't tell by the START-OF-SPEECH whether speech >> >>> or DTMF was >> >>> > heard. Similarly vice versa. >> >>> > >> >>> It's an interesting design question what part of the >> >>> policy resides at the client and what at the server, and >> >>> who makes the "final decision" about whether what was >> >>> heard was relevant to the control channel. Right now we >> >>> (IMO) have a weird partitioning in many cases where the >> >>> client basically says "do what I mean", but there are no >> >>> constraints of what the server actually does, and no >> >>> normalized basis for the client to figure out what to set >> >>> various magic numbers to (e.g. sensitivity). >> >>> >> >>> In this case the only thing the client needs to decide is >> >>> whether to kill the prompt because the server thinks >> >>> something that would interfere with the feedback >> >>> ear/mouth/finger control happened. What this says to me is >> >>> that it isn't necessarily a good idea for the client to >> >>> have more knobs to control the server (especially if those >> >>> knows are just more value/policy input ungrounded in any >> >>> physics/ acoustics). On the other hand, having the server >> >>> tell the client more about what it thinks is going on is >> >>> probably valuable. >> >>> >> >>> So, Coming to the point after this long rambling >> >>> introduction, I think it would in fact be useful for the >> >>> START-Of-SPEECH event to indicate some extra information, >> >>> for example: >> >>> a) I got something enough above the noise floor to qualify >> >>> for exceeding the "Sensisitvity" parameter you sent in on >> >>> the request but I really can't tell what it is (could be a >> >>> hippopatmus fart, or a siren in the background, or captain >> >>> crunch trying to whistle DTMF). >> >>> b) I think I'm hearing speech >> >>> c) I think I'm hearing DTMF >> >>> >> >>> Though I agree with your former part of your response. I am not >> >>> sure I >> >>> agree with your proposed solution. >> >>> The way I see this problem is that, it is more of what >> constitues a >> >>> barge-in event. This boils down to whether it is speech, DTMF or >> >>> both. >> >>> This is inturn boils down to what type of recognizer >> resource we are >> >>> using, dtmf-recog, speech-recog, and speech-only-recog(we >> don't have >> >>> this and I don't think we should add it, but think of this as a >> >>> place >> >>> holder that explains the concept). >> >>> >> >>> A client knowing what type of barge-in happenned, does >> not impact >> >>> the >> >>> barge-in operation itself as it may be too late(for the optimized >> >>> barge-in case). It may have other use cases, and if we >> can identify >> >>> them, I don't mind adding support for the START-OF-SPEECH event >> >>> to say >> >>> what type of barge-in happenned. But that itself does not >> solve the >> >>> original problem raised. Refer to my previous response. >> >>> >> >>> The solution lies in defining what what is a barge-in event. >> >>> That boils >> >>> down to what type of recognition is happenning, >> dtmf-only, speech- >> >>> dtmf >> >>> or speech-only. We do not support speech-only as a resource >> >>> today, the >> >>> question is do we need a header to force it. >> >>> >> >>> Sarvi >> >>> >> >>> >> >>> > 2. You would benefit from the client knowing >> what caused >> >>> the >> >>> > barge-in, DTMF Vs speech. >> >>> > >> >>> > DB> See previous comment. And previous e-mail: either add an >> >>> > inputmodes header (taking value speech, dtmf, both) to >> >>> the RECOGNIZE >> >>> > request or add a header to the START-OF-SPEECH event >> >>> indicating DTMF >> >>> > or speech. >> >>> > >> >>> I'm leaning in your direction on this latter point - as >> >>> should be evident from what I wrote above. >> >>> >> >>> > Sarvi >> >>> > >> >>> > -----Original Message----- >> >>> > From: speechsc-bounces@ietf.org >> >>> > [mailto:speechsc-bounces@ietf.org] On Behalf Of >> David R >> >>> Oran >> >>> > Sent: Tuesday, July 05, 2005 5:24 AM >> >>> > To: Klaus Reifenrath >> >>> > Cc: 'speechsc@ietf.org' >> >>> > Subject: Re: [Speechsc] START-OF-SPEECH in >> DTMF-only mode >> >>> > >> >>> > >> >>> > On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote: >> >>> > >> >>> > > The current spec is not clear when >> START-OF-SPEECH need >> >>> > to be send in >> >>> > > the following scenarios: >> >>> > > A) The client requested a DTMF Recognizer. Is the >> >>> > START-OF-SPEECH >> >>> > > event send to the client also if speech was detected? >> >>> > I suspect so, since one of the prime purposes >> is to enable >> >>> > client- mediated barge-in handling. However, if the >> >>> > recognizer is in fact only capable of recognizing DTMF >> >>> > then it may in fact not report anythin unless it's using >> >>> > some primitive thresholding machinery, like a SN >> >>> threshold. >> >>> > > B) The client requested a Speech Recognizer, but only >> >>> > activated DTMF >> >>> > > grammars. Is the START-OF-SPEECH event send to the >> >>> > client also if >> >>> > > speech was detected? >> >>> > Again, I'd say yes, for the same reason as above. >> >>> > > I think in both cases START-OF-SPEECH should only >> >>> be send after >> >>> > > detecting a DTMF digit (see Figure 12 of VoiceXML >> >>> 2.0: http:// >> >>> > > www.w3.org/TR/voicexml20/#dmlATiming). >> >>> > We seem to have reached different conclusions. I'd be >> >>> > interested in why you think my analysis above is wrong. >> >>> > >> >>> > Dave. >> >>> > >> >>> > > Klaus >> >>> > > >> >>> > > _______________________________________________ >> >>> > > Speechsc mailing list >> >>> > > Speechsc@ietf.org >> >>> > > https://www1.ietf.org/mailman/listinfo/speechsc >> >>> > > >> >>> > >> >>> > _______________________________________________ >> >>> > Speechsc mailing list >> >>> > Speechsc@ietf.org >> >>> > https://www1.ietf.org/mailman/listinfo/speechsc >> >>> > >> >>> > >> >>> > _______________________________________________ >> >>> > Speechsc mailing list >> >>> > Speechsc@ietf.org >> >>> > https://www1.ietf.org/mailman/listinfo/speechsc >> >>> > >> >>> >> >>> >> >>> _______________________________________________ >> >>> Speechsc mailing list >> >>> Speechsc@ietf.org >> >>> https://www1.ietf.org/mailman/listinfo/speechsc >> >>> >> >>> >> >> >> >> _______________________________________________ >> >> Speechsc mailing list >> >> Speechsc@ietf.org >> >> https://www1.ietf.org/mailman/listinfo/speechsc >> > >> >> _______________________________________________ >> Speechsc mailing list >> Speechsc@ietf.org >> https://www1.ietf.org/mailman/listinfo/speechsc >> > > > _______________________________________________ > Speechsc mailing list > Speechsc@ietf.org > https://www1.ietf.org/mailman/listinfo/speechsc > _______________________________________________ Speechsc mailing list Speechsc@ietf.org https://www1.ietf.org/mailman/listinfo/speechsc From speechsc-bounces@ietf.org Thu Jul 07 09:22:16 2005 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqWKV-0001mP-I7; Thu, 07 Jul 2005 09:22:16 -0400 Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1DqWKS-0001kD-HV for speechsc@megatron.ietf.org; Thu, 07 Jul 2005 09:22:13 -0400 Received: from ietf-mx.ietf.org (ietf-mx [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA05416 for ; Thu, 7 Jul 2005 09:22:10 -0400 (EDT) Received: from letter.nuance.com ([207.107.210.132]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1DqWlb-0007Mc-Kf for speechsc@ietf.org; Thu, 07 Jul 2005 09:50:18 -0400 Received: from postcard.nuance.com ([10.3.6.20]:49291) by letter.nuance.com with esmtp id 1DqWKA-0002eG-MD; Thu, 07 Jul 2005 06:21:54 -0700 Received: from mtb1exch01.nuance.com ([10.3.2.6]) by postcard.nuance.com with Microsoft SMTPSVC(6.0.3790.0); Thu, 7 Jul 2005 09:21:31 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Date: Thu, 7 Jul 2005 09:21:48 -0400 Message-ID: <7DE7C4EF3B7C8B4B82955191378290D802ED3F88@mtb1exch01.nuance.com> Thread-Topic: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Thread-Index: AcWC9PY0LPkvxBeXQQOVCo/nDyd3egAAaF2g From: "Pierre Forgues" To: "Klaus Reifenrath" , "Eric Burger" , X-OriginalArrivalTime: 07 Jul 2005 13:21:31.0325 (UTC) FILETIME=[CA0B96D0:01C582F6] X-FromHost: postcard.nuance.com [10.3.6.20]:49291 Lines: 607 X-Spam-Score: 0.0 (/) X-Scan-Signature: 9668dd9718e12afaf579fddf1143437a Content-Transfer-Encoding: quoted-printable Cc: X-BeenThere: speechsc@ietf.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Speech Services Control Working Group List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: speechsc-bounces@ietf.org Errors-To: speechsc-bounces@ietf.org I agree with Klaus's position. =20 Moreover, when the SOS event is generated, it is possible exact nature of the barge-in method is not known at that time. It is critical to notify the user as soon as possible to stop the prompt or this impacts recognition accuracy. In some cases, the resource might not even have exact determination at that ninstant in time whether this is DTMF or speech. /Pierre -----Original Message----- From: speechsc-bounces@ietf.org [mailto:speechsc-bounces@ietf.org] On Behalf Of Klaus Reifenrath Sent: Thursday, July 07, 2005 9:04 AM To: 'Eric Burger'; speechsc@ietf.org Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) I) START-XYZ for the new event is misleading if we allow the event to be generated multiple times. Do we really need to introduce a new (optional) event? Why should a DTMF only resource do speech detection?=20 II) 1. START-OF-SPEECH (and optimised bargin) is only generated for the =20 input type that is being listened for 2. A speechrecog listens for DTMF if DTMF grammars are active, =20 speech if speech grammars are active, or speech and DTMF if both =20 grammar types are active. I personally like this, BUT doesn't it conflict with VoiceXML 2.0 section 4.1.5.1 (bargein type speech): "The prompt will be stopped as soon as speech or DTMF input is detected. The prompt is stopped irrespective of whether or not the input matches a grammar and irrespective of which grammars are active." =20 Klaus -----Original Message----- From: Eric Burger [mailto:eburger@brooktrout.com]=20 Sent: Donnerstag, 7. Juli 2005 14:32 To: speechsc@ietf.org Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?) Can we declare consensus?=20 > -----Original Message----- > From: speechsc-bounces@ietf.org [mailto:speechsc-bounces@ietf.org] > Sent: Wednesday, July 06, 2005 10:45 AM > To: Dave Burke > Cc: speechsc@ietf.org; Shanmugham, Saravanan; Klaus Reifenrath > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> > summary(?) >=20 > I think we're getting close. I though about snipping out some pieces=20 > to cut down the text, but I realized the context is still needed. See=20 > inline. >=20 > On Jul 6, 2005, at 10:29 AM, Dave Burke wrote: >=20 > > Inline. > > > > Dave > > > > ----- Original Message ----- From: "David R Oran" > > To: "Dave Burke" > > Cc: ; "Shanmugham, Saravanan" ;=20 > > "Klaus Reifenrath" > > Sent: Wednesday, July 06, 2005 1:07 PM > > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary > > (?) > > > > > > > >> > >> On Jul 6, 2005, at 5:31 AM, Dave Burke wrote: > >> > >> > >>> + Attempting to summarise: > >>> > >>> 1. START-OF-SPEECH is useful for the client to know when > to stop =20 > >>> playing prompts the in non-optimised case 2. START-OF-SPEECH is=20 > >>> useful for the client to calculate the bargin time (e.g. VoiceXML > >>> 2.1