Subject: [rtcweb] Summary of DTMF Issues

From: Cullen Jennings <[email-masked]>
Date: Tue, 2 Feb 2016 10:20:43 -0800

I had a phone call with Roman to try and get to make sure I understood the DTMF issues he is concerned with. It was very productive and I am going  to try and summarize my understanding of the issues here (and Roman feel free to correct me if I got any of this wrong). 

Three basic issues

1) Support for A-D (been much discussed). This may be the least important 

2) Which specification certain things are specified in

3) timing of the DTMF event packets (this is probably the most important)

Let me outline each of these… 

——————————

Issue 1 - DTMF A-D

Roman believes that the DTMF A-D tones, while not used all the time, are used for signalling between some call centre IVR applications (from one IVR to another) and that they are supported by the major US carriers representing half or more of the market. (Rowan can you put in any more here about where it works - was this on IP connection of ISDN connections ?)

Other people seem to think they are not supported on the majority of ISDN trunks thus are very risking to use from end user (like browser) as they encouraged applications that failed. 

I have not heard anyone say this is the end of the world one way or the other. It is worth nothing that they were required in RFC 2833 but became optional in 4733. 

--------------

Issue 2 - Which spec should some of the things be in

Right now the normative text on tone duration and gaps between tones is in the W3C spec. Roman thinks what the text says is correct but it is in the wrong specification and should instead be normatively defined in an IETF spec (and the W3C spec can point at that or have a non normative summary of it)

Right now the IETF specs normatively define the range theses options can cover, and the W3C spec defines an API to set them along with default values for browsers if they are not set. All within the IETF defined ranges. So the IETF says the range applications need to be in and the W3C say for the browser application what the default is for the browser application. 

I think the best approach here might just be to get the WebRTC and RTCWeb chairs to have a quick phone call and see if they can come up with a proposal they think everyone can live with and float it to both lists. (note I’m just saying this my individual contributor hat on - I have not talked to any other chairs about it and have no idea what they will say )

-----------------

The 3rd issues the biggest issue and has to do with when events are sent and if audio is suppressed while the events are being sent. 

First let me get the easy part of this … If you receive unsuppored audio events , they should just be ignored (other than updaing stats) and not cause any problem like audio artifacts , stopped connection etc. I think everyone agrees with this but but we need to check that this is clear enough in the various specifications. (I sort of hope this got covered in 4733 but if not we might need a clarifying statement in one of the rtcweb specs).

Before I get into the harder part of this issue about what senders should do, let me explain a bit about what most receivers do. 

When rendering a received DTMF even into an auto stream, the DTM overwrites the audio for the duration of the DTMF tone. Every PSTN GW or IP phone that I have seen the code of for this does this simply by decoding the audio stream into the jitter buffer, computing where in the audio stream the DTMF tone would go then synthesizing a DTMF tone it writing it over the audio samples in the jitter buffer. The varies specs require this. 

What senders do is a bit broader. Lots of GWs and ATA will use a DSP to detect the tones in the audio stream, suppress them out of the audio stream, then send the 4733 event of the detected tone. The reason to suppress them is that if the far end is also running a DSP to listen for in band tones as well as receiving out of band tones, you can get double detection near the start of the tone. The normal VAD processing will typically cause the no audio packets to be sent for the time of the surpassed audio while the 4733 events are being sent. 

On the other hand lots of IP phones that are configured for 4733 do not generate any inland audio tones. When they user hits they button, they just generate 4733 events along with the timing information for them. This often totally uncoupled from the microphone audio path that just keeps sending audio from the microphone at the same time. One of the advantages of this is that since they 4733 tones are out of band, they don’t destroy the inland audio. So if you are on a conference call and using DTMF tones to do something like start or stop a recording, the person can keep talking while pressing the DTMF tones that the conference bridge will get the 4733 events and people listening to the call will continue to hear the person speaking with no gaps in the audio around the time that the keys were pressed to start or stop recording. 

Note that given receivers work as described above, both of these sender strategies work out fine. 

Roman would like to reduce the variance of what senders are allowed to do. This proposal applies just as much to SIP or any other applications of 4733  and is not specific to WebRTC. 

I’m going to outline 3 options. Option 0 is not what Roman wants to do but I include it just as a sort of baseline of roughly what we have today. Roman prefers option 2. 

Option 0

The sender can send the 4733 DTMF events at the time it feels most appropriate and if their is in band DTMF tones, it SHOULD suppress the in-band DTMF tones. (note a browser would never have in-band tones it needed to supress) 

Option 1

The sending endpoint MUST send silence in audio during the times that DTMF tones are being sent. 

Option 2

The  sending endpoint MUST not send any audio packet for he times that DTMF tones are being sent. The sender would do this by sliding the start of the DTMF tones forward (or backwards) until it aligned with the boundary of audio packet. The gaps between DTMF tones would be expanded to make the the DTM tones end on the boundary of an audio packet. The appropriate audio packets would then be suppressed. I suspect that for things like an ATA, it would be even more complicated in that one would need to make sure that if a DTMF tone start was slid forward, the appropriate in band audio signal in the first packet got surpassed in some way. 

I have not looked at the code path for FF and Chrome but I suspect they handle sending tones much like the IP phones in that the path of the DTMF is pretty much independent from the path for the audio. Given they are detection inland audio tones and replacing them it seems reasonable that the current implication behave more like an IP phone than a PSTN GW or ATA.

---

From: Adam Roach <[email-masked]>
Date: Tue, 2 Feb 2016 15:27:37 -0600

> Issue 1 - DTMF A-D

For the record, I am of the opinion -- although only weakly -- that we 
should go ahead and require support for these specific four tones in 
WebRTC browsers.

This one is a little tricky. While I understand questions around SDO 
ownership of decisions, I think the dominating factor here should be the 
document constituencies. The IETF documents are of use to people 
implementing browsers, WebRTC devices, and WebRTC-compatible devices.

The W3C documents are aimed at people creating browsers as well as 
people writing webapps.

I would not expect webapp developers to ever check the IETF documents, 
nor would I expect WebRTC devices to check W3C's API documents.

Following this principle to its conclusion: those things that are 
applicable to webapps need to be in the W3C documents; those things that 
are applicable to WebRTC devices need to be in the IETF documents, and 
those things that apply to both need to be repeated in both sets of 
documents and rigorously consistent with each other.

> Right now the normative text on tone duration and gaps between tones is in the W3C spec. Roman thinks what the text says is correct but it is in the wrong specification and should instead be normat…

I think this is fine, but note that the API has clamping and default 
behavior based around these timing values. We don't want to change the 
API behavior, only the document that is authoritative for the timing values.

> ...

Doesn't that really point to this being an issue for a different WG?

> The  sending endpoint MUST not send any audio packet for he times that DTMF tones are being sent. The sender would do this by sliding the start of the DTMF tones forward (or backwards) until it ali…

Option 2 seems almost unimplementably complex. It also has the property 
of being what no one does today; so even if it were as easy to implement 
as the others, it would maximize the amount of work necessary to bring 
implementations into compliance.

Option 1 seems to defeat the use case -- which I find useful -- that 
allows conference participants to send DTMF to a focus without having 
their voice audio unnecessarily suppressed.

Option 0 is, as you say (and my experience is consistent with this) the 
status quo. Your description doesn't outline any concrete issues with 
this mode of operation, so it's not clear why any IETF WG would seek to 
change things.

I mean, "here's how things work right now, there's nothing broken about 
it, should we continue to do it?" doesn't seem like a compelling 
rallying cry.

Can you expand -- or get Roman to expand -- on the rationale for wanting 
to change the current behavior?

> I have not looked at the code path for FF

If you find a DTMF-sending code path for Firefox, please let me know.

/a

---

From: Martin Thomson <[email-masked]>
Date: Wed, 3 Feb 2016 15:16:33 +1100

On 3 February 2016 at 08:27, Adam Roach <[email-masked]> wrote:
> should go ahead and require support for these specific four tones in WebRTC
> browsers.

This costs us little to specify and implement.  If it is useful, then
it will have been worth it.  And if it's a latent footgun, then we can
put it in the pile with all the others.

>> Issue 2

Like Adam, I don't mind if the normative text is moved as long as it
remains correct.

>> Issue 3

Option 0 please.

---

From: Tim Panton <[email-masked]>
Date: Thu, 4 Feb 2016 10:47:13 +0000

> The  sending endpoint MUST not send any audio packet for he times that DTMF tones are being sent. The sender would do this by sliding the start of the DTMF tones forward (or backwards) until it ali…

I’m not keen on this. (Assuming I understand it correctly) - 
My worry is that recipients that don’t care about DTMF now have a discontinuous audio stream - their jitterbuffers and clock de-skews may try to _fix_
unless they understand DTMF. Which leads to the situation I’ve been trying to avoid - i.e. that every webrtc babymonitor has to understand DTMF, just so 
it can ignore DTMF correctly.

Please tell me I am wrong.

Tim.

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 4 Feb 2016 10:49:34 -0500

> Issue 1 - DTMF A-D

I have a weak preference for including A-D since it seems simple enough.

> Issue 2 - Which spec should some of the things be in

No strong opinion here beyond "please let me know what I need to add
to the draft".

> The 3rd issues the biggest issue and has to do with when events
> are sent and if audio is suppressed while the events are being
> sent.

I strongly prefer Option 0 here. I don't think options 1 and 2 can be
implemented reliably enough to be useful and Option 0 has the
advantage of clean audio in the case where the sender only sends DTMF
as events (e.g. for videoconferencing bridges as you mentioned).

One thing to consider about options 1 and 2 is that using a codec like
Opus (though it's also possible with G.711), the frame size can go as
low as 2.5 ms, making it nearly impossible to detect tones before they
are long enough for the receiver to also detect them from the in-band
audio. So I'm fine with "SHOULD suppress the in-band DTMF tones" (when
possible), but making it a MUST would just not work.

	Jean-Marc

---

From: Roman Shpount <[email-masked]>
Date: Thu, 4 Feb 2016 19:01:08 -0500

Cullen,

Thank you very much for your great summary.
Here is a bit more details/information from me.

*1) Support for A-D (been much discussed). This may be the least important*

I am in favor of supporting this even though this is not as important to me
as it might seem.

I have done a comprehensive test of A-D DTMF tone support by US and Canada
phone operators. Based on my test DTMF A-D tones are universally supported.
Here are more details for those who are interested:

During this test I got two geographic numbers from two US CLEC (competitive
local exchange carrier), one geographic number from Canadian CLEC, and toll
free numbers from AT&T, MCI/Verizon and Level3.

All of those numbers were terminated to my IVR equipment via VoIP trunks
configured with these carriers.

Calls to the first US CLEC number can terminate via LEC PSTN switch but
most of the time, since this CLEC also provides local tandem and long
distance service are terminated via direct interconnect with long distance
carrier. This means that call is often transmitted VoIP from origination to
termination typically going though several SBCs due to the magic of least
cost routing.

Calls to the second US CLEC number and calls to Canadian CLEC always go
through LEC PSTN switch and CLEC VoIP gateway.

Calls to AT&T toll free numbers are handed over to AT&T legacy long
distance network and then go through a VoIP gateway before being handed
over to my equipment.

Calls to MCI/Verizon and Level3 toll free numbers go through more modern
long distance network and can be transmitetd via VoIP or hit PSTN
interconnects and VoIP gateways.

I have placed test calls using wholesale VoIP trunks through the following
long distance carriers (pretty much everybody): 360Networks, 382com, ANI
Networks, ANPI, AT&T, Brightlink, Broadvox, CenturyLink, Comcast, Comcast
PSTN, Earthlink, GlobalCrossing , Hypercube, Impact Telecom, Intelepeer,
Intelepeer PSTN, Inteliquent, ISPTelecom, Level3 IP, Level3 PSTN, O1,
Onvoy, Peerless, Puerto Rico Telephone, Teliphone, TNCI, Verizon, Windstream

Both inbound and outbound configurations are typical of how VoIP phone
service operators or IVR and conferencing providers get their phone numbers
or terminate outbound calls. Enterprises typically go through an additional
hosted PBX or trunk management system, but this should not change DTMF
support.

During the test my IVR test system would place a call, wait for welcome
prompt to finish playing, send DTMF digits *1234567890ABCD# and wait for
the test system to send the same digits back. Based on this test, all DTMF
digits, including A-D were successfully transmitted in both directions.
Based on the variety of inbound and outbound carriers tested, this should
demonstrate that majority of US long distance operators and their
interconnects support A-D DTMF digits.
___________________________________________

*2) Which specification certain things are specified in*

I specifically referred to https://www.w3.org/TR/webrtc/#rtcdtmfsender

The duration parameter indicates the duration in ms to use for each
character passed in the tones parameters. The duration cannot be more than
6000 ms or less than 40 ms. The default duration is 100 ms for each tone.

The interToneGap parameter indicates the gap between tones. It must be at
least 30 ms. The default value is 70 ms.

There are multiple ITU, ETSI and other normative documents that define DTMF
tone and gap duration. I do not think there are any IETF documents that
specify actual tone duration. I think it would be best to repeat this
information in rtcweb-audio draft so that implementer will get the same
consistent values from either document. This is why I propose to add the
following text to rtcweb-audio:

Generated events MUST have duration of no more than 6000 ms and no
less than 40 ms with the recommended default duration of 100 ms for each
tone. The gap between events MUST be no less then 30 ms with the
recommended default duration of 70 ms.

__________________________________________

*3) Timing of the DTMF event packets (this is probably the most important)*

This is, indeed, the most important decision.

First of all, webrtc end point should be able to receive any
telephone-events. These events should be ignored (other
than updating stats) and should not cause any problem like audio artifacts
or stopped connections. Webrtc end point should be able to deal with
discontinued audio transmission during these events. There is no mechanism
to stop PSTN network from generating telephone-events or
from controlling if non-event audio packets would or would not be sent
during the telephone-event. During my test described earlier half of the
providers sent audio during telephone-events and half did not.

First decision that we need to make is regarding timing of the
telephone-events. We can specify that:

a. telephone-events should start and stop on regular non-event audio packet
boundaries.
b. telephone-events can start and stop at any time and send updates at any
reasonable regular interval

Option a. has a benefit of better interop with existing equipment, since in
packet audio is suppressed on whole packet. It also more friendly to jitter
buffers.

Implementation option a. transmission logic is actually trivial and maps
well on regular audio transmission algorithm. Here is one of the proposed
algorithms:

When sending an audio packet:

1. If not currently sending telephone-event or inter-tone gap audio, check
if new telephone-event needs to be sent. If new tone is in the queue,
remove it from the queue, set the already transmitted telephone-event
duration to 0, record the telephone-event start timestamp and set the tone
start flag.

2. Is sending a telephone-event, increment the transmitted telephone-event
duration by ptime. Set duration in the telephone event packet with this
duration. If tone start flag is set, set the mark bit on telephone event
packet. If transmitted telephone-event duration is equal or greater
then desired
tone duration set the end of tone flag; schedule two re-transmissions of
end telephone-event with small delay between each re-transmission
(typically 5-10 ms); set transmitted gap length to 0; set start gap flag to
true; and switch to send gap audio mode. Clear tone start flag. Send the
telephone-event packet.

3. If sending a gap audio, increment the transmitted gap duration by ptime.
If start gap flag is set, set the the mark bit on the packet. Send the
audio packet of regular ptime length. Clear the gap start flag. If
transmitted gap duration is greater then desired gap duration switch to
normal audio transmission mode.

Above described algorithm can work with or without audio being transmitted
during the telephone-event. If audio is not sent during the
telephone-event, codec state will need to be reset when gap audio is
starting to be transmitted. If background audio and not silence is send
during the telephone-event, mark bit does not need to be set when gap audio
is started.

Second decision is regarding what needs to be sent during telephone-event.
There are 3 options here:

a. Background audio
b. Silence
c. Nothing

There are systems that implement all three options.

Sending background audio is typically done by cheaper IP and soft phones.
This is an easiest to implement but the lowest quality option. Background
audio has a downside of reducing the DTMF tone recognizer performance if it
gets rendered into in-band audio by the gateway and get's combined with
generated DTMF tone. For DTMF recognizer background audio is basically
noise. Furthermore, a lot of DTMF keypad implementations play DTMF tone as
a feedback to the user. This applies to some WebRTC DTMF keyboards as well,
where an audio file with DTMF tone is played to the user when the keypad
digit is played. This is causing double DTMF issues already. Furthermore, I
am not aware of any conferencing systems that renders audio during the
telephone event into the mixer. Most conferencing systems are configured to
suppress this audio since it is either DTMF or sound of persons finger
punching the conference phone microphone when dialing the digits.

Silence is commonly used by a lot of gateways and SBCs.

For a lot of SBCs and gateways sending silence or no audio packets is a
configuration option. In essence, they treat sending no audio packets
as discontinuous transmission, since audio during telephone-event is
assumed not being rendered by the terminated gateway. Thus, it
is omitted due to being completely redundant.

Main argument for sending nothing are things that Olle brought up in
https://www.ietf.org/mail-archive/web/rtcweb/current/msg02963.html . The
problem is that some gateways do not handle telephone-events for the same
timestamps as audio and discard packets in random. Based on my testing this
does not occur with any of the carriers that I am using, so this is likely
fixed.

Systems I build and operate typically do not send packets when sending
telephone events and use above described algorithm to align
telephone-events and audio packets. This seems to work well with wide
selection of carriers and equipment. This is why I prefer this option, but
I am not dead set against other options. We routinely test DTMF
transmission performance to our conferencing service across multiple
carriers in over 60 countries and I am willing to do an interop test to see
how well the options we pick will perform.

Regards,
_____________
Roman Shpount

---

From: Martin Thomson <[email-masked]>
Date: Fri, 5 Feb 2016 22:36:59 +1100

> There are 3 options here:

I would say d. Audio (or is that the same as a. ?)  For the same
reasons Adam stated: we have cases that b and c break, namely the use
of telephone-event as signaling.

If some systems have specialized requirements around this, it isn't
very hard to suppress audio and get option b.  Yes, sending nothing is
hard, but if you don't synchronize to audio packet boundaries, you
won't get two packets with the same timestamp and won't that improve
the chances that you get the tone through?

I can see why you might suppress CN packets if those are enabled on
the basis that it's largely harmless.  But I'm guessing that the
affected systems won't like CN.

(BTW, I don't really want to see a knob for this.)

---

From: Roman Shpount <[email-masked]>
Date: Fri, 5 Feb 2016 16:42:32 -0500

> of telephone-event as signaling.

This is option a. By background audio I mean audio from the microphone.

If you read https://tools.ietf.org/html/rfc4733#section-2.5.2 you will see
that "it is RECOMMENDED that gateways render only the telephone-event
payload once it is received". This means that audio during the
telephone-event is normally ignored and can be safely replaced by silence
or not sent at all.

I do not think there is a lot of value in the background audio during the
telephone event. Systems described by Adam before, the ones that convert
from telephone-event audio to DTMF in signaling channel, normally actively
suppress audio covered by the telephone-event. The reason for this is that
if audio ever gets reassembled back into in-band DTMF tones, since
signaling is not time synchronized with audio, two DTMF tones can be
generated in the resulting audio stream -- one with original DTMF tone and
another with DTMF tone generated based on signaling.

Finally, I do not think any systems suppress just the DTMF tone from the
audio using some sort of DSP filter. When DTMF tones are suppressed, all
audio for the duration of the tone is suppressed. When DTMF tone is
detected in-band, it must represent the majority of audio energy so
filtering out the tone would be doing additional work for no reason.

> the chances that you get the tone through?

Not synchronizing packets on audio boundaries will decrease your chances
for the tone getting through. The reason for this are gateways that time
packet playback based on sequence numbers and not RTP timestamps. With
telephone-event packet this system assumes that DTMF tone starts after the
end of the audio packet with immediately previous sequence number and it is
played until telephone-event with end bit is received. Essentially such
systems ignore RTP timestamps and use sequnce number * ptime instead.

> affected systems won't like CN.

I never seen a system which sends CN packets during telephone-event. I saw
systems that send regular audio packets and systems which send nothing. CN
immediately before the telephone-event or during the telephone-event would
be completely redundant.

P.S. There is one more option for the audio during telephone-event --
generate the DTMF tone and encode it using the audio codec. I hope no one
is in favor of this. This is extra work that serves no purpose. I think I
disliked it so much I forgot to include it in the list of available options.
_____________
Roman Shpount