Subject: Re: [dispatch] Yet another problem to dispatch

From: "Roni Even" <[email-masked]>
Date: Fri, 5 Jun 2009 00:35:15 +0300

Emil,
I think that John means that what you call loudest speaker is not
necessarily a speaker but may represent some noise which is high enough to
qualify as one of the N loudest speakers.
The conferencing bridge try to address this problem by defining some
thresholds for accepting a user to the mix. 

I also looked at the draft and would like to see a discussion about the
current options to receive the active speakers indications. I am not so sure
about the benefit of displaying sound level, note that the mixers usually do
some normalization on the volume of the mixed streams to enhance the loudest
speaker and to have a common level regardless of the source level in order
to prevent fluctuation of the voice that may be caused by different
microphones, encoders and background noise.  

Roni Even

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Emil Ivov
Sent: Thursday, May 28, 2009 7:29 PM
To: [email-masked]
Subject: Re: [dispatch] Yet another problem to dispatch

Hey John,

Elwell, John wrote:
> Emil,
> 
> But then you would need some sort of throttling, to prevent the UI
> constantly changing the list of names whose volumes are displayed. If
> the mixer were to do this throttling it would reduce the frequency of
> messages, which might allow a bigger choice of potential solutions.

I am not sure I understand exactly what you mean by throttling. Mixers
could be implemented to only include sound level information for the
loudest speakers thus having the possibility to ignore silent and almost
silent ones. It should be up to the mixer to determine the exact
threshold since that's where bandwidth would be most precious. Is this
what you are suggesting?

Then, once a client gets this information it can consider (and I guess
that's in line with 3550) that all CSRCs not present in a particular
packet have been silent for the corresponding period of time.

As to how exactly the client would render the sound level information,
whether or not it would display all of it, and whether it would somehow
rearrange participants, that should really be up to the UI implementors.

Does this make sense?

Cheers
Emil

> 
> These are just initial thoughts on my part, not firm opinions.
> 
> John
> 
> 
>> -----Original Message-----
>> From: Emil Ivov [mailto:[email-masked]] On Behalf 
>> Of Emil Ivov
>> Sent: 28 May 2009 14:02
>> To: Elwell, John
>> Cc: Enrico Marocco; Cullen Jennings; [email-masked]
>> Subject: Re: [dispatch] Yet another problem to dispatch
>>
>> Hey John,
>>
>> Elwell, John wrote:
>>> Background noise in conferences is certainly a big issue. 
>> The web-type
>>> interface that Cullen mentions does in fact allow the moderator to
>>> momentarily mute each suspected noisy party in turn to 
>> discover who is
>>> the culprit. Clearly this is not scalable to large 
>> conferences. But also
>>> a simultaneous display of noise levels (e.g., in the form 
>> of bar graphs)
>>> is not scalable. 
>> One way to address this would be to allow mixer implementors to only
>> include sound levels for the N loudest participants. UI implementors
>> could then use a similar algorithm when presenting the information.
>>
>> How does this sound?
>>
>> Cheers
>> Emil
>>
>>
>>> Whilst it might be easy to find protocol solutions, we
>>> need to be confident that people will find ways of 
>> exploiting them in a
>>> reasonable way at the user interface. Certainly worth exploring.
>>>
>>> John
>>>
>>>
>>>> -----Original Message-----
>>>> From: [email-masked] 
>>>> [mailto:[email-masked]] On Behalf Of Enrico Marocco
>>>> Sent: 27 May 2009 21:02
>>>> To: Cullen Jennings
>>>> Cc: [email-masked]
>>>> Subject: Re: [dispatch] Yet another problem to dispatch
>>>>
>>>> Cullen Jennings wrote:
>>>>> I find this "active speaker" problem very interesting. Often the  
>>>>> device that wants to display the active speaker information 
>>>> is not the  
>>>>> same as the device dealing with the RTP. For example, most the  
>>>>> conference systems I am using a phone but there is a web 
>> interface  
>>>>> that is dealing with active speaker information on my 
>>>> browser which is  
>>>>> no on my phone. Be interesting to look at all the existing 
>>>> approaches  
>>>>> to this.
>>>> Yes, I'm familiar with that kind of systems. However, the 
>>>> information we
>>>> are considering -- instantaneous sound levels of each 
>>>> participant -- is
>>>> definitely changing more rapidly than just active speaker 
>>>> indication and
>>>> also has more stringent synchronization requirements. 
>> That's why our
>>>> favorite approach would be to carry it in the media streams 
>>>> themselves,
>>>>  with some sort of a modulation of the RTP CSRC fields 
>> that could be
>>>> considered either an hack or an elegant solution. I would be very
>>>> interested in hearing any opinions about that.
>>>>
>>>> -- 
>>>> Ciao,
>>>> Enrico
>>>>
>>> _______________________________________________
>>> dispatch mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/dispatch
>>>
>> -- 
>> Emil Ivov, Ph.D.                               30a rue de la Patrie
>> Project Lead                                   67300 Schiltigheim
>> SIP Communicator
>> [email-masked]                     FAX:   
>> +33.1.77.62.47.31
>> http://sip-communicator.org                    PHONE: 
>> +33.1.77.62.43.30
>>
> 

-- 
Emil Ivov, Ph.D.                               30a rue de la Patrie
Project Lead                                   67300 Schiltigheim
SIP Communicator
[email-masked]                     FAX:   +33.1.77.62.47.31
http://sip-communicator.org                    PHONE: +33.1.77.62.43.30
_______________________________________________
dispatch mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dispatch

---

From: Emil Ivov <[email-masked]>
Date: Fri, 05 Jun 2009 11:10:28 +0200

Hey Roni,

> qualify as one of the N loudest speakers.

Indeed, and that's one of the reasons why sound level indicators are so
useful. It is far easier for a human to determine what is background
noise and what is speech or other activity that is part of the conversation.

> thresholds for accepting a user to the mix. 

And they can continue doing so. Are you saying that there would be some
kind of conflict between these thresholds and the fact that mixers would
include sound level information for what they are sending?

> about the benefit of displaying sound level, 

The problem with active speaker designation comes from its binary
nature. Simply picking the one loudest media source (even after
normalization) and dubbing it "active speaker" does not always
correspond to what's actually happening in a call. Sound level
indicators would help identify sources of background noise that got
through the filtering on the mixer.

> note that the mixers usually do

I am not sure why this is a problem. After doing all of the above a
mixer would simply include sound level information for the data it is
about to send (i.e. the normalized streams).

Cheers
Emil

-- 
Emil Ivov, Ph.D.                               30a rue de la Patrie
Project Lead                                   67300 Schiltigheim, France
SIP Communicator
[email-masked]                     FAX:   +33.1.77.62.47.31
http://sip-communicator.org                    PHONE: +33.1.77.62.43.30

---

From: "Roni Even" <[email-masked]>
Date: Fri, 5 Jun 2009 12:25:14 +0300

Hi Emil,
Inline

-----Original Message-----
From: Emil Ivov [mailto:[email-masked]] On Behalf Of Emil Ivov
Sent: Friday, June 05, 2009 12:10 PM
To: Roni Even
Cc: [email-masked]
Subject: Re: [dispatch] Yet another problem to dispatch

Hey Roni,

RE: the information may not be relevant since it just describes the mix, the
user gets a mixed content and not the individual streams.

RE: I am just saying that in my view it is an interesting information which
is useless.

RE: again the stream is already mixed.

RE: and what does it help???

Cheers
Emil

---

From: Enrico Marocco <[email-masked]>
Date: Fri, 5 Jun 2009 11:30:51 +0200

Roni Even wrote:
> about the benefit of displaying sound level

There are basically three reasons, apparently we should have done a
better job at listing them in the draft:

1. It's cool, Skype does it. This may sound like a stupid reason, but
you'll see it's not the case if you try to convince people in a service
provider marketing dept. to plan an investment for a service that will
have less features and a worse UI than what users can get for free on
the Internet.

2. It is very useful in conferences with a fairly high number of
participants to debug sources of background noise, like people typing or
breathing loudly in the mic.

3. It is useful in multiparty conversations where participant know each
other only by name and cannot distinguish individual voices.

-- 
Ciao,
Enrico

---

From: Enrico Marocco <[email-masked]>
Date: Fri, 5 Jun 2009 11:43:48 +0200

> user gets a mixed content and not the individual streams.

Uh? The information will represent the sound level of each individual
participant. If the streams are normalized by the mixer, the sound level
values will be normalized as well: the mixer can do that easily and the
resulting information will be consistent with the mixed stream.

> RE: I am just saying that in my view it is an interesting information which

Useless because you don't think displaying the sound levels of the
voices you hear in a multiparty conference has any value for the user,
or useless because you think a mixer can't do that reliably?

> RE: again the stream is already mixed.

Probably you use mixers way better than mine, but in my conferences I
keep hearing people typing and breathing all the time ;-)

-- 
Ciao,
Enrico

---

From: "Roni Even" <[email-masked]>
Date: Fri, 5 Jun 2009 16:19:25 +0300

Enrico,

Skype also supports starting a multipoint audio conference from the UI. I
would consider this more practical than showing the sound level. XCON
defined this functionality in a standard draft enabling the creation of
audio and video conferences in an multivendor environment. Does your client
support this functionality. Do you see this as a mandatory requirement from
service providers.

My concern is that we develop solutions that are not deployed since they
have no high priority for developers and customers. Doing a "cool" solution
in a proprietary environment is simpler since you control the clients as
well as the servers and do not worry about interoperability with other
vendors. 
Roni Even

-----Original Message-----
From: Enrico Marocco [mailto:[email-masked]] 
Sent: Friday, June 05, 2009 12:31 PM
To: Roni Even
Cc: 'Emil Ivov'; [email-masked]
Subject: Re: [dispatch] Yet another problem to dispatch

-- 
Ciao,
Enrico

---

From: Emil Ivov <[email-masked]>
Date: Fri, 05 Jun 2009 15:39:38 +0200

Roni,

Roni Even wrote:
> support this functionality. 

I believe I already mentioned this in the beginning of this thread but
probably not clearly enough: this whole work has started because of the
ongoing effort in the SIP Communicator community to implement support
for conferences established and (in some cases) hosted by the SIP
Communicator client.

> Do you see this as a mandatory requirement from service providers.

I guess Enrico would better answer this one but I can certainly imagine
how this would become a trendy feature.

> well as the servers and do not worry about interoperability with other

Well you can relax then. This is of very high priority to us ( a non
proprietary open source client with a significant community ) and it is
certainly not a mere whim for a coolish featury. We are also very
concerned about interoperability which is why we have started this whole
discussion.

Emil

---

From: Enrico Marocco <[email-masked]>
Date: Fri, 5 Jun 2009 17:05:18 +0200

> support this functionality. Do you see this as a mandatory requirement from

Well, from my very limited perspective I actually see an increasing
interest in Internet conferencing -- the technology seems to be finally
usable if not mature, plus the shrinking of travel budgets pushes in
that direction -- and features available in free services/applications
are probably to be considered as a must-have for any commercial offering.

> well as the servers and do not worry about interoperability with other

The extension needed to carry such information is trivial and folks at
SIP Communicator are actually going to implement that. Unfortunately
such an extension does not exist yet and the IETF community can provide
guidance to define and document it, rather than push them toward a
proprietary solution.

-- 
Ciao,
Enrico

---

From: "Roni Even" <[email-masked]>
Date: Tue, 9 Jun 2009 00:06:10 +0300

Cullen,
In the PSTN integration I am aware of two methods of associating the PSTN
call with the web UI. The first is when the we UI is used to dial out to the
user using the PBX telephony API. The second is that when a user is defined
in the UI he gets a PIN number so when he dials in to the conference bridge
the user must enter this PIN number and using the telephony API it is
conveyed to the we UI.
There are other solutions
Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Cullen Jennings
Sent: Wednesday, May 27, 2009 8:15 PM
To: Emil Ivov
Cc: [email-masked]
Subject: Re: [dispatch] Yet another problem to dispatch

I find this "active speaker" problem very interesting. Often the  
device that wants to display the active speaker information is not the  
same as the device dealing with the RTP. For example, most the  
conference systems I am using a phone but there is a web interface  
that is dealing with active speaker information on my browser which is  
no on my phone. Be interesting to look at all the existing approaches  
to this.

Cullen <in my individual contributor role>

On May 21, 2009, at 4:45 PM, Emil Ivov wrote:

> Hey folks,
>
> We have been working on client side conference calls  for SIP
> Communicator lately and were thinking about implementing participant
> sound level indicators. The feature was inspired by Skype's Mac OS X  
> GUI
> which you can have a look at over here:
>
> http://sip-communicator.org/skypeconf/skypeconf.png
>
> Since we couldn't find an existing IETF mechanism for a mixer to
> dispatch information on sound level to conference participants, Enrico
> and I thought that this may be an issue that is worth addressing  
> through
> dispatch.
>
> After talking to some of you privately and discussing it among  
> ourselves
> we have decided to come up with a document presenting the issue in  
> some
> more detail. We are also including short descriptions of what we
> consider to be possible approaches for resolving it.
>
> http://www.ietf.org/internet-drafts/draft-ivov-dispatch-slic-ps-00.txt
>
> Comments are most welcome, as well as any show of interest or
> indications of what you think would be the right place(s) to work on  
> the
> subject.
>
> Thanks,
> Emil