Subject: [sidr] Interim Meeting (Apr 30, 2012) fallout/lessons/room-foo

From: Chris Morrow <[email-masked]>
Date: Thu, 03 May 2012 00:08:35 -0400

Howdy,
for the folks that attended in person, and remotely I think we (chairs)
would like to get some feedback on how the meeting was done. I think we
know of a few stumbling blocks:
  1) late start/technology fail with the webex (probably webex
      operations failures more than anything - my fault)

  2) audio issues (occasionally the webex audio would cut out)

  3) local dial-in for webex-call for non-US participants

  4) in-room audio failures with respect to typing noise

  5) lack of slideware (though really, most of the discussion was based
     on agenda notes, and slideware wasn't going to be particularly
     helpful, or so the thought went)

  6) microphone discipline for in-room vs external folks, often the
     in-room folk displayed (me too) an inability to wait their turn
    (jump into the conversation directly without queuing up, etc)

(there are likely others, I'd like to get those from the remote/local
folks as well)

We tried in this meeting to use:
  o webex for shared-notes/slides

  o webex 'voip' audio for those that could make that work

  o webex PSTN dialin (us-toll-free, us-toll only) for those which
    could not use webex 'voip'

  o webex PSTN for in-room audio/bridge-to-webex

  o jabber for in-meeting communication, disambiguation of speakers,
    questions and hand-raising/remote-questions.

I think the things that worked well were:
  o PSTN dial-in (especially the addition of local-to-callers numbers
    in non-US locations)

  o shared-application for note-taking

  o jabber for questions/hand-raising/etc

I think for this experience webex came out poorly (for me frustrating at
best). I'd like to see us either give the meetecho product a whirl
(which I think the secretariat supports for us) or go a bit more less
all encompassing:
  o etherpad/shared-doc/etc for notes
  o shared slides prior to the meeting (no slides, no slot. potentially)
  o jabber
  o pstn-bridge

and force some better mic/floor discipline.

I'd also (and sandy as well) would like some feedback on this message,
the meeting, and suggestions for what a direction forward might be.

(for sizing I believe we ended up with ~21 folks in-room and 'some
number' maybe 10? remote)

-Chris
co-chair-2-of-3

---

From: Randy Bush <[email-masked]>
Date: Wed, 02 May 2012 21:34:48 -1000

>       operations failures more than anything - my fault)

http://en.wikipedia.org/wiki/Sound_check

>      (jump into the conversation directly without queuing up, etc)

like it or not, when there are 20 people in the room, the need for queue
discipline is not obvious.  so it becomes more critical that remote folk
can break in equally.

i would like to hear from folk who participated remotely.  ruediger
seemed to be able to participate as if he was in the room.  which of the
19 semi-functional technologies was he using?  (and he is unlikely to
get/answer email)

in $dayjob, this kind of meeting has video of the participants, which,
imiho, aids equality.

randy

---

From: Christopher Morrow <[email-masked]>
Date: Thu, 3 May 2012 09:52:43 -0400

> in $dayjob, this kind of meeting has video of the participants, which,

agreed, videoconference works quite well for office things (sometimes
2-3 large rooms + 20 or so remote folks at desks)

anyways, yes more from remote people.

thanks!
-chris

---

From: Samuel Weiler <[email-masked]>
Date: Thu, 3 May 2012 15:43:35 -0400 (EDT)

On Thu, 3 May 2012, Chris Morrow wrote:

> I'd also (and sandy as well) would like some feedback on this 
> message, the meeting, and suggestions for what a direction forward 
> might be.

I will observe that my support for having interims was conditioned on 
improving the remote participation experience[1].  I think we did not 
hit the bar on Monday.  In the absence of a concrete plan to make 
further progress, I feel I must withdraw my support for having 
interims.

Knowing one voice in opposition may not change the consensus call for 
having interims, I'm still going to try to make some constructive 
suggestions.

>      operations failures more than anything - my fault)

I heard grumbling in the room in Reston re: having taken the trouble 
to travel there only to be faced with a seriously delayed WG meeting. 
There was talk of wandering off to another location to "get work 
done".  We may need to deeply consider the plan for what happens if we 
fail at making the tech work in the future.

>  2) audio issues (occasionally the webex audio would cut out)

I believe some people were using the WebEx "integrated" audio, and the 
source for that in Reston was not the speakerphone but was instead a 
single laptop's built-in microphone.  That audio was bad and, as you 
report, would frequently just stop entirely.  As best as I could tell, 
the integrated audio and PSTN call were not connected, and I'm not 
sure that anyone on the integrated channel could talk back to the 
room.

The PSTN audio outbound from Reston was decent, though not as good as 
I was hoping for.

>  3) local dial-in for webex-call for non-US participants

I don't recall seeing a toll-free number for the US.  Such would be 
helpful in the future.

I found Sandy's live notes to be very helpful.  They would have been 
better if I could have paged back through them on my own schedule and 
if they hadn't kept trying to take control of my screen.  Jabber 
worked fine.  For those in the room, I think having tables/desks would 
be more comfortable -- I don't like having my "laptop" on my lap for 
the duration of a normal WG meeting, and doing that for eight hours is 
too much.

As for future suggestions re: remote participation, I refer back to my 
note of 27 March: 
http://www.ietf.org/mail-archive/web/sidr/current/msg04288.html

I continue to believe that advance testing, delegation of the tech 
issues, and providing one microphone per person will help improve the 
experience.  There was a suggestion on Monday to use multiple jabber 
rooms: one with the (live) notes, and one for 
discussion/questions/etc.  Assuming we don't find a better shared 
note-taking tool, I'm fine with trying that.

Video seems like it might help with integrating the remote folks, 
though I have little experience doing video with large numbers of 
remote participants.

-- Sam

[1] http://www.ietf.org/mail-archive/web/sidr/current/msg04268.html

---

From: "Murphy, Sandra" <[email-masked]>
Date: Thu, 3 May 2012 22:06:03 +0000

I actually was pleased with the amount of useful participation we had from remote participants.  Of course, that does not mean that all participants were as pleased with their experience.

I'm glad that you found the note-taking useful.  I'm sorry that the typing of the note taking disturbed the audio.  At the time, the choice seemed to be (a) note taking noise and active telecon, (b) no note taking noise and no telecon.

Why?  The idea was that the remote participants would be participating via webex and so the webex should be used to broadcast the note taking to the remote participants.  To get the note-taking input into the webex broadcast meant that the note taking had to be on the webex host.  (See below for "in hindsight") Muting the host in webex wasn't possible because muting the host cut off the webex audio for the remote participants.

I am not at all sure why the typing noise was coming through.  There have been suggestions that webex finds a way around a platform's audio settings and can ignore the mute.  If true, then despite the fact that the host platform was configured to be mute, the webex was working against that.

In hindsight, note taking into etherpad on the web (as provided at recent ietf meetings), would have allowed the webex host platform to broadcast the notes without being the note taking platform.  etherpad -> web -> webex -> world.  Of course, the world can watch the etherpad directly.

(Powerpoint was chosen as the note taking tool on Monday with the thought that drawings done on the whiteboard could be rendered in powerpoint.  That was a futile thought (creating diagrams in ppt is just too slow) and meant the remote participants could not scroll back to see what had been typed before.)

The audio cutting out at various times was the result of various attempts to get the typing noise fixed.  Different people in the room had various suggestions, none of which worked.

The desktop sharing ceased a handful of times as webex reported problems.   I was told that each time sharing restarted, it took over people's screens.  From my experience in other webex sessions, that is a common feature of webex.  It may be a configurable option, who knows.

These interim sessions are intended to be intense discussions of thorny issues.  So we need:

real time audio in both directions with world wide access
real time view of presentations
real time note taking
real time chat
real time video?

One final note.  Whatever tools we use have to provide a way to archive the results.

Webex can provide all of those, but is proving to be a bit problematic, in some new way each time.

Webex has been used for our meetings before, but this time was the first time webex was being used to transmit the note taking and the first time integrated conference was used.

real time audio
    webex provides a teleconference number and an integrated conference audio (aka voip).  this was the first time 
    (I think) that the integrated conference audio was in use in a sidr meeting.  the webex telecon number (for ietf) 
    is US only, so we had some people connected by webex's integrated conference audio. On Monday, we eventually
    used an volunteered telecon number with local access worldwide and dialed that into the webex telecon.  If the
    integrated conference audio is not used, the meeting room needs a way to dial into the telecon (whichever).
real time view of presentations
    on Monday, we were using webex to broadcast the presentations
    we could also just upload the presentations to the ietf and be rigorous about letting remote people know what
    slide we are on
    (uploading slides means people can scroll back at will, and they need an app to view the slides)
real time note taking
    on Monday, this was done on the webex host platform.  see discussion about typing noise.
    we could also use jabber for note taking, as was done in San Diego.  but, as has been said, using one jabber room
    for both chatting and note taking is not ideal.  support for multiple jabber rooms in the ietf could be explored.
    we could also use etherpad for note taking.  etherpad also allows for remote participation.
    either jabber or etherpad allows for scrolling back.
    as a matter of fact, webex is generally limiting the participant to whatever is being seen on the shared space
real time chat
    jabber room support is provided and expected
    webex has a chat feature
real time video
    if it can be displayed on a desktop, it can be shared in webex
    various other apps can share video

Any combination of these are possible.  Do we want webex to do everything or do we want to use a collection of different tools?

--Sandy, speaking as wg co-chair

________________________________________
From: [email-masked] [[email-masked]] on behalf of Samuel Weiler [[email-masked]]
Sent: Thursday, May 03, 2012 3:43 PM
To: Chris Morrow
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [sidr] Interim Meeting (Apr 30, 2012) fallout/lessons/room-foo

-- Sam

_______________________________________________
sidr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/sidr

---

From: "John G. Scudder" <[email-masked]>
Date: Fri, 4 May 2012 15:42:37 -0400

A few things in addition to what others have said:

>  o shared slides prior to the meeting (no slides, no slot. potentially)

We covered a fair amount of ground at the interim without using slides at all. Based on that, I'm not sure where this (possible) requirement comes from? I, for one, am not at all keen to prepare slide decks just for the fun of it, and I looked at lack of slides at the interim as a plus, not a minus. Yes, if slides are used -- and I'm not against it when useful/desired -- they should be distributed. But I can't see making them required.

That brings me to one other point, the whiteboard. I think we should encourage use of a whiteboard (or easel, etc). But this implies a need to be able to share it with remote participants. We did the best we could last time, by snapping photos of the whiteboard and sending them out in real time, but there's gotta be a better way.

Ideally remote participants would be able to mark on the whiteboard too, but while the technology does exist, I don't see it really happening. At least, let's focus on getting the audio right first.

--John

---

From: Chris Morrow <[email-masked]>
Date: Fri, 04 May 2012 16:07:08 -0400

On 05/04/2012 03:42 PM, John G. Scudder wrote:
> We covered a fair amount of ground at the interim without using
> slides at all. Based on that, I'm not sure where this (possible)
> requirement comes from? I, for one, am not at all keen to prepare

There were several remote people who said: "Where are the slides, yo?"

I think I also translated this from 'regular meeting' in my head :( For
the interim I believe if there are slides to be presented we need those
posted to the SIDR wg page on tools.ietf before the meeting starts.

I don't think that for the interim slides (in this case) would have been
particularly helpful. In past meetings we've had success with slides
(Sharon's given a few presentations that were slide-heavy, as has Sriram).

> slide decks just for the fun of it, and I looked at lack of slides at
> the interim as a plus, not a minus. Yes, if slides are used -- and
> I'm not against it when useful/desired -- they should be distributed.
> But I can't see making them required.

yes, this was my thought process jump, oops!

> That brings me to one other point, the whiteboard. I think we should
> encourage use of a whiteboard (or easel, etc). But this implies a
> need to be able to share it with remote participants. We did the best
> we could last time, by snapping photos of the whiteboard and sending
> them out in real time, but there's gotta be a better way.

so far I've not seen a workable/useful shared whiteboard app :(
I'm also not super enamoured with using my personal flickr account :)
who knows what I may post there in the future.

> Ideally remote participants would be able to mark on the whiteboard
> too, but while the technology does exist, I don't see it really
> happening. At least, let's focus on getting the audio right first.

yes.

Thanks for your thoughts,
-chris

---

From: Randy Bush <[email-masked]>
Date: Fri, 04 May 2012 12:17:25 -1000

> We covered a fair amount of ground at the interim without using slides
> at all. Based on that, I'm not sure where this (possible) requirement
> comes from? I, for one, am not at all keen to prepare slide decks just
> for the fun of it, and I looked at lack of slides at the interim as a
> plus, not a minus. Yes, if slides are used -- and I'm not against it
> when useful/desired -- they should be distributed. But I can't see
> making them required.

<aol>