Online Balloting System TESTing

["Wijnen, Bert (Bert)" <bwijnen@lucent.com>]

IESG colleagues,

It seems that the online-balloting system is (very close to) good 
enuf to start using it. As one may imagine, automating and/or
protecting against all corner cases of the balloting process is a
nightmare. The test team has done a lot of testing, and we think
we're getting ready for deployment for the Sept 18 telechat.
We intend to decide Tuesday or Wednesday next week.

All IESG members get a chance to also play with the TEST system
if they want, so that they can check for them selves if it
is acceptable. You can start doing so at any time by going to
   https://datatracker.ietf.org/cgi-bin/devel/idtracker.cgi
The test system always has "Testing Mode" in the top left corner.
It is a complete duplicate of our ID-tracker db, so you can 
play with it at your hearts content (add docs, send last call,
send ballot etc). The test system sends all emails that it would 
normally (when live) send out (to doc authors, to ietf announce list,
to ADs, to IESG etc) to a special list of people (testers). If you tell 
Michael that you want to be added, then you will get these messages too.
This is so that you can verify that the email generated by the 
system is correct (content and distribution). You can also test 
without being added to this list, but then you cannot see the actual
emails that the live system would send out as a result of your actions.

Before you start, pls read the following. As stated above, you can
easily screw up the system if you really try. But if we all play
by the (reasonable) rules/procedures, then the system works fine.
The idea is also that we should not imemdiately complain about
NITS, but rather check if we see any SHOWSTOPPERS or MAJOR issues.

So here we go.

For now, we should assume the following steps/states are the 
normal sequence for any ID to be balloted:
 
- Publication Requested (normally by secretariat, AD can do it too)
- AD Review (by AD)
  with some back and forth between AD and WG
- Prepare for Last Call (by AD)
  AD makes sure Last Call text is OK.
  - you can add/change Last Call text
  - you can save it.
  - you can issue Last Call (but it means Request Last Call, which
    mean an email to secretariat to actually issue it)
  - you can prepare Ballot text
  - you can issue Ballot, probably better to wait till Last Call ends.
  - you can edit/change Draft Announcement text
- Last Call Requested (by AD, this will trigger a notification to
  secretariat to take action.)
- In Last Call (by secretariat. An AD can do it, but that does not
  cause anything to happen. The secretariat is supposed to send out
  the message based on the AD changing the state to Last Call Requested,
  and the secretariat then sets the state to In Last Call after the
  IETF Last Call has indeed be issued.)
- Ballot write up (by AD)
- Issue Ballot (by AD)
- Put on Agenda (by AD)
- IESG Evaluation (by AD)
- Votes recorded online (by all ADs or by secretariat when mailed in)
  ADs can record/update comments (at any time)
  ADs can record/update DISCUSS text (when voting DISCUSS)
  - Make sure your vote is marked DISCUSS, otherwise your discuss
    comments does not show up on the ballot page. So if you change
    your DISCUSS to some other vote, then your discuss-text is
    not displayed on the "web ballot" or "IESG Discussion" page.
    It will have been recorded in the log of the ID-tracker, where
    you can always find it back.
  - Note that there is only one DISCUSS and one COMMENT entry per
    AD, so whatever you want to add/modify needs to be an update
    to the current text (if any)
- other states as normal after an IESG telechat
  
Things to be aware of:

- ADs can change/issue ballots (possibly last calls etc) for
  documents that are being shepherded by others. So watch out.
  I tested the changing of a ballot write-up for a doc shepherded
  by Thomas and to (re-)issue the ballot. It worked. So we should
  all be carefull to not step on each others toes.
  Good thing is that the issuing of ballot gets logged.
- You can bypass the normal steps (as is sometimes required).
  It is not clear at the moment what exactly happens in all cases.
  So pls be patient and check frequently. Let us assemble a list
  of "abnormal" things we do, what the effects/results are, and
  what (if anything) needs to be changed for that.
- The DEFER handling is a bit different than we may have been
  used to and to what I described. In order to not complicate
  the DEFER processing, the process is that:

  - One AD can press the DEFER Button. That means we get a DEFER
    till next telechat. It will not be cleared earlier. So if
    anyone else also wants a DEFER, they have it implicit.
  - The process will not CLEAR a defer till the next meeting,
    so even if the AD who took the DEFER changes early and casts
    a noObj or a Yes, even then the doc will stay in DEFER till
    next meeting.
  - If a quick clearing of DEFER is needed, we must handle it on
    the IESG mailing list.
  - I would recommend that if other ADs want a "DEFER me too"
    that they record it in a comments block, so that we at least
    can see on the ballot/comment writeups as to who did really
    want a DEFER as well.
    
Happy testing,
Bert (for the balllot-testing team)