Subject: Re: PMP> Printer Interoperability issues

From: Lloyd Young <[email-masked]>
Date: 20 Dec 96 17:40:54 EST

For the time being I will follow Jeff's lead and send e-mail to both PWG
and PMP distribution lists.

Jeff,
The reason that the PWG meeting is right away the interoperability 
testing was not to encourage kibitizers. It was done because some 
number of people were going to have to attend both the interoperability 
testing and the PWG meetings and did not want to travel to two
different locations.

I am concerned if people think they can come to the interoperability
testing just to observe. That is not the intent of the interoperability
testing and will be counter productive to getting the testing completed.
There really are two questions. First, should kibitizer's be allowed to 
attend the testing? They will be put to work if they are allowed to attend.
Secondly, should they help share in the cost of testing?

Lloyd
--------------- Jeff's original note with some deletions to short size 
-----------------------------
To: Lloyd Young
cc: pmp%pwg.org @ interlock.lexmark.com @ SMTP, pwg%pwg.org @ 
interlock.lexmark.com @ SMTP 
From: jds%underscore.com @ interlock.lexmark.com (Jeff Schnitzer) @ SMTP
Date: 12/20/96 03:06:32 PM
Subject: Re: PMP> Printer Interoperability issues

Lloyd,

One reason we wanted to have the PWG meeting right after the
interoperability testing was to encourage kibitzers.  Requiring
a significant fee of people just observing is counter-productive.
We want to increase attendance at the testing so that people will
have first-hand knowledge of the resulting issues when they are
discussed later in the week.  

Offering a chance of hands-on testing in exchange for a share of
the cost sounds reasonable...  But people who are just there to 
observe should not be charged a fee, and if the test staff is to 
be volunteer then the volunteers should not be charged.

/Jeff

---

From: JK Martin <[email-masked]>
Date: Sat, 21 Dec 1996 03:34:14 -0500

Lloyd,

> testing and the PWG meetings and did not want to travel to two

I don't think anyone wishes to deny access to the testing effort
under any circumstances...right??  Or are we mistaken here?

Funny, but I was just hoping to have a repeat performance of what
took place in Portland, Oregon, last July.  As I recall, there were
no such statements made to the effect of excluding folks who just
wanted to watch.  And (as I recall, anyway), having kibitzers at
that event did not in any way impede the effort.

Am I mistaken in these recollections?

> Secondly, should they help share in the cost of testing?

This is all very strange.  What is wrong with having PWG participants
observe the action?

Please, please, PLEASE remember that, unlike Portland, this time we
KNOW IN ADVANCE that we have some issues to work out, particularly
those dealing with the critical Alert Table definitions.  The more
people who see the actual results (in person) should serve to reduce
the amount of time needed to relay the results to those interested
in helping formulate the solutions to the known problems.

	...jay

----------------------------------------------------------------------
--  JK Martin               |  Email:   [email-masked]          --
--  Underscore, Inc.        |  Voice:   (603) 889-7000              --
--  41C Sagamore Park Road  |  Fax:     (603) 889-2699              --
--  Hudson, NH 03015-4915   |  Web:     http://www.underscore.com   --
----------------------------------------------------------------------

---

From: Chris Wellens <[email-masked]>
Date: Mon, 23 Dec 1996 19:20:34 -0800 (PST)

Here is one long email intended to answer several questions
arising over the past two weeks on the mailing list.  Consider
this an ex cathedra communication.

The stated goal of the Printer MIB Working Group is to
advance from "Proposed Standard" to "Draft Standard".

In order to achieve this goal, we need the following: 

(1)  The revised draft must include a list of the 
substantive changes to RFC1759 and the justification for 
each change (the changes *must* be for clarification and 
correction, they *cannot* be new features).

(2)  A comprehensive report on the implementation 
experiences demonstrating/proving interoperability.

As soon as these two criteria are satisfied, the Chair
can make the recommendation to the Area Director and 
the IESG that the "Proposed Standard" be advanced to
"Draft Standard".

Our status:

In the case of (1) above, I sent out an email with the
subject line "Change Document" on Monday 12/9/96.  It
was distributed at the IETF meeting and feedback
requested.  So far, only Tom Hastings has pointed out
some missing items.  We need to get more feedback and
agreement that we have a correct document.

Without a correct document, the interoperability
testing cannot go forward, because we don't know what
we are testing; we actually have two specs, not one,
and lack a clear statement as to the differences.

The deadline to get this done is now 1/13/97.

In the case of (2) above, there are numerous ways of
getting a comprehensive report on the implementation
experiences:

	(2a)  all of the implementations could be
	sent to an independent lab that could run a
	set of tests and publish a report (UNH, LanQuest ...)

	(2b)  all of the implementors could get
	together, run a set of tests against their
	implementations and discuss how the spec
	should be changed as a result

	(2c) an independent consultant with expertise
	in SNMP MIBs could run a set of tests against
	each implementation, provide the results and
	make recommendations 

	(2d) various permutations and combinations of 
	the above

While the Portland Bake-Off seems to have been a
positive experience for many, it did not satisfy (2)
above.  If it *had* provided a comprehensive report on
implementation experiences, then RFC1759 would already
be advanced.  

The next effort has to be planned and designed to achieve (2)
above.  So far we have been converging on (2b), with some 
outstanding issues:

(3)  Non-Disclosure.  There seems to be a lot of confusion on
this.  The goal is to keep the specific test results of a given
company's product confidential, not to keep everything
confidential.  

We should be discovering problems from the tests results, such as "four
printers implemented prtFooBar this way, and two printers implemented
prtFooBar another way." Then we have some discussions, and from the
discussions we figure out how to clarify the information around
prtFooBar in the draft.  Obviously we do not keep all the results
confidential, since then we would not be able to have these
discussions.

I know some of you are concerned that your company will not sign
a non-disclosure agreement.  Some of you are unaware that your
company, has in fact, already signed such an agreement with
InterWorking Labs in another division for this same kind of
testing.  The non-disclosure agreement is at
www.iwl.com/Legal/tsla.html

(4)  Not enough activity on the mailing list.  I've noticed that
some implementation experience has been posted to the mailing
list with requests for feedback and comments.  None seems to be
forthcoming.  I do not know why this is the case.  We should be
trying to resolve these corrections and clarifications on the
email list in lieu of the face to face meetings.

(5)  Everyone who attends will be a test engineer for three days.
Remember that the guiding principle of the IETF is "rough consensus and
running code".  Talk is cheap; running code is not.  If you do not have
a printer with RFC1759 implemented, we will help you borrow one.  We
want the communication and discussion about what is broken and needs to
be fixed in RFC1759 to directly arise from the testing
experience.

Finally, I am sure that not everyone is going to like everything in 
this email.  Rather than nitpick on the style and the process of
what we have to do, I would ask that you nitpick on the
implementation experience of RFC1759.  Let's continue to find
and suggest fixes to the errors and ambiguities found therein,
and focus on what we can do to demonstrate/prove
interoperability.

Have a happy holiday!