Subject: PMP> Interop Testing

From: "Harry Lewis <[email-masked]>" <[email-masked]>
Date: Tue, 24 Dec 96 08:32:29 MST

Without indenting and repeating the entire thread... I find the comments
about fee structure and attendance for the interop test very interesting. I
think interop testing is a necessary part of forging the standard. Even our
first, ad-hoc, test moved us closer to center. And I think our 2nd test needs
better planning, coordination and administration to yield more useful results.

But... what I find interesting about the current debate is that, for the first
time in the PWG, we are imposing fees and distinguishing between members. In
the past, we have always kept our costs to a minimum and shared them evenly.
For the upcoming test, the perception is that printer mfgrs will benefit
more than "kibitzers" (mostly software, NIC and related solutions providers).

With (at least) 6 printer vendor implementations and more than one software
solution already demonstrated, I would hope it's not the IETF who's mandating
a price tag for proof of interoperability in order to advance the Printer
MIB! To help settle the debates (and justify the cost) can we get a clearer
understanding of the benefits? For instance, here's what I hope to see
derived from the test:

 1. A controlled and thorough test of all printer MIB variables including
    dynamics associated with status, configuration, console and alerts.

 2. A test for SNMP compliance.

 3. An overview comparing each vendor's implementation.

 4. A report to each vendor who payed significantly greater fees
    (if this be the case) comparing their product in more detail
    with prioritized, recommendations.

 5. Confidentiality among PWG members *who attended* (including "kibitzers)
    "Pass/fail" specifics not discussed on reflector except in general terms.

 6. Press - not regarding details - but highlighting the fact that the
    PWG and/or Printer MIB has engaged in a "2nd stage" of interoperability
    testing... designed to make the Printer MIB a more useful and reliable
    standard.

I think 1,2 and 3 are what we all expect. However, without something
like 4, it's hard to justify some footing a larger bill than others. 5
is necessary to prevent participants from "shying away". Yet, an event
such as this really should not go unnoticed (6)!

Maybe I've missed reading the actual test plan. If it's posted, could I
please have a reminder of where? If it's still in development, I'd like
to hear other's comments about what we have in mind for the test.

---

From: Chris Wellens <[email-masked]>
Date: Sat, 28 Dec 1996 13:49:26 -0800 (PST)

Hi Harry,

I completely agree with you that we should endeavor to achieve
the goal of performing the interoperability testing at the
lowest possible cost.

No one is mandating that anyone pay anything.  In order to 
use a facility with an operational network, support staff, 
tables, chairs, phones, security, etc. usually a fee is 
charged.  If one of the vendors could host it at their 
facility or you know of a facility that would host it at 
no charge to the participants, that would be fantastic!

The prices that have been discussed so far only
cover Stardust's charges for facility use, and do not
cover any of the other items you listed.  

I will republish the first draft of the test plan, but
it will not be that useful, because we still don't
know what we are testing; we have two specs, not one,
and we lack a clear statement as to the differences.

I think the six items you listed that you would like
to see derived from the test event are fine.  Do you
want to create an RFP and solicit some bids for
fulfilling these requirements?  Or, ask members of the
Printer MIB Working Group if they would be willing to
contribute/donate some of the items?