Subject: IPP> MOD - RESEND: Proposal for IPP to meet IESG Language and CharSet requirements

From: Tom Hastings <[email-masked]>
Date: Tue, 7 Oct 1997 18:04:00 PDT

We've corrected the proposal to map the IPP "content-charset" attribute
to the HTTP/1.1 'charset' parameter of the 'application/ipp' media type 
"Content-Type header.  I also added line numbers to the .pdf file (three
pages) 
and (red) revision marks show the minor changes.

The updated files are now in:

ftp://ftp.pwg.org/pub/pwg/ipp/new_MOD/
-rw-r--r--   1 pwg      pwg        31744 Oct  8 00:47 langchar.doc
-rw-r--r--   1 pwg      pwg        14874 Oct  8 00:47 langchar.pdf
-rw-r--r--   1 pwg      pwg         9439 Oct  8 00:47 langchar.txt

This mail note and these files contain a proposal that Ira and I took on 
as an action item at the 10/01/97 IPP
telecon to propose clarifications and additions to the 9/26/97 Model
document that are required in order to enter on the standard track with
respect to the Language and CharSet policies of the IETF that are currently
on standards track themselves.  See RFC 1766 and 2184 and
<draft-alvestrand-charset-policy-01.txt>.  RFC 2130 (IAB Character Set
Workshop, Feb 1997) provides background material.

We'd like to discuss this proposal at the telecon, this Wednesday 10/8.  
If there is agreement on the points, we'll draft the spec for the attributes.

The text herein is the specific points for the Internationalization
Considerations section of the IPP Model document.

Send comments before the telecon if possible.

Thanks,
Tom

Subj:  Proposal for IPP to meet IESG Language and CharSet
requirements
From: Tom Hastings and Ira McDonald
Date:  10/7/97
File:    langchar.doc

This document contains a proposal that Ira and I took on at
the 10/01/97 telecon to propose clarifications and additions
to the 9/26/97 Model document that are required in order to
enter on the standard track with respect to the Language and
CharSet policies of the IETF that are currently on standards
track themselves.  See RFC 1766 and 2184 and <draft-
alvestrand-charset-policy-01.txt>.  RFC 2130 (IAB Character
Set Workshop, Feb 1997) provides background material.

NOTE - For clarity of the Model and Semantics document and
the mapping in the protocol document, we have included as
operational attributes in the Model specification any
attribute that is mapped to HTTP/1.1 Protocol as headers.
Any such mapping to HTTP/1.1 is indicated as a parenthetical
remark in the Model specification.  We think that URIs
should also be brought back into the Model document as
operational attributes that are mapped to the appropriate
place in the HTTP/1.1 protocol as well.

1.   Summary of Changes

This proposal adds/clarifies attributes so that the client
supplies the language and charset of the request and the
Printer object responds in the requested language and
charset, if supported, or in the Printer's default language
and charset which can be queried.  This functionality is
provided by two Job Template attributes:  (1) "content-
natural-language" and (2) "content-charset" which are also
operational attributes in query operations.  As with any Job
Template attribute, there are also the corresponding Printer
"content-natural-language-default", "content-charset-
default", "content-natural-language-supported", and "content-
charset-supported" attributes.  There are also two "document-
natural-language" and "document-charset" operational
attributes for independently specifying the content of
documents for those media types that need such.  This
proposal is intended to replace existing natural language
and charset attributes in the Model and Semantics document.

2.   Specific Internationalization Provisions

The following provisions are listed for inclusion in the
Internationalization Considerations section of the IPP Model
specification.  They also serve as a detailed summary of the
capabilities.

The IPP Model and Semantics specification meets the IETF
Policy on Character Sets and Languages [IETF-Pol] by
providing the following internationalization capabilities:

1.   Clients SHALL supply the HTTP/1.1 Content-Type header
  with the value:

      application/ipp; charset=xxx

  where xxx specifies the charset used by the attributes with
  attribute syntax 'text' and 'name' in the IPP request body
  (independent of the document content) and SHALL be a charset
  registered with IANA [REG]
  
  The Printer object SHALL support charset=utf-8 [28].
  Support of other charsets is OPTIONAL, but all supported
  charsets SHALL be ones in which the code points from decimal
  20 to 127 are US-ASCII [US-ASCII].
  
  For conforming IPP Printer objects, the utf-8 charset SHALL
  be restricted to mean conformance level 2 of ISO 10646 [ISO-
  10646], so that accented letters SHALL NOT be represented
  with non-spacing accents.

2.   The term "natural language" is used to avoid confusion
  with "printer language", since "printer language" is the
  name given by IANA to the registration of print document
  formats as used in the Printer MIB, RFC 1759 [1].

3.   The client SHALL supply and the Printer object SHALL
  support the single-valued "content-natural-language" and
  "content-charset" Job Template attributes for correct
  interpretation of empty language and charset tags in IPP
  attributes with attribute syntax 'text' and 'name'.   (The
  protocol SHALL convey these attributes via the HTTP/1.1
  "Content-Language" and the 'charset' parameter of the
  'application/ipp' media type "Content-Type" headers (see 1.
  above) [23]).
  
  Each value of an attribute in a request or a response with
  attribute syntax 'text' or 'name' MAY be in any language
  and/or charset and SHALL be tagged using the syntax and
  mechanism in RFC 2184 [RFC-2184], to indicate the language
  and charset of the value.
  
  If the natural language or charset is the same as that
  supplied by the client in the request, then the empty tag
  may be used consisting of only a single ' character for the
  language or charset.
  
  Examples:  If the client supplies "content-natural-language"
  as 'en' and the "content-charset" as 'us-ascii', the
  following are valid representations of the "job-name"
  attribute in the request and in the response:
  
  us-ascii'en-us'Monthly Report
  us-ascii''Monthly Report
  'en-us'Monthly Report
  ''Monthly Report                   -- the two ' characters SHALL be present
  iso-8859-1'fr'Rapport Mensuel

4.   As with any supported Job Template attribute, the
  Printer object SHALL support the corresponding "content-
  natural-language-default", "content-natural-language-
  supported", "content-charset-default", and "content-charset-
  supported" Printer attributes.  However, these Job Template
  attributes are MANDATORY for the Printer to support.

5.   IPP Printer objects NEED NOT support HTTP/1.1 Accept
  headers and IPP/1.0 does not address the processing
  semantics for HTTP/1.1 Accept headers.

6.   For the create operations the Printer object shall
  store in the Job Object any 'text' and 'name' Job Template
  attributes in the language and charset as supplied by the
  client, along with the values of the "content-natural-
  language" and "content-charset" attributes.  In a subsequent
  query request, the Printer object NEED not convert any
  'text' or 'name' attributes that had been supplied in the
  create request to the natural language or charset of the
  requester.  However, any 'text' or 'name' attributes
  generated by the Printer object SHALL be returned in the
  natural language and charset specified by the requester in
  the "content-natural-language" and "content-charset"
  operational attributes of the query.  However, if the
  specified natural language or charset is not supported by
  the Printer, the Printer SHALL respond in its default
  natural language or charset, rather than returning an error.

7.   Notifications SHALL always include non-empty language
  and charset tags according to RFC 2184, since the recipient
  might not know what empty tags mean.

8.   Type 4 keywords are often names defined by a system
  administrator and so may be in any language or charset.
  Therefore, attributes with attribute syntax 'type4 keyword'
  are allowed to have a mixture of 'keyword' tokens defined by
  the standard and/or registered as keywords (in U.S. English
  and US-ASCII) and 'name' tokens defined by the system
  administrator.  Such names SHALL follow the syntax for names
  and SHALL be tagged using the syntax and mechanism of RFC
  2184, while the keywords SHALL not be tagged (since keywords
  SHALL always be in US-English and US-ASCII).

9.   No IPP Printer implementation NEED perform actual
  translation of natural language text and name values.  A
  Printer object that supports multiple languages, often has
  separate catalogs of messages, one for each natural
  language.

10.  The natural language and charset used in certain
  document formats may be specified separately and
  independently from that used by attributes that are part of
  the IPP protocol.  Therefore, in order to indicate the
  natural language and charset of these document-formats, the
  "document-natural-language" and "document-charset" operation
  attributes are defined for use in the create, Send-Document,
  and Send-URI operations.  For those media types that are not
  defined to include a charset parameter, such as
  application/octet-stream, the "document-charset" attribute
  MAY be used to convey the charset.  For those media types
  that allow the charset to be specified as a parameter, such
  as 'text/plain', the client SHALL also supply the "document-
  charset" attribute with the same value.  Then the Printer
  object NEED not parse the "document-format" media type
  attribute value.
  
  The Printer object SHALL support the "document-natural-
  language" and "document-charset" operational attributes, if
  the Printer object supports document formats which require
  them in order to be unambiguous.

3.   Additions to the References section

[ISO-10646] ISO/IEC 10646-1:1993, "Information technology --
Universal Multiple-Octet Coded Character Set (UCS) - Part 1:
Architecture and Basic Multilingual Plane, JTC1/SC2."

[REG] N. Freed, J. Postel:  IANA CharSet Registration
Procedures, Work in Progress (draft-freed-charset-reg-
02.txt).

[US-ASCII] Coded Character Set - 7-bit American Standard
Code for Information Interchange, ANSI X3.4-1986.

[IETF-Pol] H. Alvestrand, "IETF Policy on Character Sets and
Languages, work in progress <draft-alvestrand-charset-
pollicy-01.txt>, August 29, 1997.

[RFC-2184] N. Fried, K. Moore, MIME Parameter Value and
Encoded Word Extensions: Character Sets, Languages, and
Continuations, RFC 2184, August 1997.

---

From: Tom Hastings <[email-masked]>
Date: Wed, 8 Oct 1997 16:09:20 PDT

At 02:25 PM 10/8/97 PDT, Larry Masinter wrote:
>Tom, I didn't understand the rationale for having a 'document-charset'
>rather than using the 'charset=..' parameter of the document format.

We would have preferred to use the 'charset=..' parameter of the
document format, but there are a number of media types that are
registered that do NOT specify that the 'charset=..' paramater may
be used with them.  For example, 'application/octet-stream'
and 'application/vnd.hp-PCL'.  While PCL has an optional instruction
inside the document data that specifies the codeset, it is optional,
even when the codeset is different from US-ASCII (or HP Roman-8).

Therefore, we had to introduce the IPP "document-charset" (operational)
attribute.  Rather than making it optional, we decided it would simplify
the recipient, if it was required for the sender to always included
and had to agree with the media type charset parameter, if present.

>You said something like:
>> Then the Printer
>(which doesn't seem to be a good use of "NEED", did you mean
>just "MAY" instead of "NEED NOT"?), but I don't get it. The
>Printer (object?) has to look at the document-format if it
>needs to know the document format, doesn't it?

Yes, the Printer does have to look at the value to make sure that it is
supported, but it doesn't have to parse any further once it reaches the
';' character.

We'll just delete the sentence.

>HTTP/1.1 support for ACCEPT headers is optional, but 
>Accept-language and Accept-charset is useful, e.g., when
>determining the language & charsets useful for HTTP
>warnings & error messages. I don't know if "does not
>address the processing semantics for HTTP/1.1 Accept headers"
>is useful advice. Is there an ambiguity?

We thought it was simpler to avoid the complexity of accept headers.
What we have is complicated enough.

Also in IPP, the client can query the server to see what languages
and char sets the server supports before creating a job or making a query.

Finally, IPP, unlike HTTP, the client is supplying language and charset
attributes as input and these attributes are stored on the job object.
Later queries of the job object and notifications use the value stored
with the job (or the Printer default for messages generated by the
Printer, if the stored language or charset is not supported).

Sound ok not to specify the semantics of accept headers?
Should we say that IPP implementations SHALL ignore the accept headers,
if present?

>
>Larry
>-- 
>http://www.parc.xerox.com/masinter
>
>

---

From: Larry Masinter <[email-masked]>
Date: Wed, 8 Oct 1997 16:33:42 PDT

> For example, 'application/octet-stream'

For 'application/vnd.hp-PCL': if it is necessary to know the codeset
in order to properly interpret application/vnd.hp-pcl, then the
definition of application/vnd.hp-pcl is deficient. If you can't
convince HP to fix their registration, you could register an
alternative value. It may well be that the range of available
'codeset' for some printer types is different than the range
of registered Internet 'charset' values; after all, 'charset'
combines both the coded character set and the encoding scheme,
while some printer languages may employ a different mechanism
for mapping between embedded integers and visual representations.

For 'application/octet-stream': either this is for auto-detection
or it is not. If the sender knows enough about the sender's environment
to determine what 'charset' is used, shouldn't the sender also
know what document format is actually used? Under what circumstances
would one be known and not the other?

As for the discussion on Accept headers: it seems riskier to
layer on top of HTTP but then to declare that you're ignoring
some of the features. Presumably you're going to go through
proxy gateways and other HTTP-munging intermediaries, and if
so, it's very risky to claim that you're changing the semantics.
I'm not sure why you're focusing on the 'accept headers' here,
but not also supplying advice for any other HTTP header. Do you
support etags? COntent-ID? etc. etc. Accept headers are merely
advisory as it stands, and you're not adding any additional
requirements nor are you removing them. So why say anything
at all?

> What we have is complicated enough.

People are hoping to avoid complexity by reusing existing
HTTP implementations, and so any *change* from HTTP adds
complexity.

If you want to point out that Accept headers have no special
significance in IPP, and will likely be ignored by IPP printers,
well, that's fine. Accept headers are ONLY for client to
server, and the content negotiation you need for IPP is
for server to client, so they don't apply, anyway.

It would be great to extend HTTP to allow for reverse content
negotiation (e.g., for POST or PUT) but that's some other
work that will likely be ongoing.

We're trying to put together a separate working group on
'content negotiation' more generally; perhaps there will
be something done in time for IPP version 2, future versions
of HTTP, fax, etc.

Larry

---

From: Ned Freed <[email-masked]>
Date: Thu, 09 Oct 1997 00:05:30 -0700 (PDT)

> definition of application/vnd.hp-pcl is deficient.

I agree.

> If you can't

A codeset parameter would also be a possibility.

> would one be known and not the other?

I agree with the logic here as well. But as far as application/octet-stream
goes, I suggest that before any attempt is made to standardize the processing
of this type that some consideration be given to IETF interoperability rules
for standards-track protocols. The IETF doesn't let standards advance with
demonstrable interoperability problems or without multiple interoperable
implementations. And any idea that the addition of a charset parameter is
sufficient to make complete arbitrary binary data renderable, even with the
best auto-detection logic in the world in place, just isn't going to fly.

As such, I strongly recommend that any attempt to standardize handling of
application/octet-stream, either with supplemental information or without, and
either with auto-detection or without, be abandoned. This would be OK as
an informational document or as an experimental one, but not in a standard.

				Ned

---

From: Randy Turner <[email-masked]>
Date: Thu, 09 Oct 1997 07:58:44 -0700

> > would one be known and not the other?

I agree that in the "application/octet-stream" case, 99.9% of the clients willnot know the charset, or anything else about the
document(s) to be printed.

> an informational document or as an experimental one, but not in a standard.

While I understand the spirit of the comments related to standardization of
"application/octet-stream", it is not our responsibiltiy to enforce absolute
interoperability onto an existing printing environment that inherently has an
interoperability problem. There are a million non-pathological scenarios
wherein a user has an existing file that is to be printed and may or may not
know the format of the file. In my experience with MIME, this has been
how mail clients tag attachments when they're not sure the format of a
particular attachment.  This is exactly how we would use
the "application/octet-stream" in the IPP case.

In order to make our use of "application/octet-stream" more palatable to
the IESG, we could use proper wordsmithing to come up with something
I think would work. IPP could not dictate what "application/octet" stream
means to a server that receives such a document, but only say that it is
implementaiton dependent what happens when a server receives such a
document, and maybe provide examples (such as auto-sense). In this
case we're not officially standardizing how "application/octet-stream"
is to be handled by a server, only suggesting. And while this may indeed
present interoperability problems, I think it is worth keeping the use of
"application/octet-stream" at least mentioned in the documents. With
proper wordsmithing, I don't think IPP implementations will use
"application/octet-stream" any different than it is being used today in
the SMTP/MIME world.

Randy

---

From: Jay Martin <[email-masked]>
Date: Thu, 09 Oct 1997 11:28:23 -0400

Randy's comments are right on the mark, IMHO.

The printer industry has been successfully dealing with auto-sensed
data for quite sometime now.  It should make no difference what the
official MIME type is, whether it be "application/octet-string" or
"something/go-fish".  A rose by any other color, etc...

Perhaps I'm missing something here, but it seems painfully obvious
to me that we have put a big stake in the ground to support legacy
printing systems as part of the philosophy/strategy of wide-scale
IPP deployment.  As such, auto-sensing is absolutely mandatory to
for the interoperable implementations to succeed.

The less baggage we attach to the auto-sense situation, the better.
It just seems that the rest of the world has long embraced the MIME
type "application/octet-stream" to be "unknown type, go fish", and
I don't understand why we can't follow that lead.  (Sorry if I'm
not up on the latest, greatest IETF trends in this area.)

	...jay

----------------------------------------------------------------------
--  JK Martin               |  Email:   [email-masked]          --
--  Underscore, Inc.        |  Voice:   (603) 889-7000              --
--  41C Sagamore Park Road  |  Fax:     (603) 889-2699              --
--  Hudson, NH 03051-4915   |  Web:     http://www.underscore.com   --
----------------------------------------------------------------------

Subject: Re: IPP> Proposal for IPP to meet IESG Language/CharSet requirements
From: Randy Turner <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: Larry Masinter <[email-masked]>, [email-masked]
Date: Thu, 09 Oct 1997 07:58:44 -0700

Randy

---

From: Carl-Uno Manros <[email-masked]>
Date: Thu, 9 Oct 1997 09:58:14 PDT

>an informational document or as an experimental one, but not in a standard.

Ned,

FYI, many existing proprietary products successfully supports the
auto-detection feature. It may obviously not always work, but works most of
the time. A scenario where this is very helpful, is if the user gets a file
of some unknown file type e.g. as attachment in a message or found on a web
page and wants to attempt printing it.  In another scenario, a user might
know that a file is Postscript, but not whether it is Postscript 1, 2 or 3.
The auto-sensing can often help sort that out once the file arrives at the
printer.

We obviously do not want to make IPP less powerful than existing products
in this respect. It seems to me though, that we should define our own MIME
type for a document file that we want to have auto-sensed.

Carl-Uno

Carl-Uno Manros
Principal Engineer - Advanced Printing Standards - Xerox Corporation
701 S. Aviation Blvd., El Segundo, CA, M/S: ESAE-231
Phone +1-310-333 8273, Fax +1-310-333 5514
Email: [email-masked]

---

From: Ned Freed <[email-masked]>
Date: Thu, 09 Oct 1997 22:53:33 -0700 (PDT)

> the time.

FYI, several existing proprietary mail systems use private labelling
methodologies in RFC822 messages rather than MIME. The nature of these systems
is such that they obviously don't always work, but they do work much of the
time. (GTW, even when they don't work, the result is fairly harmless. The same
thing cannot be said of auto-detect failures on printers...)

And guess what? Some of these were part of the original MIME specification.
Heck, at least one of them was presented as an alterative of MIME. 

But there isn't a trace of any of them in any standards-track IETF document
right now. There are some informational and experimental RFCs but that is all.
Why is this the case? Because had they been included the specification that
included them would have never made it to draft standard.

> A scenario where this is very helpful, is if the user gets a file

I'm well aware that auto-detect mechanisms are widely used today -- in fact
I've developed print spooling software myself that does this. And I'm well
aware that that this can work -- most of the time, that is. But unless I'm
mistakem, this WG is trying to develop a standard. And when you build a
standard, most of the time just isn't good enough.

I will also reiterate that I see no problem with having an informational
supplement that describes this approach.

> type for a document file that we want to have auto-sensed.

On the contrary, I think you absolutely do want to, at least insofar as
standards track documents are concerned. If you want to have an informational
document describing this, that's fine. But 

In any case, I've said my piece. If this WG wants to include this sort  of
thing in their documents, that's fine. But don't be surprised when objections
are raised that cause the documents to get tossed back at you for revision
later on.

				Ned