Subject: Re: IPP> Re: MOD - Comment on RFC 2044: need to specify the ISO 10646 conformance level

From: [email-masked] (Ira Mcdonald x10962)
Date: Tue, 14 Oct 1997 12:20:38 PDT

Hello M. Yergeau,

The IPP working group has chosen to base the IPP project (partly)
on ISO 10175 - DPA (Document Printing Application).  In order for
"text" strings specified in IPP to be transferred to and from
strings in DPA implementations (there are a number), it is helpful
to observe the DPA limitation to ISO 10646 Level 2.  

Perhaps this was an unfortunate choice in the ISO DPA standard.
The rationale was in order to help support correct comparison
operations for string attributes.

Certainly, access to the full repertoire of ISO 10646 Level 3
is highly desirable (for the reasons you stated in your note,
below).

But if IPP/1.0 is to be rapidly adopted and deployed by the
printing industry vendors, it must avoid unreasonable implementation
costs for   embedded systems.

I recently read the MIME media-type registration for UTF-8 and
it seemed somewhat ambiguous about which level of ISO 10646
is mandatory.

The IPP working group has approximately three (3) weeks left
to finish their Model and Semantics spec and Protocol spec
and advance them to the IESG (per a recent note from Keith
Moore, one of our IETF Applications Area Directors), if the
IPP/1.0 specs are to be adopted within calendar 1997.  There
is a very real urgency about completing IPP/1.o specs soon.
Otherwise, it is probable that single-vendor or small
consortium implementations of something "similar" to IPP,
but NOT interoperable.  And the "window of opportunity"
to solve the problem of a robust replacement for RFC 1179
will have passed.

Regards,
- Ira McDonald (outside consultant at Xerox)
------------------------------------------------------
>From [email-masked] Tue Oct 14 14:18:05 1997
Date: Tue, 14 Oct 1997 06:51:59 PDT
To: Tom Hastings <[email-masked]>
From: Francois Yergeau <[email-masked]>
Subject: IPP> Re: MOD - Comment on RFC 2044: need to specify the ISO 10646 
  conformance level
Cc: [email-masked], Keith Moore <[email-masked]>,
        [email-masked]
Sender: [email-masked]
Status: R

=C0 21:52 08/10/97 PDT, Tom Hastings a =E9crit :
>For the purposed of Internet Protocols, we suspect that level 2 is
sufficient.
>But the UTF-8 definition in RFC 2144 is silent on this matter.  We suggest
>that
>a refision to RFC 2144 be issues that indicates that utf-8 means just
level 2.

A restriction to level 2, which excludes most combining characters, would
severely restrict the expressive power of ISO 10646, and in consequence the
ability of the protocols that use it (in the UTF-8 form) to represent the
textual content that they need to represent for truly world-wide operation.
 IMHO, such a restriction is too serious to be entertained solely on the
basis of "we suspect that level 2 is sufficient."  ISO has not found it
sufficient, it has level 3; Unicode has *only* level 3.  And I think the
purposes of Internet protocols (at least those that carry text) are the
same as the purposes underlying ISO 10646 and Unicode: to enable
communication in all the world's languages.

>Alternatively, register a new value, say, 'utf-8-level-2'.

There is actually a revision of RFC 2044 underway.  The latest draft
(ftp://ds.internic.net/internet-drafts/draft-yergeau-utf8-rev-01.txt) has a
discussion of version-specific labels, which you may find relevant to your
proposal of a level-specific label.  Please take a look, and come back with
your proposal -- preferably to the ietf-charset list, as suggested by
Harald -- if you still think it is appropriate.

Regards,

--=20
Fran=E7ois Yergeau <[email-masked]>
Alis Technologies inc., Montr=E9al
T=E9l : +1 (514) 747-2547
Fax : +1 (514) 747-2561

---

From: Francois Yergeau <[email-masked]>
Date: Tue, 14 Oct 1997 18:54:41 -0400

>to observe the DPA limitation to ISO 10646 Level 2.  

Makes sense.

>Perhaps this was an unfortunate choice in the ISO DPA standard.

Indeed!

>it seemed somewhat ambiguous about which level of ISO 10646

The charset label (not media-type) registration for UTF-8 does not mandate
transmission of one level or another.  The charset label is only an
announcement mechanism that tells a receiving application how to turn the
received bytes into characters.

As such, there is nothing to stop a given protocol, say IPP, from mandating
that all strings be encoded at level 2.  The "UTF-8" label is then
perfectly adequate -- the receiving end knows how to decode -- and the
(overstated, IMHO) burden of level 3 does not come to bear on that protocol.

The rationale for not having level-specific labels is that it does not seem
to help anybody.  In your case, if the transmitters are restricted to level
2 by the protocol definition, the receiving end gains nothing from a label
that says level 2 -- they already know that.  In general, a receiver that
is prepared to receive level 3 will gain nothing from a level 2 label.  In
the opposite case (level 2 receiver, level 3 label) the only gain is a
slightly earlier warning that the receiver may fail; even that gain may be
illusory, as a level 3 label does not guarantee that level 3 combinations
actually occur -- the level 2 receiver may well succeed if it tries.

There has been ample discussion of this (and the related versioning) issue
on various IETF, ISO and Unicode lists since early 1996, and the consensus
is that this meager gain in one case is to small to bother, especially when
faced with a potential proliferation of labels for all version-level
combinations, and the possibility that it leads to less interoperability in
some cases.

>Return-Path: <[email-masked]>
>Received: from zombi (zombi.eso.mc.xerox.com) by snorkel.eso.mc.xerox.com
(4.1/XeroxClient-1.1)
>	id AA18881; Tue, 14 Oct 97 14:18:04 EDT
>Received: from alpha.Xerox.COM by zombi (4.1/SMI-4.1)
>	id AB07428; Tue, 14 Oct 97 14:13:51 EDT
>Received: from lists.underscore.com ([199.125.85.31]) by alpha.xerox.com
with SMTP id <55990(1)>; Tue, 14 Oct 1997 11:13:13 PDT
>Received: from localhost (daemon@localhost) by lists.underscore.com
(8.7.5/8.7.3) with SMTP id OAA09275 for <[email-masked]>; Tue,
14 Oct 1997 14:09:44 -0400 (EDT)
>Received: by pwg.org (bulk_mailer v1.5); Tue, 14 Oct 1997 14:05:46 -0400
>Received: (from daemon@localhost) by lists.underscore.com (8.7.5/8.7.3) id
NAA08865 for ipp-outgoing; Tue, 14 Oct 1997 13:55:26 -0400 (EDT)
>Message-Id: <[email-masked]>
>X-Sender: [email-masked]
>X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
>In-Reply-To: <3.0.1.32.19971008215215.00f613b0@garfield>
>Mime-Version: 1.0
>Content-Type: text/plain; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
-- 
François Yergeau <[email-masked]>
Alis Technologies inc., Montréal
Tél : +1 (514) 747-2547
Fax : +1 (514) 747-2561

---

From: Francois Yergeau <[email-masked]>
Date: Tue, 14 Oct 1997 19:01:28 -0400

I'm sorry, I pressed the wrong button and sent an unfinished message.  I
wanted to add a little something:

>and advance them to the IESG (per a recent note from Keith

There is nothing to stop you from mandating level 2, but that would be a
mistakeIMHO, for the reasons I gave in my reply to Tom Hastings.

If you are sure that you *must* have that restriction early on, please
entertain the possibility of having 10646 level 3 as an option in IPP.  I
know nothing about IPP -- negotiation, etc. -- but at least such an option
would give implementers the possibility of advertising that their
implementation supports it, instead of restricting forever to level 2.

Regards,

---

From: [email-masked]
Date: Wed, 15 Oct 1997 09:41:31 +0200

Ira,
I think an appropriate solution for IPP is to put the following note into
the spec:

NOTE: For the case where IPP requests are gatewayed into DPA, the text
strings in the protocol must be limited to conform to ISO 10646 level 2.
This means that the following characters are dropped when copying strings
from an IPP PDU into a DPA PDU:

0483	COMBINING CYRILLIC TITLO
0484	COMBINING CYRILLIC PALATALIZATION
0485	COMBINING CYRILLIC DASIA PNEUMATA
0486	COMBINING CYRILLIC PSILI PNEUMATA
093C	DEVANAGARI SIGN NUKTA
0953	DEVANAGARI GRAVE ACCENT
0954	DEVANAGARI ACUTE ACCENT
09BC	BENGALI SIGN NUKTA
09D7	BENGALI AU LENGTH MARK
0A3C	GURMUKHI SIGN NUKTA
0A70	GURMUKHI TIPPI
0A71	GURMUKHI ADDAK
0ABC	GUJARTI SIGN NUKTA
0B3C	ORIYA SIGN NUKTA
0B56	ORIYA AI LENGTH MARK
0B57	ORIYA AU LENGTH MARK
0BD7	TAMIL AU LENGTH MARK
0C55	TELUGU LENGTH MARK
0C56	TELUGU AI LENGTH MARK
0CD5	KANNADA LENGTH MARK
0CD6	KANNADA AI LENGTH MARK
0D57	MALAYALAM AU LENGTH MARK
302A	IDEOGRAPHIC LEVEL TONE MARK
302B	IDEOGRAPHIC RISING TONE MARK
302C	IDEOGRAPHIC DEPARTING TONE MARK
302D	IDEOGRAPHIC ENTERING TONE MARK
302E	HANGUL SINGLE DOT TONE MARK
302F	HANGUL DOUBLE DOT TONE MARK
3099	COMBINING KATAKANA-HIRAGANA VOICED SOUND MARK
309A	COMBINING KATAKANA-HIRAGANA SEMI-VOICED SOUND MARK

It always helps to be explicit about those things....

                       Harald A

---

From: [email-masked]
Date: Wed, 15 Oct 1997 11:48:26 +0200

Oops - I missed the following text in the description of implementation
level 2:

The characters in the subset collections COMBINING DIACRITICAL MARKS
(0300 to 036F), COMBINING DIACRITICAL MARKS FOR SYMBOLS (20D0 to
20FF), HANGUL JAMO (1100 to 11FF) and COMBINING HALF MARKS (FE20 to
FE2F) are not allowed in implementation level 2.

So the difference between Level 1 and Level 2 is that in Level 2
you get the vowel and point marks required for various languages,
but still can't use the diacritical marks for Western languages.

My mistake....

                           Harald A