Subject: IPP> Re: Unsigned integer count for attribute name keywords and attribute values

From: [email-masked] (Robert Herriot)
Date: Tue, 20 May 1997 12:37:21 -0700

> From [email-masked] Tue May 20 11:48:48 1997
> 
> Paul,
> 
> Bob and I were talking about the two-octet binary integer count that
> we agreed to for IPP encoding.  The following details need to be 
> carefully specified in the IPP encoding document, in order to get proper
> interoperability:
> 
> 1. The maximum value for attribute name keywords is 255 octets, since
> attribute name keywords are US ASCII.  The minimum is 1.
> From the Model document, page 43.

Did we agree to one or two bytes for the length of attribute names?  I
thought two, just in case we ever use Unicode rather than UTF-8 to
encode attribute names.  We wouldn't want the attributes to then be
limited to 127 characters.

> 
> 2. The maximum value for attribute values is 4095 characters times the 
> worst case explosion factor for UTF-8, which is 3 or 4.  In any case,
> the high order sign bit shall be 0.

The maximum length of a 2 byte Unicode character is 3 bytes in UTF-8,
A 4 byte "Unicode" character can be as much as 6 bytes. There
are currently no such characters, as far as I know.  

I don't understand why we should make that statement that the high
order sign bit needs to be 0. I would expect that it is a 16 bit
unsigned integer. In fact, I would expect that all of the lengths
are unsigned integers, be they 2 bytes or 4 bytes, but we need to
make that clear.

> 
> 3. The two-octet integer is not padded, so RISC machines that require
> integers to be aligned on 2, 4, or 8 byte boundaries, must pick up the
> two-octets whereever they are.  We don't want some clients padding the
> data to meet their server's alignment requirements and then such servers 
> not being able to accept unaligned data from other clients.  We should
> probably outlaw the ASCII NUL (decimal 0) in attribute names and values
> as well, just to make sure.  The current Model document lists the 
> abstract characters that are allowed in keywords, but the encoding document
> could specify that ASCII NUL could be introduced.  I think we need to
> specifically outlaw ASCII NUL in the encoding document.

Padding is a host issue and not a protocol issue. Since our protocol is
based on precise locations of bytes in the stream, the protocol
definition implies that there can be no padding.  Because each name and
value is specified by a length, it shouldn't matter if an ASCII NUL is
present.  In fact, I would NOT want to have two rules for termination
of a name, such as length or NULL, whichever comes first. This gets us
back into the HTTP question of Content-length versus boundary-string
and which one wins. If a name or value contains a NUL, then it must be
part of the name or value.  This rule is especially important if we
later allow other encodings such as Unicode which has numerous NUL
bytes, e.g. in every ASCII character.

> 
> 4. The first of the two octets is the most significant, as in all 
> protocols.  So-called Big Endian.
> 
> Ok?
> 
> Thanks,
> Tom
> 
>

---

From: Tom Hastings <[email-masked]>
Date: Tue, 20 May 1997 22:30:59 PDT

>> From [email-masked] Tue May 20 11:48:48 1997
Snip...

>bytes, e.g. in every ASCII character.

I agree we don't want two rules and I wasn't suggesting such.  If NUL is
included in character coded data, it would contribute to the count like
any other character.  If NUL is included in character coded data, it would
not terminate the string.

In UTF-8, as opposed to Unicode (which is two-octets for every characters),
there are no all-zero octets, except for the Unicode NUL character.  All 
other characters have all non-zero octets.  That is one of the reasons
for UTF-8: to allow the C NUL terminated string to be processed by all
existing software.  But for our application we don't want NUL terminated
strings, since we have a count instead.

To be more clear, we should outlaw ASCII NUL character in ASCII coded strings 
and outlaw the UTF-8 NUL *character* in UTF-8 strings.  Otherwise, some client 
may be putting in such padding (within the count) in order to align the 
two-octet attribute name or attribute value that follow, so that the 
particular server that the client was built with could pick up the integers 
as (aligned) integers, because they were aligned in memory as the data was 
unmarshalled into memory, rather than picking up the two-octet integers as 
individual octets.

Another reason to outlaw these NUL characters, is to prevent servers from
having to filter them out before doing string compares.  If one client starts
to embed NULs and some servers filter them out, then we get interoperability
problems with the servers that don't.

In short, this is a small problem with a simple fix:  Just add a sentence
that forbids the counted ASCII and UTF-8 strings to contain the NUL character
when representing attribute names and attribute values in IPP.

Tom

---

From: [email-masked] (Robert Herriot)
Date: Wed, 21 May 1997 12:50:21 -0700

> From [email-masked] Tue May 20 22:35:24 1997
> 

I don't agree with most of your reasons for outlawing NUL. 

Padding is not an issue because we specify precisely the byte position
of each item in the protocol. Alignment is a notion that each platform
deals with as it parses the protocol; it is not a protocol notion.

Because the server has a length, it can easily compare the strings using a 
a length based compare (e.g. memcmp in Unix 95).  In addition, many 
servers will convert the UTF-8 to some other encoding.

The real question is what are allowable characters (not encodings) for
names and values.  We have talked about key-words being limited to
letters, digits, hyphen and underscore. That eliminates far more than
NUL.  For values, we have the same question except the possibilities
are more numerous.  We have said that values are text.  Do we want to
say that there are no exceptions. Because of the format of the
protocol, an attribute could be an octet string for certain well-known
attributes.  For attributes with text values, I think that it is
reasonable to eliminate certain control characters once the value is
decoded, but decoding is the key.  If we later allow Unicode encoded
values in addition to UTF-8, then some octets will be NULs, even if
we eliminate the Unicode character equivalent to NUL.

Bob Herriot

---

From: Tom Hastings <[email-masked]>
Date: Wed, 28 May 1997 12:20:13 PDT

Gee.  I thought that we agreed that attribute values were keywords, not
numbers. That is why there is a count for every attribute value.  Ok?

Sorry I didn't see this  mail message until now.

Tom

At 10:46 05/21/97 PDT, Paul Moore wrote:
>The length ot the attribute name is 2 bytes.
>
>The one that I think is much more interesting - which we did not drill
>into - is keyword values for attributes. I still have those as 2 byte
>enumerations. I think this is the right thing to do and Tom agreed at
>the meeting but there was not a lot of discusion.
>
>Eg Operation has two values 1 (which means Validate) and 2 (meaning
>printjob).
>DocumentFormat uses the rfc1759 values
>etc.

So these values should be key words, not enums, ok?

Tom

>
>> -----Original Message-----
>> From:	Scott Isaacson [SMTP:[email-masked]]
>> Sent:	Wednesday, May 21, 1997 9:32 AM
>> To:	[email-masked]; [email-masked]; Paul
>> Moore
>> Cc:	[email-masked]
>> Subject:	IPP> Re: Unsigned integer count for attribute name
>> keywordsandattribute values
>> 
>> 
>> 
>> >>> Robert Herriot <[email-masked]> 05/20 1:37 PM >>>
>> 
>> I thought all lengths would be 2 or 4 bytes, never just 1.  I thought
>> we
>> limited attribute names to be the same as keywords now:  US ASCII
>> characters
>> 
>> (a-zA-Z), digits (0-9), hyphen (-) and underscore (_).   We all know
>> the
>> consequences of not being able to internationalize these attribute
>> names,
>> but why are you suggesting maybe someday worrying about UTF-8 for
>> attribute
>> names?
>> 
>> Scott
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>
>