Subject: IPPDEV> Re: magic number

From: [email-masked] (Robert Herriot)
Date: Thu, 6 Nov 1997 13:32:43 -0800

I am surprised by your first example.  When I snooped the network
traffic for the same URL, I got a 500 error, though it came from a
local proxy.  If you got a 200 by snooping, then that seems like an
error.  If such is really possible then this seems like yet another
problem with choosing to layer on top of HTTP.

But even if this response can really happen, doesn't the string
'Content-Type:  text/html' act as a magic number and indicate to the
client that there something unexpected because the expected
Content-Type is application/ipp?  The client, without and HTML
displayer cannot tell the user what the problem is, but it is certainly
an error.

Also, why were so many there so many '\r\n's in your second example?
You imply that this is a real example, but my reading of RFC2068 is
that the entity immediately follows the empty line, so the additional
'\r\n' are part of the application/ipp entity which is an error.  But I
see your point that if there are buggy clients and servers out there,
these extra characters would look like version 13.8 and might be valid 
eventually. So you may have a point about magic numbers.

Bob Herriot  
> From [email-masked] Thu Nov  6 08:15:56 1997
> 
> I still wish we had some kind of magic number or string to identify valid IPP
> messages with high confidence.  Talking IPP through off-the-shelf Web servers
> we sometimes see this kind of thing:
> 
> HTTP/1.1 200 Document follows. \r\n
> Server: XXXXXXXXXXXXX\r\n
> Date: Wed, 05 Nov 1997 16:30:37 GMT\r\n
> Accept-Ranges: bytes\r\n
> Content-Type: text/html\r\n
> Content-Length: 190\r\n
> Last-Modified: Wed, 05 Nov 1997 16:30:37 GMT\r\n
> \r\n
> <HTML>\n
> <HEAD>\n
> <TITLE>Error</TITLE>\n
> </HEAD>\n
> <BODY>\n
> <H1>Error 500</H1>\n
> \n
> Internal error\n
> \n
> <P><HR><ADDRESS><A HREF="/">XXXXXXXXXXX  Server X.X.X.X </A></ADDRESS>\n
> </BODY>\n
> </HTML>\n
> <HTML><HEAD><TITLE>Error</TITLE></HEAD><BODY><H1>Error 501</H1>The request is
> not valid or not recognized "INVALID-METHOD"<P><HR><ADDRESS><A HREF="
> http://karlk.penn.boulder.ibm.com:8080/">XXXXXXXXXXXXXXXX Server
> XXXXX</A></ADDRESS></BODY></HTML>
> 
> Even though there was an Error 500 in the server, the response has been wrapped
> in a document and sent with return code 200.  So the IPP client has apply
> various heuristics to see if it has got a good IPP response.
> 
> And even when there isn't an internal error in the server we often see this:
> 
> HTTP/1.1 200 OK\r\n
> Server: XXXXXXXXXXXXXXXXXXXX\r\n
> Date: Wed, 5 Nov 1997 16:40:10 GMT\r\n
> Content-Type: application/ipp\r\n
> Last-Modified: Wed, 5 Nov 1997 16:40:10 GMT\r\n
> Content-Length: 126\r\n
> \r\n
> \r\n
> \r\n
> \x01\x00\x00\x00\x02*\x00\bjob-name\x00\x04
> Job1*\x00\tjob-state\x00\x04Job1*\x00\x00\x00\x04Job2*\x00\x11job-state-reasons\x00\x04Job1*\x00\x11job-state-message\x00\x02YO\x05)\x00\x05sides\x00\x0bunsupported\x03
> 
> Notice all the CRLF pairs following the HTTP header.  So the client again is
> forced to use heuristics, and I hope we never get to version 13.10, because the
> start of the IPP message will look just like another CRLF!
> 
> Does anyone have a good algorithm worked out for determining the validity of an
> IPP response and finding the true start of the message?
> 
> Will this problem go away with mandatory SSL V3 "framing"?
> 
>   -Carl Kugler

---

From: [email-masked] (Robert Herriot)
Date: Fri, 7 Nov 1997 15:28:21 -0800

In your first example below, you justify its behavior by saying that it
provides some HTML text to describe the error, but the proxy I deal with 
generated similar HTML but it also returned a status of 500 which seems more
accurate than a 200 status that yours generates.  

With regard to your quote about extra CRLF before Request-Lines, that
quote applies to CRLFs preceding the "HTTP/1.1 200 OK CRLF" and not to
extra CRLFs before the message body.  All the language that I can find
in RFC 2068 says that the message-body must start immediately after an
empty line (i.e. one with just CRLF). So it would seem to me that this
server is nonconforming.

Bob Herriot

> From [email-masked] Fri Nov  7 09:48:50 1997
> 
> My first example results from a feature of our web server that allows
> administrators to customize the error messages the server returns to clients
> when an error condition is encountered.  Instead of returning a raw error 500
> to the client, the server generates an HTML document describing the error.  I
> agree that verifying Content-Type == application/ipp is a useful validity check
> in this case.
> 
> The second example,
> 
> HTTP/1.1 200 OK\r\n
> Server: XXXXXXXXX\r\n
> Date: Fri, 7 Nov 1997 17:27:48 GMT\r\n
> Last-Modified: Fri, 7 Nov 1997 17:27:48 GMT\r\n
> Content-Length: 126\r\n
> \r\n
> \r\n
> \r\n
> \x01\x00\x00\x00\x02*\x00\bjob-name\x00\x04Job1*\x00\t
> job-state\x00\x04Job1*\x00\x00\x00\x04Job2*\x00\x11job-state-reasons\x00\x04Job1*\x00\x11job-state-message\x00\x02YO\x05)\x00\x05sides\x00\x0bunsupported\x03
> 
> seems to be the kind of thing RFC2068 is talking about here:
> 
>   In the interest of robustness, servers SHOULD ignore any empty
>    line(s) received where a Request-Line is expected. In other words, if
>    the server is reading the protocol stream at the beginning of a
>    message and receives a CRLF first, it should ignore the CRLF.
> Fielding, et. al.           Standards Track                    [Page 30]*,
> RFC 2068                        HTTP/1.1                    January 1997
>      Note: certain buggy HTTP/1.0 client implementations generate an
>      extra CRLF's after a POST request. To restate what is explicitly
>      forbidden by the BNF, an HTTP/1.1 client must not preface or follow
>      a request with an extra CRLF.
> 
> except that in this case it's the server generating extra CRLF's.  I agree that
> it shouldn't, but this is an example of a real-world problem encountered with a
> commercial off-the-shelf web server.
> 
>   -Carl
> 
> 
> 
> 
> 
> [email-masked] on 11/06/97 02:34:51 PM
> Please respond to [email-masked] @ internet
> To: Carl Kugler/Boulder/IBM@ibmus, [email-masked] @ internet
> cc:
> Subject: Re: magic number
> 
> > Even though there was an Error 500 in the server, the response has been
> wrapped > in a document and sent with return code 200.  So the IPP client has
> apply > various heuristics to see if it has got a good IPP response. > > And
> even when there isn't an internal error in the server we often see this: > >
> HTTP/1.1 200 OK\r\n > Server: XXXXXXXXXXXXXXXXXXXX\r\n > Date: Wed, 5 Nov 1997
> 16:40:10 GMT\r\n > Content-Type: application/ipp\r\n > Last-Modified: Wed, 5 Nov
> 1997 16:40:10 GMT\r\n > Content-Length: 126\r\n > \r\n > \r\n > \r\n >
> \x01\x00\x00\x00\x02*\x00\bjob-name\x00\x04 >
> Job1*\x00\tjob-state\x00\x04Job1*\x00\x00\x00\x04Job2*\x00\x11job-state-reasons\
> x00\x04Job1*\x00\x11job-state-message\x00\x02YO\x05)\x00\x05sides\x00\x0bunsuppo
> rted\x03 > > Notice all the CRLF pairs following the HTTP header.  So the
> client again is > forced to use heuristics, and I hope we never get to version
> 13.10, because the > start of the IPP message will look just like another CRLF!
> >
>  > Does anyone have a good algorithm worked out for determining the validity of
> an > IPP response and finding the true start of the message? > > Will this
> problem go away with mandatory SSL V3 "framing"? > >   -Carl Kugler
> 
>

---

From: Carl Kugler <[email-masked]>
Date: Fri, 7 Nov 1997 18:51:51 -0500

Re:  error 500 wrapped in document with code 200 header:  I'm don't mean to
justify it -- I'm just reporting what I've observed.

Re, CRLFs:  agreed, it's non-conforming.  But in the interest of robustness, an
implementation might need to allow for it.

-Carl

Please respond to [email-masked] @ internet

To: Carl Kugler/Boulder/IBM@ibmus, [email-masked] @ internet
cc: [email-masked] @ internet
Subject: Re: IPPDEV> Re: magic number

Bob Herriot

> agree that verifying Content-Type == application/ipp is a useful validity
check > in this case. > > The second example, > > HTTP/1.1 200 OK\r\n >
Server: XXXXXXXXX\r\n > Date: Fri, 7 Nov 1997 17:27:48 GMT\r\n > Content-Type:
application/ipp\r\n > Last-Modified: Fri, 7 Nov 1997 17:27:48 GMT\r\n >
Content-Length: 126\r\n > \r\n > \r\n > \r\n >
\x01\x00\x00\x00\x02*\x00\bjob-name\x00\x04Job1*\x00\t >
job-state\x00\x04Job1*\x00\x00\x00\x04Job2*\x00\x11job-state-reasons\x00\x04Job1
*\x00\x11job-state-message\x00\x02YO\x05)\x00\x05sides\x00\x0bunsupported\x03 >
seems to be the kind of thing RFC2068 is talking about here: > >   In the
interest of robustness, servers SHOULD ignore any empty >    line(s) received
where a Request-Line is expected. In other words, if >    the server is reading
the protocol stream at the beginning of a >    message and receives a CRLF
first, it should ignore the CRLF. > Fielding, et. al.           Standards Track
[Page 30]*, > RFC 2068                        HTTP/1.1
January 1997 >      Note: certain buggy HTTP/1.0 client implementations generate
an >      extra CRLF's after a POST request. To restate what is explicitly >
forbidden by the BNF, an HTTP/1.1 client must not preface or follow >      a
request with an extra CRLF. > > except that in this case it's the server
generating extra CRLF's.  I agree that > it shouldn't, but this is an example of
a real-world problem encountered with a > commercial off-the-shelf web server. >
 >   -Carl > > > > > > [email-masked] on 11/06/97 02:34:51 PM >
Please respond to [email-masked] @ internet > To: Carl
Kugler/Boulder/IBM@ibmus, [email-masked] @ internet > cc: > Subject: Re: magic
number > > I am surprised by your first example.  When I snooped the network >
traffic for the same URL, I got a 500 error, though it came from a > local
proxy.  If you got a 200 by snooping, then that seems like an > error.  If such
is really possible then this seems like yet another > problem with choosing to
layer on top of HTTP. > > But even if this response can really happen, doesn't
the string > 'Content-Type:  text/html' act as a magic number and indicate to
the > client that there something unexpected because the expected > Content-Type
is application/ipp?  The client, without and HTML > displayer cannot tell the
user what the problem is, but it is certainly > an error. > > Also, why were so
many there so many '\r\n's in your second example? > You imply that this is a
real example, but my reading of RFC2068 is > that the entity immediately follows
the empty line, so the additional > '\r\n' are part of the application/ipp
entity which is an error.  But I > see your point that if there are buggy
clients and servers out there, > these extra characters would look like version
13.8 and might be valid > eventually. So you may have a point about magic
numbers. > > Bob Herriot > > From [email-masked] Thu Nov  6 08:15:56 1997 >
> > > I still wish we had some kind of magic number or string to identify valid
IPP > > messages with high confidence.  Talking IPP through off-the-shelf Web
servers > > we sometimes see this kind of thing: > > > > HTTP/1.1 200 Document
follows. \r\n > > Server: XXXXXXXXXXXXX\r\n > > Date: Wed, 05 Nov 1997 16:30:37
GMT\r\n > > Accept-Ranges: bytes\r\n > > Content-Type: text/html\r\n > >
Content-Length: 190\r\n > > Last-Modified: Wed, 05 Nov 1997 16:30:37 GMT\r\n > >
\r\n > > <HTML>\n > > <HEAD>\n > > <TITLE>Error</TITLE>\n > > </HEAD>\n > >
<BODY>\n > > <H1>Error 500</H1>\n > > \n > > Internal error\n > > \n > >
<P><HR><ADDRESS><A HREF="/">XXXXXXXXXXX  Server X.X.X.X </A></ADDRESS>\n > >
</BODY>\n > > </HTML>\n > >
<HTML><HEAD><TITLE>Error</TITLE></HEAD><BODY><H1>Error 501</H1>The request is >
> not valid or not recognized "INVALID-METHOD"<P><HR><ADDRESS><A HREF=" > >
http://karlk.penn.boulder.ibm.com:8080/">XXXXXXXXXXXXXXXX Server > >
XXXXX</A></ADDRESS></BODY></HTML> > > > > Even though there was an Error 500 in
the server, the response has been > wrapped > in a document and sent with return
code 200.  So the IPP client has > apply > various heuristics to see if it has
got a good IPP response. > > And > even when there isn't an internal error in
the server we often see this: > > > HTTP/1.1 200 OK\r\n > Server:
XXXXXXXXXXXXXXXXXXXX\r\n > Date: Wed, 5 Nov 1997 > 16:40:10 GMT\r\n >
Content-Type: application/ipp\r\n > Last-Modified: Wed, 5 Nov > 1997 16:40:10
GMT\r\n > Content-Length: 126\r\n > \r\n > \r\n > \r\n > >
\x01\x00\x00\x00\x02*\x00\bjob-name\x00\x04 > >
Job1*\x00\tjob-state\x00\x04Job1*\x00\x00\x00\x04Job2*\x00\x11job-state-reasons\
x00\x04Job1*\x00\x11job-state-message\x00\x02Y
O\x05)\x00\x05sides\x00\x0bunsuppo >
rted\x03 > > Notice all the CRLF pairs following the HTTP header.  So the >
client again is > forced to use heuristics, and I hope we never get to version >
13.10, because the > start of the IPP message will look just like another CRLF!
>
> >  > Does anyone have a good algorithm worked out for determining the validity
of > an > IPP response and finding the true start of the message? > > Will this
problem go away with mandatory SSL V3 "framing"? > >   -Carl Kugler > >