Re: Conent length with MIME header

[Steve Hole <steve.hole@messagingdirect.com>]

On Tue, 07 Nov 2000 08:46:33 -0600 dick@8760.com wrote:

> <db>
> It's true that Content-Length is not a MIME standard header, but as was pointed
> out in the HTTP spec it is needed in some cases. With regard to items 2 and 3,
> the content-length header was included primarily for efficiency purposes.
> Consider the case where an intermediary server is processing a large file, it's
> much less work for a server to extract a precise number of octets from a payload
> as opposed to performing boundary string comparisons over the content in search
> of "the end". 

One of the things that was learned about MIME handling in mail was that 
content length numbers were likely to be erroneous.   MIME is a syntactic 
representation and the composition software is required to do 
encoding/packaging from the bottom up.   Many clients do streamed decoding
of content-transfer-encoding which renders any content-length information 
useless.   In fact, it is worse than useless, it is misleading.

> I believe ebXML will work without the content-length on header and
> payload body parts, but it will result in a less efficient (IMO) operation. This
> may not be a big deal for point-to-point cases, but a large intermediary, like
> Commerce One or Ariba, is more likely to experience the difference in overhead
> when they're repackaging.
> 
> I know memory is cheap and hopefully Moore's Law will hold for at least another
> 30 years so perhaps I shouldn't be concerned ( this comes from programming
> PDP-11's when one only had a 32k memory space and overlays were a fact of life).

Memory is definitely not cheap when dealing with high volume throughput.   
It is simply a matter of scale.   In a situation where you want to build a
high volume ebXML document generator (perhaps an ebXML invoice 
representation -- read e-bills), then it is definitely an issue.   By 
mandating use of Content-Length you force the implementation to consume 
memory for presumed efficiency on the recipient side.    For mail based 
applications, I can tell you that MIME processors will decode on the fly 
and the Content-Length is not useful.

> </db>
> 
> <db>
> If the consensus is to remove Content-Length from the "definitive" parts of the
> ebXML message envelope then we will have to specify details for its use under
> the HTTP section of TRANSPORT BINDINGS (which I believe is still TBD).

This is the right thing to do.
 
> The Content-Length gives more control to system administrators by allowing them
> to block large transfers BEFORE consuming system resources. 

In the http case yes.   In the mail case no.   No MTA would ever depend on
the Content-Length header -- mostly because it will do anything to avoid 
having to parse any relayed message.    MTA's will use SMTP SIZE extension
data to do this if possible from the SMTP envelope -- situation which is 
synonymous with the HTTP envelope content length information.

> It also provides
> efficiencies for intermediaries that deal with unpacking/repacking payloads in
> (hopefully) high volumes.

In fact, I think it very unlikely that it would because of 
content-transfer-encodings.    A MIME parser MUST parse on the MIME 
boundaries ... period.   You already pay the price, and in the presence of
content-transfer-encodings you are forced to determine the size of the 
payload anyway.

It is possible that you could repackage encoded content, but the vast bulk
of applications would want to deal with the raw source payload for 
regeneration of the MIME package.

Cheers.
---
Steve Hole
Chief Technical Officer
Messaging Direct
Mailto:Steve.Hole@MessagingDirect.com
Phone: 780-424-4922