RE: Applying transformations to Payloads (compression)

["Moberg, Dale" <Dale_Moberg@stercomm.com>]

Some additional points relevant to the compression issue:

Some transports already have options for compressing data.
  Example: HTTP Content-coding has IANA values "gzip" and "compress"
Some security packaging schemes have or are defining compression options:
  Example: PGP already has compression and S/MIME IETF has a compression
content
  being defined within the IETF SMIME WG.
For file transport approaches, there are many ways of compressing and
    then sending a compressed file. With some forms of transport,
    the binary form of compressed data may trigger a partial expansion
    because of the need for content-transfer-encodings (email related
    and possibly elsewhere (SOAP?)).

There are a lot of interdependencies here to track! CPP and CPA 
might be able to specify some of these compression options.
I think the issue raised by David Welsh is whether a specific
profile for packaging involving compression needs to be 
created. If so, is it part of the security packaging, part of
the transport binding information, or what?

In view of the interactions of compression with security, the
security group should also be involved reviewing proposals;
security people like to compress before encrypting and for
signatures there can be interoperability issues pertaining
to use of different conventions for calculating the hash
(was the hash over the uncompressed or the compressed data...).

If there is a FTP binding for ebXML, then a file level compression
solution might be mentioned in discussion of the FTP binding.
Possibly this option (if you want compression, arrange
to use FTP as a transport and apply compression to the file
transferred) would meet the requirement, if there is a
compression requirement for TRP or TP.

I doubt very much that a "transport and packaging/security independent" 
approach to compression will be found because the various 
security standards and IETF transport standards
have already handled compression in their own ways.

I agree with Marty that the best that can be done is to point to a CPA
that governs which compression options, at which level, are being 
employed.

Dale Moberg

> -----Original Message-----
> From: john_ibbotson@uk.ibm.com [mailto:john_ibbotson@uk.ibm.com]
> Sent: Wednesday, January 03, 2001 9:53 AM
> To: Martin W Sachs
> Cc: ebxml-tp@lists.ebxml.org; ebxml-transport@lists.ebxml.org
> Subject: RE: Applying transformations to Payloads (compression)
> 
> 
> 
> 
> 
> I don't believe it is the job of the MSH to compress the 
> payload. This only
> adds to the complexity of the MSH. Certainly, the CPP can include
> information on compression schemes it supports but compression of the
> payload should be the job of the application which then passes the
> compressed stream to the MSH for enveloping. Further 
> compression may occur
> at the transport layer but again, that isn't the job of the MSH.
> John
> 
> XML Technology and Messaging,
> IBM UK Ltd, Hursley Park,
> Winchester, SO21 2JN
> 
> Tel: (work) +44 (0)1962 815188        (home) +44 (0)1722 781271
> Fax: +44 (0)1962 816898
> Notes Id: John Ibbotson/UK/IBM
> email: john_ibbotson@uk.ibm.com
> 
> 
> Martin W Sachs <mwsachs@us.ibm.com> on 01/03/2001 02:42:29 PM
> 
> Please respond to Martin W Sachs <mwsachs@us.ibm.com>
> 
> To:   "Hudson, Darren" <dhudson@spaceworks.com>
> cc:   "'Welsh, David'" <David.Welsh@nordstrom.com>,
>       "'rawlins@metronet.com'" <rawlins@metronet.com>,
>       ebxml-tp@lists.ebxml.org, "Bob Haugen (E-mail)"
>       <linkage@interaccess.com>, ebxml-transport@lists.ebxml.org
> Subject:  RE: Applying transformations to Payloads (compression)
> 
> 
> 
> 
> Re-sending posting because it was rejected because of an obscene first
> line.  (It is forbidden to mention the name of a list in the 
> first line).
> 
> Regards,
> Marty
> 
> **************************************************************
> ***********************
> 
> 
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287;  IBM tie line 863-7287
> Notes address:  Martin W Sachs/Watson/IBM
> Internet address:  mwsachs @ us.ibm.com
> **************************************************************
> ***********************
> 
> ---------------------- Forwarded by Martin W Sachs/Watson/IBM 
> on 01/03/2001
> 09:40 AM ---------------------------
> 
> Martin W Sachs
> 01/03/2001 09:12 AM
> 
> To:   "Hudson, Darren" <dhudson@spaceworks.com>
> cc:   "'Welsh, David'" <David.Welsh@nordstrom.com>,
>       "'rawlins@metronet.com'" <rawlins@metronet.com>,
>       ebxml-tp@lists.ebxml.org, "Bob Haugen (E-mail)"
>       <linkage@interaccess.com>, ebxml-transport@lists.ebxml.org
> From: Martin W Sachs/Watson/IBM@IBMUS
> Subject:  RE: Applying transformations to Payloads 
> (compression)  (Document
>       link: Martin W. Sachs)
> 
> (note that I have added ebxml-transport to the cc list).
> 
> Since no one has mentioned transformations other than compression, I
> suggest we confine his discussion to compression for now.
> 
> To support compression, we need the following:
> 
>    In the CPP and CPA
> 
>      Indication that a party uses business document compression. This
>      should be able to be specified both per message 
> definition and for
>      each party's Service (set of business transaction 
> definitions) as a
>      whole.  At a minimum the CPP/CPA would have to state 
> that compression
>      is (or may be) used and the algorithm.  If someone can provide a
>      specific suggestion for a compression definition, I will 
> add it to the
>      list of things to be done in the next iteration of the CPP-CPA
>      specification.
> 
>    In the Messaging Service specification
> 
>      The packaging information should include an indicator of 
> whether a
>      payload is compressed (and the algorithm?).  This allows 
> for carrying
>      multiple payload documents, some, but not all, of which are
>      compressed.
> 
> 
>    Regards,
>    Marty
> 
> 
> 
> **************************************************************
> ***********************
> 
> 
>    Martin W. Sachs
>    IBM T. J. Watson Research Center
>    P. O. B. 704
>    Yorktown Hts, NY 10598
>    914-784-7287;  IBM tie line 863-7287
>    Notes address:  Martin W Sachs/Watson/IBM
>    Internet address:  mwsachs @ us.ibm.com
> 
> **************************************************************
> ***********************
> 
> 
> 
> 
> "Hudson, Darren" <dhudson@spaceworks.com> on 01/03/2001 04:40:46 AM
> 
> To:   "'Welsh, David'" <David.Welsh@nordstrom.com>,
>       "'rawlins@metronet.com'" <rawlins@metronet.com>,
>       ebxml-tp@lists.ebxml.org
> cc:   "Bob Haugen (E-mail)" <linkage@interaccess.com>
> Subject:  RE: Applying transformations to Payloads
> 
> 
> 
> 
> I would also like to add to this.
> 
> In Europe we do not have the luxury of masses of bandwidth so 
> I am in full
> agreement on compressing payloads.
> 
> For example, some SME's are connected via a VAN and are 
> charged by bytes,
> they are going to be upset to find out how fat XML is compared to EDI.
> 
> There is a great need for EDI and XML to work in harmony and 
> compression is
> going to greatly help in this.
> 
> 
> Thanks,
> 
> Darren Hudson
> 
> -----Original Message-----
> From: Welsh, David [mailto:David.Welsh@nordstrom.com]
> Sent: 02 January 2001 23:54
> To: 'rawlins@metronet.com'; ebxml-tp@lists.ebxml.org
> Cc: Bob Haugen (E-mail)
> Subject: RE: Applying transformations to Payloads
> 
> 
> Well if this helps, maybe I can introduce myself.
> 
> I am the e-Fulfillment Director for http://www.nordstrom.com .
> Nordstrom.com, as a business, are the premier internet 
> business site and
> catalog business for apparel in the domestic US.
> You can read more about our business operations and approach 
> to business at
> such sites as
> http://www.redherring.com/industries/2000/1017/ind-shoptalk101
> 700.html with
> articles like :
> Shop Talk: Nordstrom.com says execute, don't innovate
> By Ken Yamada
> Redherring.com, October 17, 2000
> 
> In addition, this past weekend as a necessary business 
> function, I just
> saved hour$$ of people and data comm processing time to do a 
> full inventory
> syncronization (because I data compressed XML data payloads) between a
> remote warehouse in Chicago and my main ERP system in 
> Seattle, where every
> few minutes of every day for months now we have been getting XML based
> inventory adjustments and send PO's,.... again all in XML.
> 
> I have a business need, and we choose not to do EDI.
> 
> Thanks
> David Welsh
> Director e-Fulfillment
> Norstrom.com
> (206) 215-7293
> 
> 
> > -----Original Message-----
> > From: Mike Rawlins [mailto:rawlins@metronet.com]
> > Sent: Tuesday, January 02, 2001 3:13 PM
> > To: ebxml-tp@lists.ebxml.org
> > Subject: Re: Applying transformations to Payloads
> >
> >
> > I think I would want to hear directly from someone who has a
> > business need for compression before we
> > went to any trouble to support it.  My view on this is that
> > if someone is really concerned about data
> > volumes and compression, they won't be using XML anyway -
> > they'll probably be using traditional EDI with
> > compression on top of it.
> >
> > "Welsh, David" wrote:
> >
> > > Thanks for the attention. I certainly don't want to "throw
> > a curve at this
> > > late stage" tot he group.
> > > Is 'transforms' and 'encoding' the same thing ?
> > >
> > > Given there's several orders of magnitude of increased data
> > (text) volume
> > > size usually (well in my experience) involved (see attached
> > example from the
> > > BP group of a business process document, and you could say
> > it's not that
> > > much bigger but multiply the extra overhead by 10's of
> > thousands of orders
> > > and ...), and I also suspect given most people (especially
> > the SME's won't
> > > have the luxury of T1, DSL or high speed access to an ISP),
> > I was hoping
> > > there'd be a way to allow for compressed / non-compressed
> > payloads to be
> > > indicated.
> > >
> > > Not everyone probably could support a compressed payload,
> > and to support
> > > compression probably could be something extra in a normal
> > exhange; but if
> > > trading partners could go the extra step and support
> > compression then it
> > > seems reasonable that there'd be some realistic chance of
> > company's being
> > > more efficient; time to process, time to transmit, cost of
> > services, .. !
> > >
> > > Thanks again
> > > Dave Welsh
> > >
> > > > -----Original Message-----
> > > > From: christopher ferris [mailto:chris.ferris@east.sun.com]
> > > > Sent: Tuesday, January 02, 2001 5:43 AM
> > > > To: Burdett, David
> > > > Cc: ebxml-tp@lists.ebxml.org; 'Welsh, David'
> > > > Subject: Re: Applying transformations to Payloads
> > > >
> > > >
> > > > David,
> > > >
> > > > Not transforms per se, but there is an encoding element
> > > > that theoretically could be used.
> > > >
> > > > It might be more flexible to simply allow for some
> > > > set of transforms to be defined as they have done for
> > > > XMLDSIG which can be extended to arbitrary xslt transforms
> > > > specified by the user/programmer.
> > > >
> > > > Of course, this will need to be discussed after Marty 
> has a draft
> > > > published, hopefully, sometime very soon!
> > > >
> > > > Cheers,
> > > >
> > > > Chris
> > > >
> > > > "Burdett, David" wrote:
> > > > >
> > > > > I've been exchanging emails with David Welsh at Nordstrom
> > > > about the idea of
> > > > > compressing data inside the payload of an ebXML message.
> > > > However before you
> > > > > could do this, it would be useful if information on the
> > > > ability to accept
> > > > > this (or any other) type of transformation in the payload
> > > > was recorded in
> > > > > the CPP/CPA.
> > > > >
> > > > > Is there a facility to support this in the CPP/CPA
> > > > >
> > > > > Regards
> > > > >
> > > > > David
> > > > >
> > > > > Product Management, Commerce One
> > > > > 4400 Rosewood Drive, Pleasanton, CA 94588, USA
> > > > > Tel: +1 (925) 520 4422 (also voicemail); Pager: +1
> > (888) 936 9599
> > > > > mailto:david.burdett@commerceone.com; Web:
> > > > http://www.commerceone.com
> > > >
> > >
> > >
> > --------------------------------------------------------------
> > ----------
> > >
> > > Subject: DTD & Sample - Specification Schema for review
> > > Date: Mon, 1 Jan 2001 11:44:25 -0700
> > > From: Cory Casanave <cory-c@dataaccess.com>
> > > Reply-To: Karsten Riemer <Karsten.Riemer@East.Sun.COM>
> > > To: 'Karsten Riemer ' <Karsten.Riemer@East.Sun.COM>,
> > >      "'ebxml-bp@lists.ebxml.org '" <ebxml-bp@lists.ebxml.org>,
> > >      "'ebxml-core@lists.ebxml.org '" <ebxml-core@lists.ebxml.org>
> > >
> > >  <<ebXmlSpecifcationSample090.xml>>
> > <<ebXmlSpecificationDTD090.dtd>>
> > >  Enclosed is the DTD which matches the ebXml Specification
> > Schema 0.90 and a
> > > sample of it's use.  Note that we do not have an
> > > Automated generator or parser for this yet so it is done by
> > hand and could
> > > have errors.  It is valid XML.
> > >
> > > The following comments are derived from the process of
> > making the sample and
> > > schema;
> > >
> > > 1. Important! We must define a set of "core" types.  To
> > make the example, I
> > > assumed the following:
> > >                       Integer, String (Simple), Text (IE
> > HTML), Float,
> > > Decimal, Date, Time,
> > >                       DateTime, Currency, Duration.
> > >                       Other built-in types may be added by
> > core components
> > >
> > > 2. There is no apparent difference between "document" 
> (representing
> > > "Structured Document") and "aggregate", other than one can
> > be in a document
> > > set and one can not - so why have the distinction? (The
> > same is true for
> > > document-set, but some people seem to Really want this).
> > >
> > > 3. Having to define document sets all the time is 
> annoying.  Perhaps
> > > reference to a document type in a business transaction can
> > imply a document
> > > set.
> > >
> > > 4. It is not clear if the user can make new "Basic
> > Information Entities" or
> > > if this is just for built-in types.  It is not currently 
> in the DTD.
> > >
> > > 5. There is no way, as there is in XML, to define reusable
> > attribute types.
> > >
> > > 6. There are some times I would have wanted to put the
> > delivery attributes o
> > > the data type as well as the attribute.
> > >
> > > 7. Document set should inherit from information entity so
> > it can be used in
> > > attributes, as is done in the example.
> > >
> > > 8. I assume we want to be able to document models, I added
> > this to the DTD
> > > but it is not in the model.
> > >
> > > -----Original Message-----
> > > From: Karsten Riemer
> > > To: ebxml-bp@lists.ebxml.org; ebxml-core@lists.ebxml.org
> > > Sent: 12/30/00 7:09 PM
> > > Subject: Specification Schema for review
> > >
> > > Hi,
> > > attached is the BP Specification Schema.
> > > It will be discussed in this coming Tuesday's metamodel
> > meeting and if
> > > no
> > > major issues are raised it will be submitted to QR on
> > Wednesday 1/3. It
> > > will
> > > be accompanied by a DTD and a set of patterns. Paul, please
> > send call
> > > info for
> > > Tuesday's meeting to BP and Core lists.
> > >
> > > thanks,
> > > -karsten
> > >  <<WinZip>>
> > >
> > >
> > --------------------------------------------------------------
> > ----------
> > >                                      Name:
> > ebXmlSpecifcationSample090.xml
> > >    ebXmlSpecifcationSample090.xml    Type: XML Document (text/xml)
> > >                                  Encoding: QUOTED-PRINTABLE
> > >
> > >                                    Name:
> > ebXmlSpecificationDTD090.dtd
> > >    ebXmlSpecificationDTD090.dtd    Type: DTD File
> > (application/x-unknown-content-type-dtd_auto_file)
> > >                                Encoding: QUOTED-PRINTABLE
> >
> > --
> > Michael C. Rawlins, Rawlins EC Consulting
> > http://www.metronet.com/~rawlins/
> >
> >
> 
> 
> 
> 
> 
> 
> 
>