Re: Applying transformations to Payloads (compression)

[Martin W Sachs <mwsachs@us.ibm.com>]

I stumbled earlier by referring to MSH instead of "Message Service
Specification" or "wire format".  Let me try again.

In the CPP/CPA, we can provide for specifying compression per message
definition and globally for each party's service.  With compression on, it
would mean that all payloads sent by a given party, or all instances of a
given message, would be compressed.

I had suggested that users of compression might want more granularity.  I
suggested that with multiple payloads, some might be compressed and others
not.  Another example might be that some instances of the same message
definition might be compressed and others not.  If this level of
granularity is valuable, then it would require indicators somewhere in the
packaging information (manifest?) that indicate whether a particular
payload is or isn't compressed. The values of these indicators would be
supplied by the sending application.  Nothing in this paragraph asks the
messaging service implementation to do anything more than set an indicator
based on what the sending application told it. If this level of granularity
is not valuable, then everything can be done with definitions in the CPP
and CPA.

Regards,
Marty

*************************************************************************************

Martin W. Sachs
IBM T. J. Watson Research Center
P. O. B. 704
Yorktown Hts, NY 10598
914-784-7287;  IBM tie line 863-7287
Notes address:  Martin W Sachs/Watson/IBM
Internet address:  mwsachs @ us.ibm.com
*************************************************************************************



Christopher Ferris <chris.ferris@east.sun.com>@east.sun.com on 01/03/2001
10:53:02 AM

Sent by:  Chris.Ferris@east.sun.com


To:   ebxml-tp@lists.ebxml.org
cc:   ebxml-transport@lists.ebxml.org
Subject:  Re: Applying transformations to Payloads (compression)



If I'm not mistaken, I believe that we agreed early on
that the MSH would NOT do compression, but that the
transport (or more correctly, transfer?)  *might* apply
compression. Certainly, an application could also apply
compression to a payload object so long as the parties
agree and can both support the scheme.

Dale's proposal to define packaging profiles would
be the place to specify any compression scheme
applied to the payload by the application, and the
CPP/CPA should have some capacity to describe a
transport's capabilities and preferences w/r/t
compression (e.g. gzip stream)

I think that the issue here belongs in TP (CPP/CPA)
and that all we need be concerned with is that the
CPP/CPA provide the means to identify any particulars
related to packaging or transport.

Since we've "frozen" the CPP/CPA dtd for the time
being, so that we can get a draft of the spec written,
I suggest that we defer this dicussion until the
spec has been published to the TP team for comments
and feedback, and track any necessary changes accordingly,
including team votes on whether or not to adopt any changes.

Cheers,

Chris

"Moberg, Dale" wrote:
>
> Some additional points relevant to the compression issue:
>
> Some transports already have options for compressing data.
>   Example: HTTP Content-coding has IANA values "gzip" and "compress"
> Some security packaging schemes have or are defining compression options:
>   Example: PGP already has compression and S/MIME IETF has a compression
> content
>   being defined within the IETF SMIME WG.
> For file transport approaches, there are many ways of compressing and
>     then sending a compressed file. With some forms of transport,
>     the binary form of compressed data may trigger a partial expansion
>     because of the need for content-transfer-encodings (email related
>     and possibly elsewhere (SOAP?)).
>
> There are a lot of interdependencies here to track! CPP and CPA
> might be able to specify some of these compression options.
> I think the issue raised by David Welsh is whether a specific
> profile for packaging involving compression needs to be
> created. If so, is it part of the security packaging, part of
> the transport binding information, or what?
>
> In view of the interactions of compression with security, the
> security group should also be involved reviewing proposals;
> security people like to compress before encrypting and for
> signatures there can be interoperability issues pertaining
> to use of different conventions for calculating the hash
> (was the hash over the uncompressed or the compressed data...).
>
> If there is a FTP binding for ebXML, then a file level compression
> solution might be mentioned in discussion of the FTP binding.
> Possibly this option (if you want compression, arrange
> to use FTP as a transport and apply compression to the file
> transferred) would meet the requirement, if there is a
> compression requirement for TRP or TP.
>
> I doubt very much that a "transport and packaging/security independent"
> approach to compression will be found because the various
> security standards and IETF transport standards
> have already handled compression in their own ways.
>
> I agree with Marty that the best that can be done is to point to a CPA
> that governs which compression options, at which level, are being
> employed.
>
> Dale Moberg
>
> > -----Original Message-----
> > From: john_ibbotson@uk.ibm.com [mailto:john_ibbotson@uk.ibm.com]
> > Sent: Wednesday, January 03, 2001 9:53 AM
> > To: Martin W Sachs
> > Cc: ebxml-tp@lists.ebxml.org; ebxml-transport@lists.ebxml.org
> > Subject: RE: Applying transformations to Payloads (compression)
> >
> >
> >
> >
> >
> > I don't believe it is the job of the MSH to compress the
> > payload. This only
> > adds to the complexity of the MSH. Certainly, the CPP can include
> > information on compression schemes it supports but compression of the
> > payload should be the job of the application which then passes the
> > compressed stream to the MSH for enveloping. Further
> > compression may occur
> > at the transport layer but again, that isn't the job of the MSH.
> > John
> >
> > XML Technology and Messaging,
> > IBM UK Ltd, Hursley Park,
> > Winchester, SO21 2JN
> >
> > Tel: (work) +44 (0)1962 815188        (home) +44 (0)1722 781271
> > Fax: +44 (0)1962 816898
> > Notes Id: John Ibbotson/UK/IBM
> > email: john_ibbotson@uk.ibm.com
> >
> >
> > Martin W Sachs <mwsachs@us.ibm.com> on 01/03/2001 02:42:29 PM
> >
> > Please respond to Martin W Sachs <mwsachs@us.ibm.com>
> >
> > To:   "Hudson, Darren" <dhudson@spaceworks.com>
> > cc:   "'Welsh, David'" <David.Welsh@nordstrom.com>,
> >       "'rawlins@metronet.com'" <rawlins@metronet.com>,
> >       ebxml-tp@lists.ebxml.org, "Bob Haugen (E-mail)"
> >       <linkage@interaccess.com>, ebxml-transport@lists.ebxml.org
> > Subject:  RE: Applying transformations to Payloads (compression)
> >
> >
> >
> >
> > Re-sending posting because it was rejected because of an obscene first
> > line.  (It is forbidden to mention the name of a list in the
> > first line).
> >
> > Regards,
> > Marty
> >
> > **************************************************************
> > ***********************
> >
> >
> > Martin W. Sachs
> > IBM T. J. Watson Research Center
> > P. O. B. 704
> > Yorktown Hts, NY 10598
> > 914-784-7287;  IBM tie line 863-7287
> > Notes address:  Martin W Sachs/Watson/IBM
> > Internet address:  mwsachs @ us.ibm.com
> > **************************************************************
> > ***********************
> >
> > ---------------------- Forwarded by Martin W Sachs/Watson/IBM
> > on 01/03/2001
> > 09:40 AM ---------------------------
> >
> > Martin W Sachs
> > 01/03/2001 09:12 AM
> >
> > To:   "Hudson, Darren" <dhudson@spaceworks.com>
> > cc:   "'Welsh, David'" <David.Welsh@nordstrom.com>,
> >       "'rawlins@metronet.com'" <rawlins@metronet.com>,
> >       ebxml-tp@lists.ebxml.org, "Bob Haugen (E-mail)"
> >       <linkage@interaccess.com>, ebxml-transport@lists.ebxml.org
> > From: Martin W Sachs/Watson/IBM@IBMUS
> > Subject:  RE: Applying transformations to Payloads
> > (compression)  (Document
> >       link: Martin W. Sachs)
> >
> > (note that I have added ebxml-transport to the cc list).
> >
> > Since no one has mentioned transformations other than compression, I
> > suggest we confine his discussion to compression for now.
> >
> > To support compression, we need the following:
> >
> >    In the CPP and CPA
> >
> >      Indication that a party uses business document compression. This
> >      should be able to be specified both per message
> > definition and for
> >      each party's Service (set of business transaction
> > definitions) as a
> >      whole.  At a minimum the CPP/CPA would have to state
> > that compression
> >      is (or may be) used and the algorithm.  If someone can provide a
> >      specific suggestion for a compression definition, I will
> > add it to the
> >      list of things to be done in the next iteration of the CPP-CPA
> >      specification.
> >
> >    In the Messaging Service specification
> >
> >      The packaging information should include an indicator of
> > whether a
> >      payload is compressed (and the algorithm?).  This allows
> > for carrying
> >      multiple payload documents, some, but not all, of which are
> >      compressed.
> >
> >
> >    Regards,
> >    Marty
> >
> >
> >
> > **************************************************************
> > ***********************
> >
> >
> >    Martin W. Sachs
> >    IBM T. J. Watson Research Center
> >    P. O. B. 704
> >    Yorktown Hts, NY 10598
> >    914-784-7287;  IBM tie line 863-7287
> >    Notes address:  Martin W Sachs/Watson/IBM
> >    Internet address:  mwsachs @ us.ibm.com
> >
> > **************************************************************
> > ***********************
> >
> >
> >
> >
> > "Hudson, Darren" <dhudson@spaceworks.com> on 01/03/2001 04:40:46 AM
> >
> > To:   "'Welsh, David'" <David.Welsh@nordstrom.com>,
> >       "'rawlins@metronet.com'" <rawlins@metronet.com>,
> >       ebxml-tp@lists.ebxml.org
> > cc:   "Bob Haugen (E-mail)" <linkage@interaccess.com>
> > Subject:  RE: Applying transformations to Payloads
> >
> >
> >
> >
> > I would also like to add to this.
> >
> > In Europe we do not have the luxury of masses of bandwidth so
> > I am in full
> > agreement on compressing payloads.
> >
> > For example, some SME's are connected via a VAN and are
> > charged by bytes,
> > they are going to be upset to find out how fat XML is compared to EDI.
> >
> > There is a great need for EDI and XML to work in harmony and
> > compression is
> > going to greatly help in this.
> >
> >
> > Thanks,
> >
> > Darren Hudson
> >
> > -----Original Message-----
> > From: Welsh, David [mailto:David.Welsh@nordstrom.com]
> > Sent: 02 January 2001 23:54
> > To: 'rawlins@metronet.com'; ebxml-tp@lists.ebxml.org
> > Cc: Bob Haugen (E-mail)
> > Subject: RE: Applying transformations to Payloads
> >
> >
> > Well if this helps, maybe I can introduce myself.
> >
> > I am the e-Fulfillment Director for http://www.nordstrom.com .
> > Nordstrom.com, as a business, are the premier internet
> > business site and
> > catalog business for apparel in the domestic US.
> > You can read more about our business operations and approach
> > to business at
> > such sites as
> > http://www.redherring.com/industries/2000/1017/ind-shoptalk101
> > 700.html with
> > articles like :
> > Shop Talk: Nordstrom.com says execute, don't innovate
> > By Ken Yamada
> > Redherring.com, October 17, 2000
> >
> > In addition, this past weekend as a necessary business
> > function, I just
> > saved hour$$ of people and data comm processing time to do a
> > full inventory
> > syncronization (because I data compressed XML data payloads) between a
> > remote warehouse in Chicago and my main ERP system in
> > Seattle, where every
> > few minutes of every day for months now we have been getting XML based
> > inventory adjustments and send PO's,.... again all in XML.
> >
> > I have a business need, and we choose not to do EDI.
> >
> > Thanks
> > David Welsh
> > Director e-Fulfillment
> > Norstrom.com
> > (206) 215-7293
> >
> >
> > > -----Original Message-----
> > > From: Mike Rawlins [mailto:rawlins@metronet.com]
> > > Sent: Tuesday, January 02, 2001 3:13 PM
> > > To: ebxml-tp@lists.ebxml.org
> > > Subject: Re: Applying transformations to Payloads
> > >
> > >
> > > I think I would want to hear directly from someone who has a
> > > business need for compression before we
> > > went to any trouble to support it.  My view on this is that
> > > if someone is really concerned about data
> > > volumes and compression, they won't be using XML anyway -
> > > they'll probably be using traditional EDI with
> > > compression on top of it.
> > >
> > > "Welsh, David" wrote:
> > >
> > > > Thanks for the attention. I certainly don't want to "throw
> > > a curve at this
> > > > late stage" tot he group.
> > > > Is 'transforms' and 'encoding' the same thing ?
> > > >
> > > > Given there's several orders of magnitude of increased data
> > > (text) volume
> > > > size usually (well in my experience) involved (see attached
> > > example from the
> > > > BP group of a business process document, and you could say
> > > it's not that
> > > > much bigger but multiply the extra overhead by 10's of
> > > thousands of orders
> > > > and ...), and I also suspect given most people (especially
> > > the SME's won't
> > > > have the luxury of T1, DSL or high speed access to an ISP),
> > > I was hoping
> > > > there'd be a way to allow for compressed / non-compressed
> > > payloads to be
> > > > indicated.
> > > >
> > > > Not everyone probably could support a compressed payload,
> > > and to support
> > > > compression probably could be something extra in a normal
> > > exhange; but if
> > > > trading partners could go the extra step and support
> > > compression then it
> > > > seems reasonable that there'd be some realistic chance of
> > > company's being
> > > > more efficient; time to process, time to transmit, cost of
> > > services, .. !
> > > >
> > > > Thanks again
> > > > Dave Welsh
> > > >
> > > > > -----Original Message-----
> > > > > From: christopher ferris [mailto:chris.ferris@east.sun.com]
> > > > > Sent: Tuesday, January 02, 2001 5:43 AM
> > > > > To: Burdett, David
> > > > > Cc: ebxml-tp@lists.ebxml.org; 'Welsh, David'
> > > > > Subject: Re: Applying transformations to Payloads
> > > > >
> > > > >
> > > > > David,
> > > > >
> > > > > Not transforms per se, but there is an encoding element
> > > > > that theoretically could be used.
> > > > >
> > > > > It might be more flexible to simply allow for some
> > > > > set of transforms to be defined as they have done for
> > > > > XMLDSIG which can be extended to arbitrary xslt transforms
> > > > > specified by the user/programmer.
> > > > >
> > > > > Of course, this will need to be discussed after Marty
> > has a draft
> > > > > published, hopefully, sometime very soon!
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Chris
> > > > >
> > > > > "Burdett, David" wrote:
> > > > > >
> > > > > > I've been exchanging emails with David Welsh at Nordstrom
> > > > > about the idea of
> > > > > > compressing data inside the payload of an ebXML message.
> > > > > However before you
> > > > > > could do this, it would be useful if information on the
> > > > > ability to accept
> > > > > > this (or any other) type of transformation in the payload
> > > > > was recorded in
> > > > > > the CPP/CPA.
> > > > > >
> > > > > > Is there a facility to support this in the CPP/CPA
> > > > > >
> > > > > > Regards
> > > > > >
> > > > > > David
> > > > > >
> > > > > > Product Management, Commerce One
> > > > > > 4400 Rosewood Drive, Pleasanton, CA 94588, USA
> > > > > > Tel: +1 (925) 520 4422 (also voicemail); Pager: +1
> > > (888) 936 9599
> > > > > > mailto:david.burdett@commerceone.com; Web:
> > > > > http://www.commerceone.com
> > > > >
> > > >
> > > >
> > > --------------------------------------------------------------
> > > ----------
> > > >
> > > > Subject: DTD & Sample - Specification Schema for review
> > > > Date: Mon, 1 Jan 2001 11:44:25 -0700
> > > > From: Cory Casanave <cory-c@dataaccess.com>
> > > > Reply-To: Karsten Riemer <Karsten.Riemer@East.Sun.COM>
> > > > To: 'Karsten Riemer ' <Karsten.Riemer@East.Sun.COM>,
> > > >      "'ebxml-bp@lists.ebxml.org '" <ebxml-bp@lists.ebxml.org>,
> > > >      "'ebxml-core@lists.ebxml.org '" <ebxml-core@lists.ebxml.org>
> > > >
> > > >  <<ebXmlSpecifcationSample090.xml>>
> > > <<ebXmlSpecificationDTD090.dtd>>
> > > >  Enclosed is the DTD which matches the ebXml Specification
> > > Schema 0.90 and a
> > > > sample of it's use.  Note that we do not have an
> > > > Automated generator or parser for this yet so it is done by
> > > hand and could
> > > > have errors.  It is valid XML.
> > > >
> > > > The following comments are derived from the process of
> > > making the sample and
> > > > schema;
> > > >
> > > > 1. Important! We must define a set of "core" types.  To
> > > make the example, I
> > > > assumed the following:
> > > >                       Integer, String (Simple), Text (IE
> > > HTML), Float,
> > > > Decimal, Date, Time,
> > > >                       DateTime, Currency, Duration.
> > > >                       Other built-in types may be added by
> > > core components
> > > >
> > > > 2. There is no apparent difference between "document"
> > (representing
> > > > "Structured Document") and "aggregate", other than one can
> > > be in a document
> > > > set and one can not - so why have the distinction? (The
> > > same is true for
> > > > document-set, but some people seem to Really want this).
> > > >
> > > > 3. Having to define document sets all the time is
> > annoying.  Perhaps
> > > > reference to a document type in a business transaction can
> > > imply a document
> > > > set.
> > > >
> > > > 4. It is not clear if the user can make new "Basic
> > > Information Entities" or
> > > > if this is just for built-in types.  It is not currently
> > in the DTD.
> > > >
> > > > 5. There is no way, as there is in XML, to define reusable
> > > attribute types.
> > > >
> > > > 6. There are some times I would have wanted to put the
> > > delivery attributes o
> > > > the data type as well as the attribute.
> > > >
> > > > 7. Document set should inherit from information entity so
> > > it can be used in
> > > > attributes, as is done in the example.
> > > >
> > > > 8. I assume we want to be able to document models, I added
> > > this to the DTD
> > > > but it is not in the model.
> > > >
> > > > -----Original Message-----
> > > > From: Karsten Riemer
> > > > To: ebxml-bp@lists.ebxml.org; ebxml-core@lists.ebxml.org
> > > > Sent: 12/30/00 7:09 PM
> > > > Subject: Specification Schema for review
> > > >
> > > > Hi,
> > > > attached is the BP Specification Schema.
> > > > It will be discussed in this coming Tuesday's metamodel
> > > meeting and if
> > > > no
> > > > major issues are raised it will be submitted to QR on
> > > Wednesday 1/3. It
> > > > will
> > > > be accompanied by a DTD and a set of patterns. Paul, please
> > > send call
> > > > info for
> > > > Tuesday's meeting to BP and Core lists.
> > > >
> > > > thanks,
> > > > -karsten
> > > >  <<WinZip>>
> > > >
> > > >
> > > --------------------------------------------------------------
> > > ----------
> > > >                                      Name:
> > > ebXmlSpecifcationSample090.xml
> > > >    ebXmlSpecifcationSample090.xml    Type: XML Document (text/xml)
> > > >                                  Encoding: QUOTED-PRINTABLE
> > > >
> > > >                                    Name:
> > > ebXmlSpecificationDTD090.dtd
> > > >    ebXmlSpecificationDTD090.dtd    Type: DTD File
> > > (application/x-unknown-content-type-dtd_auto_file)
> > > >                                Encoding: QUOTED-PRINTABLE
> > >
> > > --
> > > Michael C. Rawlins, Rawlins EC Consulting
> > > http://www.metronet.com/~rawlins/
> > >
> > >
> >
> >
> >
> >
> >
> >
> >
> >