Re: Expanding Reliable Messaging to meet the needs of IOTP

[Henry Lowe <hlowe@omg.org>]

Chris,

Try removing David's item 1b and have a look at it again.
I think that takes care of the layering violation problem.
Add in 1b and, I agree, you got problems.

Also, now (i.e., without 1b), you have a significantly 
more efficient MS as you've reduced the number of messages 
by 1/3 -- you were right a couple days ago in saying that 
bandwidth doesn't count anymore, but the number of messages 
is still a big deal (just in terms of context switches to  
say nothing of state access).

Best regards,
Henry
-------------------------------------------------
At 09:32 AM 08/30/2000 -0400, Christopher Ferris wrote:
>Henry,
>
>RefToMessageId is already defined in the Header. I still
>think that there is violation of the layers as outlined in
>my previous posting.
>
>Cheers,
>
>Chris
>
>Henry Lowe wrote:
>> 
>> David,
>> 
>> What you are suggesting, IMHO, does not violate layering except for
>> item 1b.  If you were happy to live without 1b, I would back this
>> as long as the "RefToMessageId" is in the ebXML Messaging Service
>> Headers spec (it isn't yet, of course RM could put it in when they
>> are integrated) and not in the underlying transport (not all under-
>> lying transports may have "RefToMessageId"s and, even if they did,
>> you might be back into layering problems).
>> 
>> Indeed, what you suggest, without 1b, probably wouldn't need any-
>> thing in the TPA telling you where to look for your ACKed message
>> ID (if it was a NORMAL message, you'd look in the "RefToMessageId"
>> and if it was a "Acknowledgement" message you'd look in the
>> "RefToMessageId", too -- nice and simple :-).
>> 
>> Of course, the question of how you get the fact that message 9876
>> had been ACKed to the application (BP) has to be dealt with without
>> violating layering, but that's easily dealt with, e.g., use the
>> NOTIFICATION service (see the Technical Architecture section on
>> TRP) to notify the BP that "message 9876 has been ACKed".  If we
>> thought about it for a few minutes, we could probably come up
>> with other alternatives.
>> 
>> Best regards,
>> Henry
>> 
>> PS: Hope all got this as I stripped off all e-mail addresses except
>> for the TRP list.  Let me know if you didn't get this ;-)
>> ------------------------------------------------------
>> At 01:06 PM 08/29/2000 -0700, David Burdett wrote:
>> >Chris
>> >
>> >In the header we have "RefToMessageId" that identifies a message that is
>> >caused "this" message to be generated.
>> >
>> >On an Acknowledgement message, this will refer to the message that is being
>> >acknowledged.
>> >
>> >Why can't we simply say in the reliable messaging spec words along the
lines
>> >of ...
>> >
>> >>>>
>> >Reliable delivery of a message to a destination is demonstrated by the
>> >receipt of the sender of either:
>> >1. A message with a Message Type of "Acknowldgement" that refers to the
>> >message that was sent either:
>> >  a) in the "RefToMessageId" field, or
>> >  b) in the payload of the acknowledgement message (method TBD)
>> >2. A message with a Message Type of "Normal" that refers in the
>> >"RefToMessageId" field to the original message that was sent
>> >
>> >The method of acknowledging a message is specified in the Trading Party
>> >Agreement or the Party Profile and is dependent on, for example:
>> >* the data communications protocol being used
>> >* the time taken to generate a "Normal" message in response to an earlier
>> >"Normal" message.
>> ><<<
>> >
>> >Note that the messaging service need not know *anything* about the business
>> >process that is being supported.
>> >
>> >I honestly don't see where the complexity is or how this sort of approach
>> >would violate the "layering".
>> >
>> >I also know that we if we don't spec this now, I will have to write our own
>> >spec (that I would publish) to the list, that does it as we really, really
>> >need it.
>> >
>> >David
>> >
>> >-----Original Message-----
>> >From: Christopher Ferris [mailto:chris.ferris@east.sun.com]
>> >Sent: Tuesday, August 29, 2000 7:03 AM
>> >To: David Burdett
>> >Cc: 'Henry Lowe'; 'mwsachs@us.ibm.com'; ebXML Transport (E-mail)
>> >Subject: Re: Expanding Reliable Messaging to meet the needs of IOTP
>> >
>> >
>> >All,
>> >
>> >I don't understand how this doesn't violate the layering.
>> >It means that the MS has to track the behavior of the BP
>> >which is IMHO out of the question.
>> >
>> >The MS shouldn't care where a given message originates
>> >and the BP shouldn't be involved in any MS-level optimizations.
>> >
>> >This is an example of early optimization which is IMHO
>> >a mistake, especially given that bandwidth is so much less
>> >of a concern than it has been in the past, and the situation
>> >is most definitely improving rapidly.
>> >
>> >This also violates the KISS principle and in my mind over
>> >complicates any implementation.
>> >
>> >Cheers,
>> >
>> >Chris
>> >
>> >
>> >David Burdett wrote:
>> >>
>> >> Henry
>> >>
>> >> You're suggesting just what I had in mind except that I think the "bit"
>> >you
>> >> mention could alternatively be held in a TPA.
>> >>
>> >> David
>> >>
>> >> -----Original Message-----
>> >> From: Henry Lowe [mailto:hlowe@omg.org]
>> >> Sent: Monday, August 28, 2000 9:43 AM
>> >> To: David Burdett
>> >> Cc: 'Christopher Ferris'; 'mwsachs@us.ibm.com'; ebXML Transport (E-mail)
>> >> Subject: RE: Expanding Reliable Messaging to meet the needs of IOTP
>> >>
>> >> I must agree with David Brudett on this.
>> >>
>> >> First, you must remember, that for the low level
>> >> "I got the message" ACK, you cannot rely on the
>> >> underlying transport -- in some cases, e.g., SMTP,
>> >> there just isn't an ACK with the underlying transport,
>> >> and in the multi-hop case, the underlying transport
>> >> ACK isn't end-to-end anyway.  What this amounts to is,
>> >> if an "I got the message" MS level ACK is important,
>> >> the MS has to send it!!
>> >>
>> >> With this in mind, what David is proposing, piggy-
>> >> backing the low level and BP ACK (where it makes sense
>> >> and is within reasonable timing constraints) is not
>> >> only simple but also message efficient and it reduces
>> >> the number of messages sent by 33% -- GOODness as it's
>> >> messages, not bytes, which counts for efficiency.
>> >>
>> >> A simple (one bit) mechanism can be defined to implement
>> >> this in the MS protocol (well, if you use XML it will be
>> >> more than one bit).  In the original message sent (request
>> >> of a request/response pair) set a flag (one bit?) whose
>> >> semantics is, if set, hold off on ACK until response from
>> >> the BP is to be sent and piggy-back the two -- in the
>> >> response header you will have a similar flag which says
>> >> this is a piggy-backed ACK.  Done is 2 messages instead
>> >> of 3.
>> >>
>> >> This doesn't involve any layering violations (which I would
>> >> guess is what Chris was thinking about) and involves a
>> >> minimal increase of state preservation (one bit).
>> >>
>> >> Best regards,
>> >> Henry
>> >>
>> >> PS: Note I'm shamelessly promoting multi-hop AGAIN.
>> >> ----------------------------------------------------
>> >> At 09:41 AM 08/25/2000 -0700, David Burdett wrote:
>> >> >Chris
>> >> >
>> >> >Two main issues ...
>> >> >
>> >> >POINT ONE
>> >> >I fundamentally disagree with you when you say ...
>> >> >
>> >> >>>> The business level response is just another message. It cannot be
>> >> >interpreted as an ack without all kinds of kludgy whacked out code which
>> >> the
>> >> >MS must process to figure out which way is up.<<<
>> >> >
>> >> >This statement can ONLY be true IF THERE IS ONLY ONE WAY TO TECHNICALLY
>> >> >SOLVE A PROBLEM which clearly is not true. I also assert that you cannot
>> >> >PROVE that something can only be solved using "kludgy whacked out code"
>> >> >since what would stop someone from solving the problem simply and
>> >> elegantly.
>> >> >Unless you can prove it is impossible to write elegant code then your
>> >> >statement is fundamentally wrong.
>> >> >
>> >> >Anyway, to stop this getting in to a slanging match ;) What I will do
>> >(but
>> >> >folks please give me time - I have a day job) is write up a solution
>> >which
>> >> >solves the problem pretty elegantly based on a set of pseudo code that I
>> >> >wrote to solve exactly this problem for IOTP.
>> >> >
>> >> >So Chris, I think the jury's out on this one.
>> >> >
>> >> >POINT TWO
>> >> >Whilst I agree that we want to follow the KISS principle. We must
support
>> >a
>> >> >reasonable set of use cases, that represent how eCommerce is actually
>> >being
>> >> >used today otherwise ebXML will not be adopted. Although synchronous
>> >> >responses are needed for IOTP, they aren't the only cases. Commerce
One's
>> >> >"RoundTrip" and Ariba's "PunchOut" which both do similar things, both
>> >need
>> >> a
>> >> >synchronous messaging system.
>> >> >
>> >> >So, again, to stop this getting in to a slanging match, the solution is
>> >to
>> >> >develop an alternative that proves that this is pretty simple to do.
>> >Again
>> >> I
>> >> >will do this, but I need the time.
>> >> >
>> >> >David
>> >> >PS I am about to go into a day-long meeting and might not be able to
>> >> >continue this conversation until next week.
>> >> >
>> >> >-----Original Message-----
>> >> >From: Christopher Ferris [mailto:chris.ferris@east.sun.com]
>> >> >Sent: Friday, August 25, 2000 7:27 AM
>> >> >To: David Burdett
>> >> >Cc: 'mwsachs@us.ibm.com'; ebXML Transport (E-mail)
>> >> >Subject: Re: Expanding Reliable Messaging to meet the needs of IOTP
>> >> >
>> >> >
>> >> >I fundamentally disagree with this notion. The business level
>> >> >response is just another message. It cannot be interpreted as
>> >> >an ack without all kinds of kludgy whacked out code which the
>> >> >MS must process to figure out which way is up.
>> >> >
>> >> >The MS should have a minimal set of functionality:
>> >> >       send/receive a mesage, match to conversation, route to service
>> >> >handler
>> >> >       send/receive an MS-level ack or error, process accordingly
>> >> >
>> >> >which could be optimized with the windowing technique to:
>> >> >       send/receive a window of messages, match to conversation, route
to
>> >> >service handler
>> >> >       send/receive an MS-level ack or error, process accordingly
>> >> >
>> >> >Suggesting that a business message could double as an ack
>> >> >would require all kinds of nasty logic at the
>> >> >MS level which would impede performance to no end because each
>> >> >receipt of a message would have to be analyzed to death.
>> >> >Not only that but the design of the choreography would be overly
>> >> complicated
>> >> >IMO.
>> >> >
>> >> >The MS should be lean and mean with a minimum of boolean logic.
>> >> >
>> >> >Chris
>> >> >
>> >> >David Burdett wrote:
>> >> >>
>> >> >> >>>However I am concerned about relying on the business-level response
>> >as
>> >> >> the confirmation of receipt of the message because business-level
>> >> timeouts
>> >> >> can be very long<<<
>> >> >>
>> >> >> I agree which is why I'm proposing that the business-level response is
>> >an
>> >> >> *alternative* as some business processes can be very short and a
>> >separate
>> >> >> acknowledgement unnecessary. We can easily include the way in which
>> >> >reliable
>> >> >> messaging receives its acknowledgements through in the TPA.
>> >> >>
>> >> >> I'm also not sure that if you are having a pseudo-real-time
>> >conversation
>> >> >> that occurs in IOTP, that using a separate acknowledgement actually
>> >works
>> >> >on
>> >> >> HTTP (can anyone advise).
>> >> >>
>> >> >> David
>> >> >> PS Sending to whole list since that is what I reckon Marty intended to
>> >> do.
>> >> >>
>> >> >> -----Original Message-----
>> >> >> From: mwsachs@us.ibm.com [mailto:mwsachs@us.ibm.com]
>> >> >> Sent: Monday, August 21, 2000 10:45 AM
>> >> >> To: David Burdett
>> >> >> Subject: Re: Expanding Reliable Messaging to meet the needs of IOTP
>> >> >>
>> >> >> Dave,
>> >> >>
>> >> >> Your suggestion has merit.  However I am concerned about relying on
the
>> >> >> business-level response as the confirmation of receipt of the message
>> >> >> because business-level timeouts can be very long (consider a request
>> >> which
>> >> >> initiates a human workflow or requires the requestee to communicate
>> >with
>> >> a
>> >> >> subcontractor first).  Waiting that long to discover than the request
>> >was
>> >> >> dropped by the network seems undesirable. However many of the
>> >transports
>> >> >> that will be used below the messaging service already have acks
>> >> prescribed
>> >> >> in their protocols.  If the implementation persists the message before
>> >> >> sending the transport-level ACK, we should be covered without an extra
>> >> >> layer of function.
>> >> >>
>> >> >> One case where the existing proposal seems to make sense is with SMTP
>> >> >since
>> >> >> SMTP has no end to end acknowledgment on a multihop path.
>> >> >>
>> >> >> Regards,
>> >> >> Marty
>> >> >>
>> >> >>
>> >>
>>
>>***************************************************************************
>> >> *
>> >> >> *********
>> >> >>
>> >> >> Martin W. Sachs
>> >> >> IBM T. J. Watson Research Center
>> >> >> P. O. B. 704
>> >> >> Yorktown Hts, NY 10598
>> >> >> 914-784-7287;  IBM tie line 863-7287
>> >> >> Notes address:  Martin W Sachs/Watson/IBM
>> >> >> Internet address:  mwsachs @ us.ibm.com
>> >> >>
>> >>
>>
>>***************************************************************************
>> >> *
>> >> >> *********
>> >> >>
>> >> >> David Burdett <david.burdett@commerceone.com> on 08/21/2000 11:30:37
AM
>> >> >>
>> >> >> To:   "ebXML Transport (E-mail)" <ebxml-transport@lists.ebxml.org>
>> >> >> cc:
>> >> >> Subject:  Expanding Reliable Messaging to meet the needs of IOTP
>> >> >>
>> >> >> Folks
>> >> >>
>> >> >> I suggest that reliable messaging should be expanded, so that, instead
>> >of
>> >> >> sending an ack, to imply that the message has been received, an
>> >> >alternative
>> >> >> would be to use the next "normal" message received instead.
>> >> >>
>> >> >> The reason I suggest this is because this is how the Internet Open
>> >> Trading
>> >> >> Protocol (RFC 2801) does it and I'm not sure that the current Reliable
>> >> >> Messaging spec will support this method of working. In outline what
>> >> >happens
>> >> >> is illustrated by the following:
>> >> >>
>> >> >> 1. Customer sends a "Payment Request" message to a Payment Handler
>> >(e.g.
>> >> a
>> >> >> bank) - using a HTTP Post
>> >> >> 2. The Payment Handler processes the Payment Request and sends a
>> >"Payment
>> >> >> Response" message on the HTTP Response
>> >> >> 3. If the Payment Response is not received in time, a timeout occurs
>> >and
>> >> >> the
>> >> >> Customer resends the Payment Request
>> >> >> 4. If the Payment Handler receives a duplicate Payment Request, then
>> >they
>> >> >> resend the Payment Response that they sent previously, but otherwise
>> >> >> ignores
>> >> >> it
>> >> >>
>> >> >> No separate acknowledgement message is needed since the receipt of the
>> >> >> Payment Response by the Customer implies that the Payment Request has
>> >> been
>> >> >> received.
>> >> >>
>> >> >> Thoughts?
>> >> >>
>> >> >> David
>> >> >>
>> >> >> Product Management, CommerceOne
>> >> >> 4400 Rosewood Drive 3rd Fl, Bldg 4, Pleasanton, CA 94588, USA
>> >> >> Tel: +1 (925) 520 4422 (also voicemail); Pager: +1 (888) 936 9599
>> >> >> mailto:david.burdett@commerceone.com; Web: http://www.commerceone.com
>> >> >
>> >> >--
>> >> >    _/_/_/_/ _/    _/ _/    _/ Christopher Ferris - Enterprise Architect
>> >> >   _/       _/    _/ _/_/  _/  Phone: 781-442-3063 or x23063
>> >> >  _/_/_/_/ _/    _/ _/ _/ _/   Email: chris.ferris@East.Sun.COM
>> >> >       _/ _/    _/ _/  _/_/    Sun Microsystems,  Mailstop: UBUR03-313
>> >> >_/_/_/_/  _/_/_/  _/    _/     1 Network Drive Burlington, MA 01803-0903
>> >
>> >--
>> >    _/_/_/_/ _/    _/ _/    _/ Christopher Ferris - Enterprise Architect
>> >   _/       _/    _/ _/_/  _/  Phone: 781-442-3063 or x23063
>> >  _/_/_/_/ _/    _/ _/ _/ _/   Email: chris.ferris@East.Sun.COM
>> >       _/ _/    _/ _/  _/_/    Sun Microsystems,  Mailstop: UBUR03-313
>> >_/_/_/_/  _/_/_/  _/    _/     1 Network Drive Burlington, MA 01803-0903
>
>-- 
>    _/_/_/_/ _/    _/ _/    _/ Christopher Ferris - Enterprise Architect
>   _/       _/    _/ _/_/  _/  Phone: 781-442-3063 or x23063
>  _/_/_/_/ _/    _/ _/ _/ _/   Email: chris.ferris@East.Sun.COM
>       _/ _/    _/ _/  _/_/    Sun Microsystems,  Mailstop: UBUR03-313
>_/_/_/_/  _/_/_/  _/    _/     1 Network Drive Burlington, MA 01803-0903