MCOP is a fast multimedia communication protocol

Protocol specification

written and maintained by
Stefan Westerfeld <stefan@space.twc.de>

=============================================================================
1. Introduction
=============================================================================

It has conceptual similarities to CORBA, but it is intended to extend it in
all ways that are required for real time multimedia operations.  

It provides a multimedia object model, which can be used for both:
communication between components in one adress space (one process), and
between components that are in different threads, processes or on
different hosts.

All in all, it will be designed for extremely high performance (so everything
shall be optimized to be blazingly fast), suitable for very communicative
multimedia applications. For instance streaming videos around is one of
the applications of MCOP, where most CORBA implementations would go down to
their knees.

The interface definitions can handle the following natively

- continous streams of data (such as audio data)
- event streams of data (such as midi events)
- real reference counting

and the most important CORBA gimmicks, like

- synchronous method invocations
- asynchronous method invocations
- constructing user defined data types
- multiple inheritance
- passing object references

=============================================================================
2. Implementation
=============================================================================

2.1. The MCOP message marshalling
=============================================================================

Design goals/ideas:

- marshalling should be easy to implement
- demarshalling requires the receiver to know what type he wants to
  demarshall
- the receiver is expected to use every information - so skipping is only
  in the protocol to a degree that

    if you know you are going to receive a block of bytes, you don't need
    to look at each byte for an end marker

    if you know you are going to receive a string, you don't need to read
    it 'til the zero byte to find out it's length while demarshalling

      however

    if you know you are going to receive a sequence of strings, you need
    to look at the length of each of them to find the end of the sequence,
    as strings have variable length. But if you use the strings for
    something useful, you'll need to do that anyway, so this is no loss.

- as little overhead as possible

Marshalling of the different types:

type  | marshalling process
------+----------------------------------------------------------------------
void    void types are marshalled by omitting them, so nothing is written
        to the stream for them

long    is marshalled as four bytes, the most significant byte first,
        so the number 10001025 (which is 0x989a81) would be marshalled as

        0x00 0x98 0x9a 0x81

enums   are marshalled like longs
         
byte    is marshalled as a single byte, so the byte 0x42 would be
        marshalled as

        0x42

string  is marshalled as a long, containing the length of the following
        string, and then the sequence of characters strings must end with
        one zero byte (which is included in the length counting).
            
        "hello" would be marshalled as

        0x00 0x00 0x00 0x06   0x68 0x65 0x6c 0x6c   0x6f 0x00
                       ^^^^
        important: include the trailing 0 byte in length counting!

boolean is marshalled as a byte, containing 0 if false or 1 if true, so
        the boolean value true is marshalled as

        0x01

float   is marshalled after the four byte IEEE754 representation - detailed
        docs how IEEE works are here:

		 * http://twister.ou.edu/workshop.docs/common-tools/numerical_comp_guide/ncg_math.doc.html 
		 * http://java.sun.com/docs/books/vmspec/2nd-edition/html/Overview.doc.html

        so the value 2.15 would be marshalled as

        0x9a 0x99 0x09 0x40

struct  a structure is marshalled by marshalling it's contents. There
        are no additional prefixes or suffixes required, so the structure

        struct test {
            string name;        // which is "hello"
            long value;         // which is 10001025  (0x989a81)
        };

        would be marshalled as

        0x00 0x00 0x00 0x06   0x68 0x65 0x6c 0x6c
        0x6f 0x00 0x00 0x98   0x9a 0x81

sequence  a sequence is marshalled by listing the number of elements that
        follow, and then marshalling the elements one by one.

        So a sequence of 3 longs a, with a[0] = 0x12345678, a[1] = 0x01
        and a[2] = 0x42 would be marshalled as

        0x00 0x00 0x00 0x03   0x12 0x34 0x56 0x78
        0x00 0x00 0x00 0x01   0x00 0x00 0x00 0x42

If you need to refer to a type, all primitive types are referred by the
names given above. Structures and enums get own names (like Header). Sequences
are referred as *<normal type>, so that a sequence of longs is "*long" and
a sequence of Header struct's is "*Header".

2.2 Messages
=============================================================================

The MCOP message header format is defined as defined by that structure

struct Header {
    long magic;          // the value 0x4d434f50, which is marshalled as MCOP
    long messageLength;
    long messageType;
};

The possible messageTypes are currently

 mcopServerHello		= 1
 mcopClientHello		= 2
 mcopAuthAccept			= 3
 mcopInvocation			= 4
 mcopReturn				= 5
 mcopOnewayInvocation   = 6

A few notes about the MCOP messaging:

- every message starts with a Header
- some messages types should be dropped by the server, as long as the
  authentication is not complete
- after receiving the header, the protocol (connection) handling can receive
  the message completely, without looking at the contents

The messageLength in the header is of course in some cases redundant,
which means that this approach is not minimal regarding the number of bytes.

However, it leads to an easy (and fast) implementation of non-blocking
messaging processing. With the help of the header, the messages can be
received by protocol handling classes in the background (non-blocking), if
there are many connections to the server, all of them can be served parallel.
You don't need to look at the message content, to receive the message (and
to determine when you are done), just at the header, so the code for that
is pretty easy.

Once a message is there, it can be demarshalled and processed in one single
pass, without caring about cases where not all data may have been received
(because the messageLength guarantees that everything is there).

2.3 Invocations
=============================================================================

To call a remote method, you need to send the following structure in the body
of an MCOP message with the messageType = 1 (mcopInvocation):

struct Invocation {
    long objectID;
    long methodID;
    long requestID;
};

after that, you send the parameters as structure, e.g. if you invoke the
method string concat(string s1, string s2), you send a structure like

struct InvocationBody {
    string s1;
    string s2;
};

if the method was declared to be oneway - that means asynchronous without
return code - then that was it. Otherwise, you'll receive as answer the
message with messageType = 2 (mcopReturn)

struct ReturnCode {
    long requestID;
    <resulttype> result;
};

where <resulttype> is the type of the result. As void types are omitted in
marshalling, you can also only write the requestID if you return from a
void method.

So our string concat(string s1, string s2) would lead to a returncode like

struct ReturnCode {
    long   requestID;
    string result;
};

2.4. Inspecting interfaces
=============================================================================

To do invocations, you need to know the methods and object supports. To do
so, the methodID 0, 1, 2 and 3 are hardwired to certain functionalities. That
is

long _lookupMethod(MethodDef methodDef);				// methodID always 0
string _interfaceName();								// methodID always 1
InterfaceDef _queryInterface(string name);				// methodID always 2
TypeDef _queryType(string name);						// methodID always 3

to read that, you of course need also

struct MethodDef {
	string  methodName;
	string  type;
	long    flags;        // set to 0 for now (will be required for streaming)
	sequence<ParamDef> signature;
};

struct ParamDef {
	string name;
	long   typeCode;
};

the parameters field contains type components which specify the types
of the parameters. The type of the returncode is specified in the MethodDef's
type field.

Strictly speaking, only the methods _lookupMethod() and _interfaceName()
differ from object to object, while the _queryInterface() and _queryType() 
are always the same.

=> What are those methodIDs?

If you do an MCOP invocation, you are expected to pass a number for the
method you are calling. The reason for that is, that numbers can be processed
much faster than strings when executing an MCOP request.

So how do you get those numbers? If you know the signature of the method,
that is a MethodDef that describes the method, (which contains name, type,
parameter names, parameter types and such), you can pass that to
_lookupMethod of the object where you wish to call a method. As _lookupMethod
is hardwired to methodID 0, you should encounter no problems doing so.

On the other hand, if you don't know the method signature, you can find which
methods are supported by using _interfaceName, _queryInterface and _queryType.

2.5. TypeDefinitions
=============================================================================

User defined datatypes are described using the TypeDef structure:

struct TypeComponent {
	string type;
	string name;
};

struct TypeDef {
	string name;

	sequence<TypeComponent> contents;
};
