Comparing IPC and capability invocation

[Volkmar Uhlig]

> -----Original Message-----
> From: Jonathan S. Shapiro [mailto:shap at eros-os.org] 
> Sent: Thursday, December 11, 2003 9:57 AM
> 
> By all means we should discuss, but let me attempt to clarify.
> 
> ALL communications are invocations on objects. The only questions that
> exist in principle are:
> 
>    1. What restrictions are imposed on the TYPE of object that can be
>       invoked?
> 
>    2. Is the object namespace extensible by user-mode code. That is,
>       can user-mode servers present objects or interfaces that appear
>       to the invoker to be "first class" in the same sense that kernel
>       supported objects are first class.
> 
> L4 imposes the restriction that the only invocable object type is
> "process" (or in some cases thread).

We only have threads, there is no notion of processes (I'm talking about
V4/X.2).


> L4 does *not* (today) provide means to allow a server to extend the
> object name space.

There is no reason, "objects" are user-managed things.  Allowing
extension doesn't bring you any benefit.  Transparency can be
implemented in a user-level library (without any overhead).  How I
understood your description is that you cache information about what
user object types you have in the kernel.  That costs you another check
on the critical path.

You argue that single method of invocation is a good thing--yes.  We
have that, it is called IPC.  

For all kernel objects (we have threads, AS, and memory) I don't see a
benefit of a unified interface.  That leads to IOCTL and that is a mess.
If you want to use a generic interface (i.e. IPC) to _all_ kernel
objects you can always place a protocol translation layer in between at
almost no cost (threads and AS are manipulated very infrequently).  This
allows for optimizations you can't perform otherwise.


> My first point is probably self-explanatory, so I will expand only on
> the second.
> 
> In L4, if a client wishes to perform an operation on a file, 
> the "name"
> of the file must be passed as an argument to an IPC. The invocation is
> something like:
> 
>    file_server->invoke(file-id, operation-id, ... other args ...)
> 
> Because "file" is not a kernel-supported object, the protocol mandates
> that the sender provide an additional argument in the IPC 
> invocation. In
> the EROS philosophy, we would argue that these objects are therefore
> "second class" and that this is bad for several reasons:
> 
>    1. The invoker should not know the server identity. That should
>       be known only to the file object.
>    2. It is difficult to transparently virtualize objects when their
>       invocation patterns are different.

1) Assume threads have zero costs.  That means you can have one thread
per object and the thread id is your identifier.

2) Assume you want to share a thread for multiple object invocations
(and threads have costs > 0).  The additional check you have to perform
on _every_ object invocation (code for checking, table access, I and
D-cache footprint etc) in EROS has probably a higher overhead than the
additional parameter on the invocation which is untouched by the kernel.
For concurrency (multiple worker threads) you can use LIPC in L4 so you
can perform load distribution in user land.

3) Virtualizing objects is only a question of protocols.  That is
something _userland_ defines.  So why should virtualization be
difficult??? (I'm puzzled).  Furthermore, how do you deal with situation
where your object space is full?  Is the object space local or global?  
We can dynamically extend and shrinl the object space (e.g. use a single
bit if we have only 2 object types).


> Next problem:
> 
> The server must then run some function:
> 
> 	get_permissions(sender-id, file-id) -> permissions
> 
> to determine what operations are permitted. Note that if this 
> operation
> is performed faithfully and correctly, it is impossible to emulate
> correctly the behavior of the UNIX I_SENDFD socket operation without
> many additional calls to a shared service -- the design of 
> the operation
> makes descriptor transfer an inherently expensive operation.

I would say that is a weak argument considering all the shortcomings of
the POSIX API.  Implementing fork within a distributed system is very
expensive--so what?  We know for more than 10 years that fork is broken.
I will look into I_SENDFD into more detail and try to give you a
satisfactory answer.

> Descriptors, which *can* be used as a foundation for certain kinds of
> security, suddenly become extremely inefficient because they cannot be
> passed without consulting a third party.

You can cache that information in user land after validation or use
shared memory with the authentication server.  That is what you
basically do in the kernel.


> The server-defined-bits portion cannot be examined by the 
> client. It is
> provided to the server during invocation. The server can 
> interpret these
> bits in any way desired: as an object id, as a facet id, as permission
> bits, as some mix of these.
> 
> The presence of these bits does not preclude invocation of the server
> qua server; the server merely assigns to itself an arbitrarily chosen
> value of "server-defined-bits" to name the server itself.

So you provide an in-kernel cache for some identifiers (call it bits)
which are unforgeable.  How much of your register real estate do you
give up for that?  What when the size is exceeded?


> Because these are architecturally insufficient to implement an
> efficient, secure, object-based operating system.

Hmm, actually that is a question of what you try to implement.  What if
you don't want an object-based OS?  Do you incur a significant overhead
with your model?  I'm curious how a Linux kernel would perform on top of
EROS--I could imagine that your security model has a measurable
overhead.  And then we have to start a discussion on generality of uKs.


> Our experience has been that relying on such clients to specify the
> intended operation is not robust. The flow of permissions in complex
> programs is not well localized, and it is very easy to write a
> subroutine designed for one purpose that does some mildly dangerous
> thing and then call it (by programmer error) in the middle of some
> sequence of code where care is required.
> 
> Tying permissions to the object descriptor does not prevent the
> programmer from passing the wrong descriptor, but it does help a great
> deal in localizing the scope of programmer attention that is 
> required to resolve these problems.

This sounds like you are suggesting kernel design based on bad
programming habits.  Are you willing to pay the overhead?  We don't.

- Volkmar