On Thu, 2004-01-01 at 02:56, Rudy Koot wrote:
> Some comments and questions to about 5 different e-mails:
Some answers, as best I can. :-)
> ===== Rudy
>
> Some of Volkmar's arguments are based on the fact that L4 is a general
> purpose microkernel, I think the discussion would be more useful if we look
> at how a microkernel with the purpose of running a capability based system
> on top, should be design. Leaving the question wheter those capabilities
> should be implemented at kernel-level or user-level open.
We already have that kernel. It's called EROS. While I do know how to
build a variant that is much closer in spirit to L4 (i.e. smaller -- we
designed such a nucleus at IBM) I do not believe that its end to end
performance will be any better.
However, I have been very carefully avoiding the direction you identify
for three reasons. The first is that I deeply respect the L4 work. The
second is that I attempted a successor to L4 after Jochen left IBM and
it became a bit sensitive. The third is that the L4 group has not chosen
to pursue capability-based design, and I am not interested in
redirecting their work.
At this time, what I am trying to achieve is a common nucleus on which
both teams can pursue their research. This will facilitate
cross-fertilization of ideas, and I believe that this will benefit both
groups. The plain fact is that the L4 and EROS communities can
successfully proceed in independent directions, each lacking a critical
mass of people. I do not believe that this is the most productive way to
proceed.
And I should acknowledge that I disagree with Volkmar very fundamentally
about the "performance uber alles" assumption. Performance is important,
but from a research perspective it is *more* important to understand
what the fundamental architectural issues are. Once we do, we can step
back and decide what to take out. At the moment, we have no common base
on which direct comparison is possible.
I don't think that the L4 kernel has all the answers, but I also think
that the EROS kernel does not have all the answers. Each, I think, has
important strengths. EROS lacks a certain elegance of minimality. L4 (as
I understand it today) lacks a credible story about security, access
control and denial of service. Perhaps it is time for both groups to
step back and make a serious attempt to learn from each other.
> ===== Rudy
>
> Any form of IPC in which the decission to allow or disallow a thread to send
> a message to another thread is vulnerable to a DoS attack. If the decission
> is mad by the kernel the IPC can be canceled immediately, but if it is made
> at user level another thread is blocked for a small amount of time.
This is exactly true, but it fails to take into account the fact that
IPC indirection can possibly suppress this behavior. I do not understand
the current IPC indirection proposal, and trying to reason without
understanding it is very slippery. Also impolite. Something I am known
for, but I do try not to do it on purpose. :-)
>From the conversations on this list, however, it appears to me that the
L4 research groups do not agree universally on what the IPC control
mechanism is, and this may be contributing to some difficulty in the
discussion. One group clearly advocates thread spaces. Volkmar (if I
understand him) currently advocates IPC indirection. The two designs
have very different implications for performance, implementation, and
security.
> ===== Shap
>
> > b) For some types of systems (capability systems), disclosure of
> > the sender id is an absolute violation of design requirements,
> > so any microkernel that relies exclusively on server-side
> > security checks based on sender-id is not a universal microkernel.
>
> Sorry, but you still owe me an example of a situation in which revealing the
> sender-id actually breaks the security of the system. The only thing I could
> think of was that it might reveal some details about the internal structure
> of an application, but if the system is secure that information hasn't got
> much use to a malicious program or user.
>From the perspective of security you are reasoning backwards. There is
never a mandate to justify withholding information. The mandate is
always the other way: if you wish to argue that a sender-id is
desirable, you must justify why its disclosure is safe and also why it
is required.
Observation: it is not, and it is not.
In answer to your question, here are some problems with disclosing
sender ID:
1. The one you mentioned: it discloses client structure, which is
something that the server is not entitled to know.
2. It mandates a client structure, because using the sender-id as
a credential strongly discourages the application developer
from breaking their applications into multiple address spaces
and separate threads. This defeats robustness and security both.
3. It is inaccurate. The server can NEVER rely on the sender-id,
because it cannot know if the immediate sender is acting in
proxy for someone else. That is, sender-id can only provide
false security.
Personally, I find (2) very ironic. Ultimately, the main justification
for fast IPC is that it makes decomposed applications easier. I find it
curious, given this, that L4 adopts an access checking model that
defeats decomposition. I also find it very curious that no L4-based
system (at least, none that I know about) has really pushed for
fine-grain decomposition. Perhaps Mungi did and I do not understand it
well enough.
As a backwards compatibility matter, and as a validation of the L4
nucleus, running Linux on an L4 is interesting. As a matter of future
research, and as a matter of forward-looking system architecture, it is
boring. The right forward-looking research question is:
*Given* a fast nucleus in the style of L4, what is the most
effective way to structure a native operating environment?
What fundamental and novel leverage do such kernels provide?
That is: do fast IPC and microkernels provide anything that is
architecturally important, or are they just clever tricks that are good
only for conference papers? Obviously, I believe that they are useful.
The problem I see is that *none* of our groups are big enough to
successfully evaluate this question of "what native environment?" It
requires a truly awesome amount of work.
> I really think this is an important point. If revealing the sender-id is
> indeed a violation of security, there wouldn't be a need to discuss wheter
> or not thread-spaces need to be implemented. They just have to.
>
> > c) More specifically, any microkernel that requires checks based
> > on sender-id is entirely unsuitable as a platform for EROS-NG.
Because (a) revealing sender-id is a security hole, and (b) EROS is a
capability system. The latter is a matter of definition.
> >Assume you add 2 more TLB entries and 5 more L2 cache lines--your
> >aftercosts for IPC go up by 2*80 + 5*80 = 560 cycles.
> >Considering overall IPC costs of 1000 cycles on a P4 with all those
> >nasty cache and TLB flushes you add an overhead of >50%.
>
> I believe these costs are exagerated. To convert a capability into sender-id
> a simple table lookup can be used. This requires a single TLB entry and a
> single L2 cache line (worst case). If IPC is frequent, which is the case in
> which you want high performance, the TLB entry is very likely to be present
> and probably even the L2 cache line.
Volkmar is combining two things in one, and seems to be forgetting that
he does so. You are correct. Once you have the capability, obtaining the
target thread id is quite fast. No table lookup is required. The
capability is kernel-protected state, and can simply contain a direct
pointer to the recipient PCB.
I think that the costs that Volkmar is identifying come from the need to
lookup the thread-id or capability in the thread mapping space. That is,
these are the costs of the address space traversal. The costs are the
same in either case (thread-id or capability).
If the kernel implementation uses flexpages, then I believe that in the
usual case the reality will be only one TLB miss, but the number of L2
references seems plausible and problematic.
In practice, I would use a cache for an L1 software probe first in order
to avoid all marginal TLB and cache misses in the usual case, but this
would result in high variance of IPC times.
> >In our case we use an object identifier in the message, which is a
> >handle to an object descriptor and do a reverse check (i.e. if the
> >thread is allowed to invoke that object). Costs: boundary check (can be
> >a simple AND) one MOV and a CMP. The permissions can go into the same
> >cache line.
Volkmar's technique is quite elegant. The only problem I see is that it
places the burden of verification on the wrong party -- it should be on
the client. That is, the design invites DoS attacks.
However, Volkmar is making an assumption that is problematic. He is
assuming that the passed ID is a pointer. From a security perspective
this is VERY bad. First, it invites memory attacks. Second, it discloses
information about the internal implementation of the server. Third, it
makes selective rescind of authority very hard to do.
One of the features of EROS is an indirection object. This object can
stand in front of a start capability transparently. The client invokes
the indirection object capability rather than the start capability.
Given this construction, a service can selectively rescind that client
capability by destroying the indirection object.
This is something that L4 presently has no means to do. Of course, L4
does not need it, because it makes no assumption about controlling
client invocations at all.
> >A possibility you did not mention is a hybrid thread id which has an
> >thread and a descriptor part. The descriptor is kernel enforced. That
> >is what we currently have in V4 (please not that V2 and X.0 are
> >completely outdated!!!)--the version part of the thread id.
>
> Sorry, but I do not see how the version part could be used as the server
> defined word. You can't have multiple versions (read server defined words)
> of the same thread-id at the same time.
I don't either, but I don't understand what Volkmar is proposing enough
to comment intelligently. I would like to, and I hope he will respond
with a pointer to a specification or more detail about where these
references are occuring. I am not sure that I have understood him
correctly.
shap
