HURDNG : Which type of OS design could we have to think nowadays ?

[Ernst Rohlicek jun.]

Hello Guillaume, Hello list,


This is a long post and I hope to have made it somewhat interesting as 
well. Please stay with me :-)


 > Here is the start point of my reflexion :
 >
 > http://lists.gnu.org/archive/html/l4-hurd/2006-04/msg00053.html

In this post, Mr. Shapiro compares roads of action for a next-generation 
Hurd and starts out with:

     "This will sound strange coming from me, but it may turn out that
     HurdNG should build it's own microkernel. I do not think this is a
     good idea, but let me try to enumerate the alternatives and
     articulate the advantages and disadvantages as I see them."

His conclusions are true: Using something existing is great, PROVIDED it 
fits reasonably well - and it must fit almost the way it came. You may 
be able to retro-fit a few bits here and there, but that's it.

It is also my experience that if you try to change a program in 
fundamental ways, it's easier to redo it rather than trying to re-orient it.

Think of changing a KDE application into a GNOME application ... and the 
closer you get toward the kernel, the more work you have with 
re-orienting it.

You have translated your design into code (created an "instance" so to 
speak) and we all know that C code is not friends with large re-writes.


Another thing is that FAR too many people shy away from starting out 
anew. We're not a commercial company with time pressure.

We have learned from the last "iteration", know a lot of the issues and 
mistakes - plus, we have time. Instead of patching and re-orienting and 
hoping, we should enter the next iteration and put to use what we've 
learned.


Another snippet from the above message:

     "If, after you look at the alternatives, you still think that
     building or modifying a micro-kernel sounds good, try to assemble a
     team. If you can't, stop and look for another way.

     In my opinion, the L4-Hurd and mainstream Hurd communities simply do
     not have the (human) resources to build both a microkernel and an OS
     before most of this list has died of old age."

Yup, there's action required... if not, well ... read again about that 
age thing.

Just wanted to point that out...


 > I believe that we can do extremely complex and reliable software with
 > a few people, thanks to the powerful formal and design tools available
 > now.
 >
 > Time is maybe our worst ennemy : more and more technologies are
 > created day by day and we need to understand and to be able to use
 > them And it's maybe our better friend : as the hardware/software
 > becomes more and more complex, the tools to develop them becomes more
 > powerful  too.
 >
 > The tools developed to understand and create complex systems give us
 > the ability to design and implement an OS design secure & reliable.

You mention some interesting methods and utilities.

I've used formal design myself (IBM's Rational - great tool) and it 
really takes away a lot of burden. You can change some relationships in 
the design view and they get translated / "instantiated" right into 
code. Usually you have to just fill in the blanks and application logic.

I wonder if the things you mentioned can be used not just for modeling
specific types of applications and relationships, but for more generic 
(Hurd servers, maybe even kernel?) programs. Could you find out about that?

Rationale: In projects you often just HAVE to re-adapt the design of 
your program after iteration 1 ... and any way of quickly re-adapting 
the code to fit the new design is just GOLD.

That's in my opinion why there is things like re-factoring, 
meta-programming, formal and model-based design and lots of scripting 
(prototyping?) languages.

Would you implement a proof-of-concept or prototype rather in Python or 
in ANSI C ? With manual memory allocation, char arrays, buffers, 
pointers, Makefiles, autotools stuff...?

Guillaume, your idea is quite on the spot, I believe. I personally 
acknowledge that you can do great prototypes with formal design and 
high-level languages. Look over to the development of the seL4 / L4.sec 
kernel:

     http://ertos.nicta.com.au/research/sel4/tech.pml

They themselves use a machine simulator, which catches all kernel 
requests and hands them to the "kernel", which is developed outside of 
the simulator in the Haskell programming language.

I'm convinced that they're able to go through design and verification 
iterations so much faster. And besides, their specification = 
implementation looks almost the same as a model description from 
model-based engineering.

I guess that later, when the algorithms have been optimized and they 
found that they've developed a useful design for their requirements, 
they might actually implement it in C (though comparison shows that the 
Haskell GHC compiler also produces very fast code), search for 
"performance" or "slow" here:

     http://www.haskell.org/haskellwiki/Introduction

But that Haskell thing just as a side note.

It's important to point out that fast prototyping, fast design 
verification (and -adaption until we know, "This is it, we have it"), 
fast creation of code instances from a model or directly use a 
high-level language for initial design creation is almost a key point.


 > -The first one is the start of any important  project : developers can
 > have the possibility to share complex ideas with minimum requirements
 > and to integrate them in a coherent environment. ( see
 > http://www.gnu.org/software/hurd/ => 23 years old and even the goals
 > are not well defined )

That's an under-estimated point ... and yet a vital one.


 > -The second feature provide any user the ability to develop on it with
 > existing materials and to have the support of developers of legacy OS.

Yes, transition is important. Going too far out with GNU Hurd means no good.


 > -The third one needs that a device driver framework needs to be
 > rapidly  envisaged. ( again http://www.gnu.org/software/hurd/ => no
 > good driver  kit at this date, the same for linux :
 > http://www.kroah.com/log/2006/05/24/, 15 years later ). Who wants to
 > have an OS that boots only a black screen with a beep sound ? If we
 > follow the Moore law, an OS design of the future would be thought for
 > quadruple play ( mobility, audio, video, Internet ) and security of
 > course.

Yes, being able to fulfill use-cases is important.

Besides, any developer for a possibly next-gen OS wants to listen to 
some MP3s :-)


 > Abstraction will be the solution,

Putting more and more layers of abstraction between the hardware and the 
program logic usually is no good.

What exactly do you mean with abstraction? Or: What kind of abstraction 
do you mean?


> I believe that HurdNG had to focus one 2 things ( lol :-), easy to tell, 
> hard to design ) : security and performance. Security because all 
> devices are becoming more and more connected and performance because of 
> the Moore's law.

Security indeed is becoming more and more important; as as 
maintainability and modularity.


> *First option :
> 
> Should we support 386 and let the OS die with this obselete hardware or 
> think to the future and enjoy the power of 64 bits and multicore that 
> are becoming standards ?
> 
> The ideas have to go more far that  the present, if the HURDNG ( forget 
> now UNIX:-) ) want to be a good option/successor to Linux.
> 
> A Single Adress Space OS ( 64 bits = 256 Tb, we have time before 
> exploiting all this power => it would be moore easy to get rid of ACL 
> because of easier implementation of persistence) and a language/main 
> library thought for massive multithread (CSP or Concurrent/Parallel 
> Haskell => get rid of C/ maybe GNU/UNIX/POSIX too :-) ) .
> 
> http://www.torsion.org/
> 
> http://www.cs.dartmouth.edu/~dfk/papers/kotz:addrtrace.pdf
> 
> http://www.wotug.org/
> 
> http://www.eros-os.org/pipermail/e-lang/1999-November/003069.html
> 
> http://tunes.org/wiki/Orthogonal_Persistence

Persistence: Well, I'm not sure whether orthogonal persistence (in my 
interpretation this is quite similar to hiberation / suspend to disk) is 
really required - better: what it's benefits in a general-purpose OS for 
non-embedded CPUs are...

After all you can still store your caps on disk, if you would like to.


Design for old HW or for the future: I would not design a new OS for a 
machine you can't buy in stores anymore since 5 years (in 2001 we had 
Athlons and Pentium 3 / 4).

Rationale: If you say that a new OS will have a lifetime of 10 years, 
then you will think back and say, "Why the hell did they let themselves 
get restricted by CPUs, which were several generations behind - already 
back then?!" Looking from today's perspective: If you design a little 
bit "too big" or "far ahead" for today's measures (64-bits where 
possible, massively threaded and laid out for SMP, among many other 
decisions), then today's design might still be ... hm, well at least 
MAKE SENSE in 10 years.


> *Second option  :
> 
> Explore new ideas, maybe too innovative (Have to carefully choose/adapt 
> the language).
> 
> http://lists.gnu.org/archive/html/l4-hurd/2006-04/msg00053.html
> 
> I would add a fourth option : Get rid of a kernel :

Well, you need need some kind of authority. "Processes can all be equal, 
but some have to be more equal."

And the Exokernel idea is probably great for very specialized purposes, 
so remember that GNU/Hurd is supposed to be a general-use OS.

If I misunderstood, we can further discuss of course.


> http://www.tunes.org

"Reflective OS" sounds tempting, though I personally think this belongs
more into AI - intelligent, integrated assistants and the like.

Of course it's okay that you mention it - I guess we all honor your
right to bring new things to the table :-)


> "Isaac is an operating system purely made by dynamicaly interconnected 
> prototype object. Each macro-object run on top of the processor.
> 
>  IsaacOS is not an interface with a standard kernel.  IsaacOS is fully 
> made with only object. No file, no process, no kernel, just and only 
> object."
> 
> http://isaacos.loria.fr/index.html

I have yet to read up on "prototype-based" languages and this specific 
language here - hm, "based on Smalltalk", "uses GCC", "fast code like 
C", "used for writing operating systems", "

Generally, I believe that we don't have to use C anymore to write 
hardware-near programs - back then in the 70s, 80s and even still in the 
90s it was fine, but not today. There are fast alternatives with better 
facilities.

BTW, think back to the 70's when C was invented. Back then it must have 
been like, "Oh my god, this is high-tech - are you sure you want to use 
that new, crazy 'C' language for your OS?" ... why are we so shy to use 
something newer and safer than C? (Safer as in eg. type safety) There 
are alternatives are in place.

As motivation, here is a link to a WORKING implementation of Unix in 
C++, which is even binary compatible with Linux:

     http://www.unixlite.org/index.html

So it is possible.


As mentioned in my reply-paragraphs above, I suggest one of the 
following for a new kernel for GNU/Hurd - should seL4 / L4.sec and 
Coyotos be too far off the goals and requirements ... wait, WHAT goals 
and requirements BTW? -

But I digress.  ;-)

Anyway, in that case, I would suggest using one or more or a mix of the 
following roads:

  1. Using an intermediate prototyping language and/or some (needs TBD) 
model-based approach

  2. Using an intermediate high(er)-level language for prototyping and 
starting implementation in C once we're sure about a (for our purposes) 
complete and sound design

  3. Writing in a high(er)-level language which can be compiled into 
fast code (as fast like C), but which does not allow you to shoot 
yourself in the foot (too often) and which allows for relatively easy 
re-orientation of the code.


For any of that to work, we need to know (and then get clearer) about 
things like: Goals, ideas, what Hurd really could do better than others 
... how your computer(s) would work differently if it would be running 
your ideal version of Hurd, according to your ideas...?

What are your ideas BTW?


>                         Best Regards,
> 
>                                    Guillaume FORTAINE
> 

Likewise, thanks for the input.


--Ernst Rohlicek jun.