Hello Guillaume, Hello list,
This is a long post and I hope to have made it somewhat interesting as well. Please stay with me :-)
Here is the start point of my reflexion :
http://lists.gnu.org/archive/html/l4-hurd/2006-04/msg00053.html
In this post, Mr. Shapiro compares roads of action for a next-generation Hurd and starts out with:
"This will sound strange coming from me, but it may turn out that HurdNG should build it's own microkernel. I do not think this is a good idea, but let me try to enumerate the alternatives and articulate the advantages and disadvantages as I see them."
His conclusions are true: Using something existing is great, PROVIDED it fits reasonably well - and it must fit almost the way it came. You may be able to retro-fit a few bits here and there, but that's it.
It is also my experience that if you try to change a program in fundamental ways, it's easier to redo it rather than trying to re-orient it.
Think of changing a KDE application into a GNOME application ... and the closer you get toward the kernel, the more work you have with re-orienting it.
You have translated your design into code (created an "instance" so to speak) and we all know that C code is not friends with large re-writes.
Another thing is that FAR too many people shy away from starting out anew. We're not a commercial company with time pressure.
We have learned from the last "iteration", know a lot of the issues and mistakes - plus, we have time. Instead of patching and re-orienting and hoping, we should enter the next iteration and put to use what we've learned.
Another snippet from the above message:
"If, after you look at the alternatives, you still think that building or modifying a micro-kernel sounds good, try to assemble a team. If you can't, stop and look for another way.
In my opinion, the L4-Hurd and mainstream Hurd communities simply do not have the (human) resources to build both a microkernel and an OS before most of this list has died of old age."
Yup, there's action required... if not, well ... read again about that age thing.
Just wanted to point that out...
I believe that we can do extremely complex and reliable software with a few people, thanks to the powerful formal and design tools available now.
Time is maybe our worst ennemy : more and more technologies are created day by day and we need to understand and to be able to use them And it's maybe our better friend : as the hardware/software becomes more and more complex, the tools to develop them becomes more powerful too.
The tools developed to understand and create complex systems give us the ability to design and implement an OS design secure & reliable.
You mention some interesting methods and utilities.
I've used formal design myself (IBM's Rational - great tool) and it really takes away a lot of burden. You can change some relationships in the design view and they get translated / "instantiated" right into code. Usually you have to just fill in the blanks and application logic.
I wonder if the things you mentioned can be used not just for modeling specific types of applications and relationships, but for more generic (Hurd servers, maybe even kernel?) programs. Could you find out about that?
Rationale: In projects you often just HAVE to re-adapt the design of your program after iteration 1 ... and any way of quickly re-adapting the code to fit the new design is just GOLD.
That's in my opinion why there is things like re-factoring, meta-programming, formal and model-based design and lots of scripting (prototyping?) languages.
Would you implement a proof-of-concept or prototype rather in Python or in ANSI C ? With manual memory allocation, char arrays, buffers, pointers, Makefiles, autotools stuff...?
Guillaume, your idea is quite on the spot, I believe. I personally acknowledge that you can do great prototypes with formal design and high-level languages. Look over to the development of the seL4 / L4.sec kernel:
http://ertos.nicta.com.au/research/sel4/tech.pml
They themselves use a machine simulator, which catches all kernel requests and hands them to the "kernel", which is developed outside of the simulator in the Haskell programming language.
I'm convinced that they're able to go through design and verification iterations so much faster. And besides, their specification = implementation looks almost the same as a model description from model-based engineering.
I guess that later, when the algorithms have been optimized and they found that they've developed a useful design for their requirements, they might actually implement it in C (though comparison shows that the Haskell GHC compiler also produces very fast code), search for "performance" or "slow" here:
http://www.haskell.org/haskellwiki/Introduction
But that Haskell thing just as a side note.
It's important to point out that fast prototyping, fast design verification (and -adaption until we know, "This is it, we have it"), fast creation of code instances from a model or directly use a high-level language for initial design creation is almost a key point.
-The first one is the start of any important project : developers can have the possibility to share complex ideas with minimum requirements and to integrate them in a coherent environment. ( see http://www.gnu.org/software/hurd/ => 23 years old and even the goals are not well defined )
That's an under-estimated point ... and yet a vital one.
-The second feature provide any user the ability to develop on it with existing materials and to have the support of developers of legacy OS.
Yes, transition is important. Going too far out with GNU Hurd means no good.
-The third one needs that a device driver framework needs to be rapidly envisaged. ( again http://www.gnu.org/software/hurd/ => no good driver kit at this date, the same for linux : http://www.kroah.com/log/2006/05/24/, 15 years later ). Who wants to have an OS that boots only a black screen with a beep sound ? If we follow the Moore law, an OS design of the future would be thought for quadruple play ( mobility, audio, video, Internet ) and security of course.
Yes, being able to fulfill use-cases is important.
Besides, any developer for a possibly next-gen OS wants to listen to some MP3s :-)
Abstraction will be the solution,
Putting more and more layers of abstraction between the hardware and the program logic usually is no good.
What exactly do you mean with abstraction? Or: What kind of abstraction do you mean?
I believe that HurdNG had to focus one 2 things ( lol :-), easy to tell, hard to design ) : security and performance. Security because all devices are becoming more and more connected and performance because of the Moore's law.
Security indeed is becoming more and more important; as as maintainability and modularity.
*First option :
Should we support 386 and let the OS die with this obselete hardware or think to the future and enjoy the power of 64 bits and multicore that are becoming standards ?
The ideas have to go more far that the present, if the HURDNG ( forget now UNIX:-) ) want to be a good option/successor to Linux.
A Single Adress Space OS ( 64 bits = 256 Tb, we have time before exploiting all this power => it would be moore easy to get rid of ACL because of easier implementation of persistence) and a language/main library thought for massive multithread (CSP or Concurrent/Parallel Haskell => get rid of C/ maybe GNU/UNIX/POSIX too :-) ) .
http://www.cs.dartmouth.edu/~dfk/papers/kotz:addrtrace.pdf
http://www.eros-os.org/pipermail/e-lang/1999-November/003069.html
Persistence: Well, I'm not sure whether orthogonal persistence (in my interpretation this is quite similar to hiberation / suspend to disk) is really required - better: what it's benefits in a general-purpose OS for non-embedded CPUs are...
After all you can still store your caps on disk, if you would like to.
Design for old HW or for the future: I would not design a new OS for a machine you can't buy in stores anymore since 5 years (in 2001 we had Athlons and Pentium 3 / 4).
Rationale: If you say that a new OS will have a lifetime of 10 years, then you will think back and say, "Why the hell did they let themselves get restricted by CPUs, which were several generations behind - already back then?!" Looking from today's perspective: If you design a little bit "too big" or "far ahead" for today's measures (64-bits where possible, massively threaded and laid out for SMP, among many other decisions), then today's design might still be ... hm, well at least MAKE SENSE in 10 years.
*Second option :
Explore new ideas, maybe too innovative (Have to carefully choose/adapt the language).
http://lists.gnu.org/archive/html/l4-hurd/2006-04/msg00053.html
I would add a fourth option : Get rid of a kernel :
Well, you need need some kind of authority. "Processes can all be equal, but some have to be more equal."
And the Exokernel idea is probably great for very specialized purposes, so remember that GNU/Hurd is supposed to be a general-use OS.
If I misunderstood, we can further discuss of course.
"Reflective OS" sounds tempting, though I personally think this belongs more into AI - intelligent, integrated assistants and the like.
Of course it's okay that you mention it - I guess we all honor your right to bring new things to the table :-)
"Isaac is an operating system purely made by dynamicaly interconnected prototype object. Each macro-object run on top of the processor.
IsaacOS is not an interface with a standard kernel. IsaacOS is fully made with only object. No file, no process, no kernel, just and only object."
I have yet to read up on "prototype-based" languages and this specific language here - hm, "based on Smalltalk", "uses GCC", "fast code like C", "used for writing operating systems", "
Generally, I believe that we don't have to use C anymore to write hardware-near programs - back then in the 70s, 80s and even still in the 90s it was fine, but not today. There are fast alternatives with better facilities.
BTW, think back to the 70's when C was invented. Back then it must have been like, "Oh my god, this is high-tech - are you sure you want to use that new, crazy 'C' language for your OS?" ... why are we so shy to use something newer and safer than C? (Safer as in eg. type safety) There are alternatives are in place.
As motivation, here is a link to a WORKING implementation of Unix in C++, which is even binary compatible with Linux:
http://www.unixlite.org/index.html
So it is possible.
As mentioned in my reply-paragraphs above, I suggest one of the following for a new kernel for GNU/Hurd - should seL4 / L4.sec and Coyotos be too far off the goals and requirements ... wait, WHAT goals and requirements BTW? -
But I digress. ;-)
Anyway, in that case, I would suggest using one or more or a mix of the following roads:
1. Using an intermediate prototyping language and/or some (needs TBD) model-based approach
2. Using an intermediate high(er)-level language for prototyping and starting implementation in C once we're sure about a (for our purposes) complete and sound design
3. Writing in a high(er)-level language which can be compiled into fast code (as fast like C), but which does not allow you to shoot yourself in the foot (too often) and which allows for relatively easy re-orientation of the code.
For any of that to work, we need to know (and then get clearer) about things like: Goals, ideas, what Hurd really could do better than others ... how your computer(s) would work differently if it would be running your ideal version of Hurd, according to your ideas...?
What are your ideas BTW?
Best Regards, Guillaume FORTAINE
Likewise, thanks for the input.
--Ernst Rohlicek jun.