First: On - let's see - Fri, 28 Mar 2014 02:02:52 +0100 - I sent a message to this list with the subject "again, must know mechanics of compile" - previously, I was able to send to this list, but as for now, I can't find that message either in the group or in the web archive [1]. (I see my old posts though.) I looked for that message because I thought I build on that thread, that's how I noticed it wasn't there (?).
Anyway, as for including header files, I found that you can indeed make a directory at the base package level, called "include" - and put a Makefile in it just like it is done for the other projects (but I'm unsure what the Makefile actually does - perhaps if you specify the include directory as a target in the *package* Makefile, you can drop it?).
And, you can't use the C++ ".hh" extension, but ".h" works, if you include them like this:
#include <l4/hs/task_scheduler.h>
if your package is named "hs". In effect, this is quite the opposite of the #include "task_scheduler.hh", and then the -I flag to the compiler.
As for -std=c++11, I found that you can pass that not with CFLAGS, but close - this works:
CXXFLAGS += -std=c++11
[1] http://os.inf.tu-dresden.de/pipermail/l4-hackers/2014/thread.html