RCU performance

[Christian Prochaska]

Hello,

when building Fiasco.OC with multiprocessor support enabled, the
unmapping of an IPC gate via 'l4_task_unmap()' appears to require at
least 3 timer interrupts to occur until completion, which can make a
noticable difference in overall execution time compared to the
uniprocessor configuration (extreme example:
https://github.com/genodelabs/genode/issues/712, in a more realistic
scenario (compiling an application) the difference was still like ~2
minutes execution time with the uniprocessor configuration vs. ~6
minutes with the multiprocessor configuration).

I've attached the debug output I got while trying to understand what's
going on and which shows some of the RCU-related activities after each
of the 3 timer interrupts (patch used for the debug messages:
https://github.com/cproc/foc/commit/fa7b6f823cb29c0fc1db389ceb956d554d9e1546.patch).

According to these messages, the quiescent period had ended after the
second timer interrupt and I wonder if it is really necessary to wait
for a third timer interrupt before executing the corresponding
callbacks. Would it work correctly with the attached patch, too?

Christian

-------------- next part --------------
*** userland ***

void Genode::Capability_map::remove(Genode::Cap_index*): calling l4_task_unmap()

*** kernel ***

Reap_list::del(): calling rcu_wait()
Context::rcu_wait(): adding 'rcu_unblock()' callback to _n list
Context::rcu_wait(): calling schedule()
idle_op()
Thread::handle_timer_interrupt(): calling Rcu::do_pending_work(0)
Rcu::do_pending_work(): calling process_callbacks(0)
Rcu_data::process_callbacks(): _n.empty() = 0, _c.empty() = 1, _d.empty() = 1, rgp->_completed = -298, CPU-local batch number = -298
Rcu_data::process_callbacks(): moving callbacks from _n list to _c list
Rcu_data::process_callbacks(): increased CPU-local batch number to -297
Rcu_glbl::start_batch(): increased current global batch number to -297
Rcu_data::check_quiescent_state(): _q_batch = -298
Rcu_data::check_quiescent_state(): _q_batch != rgp->_current, starting new grace period
Rcu_data::check_quiescent_state(): _q_batch increased to -297
Rcu_data::process_callbacks(): check_quiescent_state() returned, _n.empty() = 1, _c.empty() = 0, _d.empty() = 1, rgp->_completed = -298, CPU-local batch number = -297
Thread::handle_timer_interrupt(): resched = 0
idle_op()
Thread::handle_timer_interrupt(): calling Rcu::do_pending_work(0)
Rcu::do_pending_work(): calling process_callbacks(0)
Rcu_data::process_callbacks(): _n.empty() = 1, _c.empty() = 0, _d.empty() = 1, rgp->_completed = -298, CPU-local batch number = -297
Rcu_data::check_quiescent_state(): _q_batch = -297
Rcu_data::check_quiescent_state(): quiescent state passed, calling rgp->cpu_quiet()
Rcu_glbl::cpu_quiet(): clearing cpu 0
Rcu_glbl::cpu_quiet(): all CPUs clear, quiescent period has ended
Rcu_glbl::cpu_quiet(): global _completed is now -297
Rcu_data::process_callbacks(): check_quiescent_state() returned, _n.empty() = 1, _c.empty() = 0, _d.empty() = 1, rgp->_completed = -297, CPU-local batch number = -297
Thread::handle_timer_interrupt(): resched = 0
idle_op()
Thread::handle_timer_interrupt(): calling Rcu::do_pending_work(0)
Rcu::do_pending_work(): calling process_callbacks(0)
Rcu_data::process_callbacks(): _n.empty() = 1, _c.empty() = 0, _d.empty() = 1, rgp->_completed = -297, CPU-local batch number = -297
Rcu_data::process_callbacks(): calling _d.append(_c)
Rcu_data::check_quiescent_state(): _q_batch = -297
Rcu_data::process_callbacks(): check_quiescent_state() returned, _n.empty() = 1, _c.empty() = 1, _d.empty() = 0, rgp->_completed = -297, CPU-local batch number = -297
Rcu_data::process_callbacks(): calling do_batch()
Context::rcu_unblock()
Thread::handle_timer_interrupt(): resched = 1
Thread::handle_timer_interrupt(): calling schedule()

*** userland ***

void Genode::Capability_map::remove(Genode::Cap_index*): l4_task_unmap() returned

-------------- next part --------------
A non-text attachment was scrubbed...
Name: rcu.diff
Type: text/x-patch
Size: 726 bytes
Desc: not available
URL: <http://os.inf.tu-dresden.de/pipermail/l4-hackers/attachments/20130411/3783f3e1/attachment.bin>