enqueue and dequeue

When a CPU becomes idle, the dispatcher looks for work enqueued on other
(non-idle) CPUs. The following example uses the dequeue probe
to understand how often applications are transferred and by which CPU:

Instead of knowing which CPUs took which work, you might want to know
the CPUs on which processes and threads are waiting to run. You can use the enqueue and dequeue probes together to answer
this question:

The output is roughly what you would expect for an idle system: the
majority of the time that a runnable thread is enqueued, the run queue was
very short (three or fewer threads in length). However, given that the system
was largely idle, the exceptional data points at the bottom of the table might
be unexpected. For example, why was the run queue as long as 13 runnable threads?
To explore this question, you could write a D script that displays the contents
of the run queue when the length of the run queue is long. This problem is
complicated because D enablings cannot iterate over data structures, and therefore
cannot simply iterate over the entire run queue. Even if D enablings could
do so, you should avoid dependencies on the kernel's internal data structures.

For this type of script, you would enable the enqueue and dequeue probes and use both speculations and associative arrays.
Whenever a thread is enqueued, the script increments the length of the queue
and records the timestamp in an associative array keyed by the thread. You
cannot use a thread-local variable in this case because a thread might be
enqueued by another thread. The script then checks to see if the queue length
exceeds the maximum. If it does, the script starts a new speculation, and
records the timestamp and the new maximum. Then, when a thread is dequeued,
the script compares the enqueue timestamp to the timestamp of the longest
length: if the thread was enqueued before the timestamp
of the longest length, the thread was in the queue when the longest length
was recorded. In this case, the script speculatively traces the thread's information.
Once the kernel dequeues the last thread that was enqueued at the timestamp
of the longest length, the script commits the speculation data. This script
is shown below:

The output reveals that the long run queues are due to many runnable xterm processes. This experiment coincided with a change in virtual
desktop, and therefore the results are probably due to some sort of X event
processing.