Devel::NYTProf v3 - 200908 (OUTDATED, see 201008)

Slides of my talk on Devel::NYTProf and optimizing perl code at the Italian Perl Workshop (IPW09). It covers the new features in NYTProf v3 and a new section outlining a multi-phase approach to optimizing your perl code.

7.
CPU Time vs Real Time
• CPU time
- Measures time CPU sent executing your code
- Not (much) affected by other load on system
- Doesn’t include time spent waiting for i/o etc.
• Real time
- Measures the elapsed time-of-day
- Your time is affected by other load on system
- Includes time spent waiting for i/o etc.
CPU: only useful if high resolution
Real: Most useful most of the time. Real users wait in real time!
You want to know about slow I/O, slow db queries, delays due to thread contention, locks etc. etc

8.
Sub vs Line
• Subroutine Proﬁling
- Measures time between subroutine entry and exit
- That’s the Inclusive time. Exclusive by subtraction.
- Reasonably fast, reasonably small data ﬁles
• Problems
- Can be confused by funky control ﬂow
- No insight into where time spent within large subs
- Doesn’t measure code outside of a sub
Funky: goto &sub, next/redo/last out of a sub, even exceptions

9.
Sub vs Line
• Line/Statement proﬁling
- Measure time from start of one statement to next
- Exclusive time (except includes built-ins & xsubs)
- Fine grained detail
• Problems
- Very expensive in CPU & I/O
- Assigns too much time to some statements
- Too much detail for large subs (want time per sub)
- Hard to get overall subroutine times

11.
v1 Innovations
• Fork by Adam Kaplan of Devel::FastProf
- working at the New York Times
• HTML report borrowed from Devel::Cover
• More accurate: Discounts proﬁler overhead
including cost of writing to the ﬁle
• Test suite!

12.
v2 Innovations
• Proﬁles time per block!
- Statement times can be aggregated
to enclosing block
and enclosing sub

13.
v2 Innovations
• Dual Proﬁlers!
- Is a statement proﬁler
- and a subroutine proﬁler
- At the same time

25.
Exclusive vs. Inclusive
• Exclusive Time = Bottom up
- Detail of time spent “just here”
- Where the time actually gets spent
- Useful for localized (peephole) optimisation
• Inclusive Time = Top down
- Overview of time spent “in and below”
- Useful to prioritize structural optimizations

26.
Overall time spent in and below this sub
(in + below)
Color coding based on
Median Average Deviation
relative to rest of this ﬁle Timings for each location calling into,
or out of, the subroutine

27.
Treemap showing relative
proportions of exclusive time
Boxes represent subroutines
Colors only used to show
packages (and aren’t pretty yet)
Hover over box to see details
Click to drill-down one level
in package hierarchy

28.
Calls between packages
Generates GraphViz ﬁles that can be used to produce these diagrams

36.
“More computing sins are committed in the
name of efﬁciency (without necessarily
achieving it) than for any other single
reason - including blind stupidity.”
- W.A. Wulf
• programmer time $ > cpu time $.
• likely to introduce bugs

37.
“We should forget about small efﬁciencies,
say about 97% of the time: premature
optimization is the root of all evil.
Yet we should not pass up our
opportunities in that critical 3%.”
- Donald Knuth

38.
“We should forget about small efﬁciencies,
say about 97% of the time: premature
optimization is the root of all evil.
Yet we should not pass up our
opportunities in that critical 3%.”
- Donald Knuth