oprofile-list

Hi oprofilers,
I'm having trouble with op_to_source (both 0.0.9 and a current cvs version)
and am looking for help.
Briefly, I'm trying to op_to_source a shlib, and it only annotates 2 files
of a couple hundred which went into the shlib, dozens of which had funcs
show up in oprofpp output with hundreds or thousands of samples.
oprofpp sure seems to know a log about the library:
[1895]oprofpp -demangle -l /proj/libzz.so | wc
6617 64531 816959
but when I try to run op_to_source, it only processes two files, and
doesn't really give any useful info. It does seem to recognize those two
files and copies over the source correctly and annotate them slightly, so I
don't suspect a path or lib name problem (plus, the oprofpp output seems
completely reasonable).
[1896]op_to_source -d -i /proj/libzz.so --source-dir $SOURCEDIR --output-dir /tmp/grg/oprof-annotated-source
[1897]ls /tmp/grg/oprof-annotated-source
./ ../ note.cpp debug.cpp
[1898]diff note.cpp $SOURCEDIR | wc
8 22 127
[1899]diff note.cpp $SOURCEDIR
1,5d0
< /*
< Total samples for file : "/proj/note.cpp"
< 3 0.06506%
< */
<
29d23
< /* 3 0.06506% */
[1900]diff debug.cpp $SOURCEDIR | wc
8 27 194
[1901]diff debug.cpp $SOURCEDIR
1,7d0
< /*
< Total samples for file : "/proj/debug.cpp"
< 1914 41.51%
< */
<
< /* Repository<Data>::Length(int) 1 0.02169% */
< /* 1914 41.51% */
[1902]
Also, that the one symbol op_to_source named in debug.cpp isn't actually in
that file or referenced by anything anywhere 'below' that file.
Again, oprofpp does seem to correctly show many symbols which are in many
different files in that same source dir. Shouldn't I expect to see many of
them get annotated by op_to_source?
Perhaps I'm just using the wrong command-line features? I have tried to
play with them (e.g. adding "-w 0") with no change, and this general format
does seem to work for me on simple executables. The behavior is the same
for 0.0.9 and the cvs head.
thanks for any suggestions and assistance!
--grg

From: "Greg Galperin" <grg22@...>
Sent: Saturday, March 16, 2002 3:25 AM
> Hi oprofilers,
hi,
> I'm having trouble with op_to_source (both 0.0.9 and a current cvs
version)
> and am looking for help.
>
> Briefly, I'm trying to op_to_source a shlib, and it only annotates 2 files
> of a couple hundred which went into the shlib, dozens of which had funcs
> show up in oprofpp output with hundreds or thousands of samples.
>
> oprofpp sure seems to know a log about the library:
>
> [1895]oprofpp -demangle -l /proj/libzz.so | wc
> 6617 64531 816959
this count all symbols even those w/o any samples neither debug info,
op_to_source deals only with symbols wiich receive samples and
have debug information.
> but when I try to run op_to_source, it only processes two files, and
> doesn't really give any useful info. It does seem to recognize those two
> files and copies over the source correctly and annotate them slightly, so
I
> don't suspect a path or lib name problem (plus, the oprofpp output seems
> completely reasonable).
are you sure all object file used to build the shared libs contains
debug info ? oprofpp -l does not need debugs info but op_to_source
needs them. Try objdump -d -S on the shared libs and look for
imbeded source inside assembly.
> [1896]op_to_source -d -i /proj/libzz.so --source-dir
> $SOURCEDIR --output-dir /tmp/grg/oprof-annotated-source
[snip diff annotated source/source show small difference]
>
> Also, that the one symbol op_to_source named in debug.cpp isn't actually
> in that file or referenced by anything anywhere 'below' that file.
perhaps a corner case like an inline function or a template
instantation invoked through an implicit call to a base class
ctor. What is the symbol name?
> Again, oprofpp does seem to correctly show many symbols which are in many
> different files in that same source dir. Shouldn't I expect to see many
> of them get annotated by op_to_source?
only if debug info are present, in this case if op_to_source can't open
the source file for any reason it should warn else it fails silently (debug
info missing are a quite common case and so on op_to_source does
not complain for symbol w/o debug info)
> Perhaps I'm just using the wrong command-line features? I have tried to
> play with them (e.g. adding "-w 0") with no change, and this general
> format does seem to work for me on simple executables. The behavior
> is the same for 0.0.9 and the cvs head.
nope plain op_to_source w/o any options should shows all source
wich contains at least one sample.
As a last hint, I commited (03-13) a fix against a problem with
gcc-2.95.3 and debug info.
regards,
Philippe Elie

Philippe Elie wrote:
> > Briefly, I'm trying to op_to_source a shlib, and it only annotates 2 files
> > of a couple hundred which went into the shlib, dozens of which had funcs
> > show up in oprofpp output with hundreds or thousands of samples.
> >
> > oprofpp sure seems to know a log about the library:
> >
> > [1895]oprofpp -demangle -l /proj/libzz.so | wc
> > 6617 64531 816959
>
> this count all symbols even those w/o any samples neither debug info,
> op_to_source deals only with symbols wiich receive samples and
> have debug information.
...
> are you sure all object file used to build the shared libs contains
> debug info ? oprofpp -l does not need debugs info but op_to_source
> needs them. Try objdump -d -S on the shared libs and look for
> imbeded source inside assembly.
OK, good questions; there are 271 symbols with nonzero counts, some
with many thousands of counts:
[2150]oprofpp -demangle -l /prob/libzz.so | awk '$2+0>0' | wc -l
271
[2151]oprofpp -demangle -l /prob/libzz.so | awk '$2+0>100' | wc -l
113
[2152]oprofpp -demangle -l /prob/libzz.so | awk '$2+0>1000' | wc -l
55
[2153]oprofpp -demangle -l /prob/libzz.so | awk '$2+0>10000' | wc -l
14
[2154]oprofpp -demangle -l /prob/libzz.so | awk '$2+0>100000' | wc -l
1
[2155]
The .o files used to build the shlib were all compiled with "g++ -g3",
i.e., maximal amount of debug info. Yes, "objdump -d -S" on any of
the .o files does show embedded source code (including comments and
preprocessor directives!).
> > Also, that the one symbol op_to_source named in debug.cpp isn't actually
> > in that file or referenced by anything anywhere 'below' that file.
>
> perhaps a corner case like an inline function or a template
> instantation invoked through an implicit call to a base class
> ctor. What is the symbol name?
Probably not. The file debug.ccp literally only has one line of code
which defines a global variable. It includes one header file which in
turn includes 3 /usr/include files and defines the DebugOstream class.
Nowhere does any of this mention the symbol
< /* Repository<Data>::Length(int) 1 0.02169% */
< /* 1914 41.51% */
which shows up in the op_to_source'd file (that symbol is defined
somewhere compeletely different).
> As a last hint, I commited (03-13) a fix against a problem with
> gcc-2.95.3 and debug info.
OK, I just checked out, recompiled, reinstalled oprofile. Now I get 2
more files copied into the output directory, but those 4 files are
still relatively uninteresting files and still none of them really
have any useful annotations.
I'm using gcc-2.95.2 for compilation with -g3 (is that a problem?),
kernel 2.4.18.
thanks!
--grg

From: "Greg Galperin" <grg22@...>
To: <ph_e@...>
> OK, good questions; there are 271 symbols with nonzero counts, some
> with many thousands of counts:
>
> The .o files used to build the shlib were all compiled with "g++ -g3",
> i.e., maximal amount of debug info. Yes, "objdump -d -S" on any of
> the .o files does show embedded source code (including comments and
> preprocessor directives!).
I means objdump -d -S on the shlib itself, not on the object files
try it, if you see source file can you send the unstripped binary
and the samples files (tgz please) ? Do not send it to the list
if the compressed size is bigger than 200 Ko
> < /* Repository<Data>::Length(int) 1 0.02169% */
> < /* 1914 41.51% */
>
> which shows up in the op_to_source'd file (that symbol is defined
> somewhere compeletely different).
ok, I'll look at this also.
> I'm using gcc-2.95.2 for compilation with -g3 (is that a problem?),
> kernel 2.4.18.
I never tried with 2.95.2 but it would work, and if it don't work perhaps
a work-around exists ...
regards,
Phil

Philippe Elie wrote:
> I means objdump -d -S on the shlib itself, not on the object files
yes, tons of source code in the shlib too. (e.g. counted 12,600 lines
with // commments. saw code from dozens of different files.)
> try it, if you see source file can you send the unstripped binary
> and the samples files (tgz please) ? Do not send it to the list
> if the compressed size is bigger than 200 Ko
shlib is 85MB, compresses down to 12MB. sources contain over 100Klines
of source code... I'll spare the list. :)
My initial purpose was to check that I wasn't doing anything blatantly
wrong. I have previously successfully oprofiled exes in this
environment; I guess I'll spend some time now to see if I can tear
this down to make a simple reproducer of this problem.
thanks,
grg

From: "Greg Galperin" <grg22@...>
Sent: Saturday, March 16, 2002 10:48 PM
> shlib is 85MB, compresses down to 12MB. sources contain over 100Klines
> of source code... I'll spare the list. :)
ok
> My initial purpose was to check that I wasn't doing anything blatantly
> wrong. I have previously successfully oprofiled exes in this
> environment; I guess I'll spend some time now to see if I can tear
> this down to make a simple reproducer of this problem.
really thanks to not follow blindly my advise to send the whole
things :)
regards,
Philippe Elie

On Sat, Mar 16, 2002 at 04:48:09PM -0500, Greg Galperin wrote:
> > try it, if you see source file can you send the unstripped binary
> > and the samples files (tgz please) ? Do not send it to the list
> > if the compressed size is bigger than 200 Ko
>
> shlib is 85MB, compresses down to 12MB. sources contain over 100Klines
> of source code... I'll spare the list. :)
Greg, Phil, did this problem ever get resolved ? It's still sitting in
my inbox so ...
regards
john
--
"Please let's not resume the argument with the usual whining about how this
feature will wipe out humanity or bring us to the promised land."
- Charles Campbell on magic words in Subject: headers