Comments

Hi, I've updated this patch to trunk and rebootstrapped it, so I'm
resubmitting it, I'm also making a try to CC all relevant maintainers,
sorry for spamming but even though this patch is simple, it touches many
parts.
I'm also adding stats to pointer_{set,map} but it will come in a separate
patch. The notes quoted from earlier mail still apply:
On Sun, 8 Jul 2012, Dimitrios Apostolou wrote:
> Hi,>> This patch adds many nice stats about hash tables when gcc is run with > -fmem-report. Attached patch tested on x86, no regressions.>> Also attached is sample output of -fmem-report when compiling reload.c with > -O2 -g. Especially interesting is the extreme htab usage in var-tracking > (ofcourse we already knew that... :-) and maybe symtab's slightly high > collision rate (given it's a very hot hash table). Moreover it's notable that > since last year, collision rate for mem_attrs_htab has been reduced from > around 8 to 1, still not good enough but major improvement.>> Jan: It has negligible overhead in runtime, that's why I'm pushing it again > as it is. Having compile-time instrumentation is something different, but > still I'm not quite sure if it's worth the effort. IMHO doing separate builds > to get memory info is tedious and even I've stopped doing them. If it has no > overhead maybe it's better as a runtime option?> Any way, compile-time instrumentation can come later.>> Finally I repost some of my notes I'd like to get feedback on:>> * Is it OK that I included <assert.h> in libiberty's hashtab.c? What's the > proper way to assert stuff, since gcc_assert() is not accessible?>> * Many hash tables are created with htab_create_ggc(), for example > referenced_vars and default_defs in tree-ssa.c. I collect statistics in > delete_tree_ssa() but maybe some are not deallocated in there but > automatically garbage collected?>> * Obviously hash table sizes are inflated where two entries might reference > the same element (for example in symtab_hash) but I don't handle this.>> * Changes that reduce load factor have been backed out since they brought > controversy. I still think they are good, at least for symtab. I'll measure > numbers separately for this after this patch makes it.>>> Thanks,> Dimitris>

I'm always forgetting something, now it was the changelog, see attached
(same as old, nothing significant changed).
On Fri, 3 Aug 2012, Dimitrios Apostolou wrote:
> Hi, I've updated this patch to trunk and rebootstrapped it, so I'm > resubmitting it, I'm also making a try to CC all relevant maintainers, sorry > for spamming but even though this patch is simple, it touches many parts.>> I'm also adding stats to pointer_{set,map} but it will come in a separate > patch. The notes quoted from earlier mail still apply:>> On Sun, 8 Jul 2012, Dimitrios Apostolou wrote:>>> Hi,>> >> This patch adds many nice stats about hash tables when gcc is run with >> -fmem-report. Attached patch tested on x86, no regressions.>> >> Also attached is sample output of -fmem-report when compiling reload.c with >> -O2 -g. Especially interesting is the extreme htab usage in var-tracking >> (ofcourse we already knew that... :-) and maybe symtab's slightly high >> collision rate (given it's a very hot hash table). Moreover it's notable >> that since last year, collision rate for mem_attrs_htab has been reduced >> from around 8 to 1, still not good enough but major improvement.>> >> Jan: It has negligible overhead in runtime, that's why I'm pushing it again >> as it is. Having compile-time instrumentation is something different, but >> still I'm not quite sure if it's worth the effort. IMHO doing separate >> builds to get memory info is tedious and even I've stopped doing them. If >> it has no overhead maybe it's better as a runtime option?>> Any way, compile-time instrumentation can come later.>> >> Finally I repost some of my notes I'd like to get feedback on:>> >> * Is it OK that I included <assert.h> in libiberty's hashtab.c? What's the >> proper way to assert stuff, since gcc_assert() is not accessible?>> >> * Many hash tables are created with htab_create_ggc(), for example >> referenced_vars and default_defs in tree-ssa.c. I collect statistics in >> delete_tree_ssa() but maybe some are not deallocated in there but >> automatically garbage collected?>> >> * Obviously hash table sizes are inflated where two entries might reference >> the same element (for example in symtab_hash) but I don't handle this.>> >> * Changes that reduce load factor have been backed out since they brought >> controversy. I still think they are good, at least for symtab. I'll measure >> numbers separately for this after this patch makes it.>> >> >> Thanks,>> Dimitris>
2012-05-21 Dimitrios Apostolou <jimis@gmx.net>
Print various statistics about hash tables when called with
-fmem-report. If the tables are created once use
htab_dump_statistics(), if they are created/destroyed multiple
times then introduce global variables to track statistics.
* cgraph.c, cgraph.h:
(cgraph_dump_stats): New function to dump stats about hash tables.
* gcc/symtab.c, cgraph.h:
(symtab_dump_stats): New function to dump stats about hash tables.
* cgraph.c: (call_site_hash_{num,expands,searches,collisions}):
New globals to keep statistics about call_site_hash hash tables.
(cgraph_remove_node_callees,cgraph_remove_node): if (mem_report)
then keep statistics about hash tables.
* cselib.c, cselib.h (cselib_dump_stats): New function to dump
stats about cselib_hash_table.
* cselib.c (cselib_htab_{num,expands,searches,collisions}): New
globals to keep hash table statistics.
(cselib_finish): if (mem_report) keep hash table statistics.
* dwarf2out.c (dwarf2out_finish): Call htab_dump_statistics() if
-fmem_report.
* emit-rtl.c, emit-rtl.h (mem_attrs_dump_stats): New function to
dump statistics about mem_attrs_htab hash table.
* tree.h, tree-ssa.c (tree_ssa_dump_stats): New function to print
statistics about all referenced_vars and default_defs hash tables.
* tree-ssa.c (default_defs_{num,expands,searches,collisions})
(referenced_vars_{num,expands,searches,collisions}): new globals
to keep statistics about hash tables.
(delete_tree_ssa): Keep statistics for hash tables by
increasing the new global variables.
* tree.c (dump_tree_statistics): Call tree_ssa_dump_stats().
(print_type_hash_statistics): Used the new htab_dump_statistics()
function.
* var-tracking.c (vars_htab_{num,expands,searches,collisions})
(dropval_htab_{num,expands,searches,collisions})
(cv_htab_{num,expands,searches,collisions}): new globals to keep
hash table statistics.
(shared_hash_destroy, vt_finalize): Keep statistics by
increasing values of new global variables if -fmem-report.
* var-tracking.c, rtl.h (vt_dump_stats): New function to dump
stats about vars->htab, dropped_variables and value_chains hash
tables.
* toplev.c: Included cselib.h for cselib_dump_stats().
(dump_memory_report): Call all the above functions to provide
better statistics.
* hashtab.c, hashtab.h: Added "expands" variable to htab_t for
tracking total times the hash table was expanded.
* hashtab.c, hashtab.h (htab_dump_statistics, htab_collisions_num)
(htab_searches_num, htab_expands_num): New functions for statistics.
* hashtab.c: Included assert.h for checks in htab_dump_statistics.
* cgraph.h, varpool.c (varpool_dump_stats): New function to dump
stats about varpool_hash hash table.
* libcpp/symtab.c, symtab.h: Added "expands" variable to
hash_table type for tracking total times the hash table was
expanded.
* symtab.c (ht_dump_statistics): Beautified stats output.