Description of problem:
During install of selinux-policy-targeted package I got the following error in
post-install script:
+ semodule -b /usr/share/selinux/targeted/base.pp -s targeted
libsemanage.dbase_policydb_list: out of memory
libsemanage.semanage_exec_prog: Child process /usr/sbin/genhomedircon did not
exit cleanly.
libsemanage.semanage_install_sandbox: genhomedircon returned error code -1.
I tracked down the problem to the "static int dbase_policydb_list"
(database_policydb.c) function. It defines "size_t tmp_count" variable and
passes it to "dbase->rptable->count(handle->sepolh, dbase->policydb,
&tmp_count)" by pointer. On Alpha size_t is 64 bits long, but the function
expects "tmp_count" to be "unsigned int" (gcc issue warning on it). The problem
is if 32 higher bits of that counter contains garbage it won't be overriden by
"dbase->rptable->count" (it work with unsigned int which is 32 bits long)
function and on return from it the "tmp_count" could hold large value.
There are two ways to fix it:
- initialize tmp_count to zero
- declare it as "unsigned int" (which is more correct).
Version-Release number of selected component (if applicable):
1.5.9-1
Thank you.

Hi, I'm looking at this issue. Can you attach the full list of warnings
generated (error stream only). There should be no unexpected warnings in
libsemanage - it does have a bunch of expected ones, but any new ones that show
should be fixed.

Those are bugs:
database_activedb.c: In function 'dbase_activedb_cache':
database_activedb.c:41: warning: 'i' may be used uninitialized in this function
database_policydb.c: In function 'dbase_policydb_list':
database_policydb.c:389: warning: passing argument 3 of 'dbase->rptable->count'
from incompatible pointer type
The swig ones I haven't seen before - can't comment on those, jbrindle cc-ed.
The other ones are known and expected - the incompatible types are between sepol
and semanage, since those two libraries are currently coupled, and expected to
be upgraded together (btw, we should autotool the library, and require the
proper symbols here).

Ok, it is my understanding that size_t = unsigned int or unsigned long int, and
int <= long int, is that correct ?
http://www.delorie.com/gnu/docs/glibc/libc_679.html
So size_t is always at least as large as the result returned by count, correct?
In that case, I am inclined to change the list function to return unsigned int
as well. Since this is a smaller data type, this should not be considered an API
change.