Hello!
On Mon, Aug 27, 2012 at 10:32:50AM +0200, Gert Burger <gertburger@...> wrote:
> We have a RPC server which CPU usage increases over time, after a week
> it will max out one core.
>
> >From my testing(Using pyrasite's dump stack) it seems that all the time
> is spend in:
>
> File
> "/usr/local/lib/python2.6/dist-packages/SQLObject-1.3.1-py2.6.egg/sqlobject/cache.py",
> line 263, in allIDs
> for id, value in self.expiredCache.items():
>
> Currently the overhead for each commit and rollback is around 2-3 seconds.
>
> Almost all of our tables have caching disabled via 'sqlmeta: cacheValues
> = False'.
sqlmeta.cacheValues disables caching of attributes -- with
cacheValues=False every time a program reads and attribute
(value = row.column) SQLObject issues a SELECT query.
There is also row caching -- SQLObject caches rows in the connection
cache. The code above is related to that cache. To disable the cache
pass cache=False to connection constructor or in the DB URI:
schema://host:port/db?cache=0.
> No state is kept in memory between requests and we typically write
> significantly more rows to the DB than what is read, ie. A lot of writes
> which are not read often and a few hundred rows which are read often and
> that can be changed by other processes.
So you need mass insertion. See the related FAQ entry:
http://sqlobject.org/FAQ.html#how-to-do-mass-insertion
Oleg.
--
Oleg Broytman http://phdru.name/ phd@...
Programmers don't die, they just GOSUB without RETURN.

Hi
We have a RPC server which CPU usage increases over time, after a week
it will max out one core.
>From my testing(Using pyrasite's dump stack) it seems that all the time
is spend in:
file "/usr/lib/pymodules/python2.6/turbogears/database.py", line 275, in
commit
self.threadingLocal.connection.commit()
File
"/usr/local/lib/python2.6/dist-packages/SQLObject-1.3.1-py2.6.egg/sqlobject/dbconnection.py",
line 802, in commit
subCaches = [(sub[0], sub[1].allIDs()) for sub in
self.cache.allSubCachesByClassNames().items()]
File
"/usr/local/lib/python2.6/dist-packages/SQLObject-1.3.1-py2.6.egg/sqlobject/cache.py",
line 263, in allIDs
for id, value in self.expiredCache.items():
Currently the overhead for each commit and rollback is around 2-3 seconds.
Almost all of our tables have caching disabled via 'sqlmeta: cacheValues
= False'.
No state is kept in memory between requests and we typically write
significantly more rows to the DB than what is read, ie. A lot of writes
which are not read often and a few hundred rows which are read often and
that can be changed by other processes.
Does anyone have a suggestion on how to mitigate or further debug the
problem?
Regards
Gert Burger