Hello,
I got that error. Can some one tel me what's wrong. The database I am
using now is Postgres. For the development I use sqlite and
everything was working fine.
File "/usr/local/lib/python2.4/site-packages/
SQLObject-0.7.1dev_r1860-py2.4.egg/sqlobject/main.py", line 1266, in
_SO_fetchAlternateID
result, obj = cls._findAlternateID(name, dbName, value, connection)
File "/usr/local/lib/python2.4/site-packages/
SQLObject-0.7.1dev_r1860-py2.4.egg/sqlobject/main.py", line 1262, in
_findAlternateID
value), None
File "/usr/local/lib/python2.4/site-packages/
SQLObject-0.7.1dev_r1860-py2.4.egg/sqlobject/dbconnection.py", line
590, in _SO_selectOneAlt
return self.queryOne("SELECT %s FROM %s WHERE %s = %s" %
File "/usr/local/lib/python2.4/site-packages/
SQLObject-0.7.1dev_r1860-py2.4.egg/sqlobject/dbconnection.py", line
760, in queryOne
return self._dbConnection._queryOne(self._connection, s)
File "/usr/local/lib/python2.4/site-packages/
SQLObject-0.7.1dev_r1860-py2.4.egg/sqlobject/dbconnection.py", line
342, in _queryOne
self._executeRetry(conn, c, s)
File "/usr/local/lib/python2.4/site-packages/
SQLObject-0.7.1dev_r1860-py2.4.egg/sqlobject/dbconnection.py", line
298, in _executeRetry
return cursor.execute(query)
TypeError: argument 1 must be str, not unicode
Best
-fred-

On Thu, Sep 14, 2006 at 09:55:54AM +0900, ?$B6L1[ ?$BBg51 wrote:
> You are very kind
Thank you!
> I had already tried the patch by Dan, but nothing is changed.
Ouch! Then I don't what to do.
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.

On Thu, Sep 14, 2006 at 12:26:43AM +0200, sophana wrote:
> Also, what are the new features in sqlobject 0.8dev?
http://svn.colorstudy.com/SQLObject/docs/News.txt
Also there is a number of patches in my mailbox, and a big number of
patches and bug reports in the SF tracker. You can always help by adding
new or reviewing existing ones.
> I saw that all the unicode options are not the same.
And they will change further. Currently there are 3 (three!)
charset-related options, and rumors are that it'd be enough to have one or
two. You can help by reviewing mysqlconnection.py.
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.

Thanks Oleg,
You are very kind because you reply for my question, even if the
question is discussed in this mailing-list again and again.
I had already tried the patch by Dan, but nothing is changed.
You posted that:
1. how about connection.cache.clear() ?
2. memory is freed for python, but python doesn't return memory to OS.
I want to clear the cache only for the object I want to delete, thus I
want to avoid 1.(and 1. doesn't solve this problem).
2. is doubt, because python returns memory to OS in the following
code(as posted in previous mail).
class NormalObject:
name = "dummy"
obj_list = []
for i in xrange( 10000 ):
obj_list.append( NormalObject ) # memory grows
del obj_list[:] # memory freed
This problem is a frequently asked question, and it is natural that "I
want to free the object from memory, because it is stored in DBMS."
I treat the objects with RDBMS which has > 100,000,000 rows, thus this
is very serious problem...
On Wed, 13 Sep 2006 21:21:39 +0400
Oleg Broytmann <phd@...> wrote:
> On Thu, Sep 14, 2006 at 01:40:20AM +0900, ?$B6L1[Bg51 wrote:
> > hmm, it seems that an object inherites SQLObject is not freed, but a
> > plain object is freed.
>
> There is a patch created by Dan Pascu <dan@...> that may fix
> that. Can you test it (attached)? Does it help?
>
> Oleg.
> --
> Oleg Broytmann http://phd.pp.ru/ phd@...
> Programmers don't die, they just GOSUB without RETURN.
--
株式会社ビービット 玉越 大輝
ユーザビリティ コンサルタント
beBit,Inc. Tamakoshi Hiroki hiroki.tamakoshi@...
--------------------------------------------------------
〒105-0001 東京都港区虎ノ門1-18-1 虎ノ門10森ビル7F
TEL: 03-3509-7602 / FAX: 03-3509-7605
URL: http://www.bebit.co.jp/
--------------------------------------------------------

Hi
I finally resolved the unicode problem. I had to upgrade mysqldb to 1.2.2.
Now the problem is that I had a patch for mysqldb1.2.1 to activate a
mysql reconnection option for mysql5.
Does anyone knows if this patch is still needed in mysqldb1.2.2?
Also, what are the new features in sqlobject 0.8dev?
I saw that all the unicode options are not the same.
Regards
Sophana

On Thu, Sep 14, 2006 at 01:40:20AM +0900, ?$B6L1[Bg51 wrote:
> hmm, it seems that an object inherites SQLObject is not freed, but a
> plain object is freed.
There is a patch created by Dan Pascu <dan@...> that may fix
that. Can you test it (attached)? Does it help?
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.

On Wed, Sep 13, 2006 at 11:29:39PM +0900, ?$B6L1[ ?$BBg51 wrote:
> 3. When I delete the object, the occupied memory is freed.
The memory is freed for Python, by Python doesn't return the memory to
OS. So even after connection.cache.clear() the memory usage by the process
remains the same.
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.

Hi,
I got a memory leak-like problem.
Here's a sample code:
obj_list = []
for i in range( 100000 ):
obj = SomeObject( parameters ... )
obj_list.append( obj )
for obj in obj_list:
obj.expire()
del obj
The memory usage grows up, even if I expire(clear) the cache, and delete
the object.
My wish is as follows:
1. When an object is created, it occupies some memory and the object
data is stored in the RDBMS( This is the default function of
SQLObject).
2. The object should be cached in normal usage(This is the default).
3. When I delete the object, the occupied memory is freed.
The object data still remains in the RDBMS.
I tried a patch by Dan, which is posted at 2006-05-30 in this
mailing-list, but no change is obtained in memory usage.
I searched this mailing list and found similar problems:
"Confused about caching" by Peter, 2006-01-24
"Problem with circular references" by Dan, 2006-05-30
"High memory consumption" by Brian, 2006-08-05
I found that my problem may be the problem of garbage collection on
circular references, but I cannot find the solution.
Any good idea?
(sorry my broken english.)
--
株式会社ビービット 玉越 大輝
ユーザビリティ コンサルタント
beBit,Inc. Tamakoshi Hiroki hiroki.tamakoshi@...
--------------------------------------------------------
〒105-0001 東京都港区虎ノ門1-18-1 虎ノ門10森ビル7F
TEL: 03-3509-7602 / FAX: 03-3509-7605
URL: http://www.bebit.co.jp/
--------------------------------------------------------

On Tue, Sep 12, 2006 at 03:17:55PM +0200, Markus Gritsch wrote:
> But I'm sure my original
> statement didn't offend you too much
Not at all.
> since it contained a big wink at
> the end :D
Sure, I saw it. :)
On the serious side: thank you for the help and explanations!
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.

On 9/12/06, Oleg Broytmann <phd@...> wrote:
> On Tue, Sep 12, 2006 at 01:19:30PM +0200, Markus Gritsch wrote:
> > > I don't use MySQL, so I have to rely on patches form other
> > >people. You can add your ones. ;)
> >
> > Sounds a bit frightening to me
>
> Should I run all the backends SQLObject can connect to? Firebird, MaxDB?
> Should I buy a personal license for Oracle, Sybase, M$SQL? ;)
Of course ;) Seriously: Of course not. But I'm sure my original
statement didn't offend you too much, since it contained a big wink at
the end :D
Cheers, Markus

On Tue, Sep 12, 2006 at 01:19:30PM +0200, Markus Gritsch wrote:
> > I don't use MySQL, so I have to rely on patches form other
> >people. You can add your ones. ;)
>
> Sounds a bit frightening to me
Should I run all the backends SQLObject can connect to? Firebird, MaxDB?
Should I buy a personal license for Oracle, Sybase, M$SQL? ;)
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.

On 9/12/06, Oleg Broytmann <phd@...> wrote:
> But can one use use_unicode=0 to require MySQLdb to return non-unicode
> strings?
>From reading the paragraph describing this keyword-parameter at
http://sourceforge.net/docman/display_doc.php?docid=32071&group_id=22307#functions-and-attributes
I would say yes. However, from reading the paragraph that follows,
which describes the charset keyword-argument, it is not recommended
setting use_unicode=0 when specifying a charset.
> > I wonder if it is really necessary to specify an additional
> > sqlobject_encoding anyway. Shouldn't it be the same as charset?
>
> I don't know. I don't use MySQL, so I have to rely on patches form other
> psople. You can add your ones. ;)
Sounds a bit frightening to me ;) Maybe we should ask the person
which submitted this patch if sqlobject_encoding is redundant to
charset.
Kind regards,
Markus

On Tue, Sep 12, 2006 at 11:44:53AM +0200, Markus Gritsch wrote:
> >and output (SELECT)?
>
> I think by specifying use_unicode=1 in the connection one can require
> that all results are returned as unicode.
But can one use use_unicode=0 to require MySQLdb to return non-unicode
strings?
> I wonder if it is really necessary to specify an additional
> sqlobject_encoding anyway. Shouldn't it be the same as charset?
I don't know. I don't use MySQL, so I have to rely on patches form other
psople. You can add your ones. ;)
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.

On 9/11/06, Oleg Broytmann <phd@...> wrote:
> PS. BTW, you seems to know MySQLdb (MySQL-python) good.
Don't let yourself be fooled ;) I only stumbled across some things
while I made my app working with MySQL.
> Can you say how I can prevent MySQLdb from using unicode on input (INSERT/UPDATE)
With MySQL-python.exe-1.2.1_p2.win32-py2.4.exe and earlier it was not
possible to use unicode as the SQL expression. It was only allowed as
the parameters. Now, starting from
MySQL-python.exe-1.2.2b1.win32-py2.4.exe it was explicitly changed, so
that unicode is also allowed in the SQL. So, I think this is not the
answer you looked for, but IMO it is good to be able to give unicode
to the connector and let him encode it in the charset specified in the
connection.
> and output (SELECT)?
I think by specifying use_unicode=1 in the connection one can require
that all results are returned as unicode.
> Or at least how can I know the encoding it uses to convert strings to unicode?
I believe the charset specified in the connection is used for this.
> I have to know the encoding. Is it the same as connection.kw["charset"]? sqlobject_encoding?
I wonder if it is really necessary to specify an additional
sqlobject_encoding anyway. Shouldn't it be the same as charset?
(Beside the fact, that MySQL uses e.g. 'utf8' as the charset name
instead of 'utf-8', which would probably be the correct name.)
Kind regards,
Markus

On Mon, Sep 11, 2006 at 09:53:25PM +0200, Markus Gritsch wrote:
> Everything works fine when using an SQLite connection
A bug in SQLite, PySQLite or SQLiteConnection? Postgres (or psycopg)
does not allow unicode.
> if self.need_unicode:
> into
> if self.need_unicode and not isinstance(query, unicode):
> in mysqlconnection.py make things worse?
Yes, because it hides the real source of the problem. This make much
harder to port your programs to other non-unicode supporting backends.
The real problem is in SQLObject internals. A query now must be a
string. And rewriting the core to always use unicode would be a hard job -
a big task by itself, and raises questions about BLOBCol, PickleCol...
PS. BTW, you seems to know MySQLdb (MySQL-python) good. Can you say how I
can prevent MySQLdb from using unicode on input (INSERT/UPDATE) and output
(SELECT)? Or at least how can I know the encoding it uses to convert
strings to unicode? Unicode output from MySQLdb is a constant source of
pain for PickleCol users. There is the code in PickleValidator:
def to_python(self, value, state):
if isinstance(value, unicode):
value = value.encode("ascii")
"ascii" encoding here is wrong, and unpickler doesn't accept unicode. So
I have to know the encoding. Is it the same as connection.kw["charset"]?
sqlobject_encoding? client_encoding?
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.

On 9/11/06, Oleg Broytmann <phd@...> wrote:
> On Mon, Sep 11, 2006 at 09:26:43PM +0200, Markus Gritsch wrote:
> > one issue remains: When doing a select() using LIKE (maybe with other
> > SQLBuilder objects too) with a unicode string, one gets a traceback,
> [skip]
> > print Test.select(LIKE(Test.q.name, name))[0].name.encode('utf-8') #
>
> Please read the description of the UnicodeCol:
>
> http://sqlobject.org/SQLObject.html#column-types
>
> "...you MUST apply the encoding yourself:..."
Thank you for the link, though I do not completely understand this.
Everything works fine when using an SQLite connection in my example:
sqlhub.processConnection = connectionForURI('sqlite:/:memory:')
I understand that I have to tell the MySQL connection whether to
use_unicode, the charset used, and for all I care also SQLObject about
the sqlobject_encoding. But I really do not want to change my
application code all over the place and add an .encode(dbEncoding)
everywhere I use LIKE. As already said, this is not necessary when
using the SQLite connection. Would changing the line
if self.need_unicode:
into
if self.need_unicode and not isinstance(query, unicode):
in mysqlconnection.py make things worse?
However I do not have enough insight into SQLObject, so it would be
great if you would find a solution, as you already mentioned, for the
LIKE expression.
Kind regards,
Markus

On Mon, Sep 11, 2006 at 09:26:43PM +0200, Markus Gritsch wrote:
> one issue remains: When doing a select() using LIKE (maybe with other
> SQLBuilder objects too) with a unicode string, one gets a traceback,
[skip]
> print Test.select(LIKE(Test.q.name, name))[0].name.encode('utf-8') #
Please read the description of the UnicodeCol:
http://sqlobject.org/SQLObject.html#column-types
"...you MUST apply the encoding yourself:..."
PS. May be this can be fixed for LIKE(). I'll think about it...
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.

Hi,
one issue remains: When doing a select() using LIKE (maybe with other
SQLBuilder objects too) with a unicode string, one gets a traceback,
because the type of the variable query in
myquery = unicode(query, self.encoding)
in mysqlconnection.py is <type 'unicode'> instead of <type 'str'>. So
either SQLBuilder has to be fixed to return <type 'str'> in all cases,
or the line
if self.need_unicode:
in mysqlconnection.py has to be changed into
if self.need_unicode and not isinstance(query, unicode):
The following example can be used to reproduce the problem:
# ---- 8< ----
from sqlobject import *
sqlhub.processConnection = connectionForURI(
'mysql://markus@.../test?use_unicode=1&charset=utf8&sqlobject_encoding=utf-8';)
class Test(SQLObject):
name = UnicodeCol()
Test.dropTable(ifExists=True)
Test.createTable(ifNotExists=True)
# In HTML 4 this would be: &auml; &ouml; &uuml; &euro;
name = u'\u00E4 \u00F6 \u00FC \u20AC '
Test(name=name)
print Test.get(1).name.encode('utf-8') # ok
print Test.select(LIKE(Test.q.name, name))[0].name.encode('utf-8') # traceback
# ---- 8< ----
The traceback reads:
Traceback (most recent call last):
File "test.py", line 17, in ?
print Test.select(LIKE(Test.q.name, name))[0].name.encode('utf-8')
# traceback
File "c:\python24\lib\site-packages\sqlobject-0.7.1rc1-py2.4.egg\sqlobject\sresults.py",
line 149, in __getitem__
return list(self.clone(start=start, end=start+1))[0]
File "c:\python24\lib\site-packages\sqlobject-0.7.1rc1-py2.4.egg\sqlobject\sresults.py",
line 155, in __iter__
return iter(list(self.lazyIter()))
File "c:\python24\lib\site-packages\sqlobject-0.7.1rc1-py2.4.egg\sqlobject\sresults.py",
line 163, in lazyIter
return conn.iterSelect(self)
File "c:\python24\lib\site-packages\sqlobject-0.7.1rc1-py2.4.egg\sqlobject\dbconnection.py",
line 365, in iterSelect
select, keepConnection=False)
File "c:\python24\lib\site-packages\sqlobject-0.7.1rc1-py2.4.egg\sqlobject\dbconnection.py",
line 705, in __init__
self.dbconn._executeRetry(self.rawconn, self.cursor, self.query)
File "c:\python24\lib\site-packages\sqlobject-0.7.1rc1-py2.4.egg\sqlobject\mysql\mysqlconnection.py",
line 80, in _executeRetry
myquery = unicode(query, self.encoding)
TypeError: decoding Unicode is not supported

Hi,
I think I found the solution. The trick is to specify both, the
charset and the sqlobject_encoding as parameters in the connection
URI: ?use_unicode=1&charset=utf8&sqlobject_encoding=utf-8
By using this, the following example works fine. The data in the DB
is also stored in the desired (utf-8) encoding. To prevent from any
confusion about file-encoding and using u'', I choose to have the file
ASCII-encoded and use \u escapes in the unicode string :)
Thanks for the help,
Markus
from sqlobject import *
sqlhub.processConnection = connectionForURI(
'mysql://markus@.../test?use_unicode=1&charset=utf8&sqlobject_encoding=utf-8';)
class Test(SQLObject):
name = UnicodeCol()
Test.dropTable(ifExists=True)
Test.createTable(ifNotExists=True)
# In HTML 4 this would be: &auml; &ouml; &uuml; &euro;
Test(name=u'\u00E4 \u00F6 \u00FC \u20AC ')
print Test.get(1).name.encode('utf-8') # Needs an utf-8 capable terminal

On Mon, Sep 11, 2006 at 10:14:42AM +0200, Ivan Horvath wrote:
> after many trials it seems to me that order by clause works only for one field.
> if the orderBy parameter receives list or tuple, because i need more order clause then the system parse this as sequence and the result within brackets - and it is true this sql cannot execute in mysql (received the same error in client)
> if the orderBy parameter is string like "column1, column2";
Ah, that's the problem.
> -------------
> documentation: You can use the keyword arguments orderBy to create ORDER BY in the select statements: orderBy
> takes a string, which should be the database name of the column, or a column in the form Person.q.
> firstName. You can use "-colname" to specify descending order, or call MyClass.select().
> reversed().
> -------------
The documentation is for .select(). You are trying sqlbuilder.Select()
which is a different beast.
> but how can i make a query with more order by fields?
By patching sqlbuilder.Select() to make it more in accord with
.select().
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.

On Sun, Sep 10, 2006 at 11:01:48PM +0200, Andres Freund wrote:
> Currently sqlboject uses LIKE for all its comparisons which is case sensitive
> in some databases (eg. postgres) (I _think_ this is standard compliant, im on
> the road currently, so i cant check) others do not (eg mysql).
> At least I sometimes need a case sensitive LIKE and sometimes not. Is there
> interest in changing sqlobject to support both? Eg. either having something
> like .istartswith(str) or .startswith(str, casesensitve = False)?
Why not just
from sqlobject.sqlbuilder import func
func.lower(MyTable.q.field).startswith(str.lower())
???
And how are you going to implement
.startswith(str, casesensitve = True)
on MySQL? ;)
Oleg.
--
Oleg Broytmann http://phd.pp.ru/ phd@...
Programmers don't die, they just GOSUB without RETURN.