> Hi,> > propel_pl wrote:> > Hi.> > > > I'm a memeber of php.pl community> > I think, I can help to translate User Guide to polish language. There is a lot of propel users in Poland, and i think some of them will help. Is it possible to get trac account?> > > > Sure -- we'd be grateful for the translation help.> > Please contact me off-list with a preferred username.> > Cheers,> Hans> > --------------------​--------------------​--------------------​---------> To unsubscribe, e-mail: dev-unsubscribe@prop​el.tigris.org> For additional commands, e-mail: dev-help at propel dot tigris dot org> >

> this sounds like something that might work. what about actually> extending PDO for this?

that works...

> Whilst we are at it, how about a mapping configuration so you don't> have to touch the classes/objects you want to persist in the db?

I never do, in fact any object I generate (like Document.phpDocumentPeer.php) which you're supposedly cool to edit. I leave be... I want a fullyabstracted model. so Im never going to make it impure by putting regular hand typed codeinto it... I think the whole point is to automate the M.. the model layer.. the layerin which any bug is likely to cause data corruption. So XML -> Data Objects andthat's it. This is why I think propel lends itself to web frameworks because theseparated controller is a good place to wire these generated db objects together,simply set foreign dependency with the </foreign tag and the save and validateinherits the proper methods to deal with an abstraction of the database table. keep thetable names not pluralized, lower cased with underscores and you're wellabstracted.

> I like a domain model that isn't fully aware of all the dependencies to> whatever persistence engine is used.

On 19/07/07, Pedram Nimreezi <zenstyle at gmail dot com> wrote:> I couldn't help it, I have to respond to that...>> I currently debug propel / creole / rapid framework with ntail in eclipse...> in fact, eclipse has two good features it gives you for the 200 megs of ram> it wants to use. A file / directory explorer... and this ntail plugin that> lets> you look at ansi (colored) logs... generated by Pear::Log which has been> available since the very beginning in propel... color with ansi where> context is needed.,>> There is no reason why xdebug can't be used.. in fact... Eclipse's> file/directory> manager just launches vim for me on the file... I can then at my leisure hit> F5> and rerequest a page on the site... passing of course that> /XDEBUG_SESSION_START=> flag... this starts an interactive debugging session inside that vim> window... how> that works is a mix of vim scripting and python and is really another> discussion.>> however, that's a different usage of xdebug but my point is, you can use it> or not> use it.. you can also use eclipse or not use it... if whatever diagnostic> assertions> can be added to propel with some degree of effeciency then add it but don't> dismiss> it out of hand. in fact I like propel / creole php4 better than what's> current.... but I don't> try to make everyone use that, even though it's more stable and faster. And> works> right in php4 and php5, with other optimizations like 95% of include_once's> removed> including other optimizations that are fundamental to opcode caching... why?> because> its like putting a gag order on development....>> > If you really need the lower level debugging inside Propel then add it> > in a non obtrusive way by enabling dependency injection of classes> > that subclass the default ones but add the debugging functionality and> > then call the parent methods etc.>> that's your two cents? I think that's just a buzz word conjugation....and here's another dime: if you think they are just buzz words, howwould you then describe the best practice? Instead of commenting on my'buzz words' you could give us an alternative...>> anyway I think refactoring propel into a php extension is the futureproof> solution.this sounds like something that might work. what about actuallyextending PDO for this?Whilst we are at it, how about a mapping configuration so you don'thave to touch the classes/objects you want to persist in the db? Ilike a domain model that isn't fully aware of all the dependencies towhatever persistence engine is used. AOP maybe? Ups another 'buzzword' :-)>> and it should be able to use mysql/pgsql extensions on php4 and pdo on> php5...>> that's an opinion I will support...>>>> On 7/19/07, Christian Kassab <chris dot kassab at gmail dot com> wrote:> > Hi,> >> > why stuff so much debugging functionality into Propel? It is bloated> > as it is already. How about using debugging tools like xdebug and> > Eclipse? Yes I know you can't look into what goes on inside PDO as in> > what the final query string looks like after binding the> > variables/parameters, but thats another story...> >> > If you really need the lower level debugging inside Propel then add it> > in a non obtrusive way by enabling dependency injection of classes> > that subclass the default ones but add the debugging functionality and> > then call the parent methods etc.> >> > Cheers,> >> > Christian> >> > On 19/07/07, Hans Lellelid <hans at velum dot net> wrote:> > > Oliver Schonrock wrote:> > > > Cameron Brunner wrote:> > > >> Silence is good bad or otherwise?> > > >> > > > Don't take offense please. I like the features, but I am not sure> about> > > > the implementation:> > > >> > > > - using static $query seems untidy> > > > - using __destruct() can cause problems depending on your environment> > > > - using direct var_dump() it not always suitable.> > > >> > > > if we are going to add this to the core propel repository, my view> would> > > > be that it should meet the same criteria as the rest of the code as a> > > > generic, high quality toolkit.> > > >> > > > Part of the reason I focused on simple features was that it is not> > > > obvious how to solve some of the above problems with the more advanced> > > > features while adhering to the quality standards of the rest of the> > > > code. ie can run in any environment, etc..> > > >> > > > Opinions?> > >> > > Without having looked at the code, I second this opinion. We definitely> > > don't want to use var_dump, for instance, as this is not particularly> > > useful when building webapps that don't return HTML (i.e. that return> > > XML/XML-RPC, JSON, etc.).> > >> > > My input would be that we need the default PropelPDO to be as> > > unencumbered as possible (fast). If we're going to keep all the debug> > > stuff in a separate object, then it doesn't matter much how that> > > performs. If, however, the aim is to merge this into PropelPDO, then we> > > need to make sure that we're doing things in an optimized -- and yes,> > > "correct" -- way. We definitely don't want to open the door for subtle> > > PHP issues.> > >> > > One thing I have learned from using Propel + PDO in PHP 5.2.2 is that> > > PDO is not particularly bug-free :) I have dealt with a number of> > > segfaults that are directly traceable to PDO. Sometimes it is the> > > result of a malformed query (why this segfaults, I have no idea) --- or> > > use of PDO streams within user-defined stream wrappers. I *hate*> > > debugging segfaults in PHP; what a waste of time. There's apparently> > > some really messy stuff under the hood there :) Let's do what we can to> > > keep it from rearing its ugly head for users.> > >> > > Hans> > >> > >> --------------------​--------------------​--------------------​---------> > > To unsubscribe, e-mail:> dev-unsubscribe@prop​el.tigris.org> > > For additional commands, e-mail: dev-help at propel dot tigris dot org> > >> > >> >> >> > --> > Christian Kassab> >> > Mobile:> > +44 (0)789 4467 606> > +49 (0)163 6360 939> >> > Mail:> > chris dot kassab at binaryworx dot net> > chris dot kassab at gmail dot com> >> > Web:> > http://www.binaryworx.net> > http://www.shoreflyer.net> >> >> --------------------​--------------------​--------------------​---------> > To unsubscribe, e-mail: dev-unsubscribe@prop​el.tigris.org> > For additional commands, e-mail: dev-help at propel dot tigris dot org> >> >>>>> --> ~> Pedram Nimreezi>

This is meant to be a preproduction debug flag that enables extradebug methods so you get some real query information, not just a basichere's your queries in their prepared state and oh yea, here's whatwent into each of the ?'s but dont ask me to show you the final query.xdebug is great, i love it, not many people have it even still thoplus its trace format (kcachegrind) means that you have to open upyour trace to view it, its not just appended to the current page andthere are no easy ways to see "shit, propel took 2 seconds to hydrate,wtf is going on" and thats the aim of this

again, this is NOT for production environments, if you read the codeyou will see that its been done as lightweight as possible as far asonly when debug is enabled

On 7/20/07, Christian Kassab <chris dot kassab at gmail dot com> wrote:> Hi,>> why stuff so much debugging functionality into Propel? It is bloated> as it is already. How about using debugging tools like xdebug and> Eclipse? Yes I know you can't look into what goes on inside PDO as in> what the final query string looks like after binding the> variables/parameters, but thats another story...>> If you really need the lower level debugging inside Propel then add it> in a non obtrusive way by enabling dependency injection of classes> that subclass the default ones but add the debugging functionality and> then call the parent methods etc.>> Cheers,>> Christian>> On 19/07/07, Hans Lellelid <hans at velum dot net> wrote:> > Oliver Schonrock wrote:> > > Cameron Brunner wrote:> > >> Silence is good bad or otherwise?> > >> > > Don't take offense please. I like the features, but I am not sure about> > > the implementation:> > >> > > - using static $query seems untidy> > > - using __destruct() can cause problems depending on your environment> > > - using direct var_dump() it not always suitable.> > >> > > if we are going to add this to the core propel repository, my view would> > > be that it should meet the same criteria as the rest of the code as a> > > generic, high quality toolkit.> > >> > > Part of the reason I focused on simple features was that it is not> > > obvious how to solve some of the above problems with the more advanced> > > features while adhering to the quality standards of the rest of the> > > code. ie can run in any environment, etc..> > >> > > Opinions?> >> > Without having looked at the code, I second this opinion. We definitely> > don't want to use var_dump, for instance, as this is not particularly> > useful when building webapps that don't return HTML (i.e. that return> > XML/XML-RPC, JSON, etc.).> >> > My input would be that we need the default PropelPDO to be as> > unencumbered as possible (fast). If we're going to keep all the debug> > stuff in a separate object, then it doesn't matter much how that> > performs. If, however, the aim is to merge this into PropelPDO, then we> > need to make sure that we're doing things in an optimized -- and yes,> > "correct" -- way. We definitely don't want to open the door for subtle> > PHP issues.> >> > One thing I have learned from using Propel + PDO in PHP 5.2.2 is that> > PDO is not particularly bug-free :) I have dealt with a number of> > segfaults that are directly traceable to PDO. Sometimes it is the> > result of a malformed query (why this segfaults, I have no idea) --- or> > use of PDO streams within user-defined stream wrappers. I *hate*> > debugging segfaults in PHP; what a waste of time. There's apparently> > some really messy stuff under the hood there :) Let's do what we can to> > keep it from rearing its ugly head for users.> >> > Hans> >> > --------------------​--------------------​--------------------​---------> > To unsubscribe, e-mail: dev-unsubscribe@prop​el.tigris.org> > For additional commands, e-mail: dev-help at propel dot tigris dot org> >> >>>> --> Christian Kassab>> Mobile:> +44 (0)789 4467 606> +49 (0)163 6360 939>> Mail:> chris dot kassab at binaryworx dot net> chris dot kassab at gmail dot com>> Web:> http://www.binaryworx.net> http://www.shoreflyer.net>> --------------------​--------------------​--------------------​---------> To unsubscribe, e-mail: dev-unsubscribe@prop​el.tigris.org> For additional commands, e-mail: dev-help at propel dot tigris dot org>>

On 7/19/07, Hans Lellelid <hans at velum dot net> wrote:> Oliver Schonrock wrote:> > Cameron Brunner wrote:> >> Silence is good bad or otherwise?> >> > Don't take offense please. I like the features, but I am not sure about> > the implementation:> >> > - using static $query seems untidy> > - using __destruct() can cause problems depending on your environment> > - using direct var_dump() it not always suitable.> >> > if we are going to add this to the core propel repository, my view would> > be that it should meet the same criteria as the rest of the code as a> > generic, high quality toolkit.> >> > Part of the reason I focused on simple features was that it is not> > obvious how to solve some of the above problems with the more advanced> > features while adhering to the quality standards of the rest of the> > code. ie can run in any environment, etc..> >> > Opinions?>> Without having looked at the code, I second this opinion. We definitely> don't want to use var_dump, for instance, as this is not particularly> useful when building webapps that don't return HTML (i.e. that return> XML/XML-RPC, JSON, etc.).

See my previous message

> My input would be that we need the default PropelPDO to be as> unencumbered as possible (fast). If we're going to keep all the debug> stuff in a separate object, then it doesn't matter much how that> performs. If, however, the aim is to merge this into PropelPDO, then we> need to make sure that we're doing things in an optimized -- and yes,> "correct" -- way. We definitely don't want to open the door for subtle> PHP issues.

No way should the debug stuff be in the main propelpdo object, its allheavyweight and there are overheads for it all. i would neverrecommend this went into the 'production' connection and for thatmatter im also getting to the point where i want to take the loggingmethods out of the standard 1 and put them into the debug driver tomake it even faster

> One thing I have learned from using Propel + PDO in PHP 5.2.2 is that> PDO is not particularly bug-free :) I have dealt with a number of> segfaults that are directly traceable to PDO. Sometimes it is the> result of a malformed query (why this segfaults, I have no idea) --- or> use of PDO streams within user-defined stream wrappers. I *hate*> debugging segfaults in PHP; what a waste of time. There's apparently> some really messy stuff under the hood there :) Let's do what we can to> keep it from rearing its ugly head for users.>> Hans>> --------------------​--------------------​--------------------​---------> To unsubscribe, e-mail: dev-unsubscribe@prop​el.tigris.org> For additional commands, e-mail: dev-help at propel dot tigris dot org>>

var_dump is merely to show the data collected for now, as said before,i want a smarty style popup at the end of this all

static vs put it into the object, theres no major difference,currently it just keeps things in that method, not cruft in the objectaside from queryLog... doesnt matter if that changes and its nothingmajor to change it either

what is the issue with destruct? in a normal propel setup these seemto be being hit at the right times for what im after the data of...

On 7/19/07, Oliver Schonrock <oliver at realtsp dot com> wrote:> Cameron Brunner wrote:> > Silence is good bad or otherwise?>> Don't take offense please. I like the features, but I am not sure about> the implementation:>> - using static $query seems untidy> - using __destruct() can cause problems depending on your environment> - using direct var_dump() it not always suitable.>> if we are going to add this to the core propel repository, my view would> be that it should meet the same criteria as the rest of the code as a> generic, high quality toolkit.>> Part of the reason I focused on simple features was that it is not> obvious how to solve some of the above problems with the more advanced> features while adhering to the quality standards of the rest of the> code. ie can run in any environment, etc..>> Opinions?>> Oliver>>>>>>

I currently debug propel / creole / rapid framework with ntail in eclipse...in fact, eclipse has two good features it gives you for the 200 megs of ramit wants to use. A file / directory explorer... and this ntail plugin thatletsyou look at ansi (colored) logs... generated by Pear::Log which has beenavailable since the very beginning in propel... color with ansi wherecontext is needed.,

There is no reason why xdebug can't be used.. in fact... Eclipse'sfile/directorymanager just launches vim for me on the file... I can then at my leisure hitF5and rerequest a page on the site... passing of course that/XDEBUG_SESSION_START=flag... this starts an interactive debugging session inside that vimwindow... howthat works is a mix of vim scripting and python and is really anotherdiscussion.

however, that's a different usage of xdebug but my point is, you can use itor notuse it.. you can also use eclipse or not use it... if whatever diagnosticassertionscan be added to propel with some degree of effeciency then add it but don'tdismissit out of hand. in fact I like propel / creole php4 better than what'scurrent.... but I don'ttry to make everyone use that, even though it's more stable and faster. Andworksright in php4 and php5, with other optimizations like 95% of include_once'sremovedincluding other optimizations that are fundamental to opcode caching... why?becauseits like putting a gag order on development....

> If you really need the lower level debugging inside Propel then add it> in a non obtrusive way by enabling dependency injection of classes> that subclass the default ones but add the debugging functionality and> then call the parent methods etc.

that's your two cents? I think that's just a buzz word conjugation....

anyway I think refactoring propel into a php extension is the futureproofsolution. and it should be able to use mysql/pgsql extensions on php4 and pdo onphp5...

that's an opinion I will support...

On 7/19/07, Christian Kassab <chris dot kassab at gmail dot com> wrote:>> Hi,>> why stuff so much debugging functionality into Propel? It is bloated> as it is already. How about using debugging tools like xdebug and> Eclipse? Yes I know you can't look into what goes on inside PDO as in> what the final query string looks like after binding the> variables/parameters, but thats another story...>> If you really need the lower level debugging inside Propel then add it> in a non obtrusive way by enabling dependency injection of classes> that subclass the default ones but add the debugging functionality and> then call the parent methods etc.>> Cheers,>> Christian>> On 19/07/07, Hans Lellelid <hans at velum dot net> wrote:> > Oliver Schonrock wrote:> > > Cameron Brunner wrote:> > >> Silence is good bad or otherwise?> > >> > > Don't take offense please. I like the features, but I am not sure> about> > > the implementation:> > >> > > - using static $query seems untidy> > > - using __destruct() can cause problems depending on your environment> > > - using direct var_dump() it not always suitable.> > >> > > if we are going to add this to the core propel repository, my view> would> > > be that it should meet the same criteria as the rest of the code as a> > > generic, high quality toolkit.> > >> > > Part of the reason I focused on simple features was that it is not> > > obvious how to solve some of the above problems with the more advanced> > > features while adhering to the quality standards of the rest of the> > > code. ie can run in any environment, etc..> > >> > > Opinions?> >> > Without having looked at the code, I second this opinion. We definitely> > don't want to use var_dump, for instance, as this is not particularly> > useful when building webapps that don't return HTML (i.e. that return> > XML/XML-RPC, JSON, etc.).> >> > My input would be that we need the default PropelPDO to be as> > unencumbered as possible (fast). If we're going to keep all the debug> > stuff in a separate object, then it doesn't matter much how that> > performs. If, however, the aim is to merge this into PropelPDO, then we> > need to make sure that we're doing things in an optimized -- and yes,> > "correct" -- way. We definitely don't want to open the door for subtle> > PHP issues.> >> > One thing I have learned from using Propel + PDO in PHP 5.2.2 is that> > PDO is not particularly bug-free :) I have dealt with a number of> > segfaults that are directly traceable to PDO. Sometimes it is the> > result of a malformed query (why this segfaults, I have no idea) --- or> > use of PDO streams within user-defined stream wrappers. I *hate*> > debugging segfaults in PHP; what a waste of time. There's apparently> > some really messy stuff under the hood there :) Let's do what we can to> > keep it from rearing its ugly head for users.> >> > Hans> >> > --------------------​--------------------​--------------------​---------> > To unsubscribe, e-mail: dev-unsubscribe@prop​el.tigris.org> > For additional commands, e-mail: dev-help at propel dot tigris dot org> >> >>>> --> Christian Kassab>> Mobile:> +44 (0)789 4467 606> +49 (0)163 6360 939>> Mail:> chris dot kassab at binaryworx dot net> chris dot kassab at gmail dot com>> Web:> http://www.binaryworx.net> http://www.shoreflyer.net>> --------------------​--------------------​--------------------​---------> To unsubscribe, e-mail: dev-unsubscribe@prop​el.tigris.org> For additional commands, e-mail: dev-help at propel dot tigris dot org>>

Christian Kassab wrote:> Hi,> > why stuff so much debugging functionality into Propel? It is bloated> as it is already. How about using debugging tools like xdebug and> Eclipse? Yes I know you can't look into what goes on inside PDO as in> what the final query string looks like after binding the> variables/parameters, but thats another story...

Well, you can't even see them if you're tailing your postgres logs ifyou're using prepared statements...

> If you really need the lower level debugging inside Propel then add it> in a non obtrusive way by enabling dependency injection of classes> that subclass the default ones but add the debugging functionality and> then call the parent methods etc.

why stuff so much debugging functionality into Propel? It is bloatedas it is already. How about using debugging tools like xdebug andEclipse? Yes I know you can't look into what goes on inside PDO as inwhat the final query string looks like after binding thevariables/parameters, but thats another story...

If you really need the lower level debugging inside Propel then add itin a non obtrusive way by enabling dependency injection of classesthat subclass the default ones but add the debugging functionality andthen call the parent methods etc.

Cheers,

Christian

On 19/07/07, Hans Lellelid <hans at velum dot net> wrote:> Oliver Schonrock wrote:> > Cameron Brunner wrote:> >> Silence is good bad or otherwise?> >> > Don't take offense please. I like the features, but I am not sure about> > the implementation:> >> > - using static $query seems untidy> > - using __destruct() can cause problems depending on your environment> > - using direct var_dump() it not always suitable.> >> > if we are going to add this to the core propel repository, my view would> > be that it should meet the same criteria as the rest of the code as a> > generic, high quality toolkit.> >> > Part of the reason I focused on simple features was that it is not> > obvious how to solve some of the above problems with the more advanced> > features while adhering to the quality standards of the rest of the> > code. ie can run in any environment, etc..> >> > Opinions?>> Without having looked at the code, I second this opinion. We definitely> don't want to use var_dump, for instance, as this is not particularly> useful when building webapps that don't return HTML (i.e. that return> XML/XML-RPC, JSON, etc.).>> My input would be that we need the default PropelPDO to be as> unencumbered as possible (fast). If we're going to keep all the debug> stuff in a separate object, then it doesn't matter much how that> performs. If, however, the aim is to merge this into PropelPDO, then we> need to make sure that we're doing things in an optimized -- and yes,> "correct" -- way. We definitely don't want to open the door for subtle> PHP issues.>> One thing I have learned from using Propel + PDO in PHP 5.2.2 is that> PDO is not particularly bug-free :) I have dealt with a number of> segfaults that are directly traceable to PDO. Sometimes it is the> result of a malformed query (why this segfaults, I have no idea) --- or> use of PDO streams within user-defined stream wrappers. I *hate*> debugging segfaults in PHP; what a waste of time. There's apparently> some really messy stuff under the hood there :) Let's do what we can to> keep it from rearing its ugly head for users.>> Hans>> --------------------​--------------------​--------------------​---------> To unsubscribe, e-mail: dev-unsubscribe@prop​el.tigris.org> For additional commands, e-mail: dev-help at propel dot tigris dot org>>

Oliver Schonrock wrote:> Cameron Brunner wrote:>> Silence is good bad or otherwise?> > Don't take offense please. I like the features, but I am not sure about> the implementation:> > - using static $query seems untidy> - using __destruct() can cause problems depending on your environment> - using direct var_dump() it not always suitable.> > if we are going to add this to the core propel repository, my view would> be that it should meet the same criteria as the rest of the code as a> generic, high quality toolkit.> > Part of the reason I focused on simple features was that it is not> obvious how to solve some of the above problems with the more advanced> features while adhering to the quality standards of the rest of the> code. ie can run in any environment, etc..> > Opinions?

Without having looked at the code, I second this opinion. We definitelydon't want to use var_dump, for instance, as this is not particularlyuseful when building webapps that don't return HTML (i.e. that returnXML/XML-RPC, JSON, etc.).

My input would be that we need the default PropelPDO to be asunencumbered as possible (fast). If we're going to keep all the debugstuff in a separate object, then it doesn't matter much how thatperforms. If, however, the aim is to merge this into PropelPDO, then weneed to make sure that we're doing things in an optimized -- and yes,"correct" -- way. We definitely don't want to open the door for subtlePHP issues.

One thing I have learned from using Propel + PDO in PHP 5.2.2 is thatPDO is not particularly bug-free :) I have dealt with a number ofsegfaults that are directly traceable to PDO. Sometimes it is theresult of a malformed query (why this segfaults, I have no idea) --- oruse of PDO streams within user-defined stream wrappers. I *hate*debugging segfaults in PHP; what a waste of time. There's apparentlysome really messy stuff under the hood there :) Let's do what we can tokeep it from rearing its ugly head for users.

Don't take offense please. I like the features, but I am not sure aboutthe implementation:

- using static $query seems untidy- using __destruct() can cause problems depending on your environment- using direct var_dump() it not always suitable.

if we are going to add this to the core propel repository, my view wouldbe that it should meet the same criteria as the rest of the code as ageneric, high quality toolkit.

Part of the reason I focused on simple features was that it is notobvious how to solve some of the above problems with the more advancedfeatures while adhering to the quality standards of the rest of thecode. ie can run in any environment, etc..

On 7/14/07, Cameron Brunner <cameron.brunner@​gmail.com> wrote:> Just a note...>> TODO: add debug_backtrace() into it to have a what called this method> in the first place to cause this query to run> count the total transactions ran in the query (ideally should stay at 1)> make a nice interface for displaying them both on console and in html> (automatically colorcode queries based on performance, automatic> gradient upto red for the slowest queries)> actually test all the different ways to query pdo> ... any requests? auto EXPLAIN ANALYZE and stuff it into the debug> output as well?>>> oh, found a bug in query() where it wasn't using the new phase stuff i> wrote just now, propel doesn't use that normally tho so this patch is> still fine with a standard propel setup, feel free to try it and see> what you think so far...>> On 7/14/07, Cameron Brunner <cameron.brunner@​gmail.com> wrote:> > Try this one on for size :) Tested code even!> >> > On 7/14/07, Cameron Brunner <cameron.brunner@​gmail.com> wrote:> > > On 7/14/07, Oliver Schonrock <oliver at realtsp dot com> wrote:> > > > Cameron Brunner wrote:> > > > > A - We need to store the entire query in memory if we are to do> > > > > duplicate query detection.> > > >> > > > true, however (connected with what you say in D) the> > > > PDOStatement->queryString property only gives us the prepared statement> > > > sql not the final, "all params substituted" sql. This is a limitation in> > > > PDO and probably for good reason since the param substitution is> > > > supposed to happen in the DB server for some backends (not mysql due to> > > > query cache limitations).> > > >> > > > This means that we do not, and will not for some time, have access to> > > > the final sql string. Does "duplicate detection" make sense in this> > > > case? Is it a valid feature if we are only looking at prepared> > > > statements? There will be many "duplicated" queiries for> > > > retrieveByPk(1|2|3) for example....> > >> > > Good point, time to tear the prepared statement parser out of Creole> > > then. Duplicate query detection is a big thing in my books.> > >> > > > > B - If we dont, we incur quite an overhead to do func_get_args and> > > > > call_user_func etc so if we dont we pay a penalty even in production> > > > > hence the way i did that> > > >> > > > hmm..good point. i did some simple tests over 1000 calls:> > > >> > > > plain call a(1,2,3,4): 1.5ms> > > > plain call $a->a(1,2,3,4): 1.6ms> > > > call_user_func_array('a', array(1,2,3,4)): 3.6ms> > > > call_user_func_array(array($a, 'a'), array(1,2,3,4)): 4.6ms> > > >> > > > big percentage increase, still low given that every db query takes ~1ms> > > > no matter how trivial. But worth considering when we think about how to> > > > implement these calls. I will revisit this and rewrite as a native call.> > > >> > > > > C - true, specially if you have unbuffered queries enabled, its just a> > > > > bit of a guide on what returns a lot of rows. We could also add a> > > > > memory usage check onto the dtor, that would be about when all rows> > > > > are hydrated... we could also add the true flag to the memory usage> > > > > functions, i dont know the difference tho> > > >> > > > i am not sure the memory_usage thing is very useful due to these> > > > limitations. Should perhaps be dealt with at a profiling level further> > > > out than PDO or even outside Propel.> > >> > > Agreed that its better to do it with real profiling but a quick> > > snapshot while a system is still in development without having to jump> > > through hoops is certainly a bonus in my books. To get a better memory> > > usage we can add a memory check in the dtor for the statement and that> > > should kick in when the objects are done with hydration... ?> > >> > > > > D - We can still add these into PropelPDO and since DebugPropelPDO> > > > > extends it that will run automatically still... nice and seamless and> > > > > easy... i have had propel's name cursed at work since we couldnt see> > > > > the final sql that was being ran easily (having to replace the ?'s by> > > > > hand)> > > >> > > > I am trying to clarify what features we can realistically hope to> > > > achieve and where they should sit. In my opinion, while duplicate query> > > > detection and memory consumption are desirable, they are not really> > > > possible given where we are measuring. That then leaves query timing and> > > > counting.> > > >> > > > As far as I can see these can be easily added to the existing> > > > PropelPDO.php while adding a PropelPDOStatement.php. Making sure that we> > > > use efficient call mechanisms for the extensions.> > > >> > > > Counting could be driven by a $debug switch (although I would prefer a> > > > setter rather than a public property).> > >> > > Granted, static method would be preferable but it was a quick patch> > > (half hr before work)> > >> > > > Timing could be switched on if logging is enabled and $debug=true. The> > > > time written to the log with the sql string since these belong together> > > > and there doesn't seem to be a case for keeping the sql in memory.> > > >> > > > Moving logging to exec time rather than prepare poses several problems> > > > with backward compat and persistent connections. I will prepare a new> > > > patch which considers these options and suggests a solution, bearing> > > > performance in mind.> > >> > > Persistent connections are something that I personally discourage in> > > any situation, is there any databases that really need this these days> > > to get optimal performance? The overhead for mysql and postgres is> > > quite minimal... sqlite is the only real db that i know gains> > > something from persistence.> > >> > > > > Always plenty of options, i have been wanting a way to automatically> > > > > profile an application in development and when a 'slow' or 'expensive'> > > > > (memory wise) query is ran then to display it. I was considering a> > > > > Smarty debug style popup to be able to be enabled...> > > >> > > > we have profiling information automatically at bottom of every> > > > non-production page, just in the html. it works well.> > > >> > > > hope we are getting closer.> > > >> > > > Oliver> > > >> > > >> > > >> > >> > > You're not the only one who hopes we are getting closer to the right> > > solution here. I'm bored at the moment so i'm about to play around> > > with this and maybe more work on the PropelDateTime stuff.> > >> > > --> > > Cameron Brunner> > >> > > Want a better web browser?> > > http://www.spreadfir​efox.com/?q=affiliat​es&id=182780​&t=1> > >> >> >> > --> > Cameron Brunner> >> > Want a better web browser?> > http://www.spreadfir​efox.com/?q=affiliat​es&id=182780​&t=1> >> >>>> --> Cameron Brunner>> Want a better web browser?> http://www.spreadfir​efox.com/?q=affiliat​es&id=182780​&t=1>

TODO: add debug_backtrace() into it to have a what called this methodin the first place to cause this query to runcount the total transactions ran in the query (ideally should stay at 1)make a nice interface for displaying them both on console and in html(automatically colorcode queries based on performance, automaticgradient upto red for the slowest queries)actually test all the different ways to query pdo... any requests? auto EXPLAIN ANALYZE and stuff it into the debugoutput as well?

oh, found a bug in query() where it wasn't using the new phase stuff iwrote just now, propel doesn't use that normally tho so this patch isstill fine with a standard propel setup, feel free to try it and seewhat you think so far...

On 7/14/07, Cameron Brunner <cameron.brunner@​gmail.com> wrote:> Try this one on for size :) Tested code even!>> On 7/14/07, Cameron Brunner <cameron.brunner@​gmail.com> wrote:> > On 7/14/07, Oliver Schonrock <oliver at realtsp dot com> wrote:> > > Cameron Brunner wrote:> > > > A - We need to store the entire query in memory if we are to do> > > > duplicate query detection.> > >> > > true, however (connected with what you say in D) the> > > PDOStatement->queryString property only gives us the prepared statement> > > sql not the final, "all params substituted" sql. This is a limitation in> > > PDO and probably for good reason since the param substitution is> > > supposed to happen in the DB server for some backends (not mysql due to> > > query cache limitations).> > >> > > This means that we do not, and will not for some time, have access to> > > the final sql string. Does "duplicate detection" make sense in this> > > case? Is it a valid feature if we are only looking at prepared> > > statements? There will be many "duplicated" queiries for> > > retrieveByPk(1|2|3) for example....> >> > Good point, time to tear the prepared statement parser out of Creole> > then. Duplicate query detection is a big thing in my books.> >> > > > B - If we dont, we incur quite an overhead to do func_get_args and> > > > call_user_func etc so if we dont we pay a penalty even in production> > > > hence the way i did that> > >> > > hmm..good point. i did some simple tests over 1000 calls:> > >> > > plain call a(1,2,3,4): 1.5ms> > > plain call $a->a(1,2,3,4): 1.6ms> > > call_user_func_array('a', array(1,2,3,4)): 3.6ms> > > call_user_func_array(array($a, 'a'), array(1,2,3,4)): 4.6ms> > >> > > big percentage increase, still low given that every db query takes ~1ms> > > no matter how trivial. But worth considering when we think about how to> > > implement these calls. I will revisit this and rewrite as a native call.> > >> > > > C - true, specially if you have unbuffered queries enabled, its just a> > > > bit of a guide on what returns a lot of rows. We could also add a> > > > memory usage check onto the dtor, that would be about when all rows> > > > are hydrated... we could also add the true flag to the memory usage> > > > functions, i dont know the difference tho> > >> > > i am not sure the memory_usage thing is very useful due to these> > > limitations. Should perhaps be dealt with at a profiling level further> > > out than PDO or even outside Propel.> >> > Agreed that its better to do it with real profiling but a quick> > snapshot while a system is still in development without having to jump> > through hoops is certainly a bonus in my books. To get a better memory> > usage we can add a memory check in the dtor for the statement and that> > should kick in when the objects are done with hydration... ?> >> > > > D - We can still add these into PropelPDO and since DebugPropelPDO> > > > extends it that will run automatically still... nice and seamless and> > > > easy... i have had propel's name cursed at work since we couldnt see> > > > the final sql that was being ran easily (having to replace the ?'s by> > > > hand)> > >> > > I am trying to clarify what features we can realistically hope to> > > achieve and where they should sit. In my opinion, while duplicate query> > > detection and memory consumption are desirable, they are not really> > > possible given where we are measuring. That then leaves query timing and> > > counting.> > >> > > As far as I can see these can be easily added to the existing> > > PropelPDO.php while adding a PropelPDOStatement.php. Making sure that we> > > use efficient call mechanisms for the extensions.> > >> > > Counting could be driven by a $debug switch (although I would prefer a> > > setter rather than a public property).> >> > Granted, static method would be preferable but it was a quick patch> > (half hr before work)> >> > > Timing could be switched on if logging is enabled and $debug=true. The> > > time written to the log with the sql string since these belong together> > > and there doesn't seem to be a case for keeping the sql in memory.> > >> > > Moving logging to exec time rather than prepare poses several problems> > > with backward compat and persistent connections. I will prepare a new> > > patch which considers these options and suggests a solution, bearing> > > performance in mind.> >> > Persistent connections are something that I personally discourage in> > any situation, is there any databases that really need this these days> > to get optimal performance? The overhead for mysql and postgres is> > quite minimal... sqlite is the only real db that i know gains> > something from persistence.> >> > > > Always plenty of options, i have been wanting a way to automatically> > > > profile an application in development and when a 'slow' or 'expensive'> > > > (memory wise) query is ran then to display it. I was considering a> > > > Smarty debug style popup to be able to be enabled...> > >> > > we have profiling information automatically at bottom of every> > > non-production page, just in the html. it works well.> > >> > > hope we are getting closer.> > >> > > Oliver> > >> > >> > >> >> > You're not the only one who hopes we are getting closer to the right> > solution here. I'm bored at the moment so i'm about to play around> > with this and maybe more work on the PropelDateTime stuff.> >> > --> > Cameron Brunner> >> > Want a better web browser?> > http://www.spreadfir​efox.com/?q=affiliat​es&id=182780​&t=1> >>>> --> Cameron Brunner>> Want a better web browser?> http://www.spreadfir​efox.com/?q=affiliat​es&id=182780​&t=1>>

On 7/14/07, Cameron Brunner <cameron.brunner@​gmail.com> wrote:> On 7/14/07, Oliver Schonrock <oliver at realtsp dot com> wrote:> > Cameron Brunner wrote:> > > A - We need to store the entire query in memory if we are to do> > > duplicate query detection.> >> > true, however (connected with what you say in D) the> > PDOStatement->queryString property only gives us the prepared statement> > sql not the final, "all params substituted" sql. This is a limitation in> > PDO and probably for good reason since the param substitution is> > supposed to happen in the DB server for some backends (not mysql due to> > query cache limitations).> >> > This means that we do not, and will not for some time, have access to> > the final sql string. Does "duplicate detection" make sense in this> > case? Is it a valid feature if we are only looking at prepared> > statements? There will be many "duplicated" queiries for> > retrieveByPk(1|2|3) for example....>> Good point, time to tear the prepared statement parser out of Creole> then. Duplicate query detection is a big thing in my books.>> > > B - If we dont, we incur quite an overhead to do func_get_args and> > > call_user_func etc so if we dont we pay a penalty even in production> > > hence the way i did that> >> > hmm..good point. i did some simple tests over 1000 calls:> >> > plain call a(1,2,3,4): 1.5ms> > plain call $a->a(1,2,3,4): 1.6ms> > call_user_func_array('a', array(1,2,3,4)): 3.6ms> > call_user_func_array(array($a, 'a'), array(1,2,3,4)): 4.6ms> >> > big percentage increase, still low given that every db query takes ~1ms> > no matter how trivial. But worth considering when we think about how to> > implement these calls. I will revisit this and rewrite as a native call.> >> > > C - true, specially if you have unbuffered queries enabled, its just a> > > bit of a guide on what returns a lot of rows. We could also add a> > > memory usage check onto the dtor, that would be about when all rows> > > are hydrated... we could also add the true flag to the memory usage> > > functions, i dont know the difference tho> >> > i am not sure the memory_usage thing is very useful due to these> > limitations. Should perhaps be dealt with at a profiling level further> > out than PDO or even outside Propel.>> Agreed that its better to do it with real profiling but a quick> snapshot while a system is still in development without having to jump> through hoops is certainly a bonus in my books. To get a better memory> usage we can add a memory check in the dtor for the statement and that> should kick in when the objects are done with hydration... ?>> > > D - We can still add these into PropelPDO and since DebugPropelPDO> > > extends it that will run automatically still... nice and seamless and> > > easy... i have had propel's name cursed at work since we couldnt see> > > the final sql that was being ran easily (having to replace the ?'s by> > > hand)> >> > I am trying to clarify what features we can realistically hope to> > achieve and where they should sit. In my opinion, while duplicate query> > detection and memory consumption are desirable, they are not really> > possible given where we are measuring. That then leaves query timing and> > counting.> >> > As far as I can see these can be easily added to the existing> > PropelPDO.php while adding a PropelPDOStatement.php. Making sure that we> > use efficient call mechanisms for the extensions.> >> > Counting could be driven by a $debug switch (although I would prefer a> > setter rather than a public property).>> Granted, static method would be preferable but it was a quick patch> (half hr before work)>> > Timing could be switched on if logging is enabled and $debug=true. The> > time written to the log with the sql string since these belong together> > and there doesn't seem to be a case for keeping the sql in memory.> >> > Moving logging to exec time rather than prepare poses several problems> > with backward compat and persistent connections. I will prepare a new> > patch which considers these options and suggests a solution, bearing> > performance in mind.>> Persistent connections are something that I personally discourage in> any situation, is there any databases that really need this these days> to get optimal performance? The overhead for mysql and postgres is> quite minimal... sqlite is the only real db that i know gains> something from persistence.>> > > Always plenty of options, i have been wanting a way to automatically> > > profile an application in development and when a 'slow' or 'expensive'> > > (memory wise) query is ran then to display it. I was considering a> > > Smarty debug style popup to be able to be enabled...> >> > we have profiling information automatically at bottom of every> > non-production page, just in the html. it works well.> >> > hope we are getting closer.> >> > Oliver> >> >> >>> You're not the only one who hopes we are getting closer to the right> solution here. I'm bored at the moment so i'm about to play around> with this and maybe more work on the PropelDateTime stuff.>> --> Cameron Brunner>> Want a better web browser?> http://www.spreadfir​efox.com/?q=affiliat​es&id=182780​&t=1>

On 7/14/07, Oliver Schonrock <oliver at realtsp dot com> wrote:> Cameron Brunner wrote:> > A - We need to store the entire query in memory if we are to do> > duplicate query detection.>> true, however (connected with what you say in D) the> PDOStatement->queryString property only gives us the prepared statement> sql not the final, "all params substituted" sql. This is a limitation in> PDO and probably for good reason since the param substitution is> supposed to happen in the DB server for some backends (not mysql due to> query cache limitations).>> This means that we do not, and will not for some time, have access to> the final sql string. Does "duplicate detection" make sense in this> case? Is it a valid feature if we are only looking at prepared> statements? There will be many "duplicated" queiries for> retrieveByPk(1|2|3) for example....

Good point, time to tear the prepared statement parser out of Creolethen. Duplicate query detection is a big thing in my books.

> > B - If we dont, we incur quite an overhead to do func_get_args and> > call_user_func etc so if we dont we pay a penalty even in production> > hence the way i did that>> hmm..good point. i did some simple tests over 1000 calls:>> plain call a(1,2,3,4): 1.5ms> plain call $a->a(1,2,3,4): 1.6ms> call_user_func_array('a', array(1,2,3,4)): 3.6ms> call_user_func_array(array($a, 'a'), array(1,2,3,4)): 4.6ms>> big percentage increase, still low given that every db query takes ~1ms> no matter how trivial. But worth considering when we think about how to> implement these calls. I will revisit this and rewrite as a native call.>> > C - true, specially if you have unbuffered queries enabled, its just a> > bit of a guide on what returns a lot of rows. We could also add a> > memory usage check onto the dtor, that would be about when all rows> > are hydrated... we could also add the true flag to the memory usage> > functions, i dont know the difference tho>> i am not sure the memory_usage thing is very useful due to these> limitations. Should perhaps be dealt with at a profiling level further> out than PDO or even outside Propel.

Agreed that its better to do it with real profiling but a quicksnapshot while a system is still in development without having to jumpthrough hoops is certainly a bonus in my books. To get a better memoryusage we can add a memory check in the dtor for the statement and thatshould kick in when the objects are done with hydration... ?

> > D - We can still add these into PropelPDO and since DebugPropelPDO> > extends it that will run automatically still... nice and seamless and> > easy... i have had propel's name cursed at work since we couldnt see> > the final sql that was being ran easily (having to replace the ?'s by> > hand)>> I am trying to clarify what features we can realistically hope to> achieve and where they should sit. In my opinion, while duplicate query> detection and memory consumption are desirable, they are not really> possible given where we are measuring. That then leaves query timing and> counting.>> As far as I can see these can be easily added to the existing> PropelPDO.php while adding a PropelPDOStatement.php. Making sure that we> use efficient call mechanisms for the extensions.>> Counting could be driven by a $debug switch (although I would prefer a> setter rather than a public property).

Granted, static method would be preferable but it was a quick patch(half hr before work)

> Timing could be switched on if logging is enabled and $debug=true. The> time written to the log with the sql string since these belong together> and there doesn't seem to be a case for keeping the sql in memory.>> Moving logging to exec time rather than prepare poses several problems> with backward compat and persistent connections. I will prepare a new> patch which considers these options and suggests a solution, bearing> performance in mind.

Persistent connections are something that I personally discourage inany situation, is there any databases that really need this these daysto get optimal performance? The overhead for mysql and postgres isquite minimal... sqlite is the only real db that i know gainssomething from persistence.

> > Always plenty of options, i have been wanting a way to automatically> > profile an application in development and when a 'slow' or 'expensive'> > (memory wise) query is ran then to display it. I was considering a> > Smarty debug style popup to be able to be enabled...>> we have profiling information automatically at bottom of every> non-production page, just in the html. it works well.>> hope we are getting closer.>> Oliver>>>

You're not the only one who hopes we are getting closer to the rightsolution here. I'm bored at the moment so i'm about to play aroundwith this and maybe more work on the PropelDateTime stuff.

I'm tempted to move any query logging support into DebugPropelPDO andmake the 'production' PropelPDO as fast as possible after splittingthis off...

On 7/13/07, Hans Lellelid <hans at velum dot net> wrote:> Just chiming in to say that all this makes sense to me :) I definitely> agree that a debug flag is a good idea. I think we should keep the> default PropelPDO system "production ready". We shouldn't need to swap> the PDOStatement object for non-debug PropelPDO, right?>> I like the idea of moving the query logging into the> PropelDebugPDOStatement (or whatever it's called) class.>> Hans>> Cameron Brunner wrote:> > A - We need to store the entire query in memory if we are to do> > duplicate query detection.> > B - If we dont, we incur quite an overhead to do func_get_args and> > call_user_func etc so if we dont we pay a penalty even in production> > hence the way i did that> > C - true, specially if you have unbuffered queries enabled, its just a> > bit of a guide on what returns a lot of rows. We could also add a> > memory usage check onto the dtor, that would be about when all rows> > are hydrated... we could also add the true flag to the memory usage> > functions, i dont know the difference tho> > D - We can still add these into PropelPDO and since DebugPropelPDO> > extends it that will run automatically still... nice and seamless and> > easy... i have had propel's name cursed at work since we couldnt see> > the final sql that was being ran easily (having to replace the ?'s by> > hand)> >> > Always plenty of options, i have been wanting a way to automatically> > profile an application in development and when a 'slow' or 'expensive'> > (memory wise) query is ran then to display it. I was considering a> > Smarty debug style popup to be able to be enabled...> >> > Anyway, thats where i was going with it. I think i answered everything> > in your email but im freezing so im gonna go crawl under a blanket> > instead of sitting here. Hit me up with any questions or thoughts on> > this.> >> > On 7/13/07, Oliver Schonrock <oliver at realtsp dot com> wrote:> >> Cameron Brunner wrote:> >> > I would be more interested at this point in offering DebugPropelPDO as> >> > a connection object type and adding a full array of debug info (query> >> > timing, counting, dupe query detection) to it and have a flag that> >> > switches to it when requested. Query timing generally interests me> >> > more than the count.> >>> >> I agree with much of this. Query timing would certainly be useful.> >>> >> There are questions about the tactical details, most of which are> >> stylistic and there is no "right" answer:> >>> >> a) do we want a full array of sql strings in an array in memory? Would> >> this extra information be better in the query log which is already> >> storing the sql strings (if configured in runtime conf).> >>> >> b) Do we want to have a "debug" switch in Propel to tell it which PDO> >> class to instantiate or just add the functionality to the standard> >> PropelPDO class and then enable the extra logging/counting with> >> build|runtime config (similar to log now). If we do want to instantiate> >> different PDO classes, then would we rather pass a Classname into propel> >> via the build|runtime config? This would give the Propel user lots of> >> flexibility> >>> >> c) memory usage reporting as implemented in Cameron's patch may be> >> slightly misleading as the Stmt::execute (et al) calls don't really use> >> much memory compared to instantiating the objects from the result set.> >> Even if you have mysql_buffered_query turned on, the memory used for> >> buffering the query result is not reported by memory_get_usage(),> >> although you will see your apache process grow in ps|top (substitute> >> your own tools here for non *nix/apache users).> >>> >> d) If we are going to provide a PDOStatement extension as part of> >> propel, then this gives us the option of moving the Propel::log() call> >> from PropelPDO::prepare() to the same 3 places where I had incQueryCount> >> and Cameron had debugQueryLog() (ie 2 places in PropelPDO, and one in> >> PropelPDOStatement). This would mean we make log file entries when the> >> query is actually run rather than prepared plus allows the addition of> >> the extra info (like query time) into the log file.> >>> >> Thoughts? Opinions?> >>> >>> >> --> >>> >> Oliver Schonrock> >>> >>> >>> >> >>> --------------------​--------------------​--------------------​---------> To unsubscribe, e-mail: dev-unsubscribe@prop​el.tigris.org> For additional commands, e-mail: dev-help at propel dot tigris dot org>>

Cameron Brunner wrote:> A - We need to store the entire query in memory if we are to do> duplicate query detection.

true, however (connected with what you say in D) thePDOStatement->queryString property only gives us the prepared statementsql not the final, "all params substituted" sql. This is a limitation inPDO and probably for good reason since the param substitution issupposed to happen in the DB server for some backends (not mysql due toquery cache limitations).

This means that we do not, and will not for some time, have access tothe final sql string. Does "duplicate detection" make sense in thiscase? Is it a valid feature if we are only looking at preparedstatements? There will be many "duplicated" queiries forretrieveByPk(1|2|3) for example....

> B - If we dont, we incur quite an overhead to do func_get_args and> call_user_func etc so if we dont we pay a penalty even in production> hence the way i did that

big percentage increase, still low given that every db query takes ~1msno matter how trivial. But worth considering when we think about how toimplement these calls. I will revisit this and rewrite as a native call.

> C - true, specially if you have unbuffered queries enabled, its just a> bit of a guide on what returns a lot of rows. We could also add a> memory usage check onto the dtor, that would be about when all rows> are hydrated... we could also add the true flag to the memory usage> functions, i dont know the difference tho

i am not sure the memory_usage thing is very useful due to theselimitations. Should perhaps be dealt with at a profiling level furtherout than PDO or even outside Propel.

> D - We can still add these into PropelPDO and since DebugPropelPDO> extends it that will run automatically still... nice and seamless and> easy... i have had propel's name cursed at work since we couldnt see> the final sql that was being ran easily (having to replace the ?'s by> hand)

I am trying to clarify what features we can realistically hope toachieve and where they should sit. In my opinion, while duplicate querydetection and memory consumption are desirable, they are not reallypossible given where we are measuring. That then leaves query timing andcounting.

As far as I can see these can be easily added to the existingPropelPDO.php while adding a PropelPDOStatement.php. Making sure that weuse efficient call mechanisms for the extensions.

Counting could be driven by a $debug switch (although I would prefer asetter rather than a public property).

Timing could be switched on if logging is enabled and $debug=true. Thetime written to the log with the sql string since these belong togetherand there doesn't seem to be a case for keeping the sql in memory.

Moving logging to exec time rather than prepare poses several problemswith backward compat and persistent connections. I will prepare a newpatch which considers these options and suggests a solution, bearingperformance in mind.

> Always plenty of options, i have been wanting a way to automatically> profile an application in development and when a 'slow' or 'expensive'> (memory wise) query is ran then to display it. I was considering a> Smarty debug style popup to be able to be enabled...

we have profiling information automatically at bottom of everynon-production page, just in the html. it works well.

Just chiming in to say that all this makes sense to me :) I definitelyagree that a debug flag is a good idea. I think we should keep thedefault PropelPDO system "production ready". We shouldn't need to swapthe PDOStatement object for non-debug PropelPDO, right?

I like the idea of moving the query logging into thePropelDebugPDOStatement (or whatever it's called) class.

Hans

Cameron Brunner wrote:> A - We need to store the entire query in memory if we are to do> duplicate query detection.> B - If we dont, we incur quite an overhead to do func_get_args and> call_user_func etc so if we dont we pay a penalty even in production> hence the way i did that> C - true, specially if you have unbuffered queries enabled, its just a> bit of a guide on what returns a lot of rows. We could also add a> memory usage check onto the dtor, that would be about when all rows> are hydrated... we could also add the true flag to the memory usage> functions, i dont know the difference tho> D - We can still add these into PropelPDO and since DebugPropelPDO> extends it that will run automatically still... nice and seamless and> easy... i have had propel's name cursed at work since we couldnt see> the final sql that was being ran easily (having to replace the ?'s by> hand)> > Always plenty of options, i have been wanting a way to automatically> profile an application in development and when a 'slow' or 'expensive'> (memory wise) query is ran then to display it. I was considering a> Smarty debug style popup to be able to be enabled...> > Anyway, thats where i was going with it. I think i answered everything> in your email but im freezing so im gonna go crawl under a blanket> instead of sitting here. Hit me up with any questions or thoughts on> this.> > On 7/13/07, Oliver Schonrock <oliver at realtsp dot com> wrote:>> Cameron Brunner wrote:>> > I would be more interested at this point in offering DebugPropelPDO as>> > a connection object type and adding a full array of debug info (query>> > timing, counting, dupe query detection) to it and have a flag that>> > switches to it when requested. Query timing generally interests me>> > more than the count.>>>> I agree with much of this. Query timing would certainly be useful.>>>> There are questions about the tactical details, most of which are>> stylistic and there is no "right" answer:>>>> a) do we want a full array of sql strings in an array in memory? Would>> this extra information be better in the query log which is already>> storing the sql strings (if configured in runtime conf).>>>> b) Do we want to have a "debug" switch in Propel to tell it which PDO>> class to instantiate or just add the functionality to the standard>> PropelPDO class and then enable the extra logging/counting with>> build|runtime config (similar to log now). If we do want to instantiate>> different PDO classes, then would we rather pass a Classname into propel>> via the build|runtime config? This would give the Propel user lots of>> flexibility>>>> c) memory usage reporting as implemented in Cameron's patch may be>> slightly misleading as the Stmt::execute (et al) calls don't really use>> much memory compared to instantiating the objects from the result set.>> Even if you have mysql_buffered_query turned on, the memory used for>> buffering the query result is not reported by memory_get_usage(),>> although you will see your apache process grow in ps|top (substitute>> your own tools here for non *nix/apache users).>>>> d) If we are going to provide a PDOStatement extension as part of>> propel, then this gives us the option of moving the Propel::log() call>> from PropelPDO::prepare() to the same 3 places where I had incQueryCount>> and Cameron had debugQueryLog() (ie 2 places in PropelPDO, and one in>> PropelPDOStatement). This would mean we make log file entries when the>> query is actually run rather than prepared plus allows the addition of>> the extra info (like query time) into the log file.>>>> Thoughts? Opinions?>>>>>> -- >>>> Oliver Schonrock>>>>>>> >

Cameron Brunner schreef:> Always plenty of options, i have been wanting a way to automatically> profile an application in development and when a 'slow' or 'expensive'> (memory wise) query is ran then to display it. I was considering a> Smarty debug style popup to be able to be enabled...

A - We need to store the entire query in memory if we are to doduplicate query detection.B - If we dont, we incur quite an overhead to do func_get_args andcall_user_func etc so if we dont we pay a penalty even in productionhence the way i did thatC - true, specially if you have unbuffered queries enabled, its just abit of a guide on what returns a lot of rows. We could also add amemory usage check onto the dtor, that would be about when all rowsare hydrated... we could also add the true flag to the memory usagefunctions, i dont know the difference thoD - We can still add these into PropelPDO and since DebugPropelPDOextends it that will run automatically still... nice and seamless andeasy... i have had propel's name cursed at work since we couldnt seethe final sql that was being ran easily (having to replace the ?'s byhand)

Always plenty of options, i have been wanting a way to automaticallyprofile an application in development and when a 'slow' or 'expensive'(memory wise) query is ran then to display it. I was considering aSmarty debug style popup to be able to be enabled...

Anyway, thats where i was going with it. I think i answered everythingin your email but im freezing so im gonna go crawl under a blanketinstead of sitting here. Hit me up with any questions or thoughts onthis.

On 7/13/07, Oliver Schonrock <oliver at realtsp dot com> wrote:> Cameron Brunner wrote:> > I would be more interested at this point in offering DebugPropelPDO as> > a connection object type and adding a full array of debug info (query> > timing, counting, dupe query detection) to it and have a flag that> > switches to it when requested. Query timing generally interests me> > more than the count.>> I agree with much of this. Query timing would certainly be useful.>> There are questions about the tactical details, most of which are> stylistic and there is no "right" answer:>> a) do we want a full array of sql strings in an array in memory? Would> this extra information be better in the query log which is already> storing the sql strings (if configured in runtime conf).>> b) Do we want to have a "debug" switch in Propel to tell it which PDO> class to instantiate or just add the functionality to the standard> PropelPDO class and then enable the extra logging/counting with> build|runtime config (similar to log now). If we do want to instantiate> different PDO classes, then would we rather pass a Classname into propel> via the build|runtime config? This would give the Propel user lots of> flexibility>> c) memory usage reporting as implemented in Cameron's patch may be> slightly misleading as the Stmt::execute (et al) calls don't really use> much memory compared to instantiating the objects from the result set.> Even if you have mysql_buffered_query turned on, the memory used for> buffering the query result is not reported by memory_get_usage(),> although you will see your apache process grow in ps|top (substitute> your own tools here for non *nix/apache users).>> d) If we are going to provide a PDOStatement extension as part of> propel, then this gives us the option of moving the Propel::log() call> from PropelPDO::prepare() to the same 3 places where I had incQueryCount> and Cameron had debugQueryLog() (ie 2 places in PropelPDO, and one in> PropelPDOStatement). This would mean we make log file entries when the> query is actually run rather than prepared plus allows the addition of> the extra info (like query time) into the log file.>> Thoughts? Opinions?>>> -->> Oliver Schonrock>>>

Cameron Brunner wrote:> I would be more interested at this point in offering DebugPropelPDO as> a connection object type and adding a full array of debug info (query> timing, counting, dupe query detection) to it and have a flag that> switches to it when requested. Query timing generally interests me> more than the count.

I agree with much of this. Query timing would certainly be useful.

There are questions about the tactical details, most of which arestylistic and there is no "right" answer:

a) do we want a full array of sql strings in an array in memory? Wouldthis extra information be better in the query log which is alreadystoring the sql strings (if configured in runtime conf).

b) Do we want to have a "debug" switch in Propel to tell it which PDOclass to instantiate or just add the functionality to the standardPropelPDO class and then enable the extra logging/counting withbuild|runtime config (similar to log now). If we do want to instantiatedifferent PDO classes, then would we rather pass a Classname into propelvia the build|runtime config? This would give the Propel user lots offlexibility

c) memory usage reporting as implemented in Cameron's patch may beslightly misleading as the Stmt::execute (et al) calls don't really usemuch memory compared to instantiating the objects from the result set.Even if you have mysql_buffered_query turned on, the memory used forbuffering the query result is not reported by memory_get_usage(),although you will see your apache process grow in ps|top (substituteyour own tools here for non *nix/apache users).

d) If we are going to provide a PDOStatement extension as part ofpropel, then this gives us the option of moving the Propel::log() callfrom PropelPDO::prepare() to the same 3 places where I had incQueryCountand Cameron had debugQueryLog() (ie 2 places in PropelPDO, and one inPropelPDOStatement). This would mean we make log file entries when thequery is actually run rather than prepared plus allows the addition ofthe extra info (like query time) into the log file.