Is there any good reason why we implement urlEncode and htmlEncode on
our own, when cgi.escape and urllib.quote exist?
I like renaming these functions, but I don't see any reason we should
reimplement them. (I encountered a particular problem with urlEncode,
which always uses + for space, even when it's not appropriate --
urllib.quote has a more complete interface)
If no one objects, I'll change WebUtils.Funcs, and add optional
arguments for the complete behavior.
Ian

On Tuesday, December 24, 2002, at 12:42 AM, Stuart Donaldson wrote:
> I am trying to nock off some of the bugs as can.
>
> I just fixed the bug in HTTPResponse setCookie(). And I also found
> some other problems with the expiration date handling which I fixed.
>
> It seems like the expiration date parsing might best be placed on the
> Cookie object itself. I didn't get that agressive with the fix at this
> time. Anyone see a reason not to do this?
>
> Also, there was a missing allMatches() function referenced in
> DateInterval. I am not a Python RE module expert. Does anyone see
> anything wrong with this code? It seems to work for me, and I just
> added it to CVS. Although I think it should go in a more general
> purpose location.
Oops -- I really should have tested that function much better before I
put it in. Thanks for cleaning it up. The implementation of allMatches
you gave was correct.
> There are a couple of other patches from Edmund Lian regarding deleting
> cookies. They look reasonable to me at first glance, if I don't hear
> any comments to the contrary in the next day or so, I'll apply them as
> well.
>
Yes, that would be good -- I like hiding browser compatibility issues if
we can.
Ian

I am trying to nock off some of the bugs as can.
I just fixed the bug in HTTPResponse setCookie(). And I also found some
other problems with the expiration date handling which I fixed.
It seems like the expiration date parsing might best be placed on the
Cookie object itself. I didn't get that agressive with the fix at this
time. Anyone see a reason not to do this?
Also, there was a missing allMatches() function referenced in
DateInterval. I am not a Python RE module expert. Does anyone see
anything wrong with this code? It seems to work for me, and I just
added it to CVS. Although I think it should go in a more general purpose
location.
def allMatches(source, regex):
"""Return a list of matches for regex in source
"""
pos = 0
end = len(source)
rv = []
match = regex.search(source, pos)
while match:
rv.append(match)
match = regex.search(source, match.end() )
return rv
There are a couple of other patches from Edmund Lian regarding deleting
cookies. They look reasonable to me at first glance, if I don't hear
any comments to the contrary in the next day or so, I'll apply them as well.
So,

If the ExtraPathInfo is set, serverSideInfoForRequest uses
serverSideInfoForRequestNewAlgorithm.
However, this is currently incompatible with the psp-handler approach
where the environment WK_ABSOLUTE is set to 1. The NewAlgorithm version
fails to check the absolute setting, and tries to process the absolute
URL as a context. That fails and the default context is used.
Any reason not to just put a similar check for absolutepath in the
serverSideInfoForRequestNewAlgorithm?
Is the intention to have the serverSideInfoForRequestNewAlgorithm
replace serverSideInfoForRequest? If so, does that mean that the
ExtraPathInfo application setting will be deprecated sometime?
-Stuart-

So before I start getting too comfortable with my knowledge of Webware,
I wanted to check on a couple of things that look like possible problems.
In the case of the psp-handler, the mod_webkit sets WK_ABSOLUTE=1 and
then HTTPRequest.py urlPath()/fsPath() will return a path constructed
from the REQUEST_URI and the DOCUMENT_ROOT. This breaks when
referencing files under user directories, because it does not properly
deal with the /~user/ expansion on Apache/Unix systems.
However, it appears in the limited testing I have done, that the
SCRIPT_FILENAME always refers to the absolute server side path for the
file, including any ~expansion for the username. (note: we can't do our
own ~ expansion, without adding code to check where Apache has the user
public_html directories located, since that is a configuration option in
Apache.)
Furthermore, in both with direct URL's (ie: no ~user expansion) and with
the ~user URL's, the extra URLPath information is not properly parsed
off of the .psp pages. However the PATH_INFO environment variable does
contain the extra URL information.
So in the case of the psp-handler / WK_ABSOLUTE=1 condition, it appears
(using Apache 1.3.26 under Linux) that the urlPath() should return the
SCRIPT_FILENAME and the extraURLPath should be set to PATH_INFO, rather
than trying to parse it.
Under the normal webkit-handler it does not appear to be a problem, as
the PATH_INFO contains the URL after parsing off the location where the
webware handler resides. And the PATH_INFO must then be parsed manually
to determine the context and other information.
Has anyone else tried the psp-handler and either extraURL information,
or the ~user expansion?
-Stuart-

Stuart Donaldson [mailto:stu@...] wrote:
> The shutdown code in ThreadedAppServer.shutDown() sends a None to all
> threads to cause them to terminate, and then does a join to
> each thread.
>
> The problem is that if you have a thread that is busy, perhaps a
> server-push thread that is just sending a continuous stream
> of data, or
> a thread bound up for some other reason that they will not terminate.
>
> This causes the shutDown to lock up.
>
> A couple of questions:
>
> 1) Should there be a timeout when shutting down, after which it just
> forcibly terminates the thread, or just ignores the thread
> allowing the
> shutDown operation to continue?
There's no way to forcibly terminate a thread in Python (that I know of).
>
> 2) Should functions like flush() and write() check to see if
> the server
> is still running, or see if the server has been trying to
> shutdown for
> some time and then raise an error?
>
> 3) Should we just go out of the way to document the behavior in the
> description of how to stop an AppServer, along with shutDown,
> initiateShutDown, and probably the server-push examples?
I vote for this.
>
> 4) Something else?
>
>
> I think if we're going to leave the behavior, we should at least do
> option 3 and document it. (If it's there, I must have missed
> it.) The
> recommended usage be that any potential compute bound or long running
> server-push operation should check server().running.
>
> On another point, shouldn't server().running parameter be
> _running, and
> use a running() function to access it?
Yes, to be consistent we should be using a function, especially if we're
going to document that servlets can check it during long-running operations.
- Geoff

Patches item #655502, was opened at 2002-12-17 16:51
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=655502&group_id=4866
Category: WebKit
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Donaldson (stuartd)
Assigned to: Nobody/Anonymous (nobody)
Summary: allow subclassing of ThreadedAppServer
Initial Comment:
This patch allows for subclassing of ThreadedAppServer.
It updates bothThreadedAppServer.py, and the proposed
replacement of NewThreadedAppServer.py
An overview of how this is accomplished is as follows:
Launch.py is updated to allow specification of a module
which does not need to reside under WebKit. The
specified python module path can either be a module
(which must contain a function main) or can be a
reference to a function whch is used in place of main.
The function main is called with the commandline to
start the server.
Within ThreadedAppServer.py (and
NewThreadedAppServer.py) the function main() takes an
optional argument specifying the serverClass. The
serverClass defaults to the ThreadedAppServer defined
within the current module.
To use this, derive a subClass of ThreadedAppServer
adding or altering functionality as you wish.
Then declare a function such as main() which at least
must take the command line arguments. If you want to
parse these yourself, that's OK, but unless you are
adding functionality beyond what is provided in the
main from ThreadedAppServer, you can turn around and
call that main function, passing in your new class as
the serverClass.
New method added include:
waitShutdown() wait until the shutDown completes.
Note that much of the functionality of run() was moved
into a run() method on the ThreadedAppServer. I also
chose not to pass arguments into mainloop() but rather
to make them properties on the object. This is because
alternate mainloop() implementations (ie:
NewThreadedAppServer) may take different arguments.
Matching the properties will be simpler and more
streightforward.
I should also note this includes a fix for the bug
630432 which fixed some monitor issues on NT. Although
this has not been tested on NT.
There is a TestServer.py file which tests this. I
didn't know exactly where to put it, I don't think it
belongs in WebKit, but perhaps in a WebKit/Tests. But
that doesn't exist.
----------------------------------------------------------------------
>Comment By: Stuart Donaldson (stuartd)
Date: 2002-12-18 17:17
Message:
Logged In: YES
user_id=326269
Oops, I noticed I had some debugging code still in
Launch.py. This patch supersedes the previous patch.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=655502&group_id=4866

Patches item #656064, was opened at 2002-12-18 17:04
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=656064&group_id=4866
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Donaldson (stuartd)
Assigned to: Nobody/Anonymous (nobody)
Summary: Change timestamp in mod_webkit to float
Initial Comment:
The timestamp sent in mod_webkit to Webware is an int,
this is useful in measuring performance and
communication delays between the web server, and Webware.
This patch changes the value from an int to a float
which is in the same format used by Python's
time.time() function. This allows much finer grained
accuracy when using the timestamp to measure
communication delays and such.
Note that the marshal module in Python does not support
a binary floating point, so it is converted to a string
to do this.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=656064&group_id=4866

The shutdown code in ThreadedAppServer.shutDown() sends a None to all
threads to cause them to terminate, and then does a join to each thread.
The problem is that if you have a thread that is busy, perhaps a
server-push thread that is just sending a continuous stream of data, or
a thread bound up for some other reason that they will not terminate.
This causes the shutDown to lock up.
A couple of questions:
1) Should there be a timeout when shutting down, after which it just
forcibly terminates the thread, or just ignores the thread allowing the
shutDown operation to continue?
2) Should functions like flush() and write() check to see if the server
is still running, or see if the server has been trying to shutdown for
some time and then raise an error?
3) Should we just go out of the way to document the behavior in the
description of how to stop an AppServer, along with shutDown,
initiateShutDown, and probably the server-push examples?
4) Something else?
I think if we're going to leave the behavior, we should at least do
option 3 and document it. (If it's there, I must have missed it.) The
recommended usage be that any potential compute bound or long running
server-push operation should check server().running.
On another point, shouldn't server().running parameter be _running, and
use a running() function to access it?
-Stuart-

Stuart Donaldson [mailto:stuartd@...] wrote:
> So is Monitor.py being used extensively?
>
> It appears that it is incompatible with the AppServer shell
> script, and
> Launch.py, in that it does not support starting, or
> restarting the appserver
> with arguments specified to AppServer/Launch...
>
> Is it intended as an alternate to AppServer/Launch with reduced
> functionality of not supporting command line arguments?
>
> Seems like you could have Monitor.py replace Launch.py, and
> have a mode
> where it sticks around to provide continual monitoring, or just does a
> one-shot monitor and exit. Any time it detects the AppServer is not
> responding, it would do its thing of restarting the
> appserver. Perhaps the
> command-line arguments need to be saved in a file and
> re-loaded to insure
> their re-use?
>
> What is AsyncThreadedAppServer? Is this somethign new, or something
> obsoleted? Monitor seems to default to this. It also
> doesn't look like it
> works with NewThreadedAppServer as-is.
I can't say that I've ever tried to use Monitor.py. I haven't ever
experienced an appserver crash (even though I'm running on Windows:-) so it
wouldn't have ever helped me. But we ought to fix it if it's currently
broken in CVS.
AsyncThreadedAppServer is no longer part of Webware -- it has been removed
in CVS. It was an experimental version of the AppServer that was supposed
to be faster, but turned out to be buggy and slower, so it was removed.
- Geoff

Patches item #655502, was opened at 2002-12-17 16:51
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=655502&group_id=4866
Category: WebKit
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Donaldson (stuartd)
Assigned to: Nobody/Anonymous (nobody)
Summary: allow subclassing of ThreadedAppServer
Initial Comment:
This patch allows for subclassing of ThreadedAppServer.
It updates bothThreadedAppServer.py, and the proposed
replacement of NewThreadedAppServer.py
An overview of how this is accomplished is as follows:
Launch.py is updated to allow specification of a module
which does not need to reside under WebKit. The
specified python module path can either be a module
(which must contain a function main) or can be a
reference to a function whch is used in place of main.
The function main is called with the commandline to
start the server.
Within ThreadedAppServer.py (and
NewThreadedAppServer.py) the function main() takes an
optional argument specifying the serverClass. The
serverClass defaults to the ThreadedAppServer defined
within the current module.
To use this, derive a subClass of ThreadedAppServer
adding or altering functionality as you wish.
Then declare a function such as main() which at least
must take the command line arguments. If you want to
parse these yourself, that's OK, but unless you are
adding functionality beyond what is provided in the
main from ThreadedAppServer, you can turn around and
call that main function, passing in your new class as
the serverClass.
New method added include:
waitShutdown() wait until the shutDown completes.
Note that much of the functionality of run() was moved
into a run() method on the ThreadedAppServer. I also
chose not to pass arguments into mainloop() but rather
to make them properties on the object. This is because
alternate mainloop() implementations (ie:
NewThreadedAppServer) may take different arguments.
Matching the properties will be simpler and more
streightforward.
I should also note this includes a fix for the bug
630432 which fixed some monitor issues on NT. Although
this has not been tested on NT.
There is a TestServer.py file which tests this. I
didn't know exactly where to put it, I don't think it
belongs in WebKit, but perhaps in a WebKit/Tests. But
that doesn't exist.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=655502&group_id=4866

Bugs item #655427, was opened at 2002-12-17 13:34
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=104866&aid=655427&group_id=4866
Category: WebKit
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Peter Kropf (peterk)
Assigned to: Nobody/Anonymous (nobody)
Summary: NewThreadedAppServer & http address
Initial Comment:
In WebKit/NewThreadedAppServer.py if it's going to
start an http server, it always uses 127.0.0.1 as the
address. This causes grief when attempting to connect
to the server from systems on the network, especially if
there is more than 1 nic. Changing the '127.0.0.1' to
server.address()[0] will bind to the address that's used
for the AppServer.
In addition, it would be good to add configuration
information to AppServer.config to specify which
address and port are to be used.
BTW if you add me as a developer to the project, I'll be
glad to fix the code. Or I can send in a patch...
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=104866&aid=655427&group_id=4866

> -----Original Message-----
> From: Geoffrey Talvola [mailto:gtalvola@...]
> Stuart Donaldson [mailto:stuartd@...] wrote:
...
>
> Would it be better to make your modifications to
> NewThreadedAppServer rather
> than to ThreadedAppServer? It may save time down the road if
> NewThreadedAppServer becomes the standard.
I am actually working primarily with NewThreadedAppServer since I have only
heard positive comments towards using it to replace ThreadedAppServer. My
comments apply to both however.
I have tried my above suggestion, dealing with shutDown() and it appears
this should ONLY be called from the _closeThread thread. Any references to
it elsewhere could initiate race conditions and end up with more than one
call to shutDown().
The existing run() function along with the Monitor quit function both
reference shutDown directly. While the existing run() function reference is
probably stale code (reference the KeyboardInterrupt discussion) the Monitor
use of shutDown will probably be a race condition.
-Stuart-

Stuart Donaldson [mailto:stuartd@...] wrote:
> Given that some things work a little different between NT/Unix,
> can you put a quick test in and see if the catch KeyboardInterrupt
> is ever getting called, and let me know?
I just tried it. Looks like KeyboardInterrupt never gets raised (on
windows). It's the signal that causes the shutdown to be called.
> I can incorporate some cleanup into my subclassable
> ThreadedAppServer patch.
> Part of which will make it so experiments like Ian's
> NewThreadedAppServer
> could be done through subclassing, rather than
> re-implementation. That way
> the NT/Unix differences can be kept in as few places as
> possible. (I am
> aware that the NewThreadedAppServer also cleans up some stuff in
> ThreadedAppServer so it is a little more than just a new
> paradigm for the
> socket handling...)
>
Would it be better to make your modifications to NewThreadedAppServer rather
than to ThreadedAppServer? It may save time down the road if
NewThreadedAppServer becomes the standard.
- Geoff

Given that some things work a little different between NT/Unix,
can you put a quick test in and see if the catch KeyboardInterrupt
is ever getting called, and let me know?
I can incorporate some cleanup into my subclassable ThreadedAppServer patch.
Part of which will make it so experiments like Ian's NewThreadedAppServer
could be done through subclassing, rather than re-implementation. That way
the NT/Unix differences can be kept in as few places as possible. (I am
aware that the NewThreadedAppServer also cleans up some stuff in
ThreadedAppServer so it is a little more than just a new paradigm for the
socket handling...)
-Stuart-
> -----Original Message-----
> From: Geoffrey Talvola [mailto:gtalvola@...]
> Sent: Tuesday, December 17, 2002 9:31 AM
> To: Stuart Donaldson; Webware-Devel (E-mail)
> Subject: RE: [Webware-devel] ThreadedAppServer keybard interrupt and
> termi nation
>
>
> Stuart Donaldson [mailto:stuartd@...] wrote:
> > I am working on a patch to integrate much of the
> functionality of the
> > ThreadedAppServer.run() function into a method on
> > ThreadedAppServer so it
> > can be more easily subclassed.
> >
> > In trying to not break any existing functionality, I have a
> couple of
> > questions:
> >
> > The run function currently catches KeyboardInterrupt
> > exceptions. However
> > signals for this are also caught and directed to the shutDown
> > function. Is
> > there a case where the KeyboardInterrupt exceptions would
> get thrown?
>
> Not sure. This might be paranoia or leftovers from an
> earlier version of
> the code that didn't use signals.
>
> > Would it make more sense if, after returning from mainloop()
> > we checked to
> > see if we were still running, and then called shufDown() if needed?
>
> Sounds plausible to me.
>
> > Also, the NT code never seems to call shutDown(), it just sets
> > server.running=0. Is this a problem on NT? I don't have
> > much NT/Python
> > combined experience so I don't know why it might not want to
> > call shutDown()
> > and cleanup the queues.
>
> But the signal code does cause shutDown() to be called on
> Windows, so it's
> not necessary to call it after returning from mainloop
> because it's the call
> to shutDown() that _caused_ mainloop to exit.
>
> You do have a point that if the mainloop exits for some other
> reason than
> Ctrl-C being called, then it looks like shutDown() won't be
> called. This is
> where your proposed test to see if we're still running might help. In
> practice, I can definitely vouch that the existing code for
> Windows seems to
> work properly.
>
> This code is all kind of messy. If you can figure out how to
> clean it up
> without breaking anything, more power to you :-) It's
> difficult to do a
> good job because things work a little differently on Windows vs. Unix.
>
> - Geoff
>

So is Monitor.py being used extensively?
It appears that it is incompatible with the AppServer shell script, and
Launch.py, in that it does not support starting, or restarting the appserver
with arguments specified to AppServer/Launch...
Is it intended as an alternate to AppServer/Launch with reduced
functionality of not supporting command line arguments?
Seems like you could have Monitor.py replace Launch.py, and have a mode
where it sticks around to provide continual monitoring, or just does a
one-shot monitor and exit. Any time it detects the AppServer is not
responding, it would do its thing of restarting the appserver. Perhaps the
command-line arguments need to be saved in a file and re-loaded to insure
their re-use?
What is AsyncThreadedAppServer? Is this somethign new, or something
obsoleted? Monitor seems to default to this. It also doesn't look like it
works with NewThreadedAppServer as-is.
Stuart Donaldson
Alerton Technologies Inc.

Stuart Donaldson [mailto:stuartd@...] wrote:
> I am working on a patch to integrate much of the functionality of the
> ThreadedAppServer.run() function into a method on
> ThreadedAppServer so it
> can be more easily subclassed.
>
> In trying to not break any existing functionality, I have a couple of
> questions:
>
> The run function currently catches KeyboardInterrupt
> exceptions. However
> signals for this are also caught and directed to the shutDown
> function. Is
> there a case where the KeyboardInterrupt exceptions would get thrown?
Not sure. This might be paranoia or leftovers from an earlier version of
the code that didn't use signals.
> Would it make more sense if, after returning from mainloop()
> we checked to
> see if we were still running, and then called shufDown() if needed?
Sounds plausible to me.
> Also, the NT code never seems to call shutDown(), it just sets
> server.running=0. Is this a problem on NT? I don't have
> much NT/Python
> combined experience so I don't know why it might not want to
> call shutDown()
> and cleanup the queues.
But the signal code does cause shutDown() to be called on Windows, so it's
not necessary to call it after returning from mainloop because it's the call
to shutDown() that _caused_ mainloop to exit.
You do have a point that if the mainloop exits for some other reason than
Ctrl-C being called, then it looks like shutDown() won't be called. This is
where your proposed test to see if we're still running might help. In
practice, I can definitely vouch that the existing code for Windows seems to
work properly.
This code is all kind of messy. If you can figure out how to clean it up
without breaking anything, more power to you :-) It's difficult to do a
good job because things work a little differently on Windows vs. Unix.
- Geoff

I am working on a patch to integrate much of the functionality of the
ThreadedAppServer.run() function into a method on ThreadedAppServer so it
can be more easily subclassed.
In trying to not break any existing functionality, I have a couple of
questions:
The run function currently catches KeyboardInterrupt exceptions. However
signals for this are also caught and directed to the shutDown function. Is
there a case where the KeyboardInterrupt exceptions would get thrown?
Would it make more sense if, after returning from mainloop() we checked to
see if we were still running, and then called shufDown() if needed?
Also, the NT code never seems to call shutDown(), it just sets
server.running=0. Is this a problem on NT? I don't have much NT/Python
combined experience so I don't know why it might not want to call shutDown()
and cleanup the queues.
Stuart Donaldson

I might also suggest moving the 'shutDown' function from ThreadedAppServer
into AppServer, since it wants the AppServer instance (retrieved from
AppServer) and calls the AppServer.initiateShutdown() method which is
defined within AppServer.
I am not clear on where the signal handling should be defined, but suspect
they could remain within ThreadedAppServer, and in fact we may want to
retain a reference in ThreadedAppServer for shutDown, but this could be done
through: "from AppServer import shutDown"
-Stuart-
> -----Original Message-----
> From: Chuck Esterbrook [mailto:ChuckEsterbrook@...]
> Sent: Monday, December 16, 2002 4:52 PM
> To: webware-devel@...
> Subject: Re: [Webware-devel] Accessing the app server instance from
> anywhere
>
>
> On Monday 16 December 2002 04:38 pm, Stuart Donaldson wrote:
> > Couldn't this be done by AppServer.__init__()?
> >
> > Then require that all AppServer's must initialize the base class
> > AppServer which they likely do anyway.
>
> Er, good point. :-)
>
> I need more sugar...
>
>
> > Is there any reason to have multiple AppServer's running? Would it
> > make sense that WebKit.globalServer be a function that returns the
> > server? Then if for any reason we end up with a more complex server
> > referencing system accesses would at least be going through a
> > function which could be modified to do the right thing.
>
> I can't imagine that there will ever be more than one AppServer in a
> single process. I'm willing to work off that assumption.
>
> -Chuck
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:
> With Great Power, Comes Great Responsibility
> Learn to use your power at OSDN's High Performance Computing Channel
> http://hpc.devchannel.org/
> _______________________________________________
> Webware-devel mailing list
> Webware-devel@...
> https://lists.sourceforge.net/lists/listinfo/webware-devel
>