Re: [Webware-discuss] Possible bug?

Christoph Zwerschke wrote:
> Stephan Diehl wrote:
>> I've cheated a bit, I'm still using Webware-0.9.1. When trying to
>> upgrade my Application to Webware-0.9.2, I realized that the
>> Application breaks.
>
> It would be great if you examine what breaks the application, since I
> want to publish a new release very soon. If there is a bug, I'd like to
> fix it first. (Some bugs in 0.9.2 have already been fixed, so it's time
> to get a bug-fix release 0.9.3 out as soon as possible.)
Since my Application problem is most probably homegrown, I've followed
up the original problem with the broken downloads.
I'm doing more or less what's described in
http://wiki.w4py.org/filestreamingandcontentdisposition.html
In addition to your suggestions, I've also
replaced the
assert not self._closed, "Stream Already Closed"
line in the write method (ASStreamOut.py) with
if self._closed:
raise ConnectionAbortedError
catching the Error in ThreadedAppServer (as in your suggestion) does not
supress the Traceback. It looks like the Traceback is generated in
Application.py within the 'handleExceptionInTransaction'.
For my special case it's probably best, to put my response.write inside
some try/except.
Stephan
>
> -- Christoph
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss
>

Thread view

I'm a longtime user of WebWare. I just stumbled over some pretty strange
problem. The Application running on Webware is a kind of Document
Management system that can contain quite huge files (ISO images).
When somebody is downloading such a file and cancels the download, the
application eats up more and more memory. It looks like the application
tries to deliver the content, but, instead of delivering it to the
client (who's not longer there), caches everything in memory.
This is the newest version of WebWare.
I've done the following change:
In WebKit/ThreadedAppServer.py
741 def flush(self):
742 """Flush stream.
743
744 Calls `ASStreamOut.ASStreamOut.flush`, and if that returns True
745 (indicating the buffer is full enough) then we send data from
746 the buffer out on the socket.
747
748 """
749 result = ASStreamOut.flush(self)
750 if result: # a True return value means we can send
751 reslen = len(self._buffer)
752 sent = 0
753 while sent < reslen:
754 try:
755 sent += self._socket.send(
756 self._buffer[sent:sent+8192])
757 except socket.error, e:
758 if e[0] == errno.EPIPE: # broken pipe
759 pass
760 elif hasattr(errno, 'ECONNRESET') \
761 and e[0] == errno.ECONNRESET:
762 pass
763 else:
764 print "StreamOut Error: ", e
765 break
766 self.pop(sent)
I've changed line 759 from 'pass' to 'raise', which now gives ugly
tracebacks in the logs, but other than that, does what should happen
anyway: cancel the download.
Stephan
P.S.: I know this is quite a political issue, but is there a chance to
change from tabs to spaces in Webware code?

Stephan Diehl schrieb:
> I'm a longtime user of WebWare. I just stumbled over some pretty
> strange problem. The Application running on Webware is a kind of
> Document Management system that can contain quite huge files (ISO
> images). When somebody is downloading such a file and cancels the
> download, the application eats up more and more memory. It looks like
> the application tries to deliver the content, but, instead of
> delivering it to the client (who's not longer there), caches
> everything in memory. This is the newest version of WebWare.
>
> I've done the following change:
> In WebKit/ThreadedAppServer.py
>
> 741 def flush(self):
> 742 """Flush stream.
> ...
> 753 while sent < reslen:
> 754 try:
> 755 sent += self._socket.send(
> 756 self._buffer[sent:sent+8192])
> 757 except socket.error, e:
> 758 if e[0] == errno.EPIPE: # broken pipe
> 759 pass
> 760 elif hasattr(errno, 'ECONNRESET') \
> 761 and e[0] == errno.ECONNRESET:
> 762 pass
> 763 else:
> 764 print "StreamOut Error: ", e
> 765 break
> 766 self.pop(sent)
>
> I've changed line 759 from 'pass' to 'raise', which now gives ugly
> tracebacks in the logs, but other than that, does what should happen
> anyway: cancel the download.
Hi Stephan,
thanks for the bug report. I think you're right, this is a problem.
By the way, on Windows you get an ECONNABORTED error in the same
situation, not an EPIPE error. So I think ECONNABORTED should be treated
the same way as the other two errors in the flush method.
For a fix of the problem, I suggest the following:
In ASSStreamOut.py:
Above the definition of the ASStreamOut class, insert:
class ConnectionAbortedError(Exception):
pass
Below, replace the 2 lines beginning:
assert not self._closed, ...
with:
if self._closed: raise ConnectionAbortedError
In ThreadedAppServer.py:
Above the definition of the ThreadedAppServer class, insert:
from ASStreamOut import ConnectionAbortedError
In the TASASStreamOut.flush() method, insert this line
before the break statement:
self._closed = True
To avoid the ugly tracebacks, insert the following lines between
the call of handler.handleRequest() and the following except:,
indented with the correct amount (both "excepts" on the same level):
except ConnectionAbortedError:
if self._verbose:
sys.stdout.write('%5i %s connection aborted\n'
% (handler._requestID, timestamp()['pretty']))
If you confirm that this fixes your problems and nobody disagrees, I
will fix it this way in the trunk.
> P.S.: I know this is quite a political issue, but is there a chance to
> change from tabs to spaces in Webware code?
I think this question should go directly to Chuck, our BDFL who had been
strongly against this in the past. I vote for a change as well.
-- Christoph

Hallo Christoph,
thanks for that. I've cheated a bit, I'm still using Webware-0.9.1. When
trying to upgrade my Application to Webware-0.9.2, I realized that the
Application breaks. I'll need to investigate this first (I'm using a
homegrown JSONRpc Handler). Alternativly, I could change my
Webware-0.9.1 according to your suggestions.
I'll definatelly get back to you about this.
Stephan
Christoph Zwerschke wrote:
> Stephan Diehl schrieb:
>> I'm a longtime user of WebWare. I just stumbled over some pretty
>> strange problem. The Application running on Webware is a kind of
>> Document Management system that can contain quite huge files (ISO
>> images). When somebody is downloading such a file and cancels the
>> download, the application eats up more and more memory. It looks like
>> the application tries to deliver the content, but, instead of
>> delivering it to the client (who's not longer there), caches
>> everything in memory. This is the newest version of WebWare.
> >
> > I've done the following change:
> > In WebKit/ThreadedAppServer.py
> >
> > 741 def flush(self):
> > 742 """Flush stream.
> > ...
> > 753 while sent < reslen:
> > 754 try:
> > 755 sent += self._socket.send(
> > 756 self._buffer[sent:sent+8192])
> > 757 except socket.error, e:
> > 758 if e[0] == errno.EPIPE: # broken pipe
> > 759 pass
> > 760 elif hasattr(errno, 'ECONNRESET') \
> > 761 and e[0] == errno.ECONNRESET:
> > 762 pass
> > 763 else:
> > 764 print "StreamOut Error: ", e
> > 765 break
> > 766 self.pop(sent)
> >
> > I've changed line 759 from 'pass' to 'raise', which now gives ugly
> > tracebacks in the logs, but other than that, does what should happen
> > anyway: cancel the download.
>
> Hi Stephan,
>
> thanks for the bug report. I think you're right, this is a problem.
>
> By the way, on Windows you get an ECONNABORTED error in the same
> situation, not an EPIPE error. So I think ECONNABORTED should be treated
> the same way as the other two errors in the flush method.
>
> For a fix of the problem, I suggest the following:
>
> In ASSStreamOut.py:
>
> Above the definition of the ASStreamOut class, insert:
>
> class ConnectionAbortedError(Exception):
> pass
>
> Below, replace the 2 lines beginning:
>
> assert not self._closed, ...
>
> with:
>
> if self._closed: raise ConnectionAbortedError
>
> In ThreadedAppServer.py:
>
> Above the definition of the ThreadedAppServer class, insert:
>
>>from ASStreamOut import ConnectionAbortedError
>
> In the TASASStreamOut.flush() method, insert this line
> before the break statement:
>
> self._closed = True
>
> To avoid the ugly tracebacks, insert the following lines between
> the call of handler.handleRequest() and the following except:,
> indented with the correct amount (both "excepts" on the same level):
>
> except ConnectionAbortedError:
> if self._verbose:
> sys.stdout.write('%5i %s connection aborted\n'
> % (handler._requestID, timestamp()['pretty']))
>
> If you confirm that this fixes your problems and nobody disagrees, I
> will fix it this way in the trunk.
>
> > P.S.: I know this is quite a political issue, but is there a chance to
> > change from tabs to spaces in Webware code?
>
> I think this question should go directly to Chuck, our BDFL who had been
> strongly against this in the past. I vote for a change as well.
>
> -- Christoph
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss
>

Stephan Diehl wrote:
> I've cheated a bit, I'm still using Webware-0.9.1. When trying to
> upgrade my Application to Webware-0.9.2, I realized that the
> Application breaks.
It would be great if you examine what breaks the application, since I
want to publish a new release very soon. If there is a bug, I'd like to
fix it first. (Some bugs in 0.9.2 have already been fixed, so it's time
to get a bug-fix release 0.9.3 out as soon as possible.)
-- Christoph

Christoph Zwerschke wrote:
> Stephan Diehl wrote:
>> I've cheated a bit, I'm still using Webware-0.9.1. When trying to
>> upgrade my Application to Webware-0.9.2, I realized that the
>> Application breaks.
>
> It would be great if you examine what breaks the application, since I
> want to publish a new release very soon. If there is a bug, I'd like to
> fix it first. (Some bugs in 0.9.2 have already been fixed, so it's time
> to get a bug-fix release 0.9.3 out as soon as possible.)
Since my Application problem is most probably homegrown, I've followed
up the original problem with the broken downloads.
I'm doing more or less what's described in
http://wiki.w4py.org/filestreamingandcontentdisposition.html
In addition to your suggestions, I've also
replaced the
assert not self._closed, "Stream Already Closed"
line in the write method (ASStreamOut.py) with
if self._closed:
raise ConnectionAbortedError
catching the Error in ThreadedAppServer (as in your suggestion) does not
supress the Traceback. It looks like the Traceback is generated in
Application.py within the 'handleExceptionInTransaction'.
For my special case it's probably best, to put my response.write inside
some try/except.
Stephan
>
> -- Christoph
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss
>

Stephan Diehl wrote:
> catching the Error in ThreadedAppServer (as in your suggestion) does not
> supress the Traceback. It looks like the Traceback is generated in
> Application.py within the 'handleExceptionInTransaction'.
> For my special case it's probably best, to put my response.write inside
> some try/except.
You're right, that did not suffice. It worked only for pages < 8192
bytes, and the the transaction resources were not closed properly.
I have now checked in a much cleaner and complete fix.
-- Christoph

Christoph Zwerschke wrote:
> Stephan Diehl wrote:
>> I've cheated a bit, I'm still using Webware-0.9.1. When trying to
>> upgrade my Application to Webware-0.9.2, I realized that the
>> Application breaks.
>
> It would be great if you examine what breaks the application, since I
> want to publish a new release very soon. If there is a bug, I'd like to
> fix it first. (Some bugs in 0.9.2 have already been fixed, so it's time
> to get a bug-fix release 0.9.3 out as soon as possible.)
Never been easier to fix a breakage: I've just tried it with the svn
Webware trunk, and it works perfectly again.
Stephan
>
> -- Christoph
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss
>