Am 08.12.2012 09:51, schrieb Sophana K:
> My tasks do not access the sessions. (Why would they?)
> As it is a new behaviour, (I was using 0.9) a look at the changes could
> help...
In 0.9 and 1.0 the session sweeper loops run over the keys list, not
over the dictionaries, simply because older Python versions did not
support that. So the problem with duplicate session sweepers might well
have existed, you just didn't notice it.
> Could it be the scheduler instances sharing some global task lists?
I thought about that too, because it would be an explanation, but no,
they don't.
One explanation for the duplicate session sweeping could be that your
tasks inherit from WebKit.Tasks.SessionTask instead of TaskKit.Task. Or
that you inherit from Application and run __init__ twice.
To debug the problem, you can put some print statement in the __init__()
and run() methods of SessionTask and see when it is instantiated (should
be instantiated once at the start) and how often it is run (should run
once every 10 minutes, depending on your SessionTimeout).
-- Christoph

My tasks do not access the sessions. (Why would they?)
As it is a new behaviour, (I was using 0.9) a look at the changes could
help...
Could it be the scheduler instances sharing some global task lists?
On Fri, Dec 7, 2012 at 2:38 PM, Christoph Zwerschke <cito@...> wrote:
> Am 07.12.2012 12:16, schrieb Sophana K:
> > Yes, I'm creating a Scheduler instance.
> > I didn't know I was supposed to use the one from the application.
>
> It's ok to create your own instance. But then the two schedulers will
> run as two parallel threads which needs more ressources and you can get
> these race conditions if both schedulers have tasks which access the
> session store. It's also convenient to use the application.taskManager()
> because it's started and stopped automatically together with the
> application.
>
> > Is it normal that a scheduler instance automatically does session
> sweeping?
> > Shouldn't the application create the task explicitely?
>
> But that's excactly how it works. The application creates its own
> scheduler, and then adds just one task, the one which is used for
> session sweeping. You can replace that task with your own or add more
> tasks if you like.
>
> > I have now fixed my code to use the existing scheduler.
> > Shouldn't you remove the previous fixes then?
>
> The fixes are still helpful in case anybody changes the session store
> for whatever maybe valid reasons.
>
> It's still unclear to me what caused your problems: Do your own tasks
> modify the session store? Then that's the explanation, because you ran
> them in a parallel scheduler. If your own tasks don't change the
> session, then you must continue searching for the error cause. It looks
> like the session sweeper thread somehow was duplicated, but how this
> happens is unclear to me without seeing your code. If you just create
> your own Scheduler instance, this should not happen.
>
> -- Christoph
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss
>

Am 07.12.2012 12:16, schrieb Sophana K:
> Yes, I'm creating a Scheduler instance.
> I didn't know I was supposed to use the one from the application.
It's ok to create your own instance. But then the two schedulers will
run as two parallel threads which needs more ressources and you can get
these race conditions if both schedulers have tasks which access the
session store. It's also convenient to use the application.taskManager()
because it's started and stopped automatically together with the
application.
> Is it normal that a scheduler instance automatically does session sweeping?
> Shouldn't the application create the task explicitely?
But that's excactly how it works. The application creates its own
scheduler, and then adds just one task, the one which is used for
session sweeping. You can replace that task with your own or add more
tasks if you like.
> I have now fixed my code to use the existing scheduler.
> Shouldn't you remove the previous fixes then?
The fixes are still helpful in case anybody changes the session store
for whatever maybe valid reasons.
It's still unclear to me what caused your problems: Do your own tasks
modify the session store? Then that's the explanation, because you ran
them in a parallel scheduler. If your own tasks don't change the
session, then you must continue searching for the error cause. It looks
like the session sweeper thread somehow was duplicated, but how this
happens is unclear to me without seeing your code. If you just create
your own Scheduler instance, this should not happen.
-- Christoph

Yes, I'm creating a Scheduler instance.
I didn't know I was supposed to use the one from the application.
Is it normal that a scheduler instance automatically does session sweeping?
Shouldn't the application create the task explicitely?
I have now fixed my code to use the existing scheduler.
Shouldn't you remove the previous fixes then?
Thanks for the hint.
On Thu, Dec 6, 2012 at 12:23 PM, Christoph Zwerschke <cito@...> wrote:
> Am 06.12.2012 10:59, schrieb Sophana K:
> > I have some tasks scheduled every 30m and 24h.
> > I did nothing special about the session sweeper. Are they related to
> > taskKit?
>
> The session sweeper thread is started by the Application instance using
> its own instance of the TaskKit scheduler, available as
> Application.taskManager(). Do you use that scheduler instance or do you
> use your own? To me it looks as if you're either running the session
> sweeper twice or one of your tasks is doing something with the session
> store of the application. Otherwise you shouldn't see that error.
>
> -- Christoph
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss
>

* Christoph Zwerschke <cito@...> [06.12.2012 12:10]:
> Yes, as Sophana already answered, the trunk is stable; I want to cut a
> new release from it this month anyway.
I'll wait for the new version then. Thanks a lot.
--
Bye, Andreas

Am 06.12.2012 10:59, schrieb Sophana K:
> I have some tasks scheduled every 30m and 24h.
> I did nothing special about the session sweeper. Are they related to
> taskKit?
The session sweeper thread is started by the Application instance using
its own instance of the TaskKit scheduler, available as
Application.taskManager(). Do you use that scheduler instance or do you
use your own? To me it looks as if you're either running the session
sweeper twice or one of your tasks is doing something with the session
store of the application. Otherwise you shouldn't see that error.
-- Christoph

Am 06.12.2012 10:34, schrieb Andreas Poisel:
> I've been using the svn head on a test system for a while now without
> any problem. Would you object using svn head in a production
> environment?
Yes, as Sophana already answered, the trunk is stable; I want to cut a
new release from it this month anyway. r8235 fixes a problem with
aborting long-running threads on 64bit platforms. You can use
http://localhost:8080/Admin/ThreadControl to play with this feature and
see if requests can be properly cancelled on your machine.
-- Christoph

* Sophana K <sophana78@...> [06.12.2012 11:00]:
> Looking at the svn logs, it looks like the changes are quite minor since
> the 1.1 release.
> I think most of them correspond to some bugs reported in this mailing list.
That's my impression, too. But I have no experience with ctypes and
wanted to make sure the changes to the threading code (e.g. rev 8235)
are considered stable.
--
Bye, Andreas

I have some tasks scheduled every 30m and 24h.
I did nothing special about the session sweeper. Are they related to
taskKit?
On Fri, Nov 30, 2012 at 6:28 PM, Christoph Zwerschke <cito@...> wrote:
> Am 30.11.2012 18:01, schrieb Sophana K:
> > Isn't it strange nobody had this problem before? Seems that there
> > isn't a lot of webware 1.1 applications in production. Don't you
> > think?
>
> That problem should not appear in practice unless you have somehow two
> session sweeper tasks running at the same time. Are you doing anything
> special in this regard? I'm running Webware 1.1 apps with many users
> which use sessions heavily, and I've never seen this issue.
>
> -- Christoph
>
> (Sorry, my mailer just sent an empty reply.)
>
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> TUNE You got it built. Now make it sing. Tune shows you how.
> http://goparallel.sourceforge.net
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss
>

Hi
Looking at the svn logs, it looks like the changes are quite minor since
the 1.1 release.
I think most of them correspond to some bugs reported in this mailing list.
On Thu, Dec 6, 2012 at 10:34 AM, Andreas Poisel <a.poisel@...> wrote:
> Hi!
>
> I've been using the svn head on a test system for a while now without
> any problem. Would you object using svn head in a production
> environment?
>
> Thank you!
> --
> Bye, Andreas
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss
>

Am 30.11.2012 18:01, schrieb Sophana K:
> Isn't it strange nobody had this problem before? Seems that there
> isn't a lot of webware 1.1 applications in production. Don't you
> think?
That problem should not appear in practice unless you have somehow two
session sweeper tasks running at the same time. Are you doing anything
special in this regard? I'm running Webware 1.1 apps with many users
which use sessions heavily, and I've never seen this issue.
-- Christoph
(Sorry, my mailer just sent an empty reply.)

Am 30.11.2012 13:48, schrieb Sophana K:
> It seems the problem has now moved to another place.
> Should I try the trunk?
>
> File "/home/.../Webware-1.1/WebKit/Tasks/SessionTask.py", line 13, in run
> self._sessionstore.cleanStaleSessions(self)
> File "/home/.../Webware-1.1/WebKit/SessionDynamicStore.py", line 256, in cleanStaleSessions
> self._memoryStore.cleanStaleSessions(task)
> File "/home/.../Webware-1.1/WebKit/SessionStore.py", line 202, in cleanStaleSessions
> for key in self:
> RuntimeError: dictionary changed size during iteration
Same problem. It's now also fixed in the trunk.
-- Christoph

It seems the problem has now moved to another place.
Should I try the trunk?
File "/home/.../Webware-1.1/WebKit/Tasks/SessionTask.py", line 13, in run
self._sessionstore.cleanStaleSessions(self)
File "/home/.../Webware-1.1/WebKit/SessionDynamicStore.py", line
256, in cleanStaleSessions
self._memoryStore.cleanStaleSessions(task)
File "/home/.../Webware-1.1/WebKit/SessionStore.py", line 202, in
cleanStaleSessions
for key in self:
RuntimeError: dictionary changed size during iteration
On Wed, Nov 28, 2012 at 12:56 PM, Christoph Zwerschke <cito@...>wrote:
> Am 28.11.2012 12:10, schrieb Sophana K:
> > Look like another thread could be changing the _memoryStore during
> > this iteration. Should I change to
> > for key in list(self._memoryStore):
>
> Yes, this or "for key in self._memoryStore.keys()" should fix it. I have
> already committed this as a fix to the trunk.
>
> -- Chris
>
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> INSIGHTS What's next for parallel hardware, programming and related areas?
> Interviews and blogs by thought leaders keep you ahead of the curve.
> http://goparallel.sourceforge.net
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss
>

Am 28.11.2012 12:10, schrieb Sophana K:
> Look like another thread could be changing the _memoryStore during
> this iteration. Should I change to
> for key in list(self._memoryStore):
Yes, this or "for key in self._memoryStore.keys()" should fix it. I have
already committed this as a fix to the trunk.
-- Chris

Hi
I just upgraded to webware-1.1, and get some errors (about every day) in
the session dynamic store intervalSweep method.
Here is the traceback
File "/home/sophana/src/env1/Webware-1.1/WebKit/Tasks/SessionTask.py",
line 13, in run
self._sessionstore.cleanStaleSessions(self)
File "/home/sophana/src/env1/Webware-1.1/WebKit/SessionDynamicStore.py",
line 264, in cleanStaleSessions
self.intervalSweep()
File "/home/sophana/src/env1/Webware-1.1/WebKit/SessionDynamicStore.py",
line 288, in intervalSweep
for key in self._memoryStore:
RuntimeError: dictionary changed size during iteration
Look like another thread could be changing the _memoryStore during this
iteration.
Should I change to
for key in list(self._memoryStore):
?
Regards
Sophana

Am 05.10.2012 10:29, schrieb Alex Ivanov:
> Can you please confirm that this fix is applicable?
> We may throw an exception if wait was not successful.
> Will it be a good idea to rewrite flags usage with python's events wait/set functionality to be on the safe side?
I have fixed it now in trunk in r8233 with the simplest possible patch.
Using events would be probably better, but I want to keep the semantics
of the existing flag for backward compatibility.
-- Christoph

Am 05.10.2012 10:29, schrieb Alex Ivanov:
> First of all, thank you, Christoph, for the excellent software! I've
> been using webware since 2002 and have not yet seen a decent
> replacement to it! Outstanding performance of lightweight servlets
> implementation leaves behind its competitors with their tons of
> useless code. That's the main feature I'm using from webware.
> Upgraded recently to 1.1 and discovered a freeze when I press Ctrl+C.
> It happens only on Windows. I'm using Windows 7. Linux is not
> affected. In fact the reason of the freeze seems to be quite simple:
Thanks for the positive feedback, kudos should go to the authors.
Your analysis of the problem and fix look good to me. It's actually a
race condition, and in fact most of the time it works on my test
machine, but sometimes not. And it's not really a freeze, the server
works properly, just doesn't respond to Ctrl-C any more, right?
I also agree that a better fix would be to use a condition or event
instead of the global flag. I'll work on that and plan to have a bugfix
release this month. A couple of issues have already been fixed after
1.1, and I have one or two other open issues that I want to address.
If there are other small issues that should be addressed in a bugfix
release, let me know.
-- Christoph

Hello!
First of all, thank you, Christoph, for the excellent software! I've been using webware since 2002 and have not yet seen a decent replacement to it! Outstanding performance of lightweight servlets implementation leaves behind its competitors with their tons of useless code. That's the main feature I'm using from webware.
Upgraded recently to 1.1 and discovered a freeze when I press Ctrl+C. It happens only on Windows. I'm using Windows 7. Linux is not affected.
In fact the reason of the freeze seems to be quite simple:
ThreadedAppServer.py, line 1308:
t = Thread(target=_windowsmainloop)
t.start()
...
_windowsmainloop calls server.mainloop(), which flags self._running:
threadCheckInterval = self._maxServerThreads * 2
threadUpdateDivisor = 5 # grab stat interval
threadCheck = 0
---> self._running = 3 # server is in the main loop now
On subsequent check server._running is still 1 because the thread does not have enough time to initialize itself:
---> while server._running > 1:
try:
sleep(1) # wait for interrupt
except Exception:
if server._running < 3:
raise # shutdown
So, we skip waiting and freeze at an endless loop.
The fix may be the following:
ThreadedAppServer.py, line 1307:
# Run the server thread
t = Thread(target=_windowsmainloop)
t.start()
try:
---> # Inserting waiting loop before _running status check.
---> for i in range(30): # wait at most 3 seconds for startup
---> if server._running == 3:
---> break
---> sleep(0.1)
while server._running > 1:
try:
sleep(1) # wait for interrupt
except Exception:
if server._running < 3:
raise # shutdown
finally:
t.join()
Can you please confirm that this fix is applicable?
We may throw an exception if wait was not successful.
Will it be a good idea to rewrite flags usage with python's events wait/set functionality to be on the safe side?
Thank you very much,
Mikhail

I will give a try again with webware 1.1
It works, but not for a long time...
python3 doesn't look like a priority for me. (I admit I had some hard
time with unicode...)
Backward compatibility for webkit is the most important IMHO.
About naming convention, I don't care that much.
I'm not using plugins. A little bit of TaskKit (not a big deal)
Migrating to more modern server architectures is a big plus for me:
more scalability, comet, etc...
If it is possible to just use WebKit compatibility in order to
integrate more easily with other python libraries, it would be great.
Sophana
On Wed, Sep 19, 2012 at 8:07 PM, Christoph Zwerschke <cito@...> wrote:
> Am 19.09.2012 18:13, schrieb NoRaGen:
>> It would be nice to have the option to upgrade to a python3.x Version
>> of Webware for later purpose.
>
> Definitely, that has also been one of my goals for a new version.
>
> > Compatiblity should be given so, that API remains the same, but
> > using python3.x capability. This should be possible, I think. API
> > changes would very bad at all.
>
> The problem here is that Webware development started even before Python
> 2, so the old API is very old-fashioned concerning naming conventions,
> use of getters and setters etc. My idea was to add a tool like py2to3
> that can convert an old style Webware app to a new style app.
>
> Another idea would be to allow both styles. E.g. request could become a
> property, but if you call the request object, then it returns self, so
> if you use request as a getter function, it would still work. We already
> discussed this in the past. Method names could be changed from camelCase
> to PEP8 style, but the old names could be kept as aliases.
>
>> However we use many modules where still no python3 port is existing.
>> For now, switching to higher python version, would still not
>> effective for us.
>
> Yes, e.g. ReportLab is still one of the roadblockers.
>
> -- Chris
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss

On 20/09/2012 1:01 AM, Christoph Zwerschke wrote:
> So my question to this mailing list would be: How many people are still
> interested in having a modernized Webware version...?
I maintain a Webware app that does a lot of work, but changes rarely,
and I'm not doing any new Python web development these days (I'm using
C# so I can share code between a desktop app and web server). I only
use mod_webkit2, WebKit and MiscUtils.DBPool. My preference would be to
keep up with Python so I can keep installing Webware when CentOS
upgrades its python, but otherwise stability is good for me. That said,
none of the suggestions for updating APIs would particularly bother me -
I like WebKit and will happily do what it takes to keep my application
working.
And thanks for all the work you do keeping it going, Christoph.
Oliver

1 message has been excluded from this view by a project administrator.