Hi there,
this message is particularly for Larry Isaacs, that I know is the
WebTools developer that cares this part of WTP.

It happened many times to me (and quite often to my co-workers) that
problems are encountered while publishing to Tomcats. Which kind of
problems? The deployed folder is in some way corrupted: completely empty
(although WTP says the webapp is published) or partially empty. In these
cases, usually a "clean" on the server is needed to make things work
again (trying to perform "Publish" is useless, because WTP thinks the
server is already published), but recently, with Indigo, it seems like
the Clean operation is not so effective, so that removing the web module
from the server, publishing, re-adding the web module to the server and
publishing again is needed to get it working again.
This is obviously time consuming especially if working on large
projects. Moreover, it gives the user a feeling that all these tools are
not working at all.

I wrote some times here on the newsgroups and I saw other people
experiencing this kind of problems, but nobody could provide easy steps
to reproduce to help Larry to investigate and fix this problem. I still
don't have the steps to reproduce, however I know for sure that these
kind of things happen whenever WTP has some problems writing on the
deploy folder or (more easily) deleting contents from the deploy folder.
This may happen particularly in Windows when JARs are locked (either by
the running server itself or for other reasons). But my co.workers using
Linux recently see this problem almost always.
The problem is that we are working with a huge proprietary project and
we are using a lot of Eclipse plugins and maybe it isn't even WTP's
fault. So it's not easy to isolate a test case to reproduce and submit a
bug report.

However, the main problem is that when publishing errors occur:
- no error reporting is performed at all (maybe something is written in
the error log?) -OR-
- an error is printed, but WTP thinks the publishing has been performed
all the same, so the server gets to the "Published" state even if
contents are not really published
In any case, a corrupted state is reached and cleaning or
removing/readding the webapp from the server is needed to fix that.

So, my request is: could at least WTP be made more resilient to
publishing errors? Could errors be reported cleanly and make the
publishing fail, so that the user may retry later, without having the
server reach a corrupted state?

I may help Larry to diagnose the problem and try some patches if needed
(although it would be necessary for these patches to be against WTP
3.3.x/Indigo). I would appreciate very much any feedback on this.

Sorry for the delayed response. It was due to vacation and not
disinterest. :)

The most effective way to address the problem is to create a separate
workspace, fetch some source code for some server plug-ins, run an
Eclipse in debug mode pointing at your main workspace. When the problem
appears, add breakpoints and try to discover what is going wrong. The
disadvantage of this approach is that it requires the most time
investment on your part.

As an alternative, we can try discover what is going wrong via logging.
I will likely need to provide some patched plug-ins that add logging
where needed. This should be able to get the maximum effectiveness with
the minimum of your time. You'll just need to report what gets logged
when the problem occurs.

Part of the difficulty here is that nothing significant has changed in
the WTP Tomcat plug-in in a number of major releases of WTP. The
problem publishing behavior is coming from changes underneath the Tomcat
plug-ins. This makes it hard for the Tomcat plug-ins to provide error
reporting, since they are unaware that anything is going wrong. At the
moment I don't have any good ideas about what changed. Outside of the
Tomcat plug-ins, I don't have all that much detailed knowledge about how
WTP works. It's mostly notions about where to look to find what is
going on.

Let me know which of the above you want to try. If the second, which
version of WTP you want to try this with. I will provide patched
plug-ins with the exact same versions so you just have to swap them into
the Eclipse installation to use them.

Cheers,
Larry

On 11/22/2011 7:42 AM, Mauro Molinari wrote:
> Hi there,
> this message is particularly for Larry Isaacs, that I know is the
> WebTools developer that cares this part of WTP.
>
> It happened many times to me (and quite often to my co-workers) that
> problems are encountered while publishing to Tomcats. Which kind of
> problems? The deployed folder is in some way corrupted: completely empty
> (although WTP says the webapp is published) or partially empty. In these
> cases, usually a "clean" on the server is needed to make things work
> again (trying to perform "Publish" is useless, because WTP thinks the
> server is already published), but recently, with Indigo, it seems like
> the Clean operation is not so effective, so that removing the web module
> from the server, publishing, re-adding the web module to the server and
> publishing again is needed to get it working again.
> This is obviously time consuming especially if working on large
> projects. Moreover, it gives the user a feeling that all these tools are
> not working at all.
>
> I wrote some times here on the newsgroups and I saw other people
> experiencing this kind of problems, but nobody could provide easy steps
> to reproduce to help Larry to investigate and fix this problem. I still
> don't have the steps to reproduce, however I know for sure that these
> kind of things happen whenever WTP has some problems writing on the
> deploy folder or (more easily) deleting contents from the deploy folder.
> This may happen particularly in Windows when JARs are locked (either by
> the running server itself or for other reasons). But my co.workers using
> Linux recently see this problem almost always.
> The problem is that we are working with a huge proprietary project and
> we are using a lot of Eclipse plugins and maybe it isn't even WTP's
> fault. So it's not easy to isolate a test case to reproduce and submit a
> bug report.
>
> However, the main problem is that when publishing errors occur:
> - no error reporting is performed at all (maybe something is written in
> the error log?) -OR-
> - an error is printed, but WTP thinks the publishing has been performed
> all the same, so the server gets to the "Published" state even if
> contents are not really published
> In any case, a corrupted state is reached and cleaning or
> removing/readding the webapp from the server is needed to fix that.
>
> So, my request is: could at least WTP be made more resilient to
> publishing errors? Could errors be reported cleanly and make the
> publishing fail, so that the user may retry later, without having the
> server reach a corrupted state?
>
> I may help Larry to diagnose the problem and try some patches if needed
> (although it would be necessary for these patches to be against WTP
> 3.3.x/Indigo). I would appreciate very much any feedback on this.
>
> Thanks in advance,
> Mauro.

Hi Larry,
this time I'm the one to be sorry for the delay, I've been very busy
recently.

Anyway, I still tried to set-up a simple test project to reproduce the
publishing problems with no luck :-( It seems like that in simple
scenarios even if a published file is locked while trying to clean the
server and/or to publish the updates, the server state doesn't get
corrupted: an error may appear if there are locked files, but the server
state stays coherent.

Anyway, during my daily work I made sure once again that when the actual
problems appear, no error message is shown in Eclipse and no error is
printed in the error log. So, it's even more difficult to understand
what's going on.

One thing I'm wondering, but it's not strictly bound to this problem: I
personally think that when you issue a "Clean" on the server, it would
be reasonable for WTP to stop the server before doing the clean
operation. In fact, it may happen that the clean fails because Tomcat
itself is holding locks on JAR files and I can't think of any scenario
in which cleaning without a previous stop of the server would be useful.
What do you think about this?

Regarding your proposal, I would prefer if you could provide a WebTools
version 3.3.1 with enhanced logging for test purposes. In this way I may
install it on my co-workers' systems, so we may reproduce the problem
faster. In fact, I personally see these kind of publishing problems not
so frequently, but other colleagues see them almost always, instead.

If we are able to gather enough information to setup a simple test case
that reproduces the problem or to identify the steps that may lead to
reproduce the problem in our environment, I may then be able to debug
(but please note that I never did that on an Eclipse project!)

I have created Bug 365748[1] to collect information on this problem.
Add yourself as a CC and supply what information you can for the 4 items
listed. If you have information on multiple instances, that's fine.
Add each of them. I'll study the lower level workings of the server
plug-ins and see what to do about enabling logging and providing custom
diagnostic versions to get more useful information.

On 12/4/2011 4:40 PM, Mauro Molinari wrote:
> Hi Larry,
> this time I'm the one to be sorry for the delay, I've been very busy
> recently.
>
> Anyway, I still tried to set-up a simple test project to reproduce the
> publishing problems with no luck :-( It seems like that in simple
> scenarios even if a published file is locked while trying to clean the
> server and/or to publish the updates, the server state doesn't get
> corrupted: an error may appear if there are locked files, but the server
> state stays coherent.
>
> Anyway, during my daily work I made sure once again that when the actual
> problems appear, no error message is shown in Eclipse and no error is
> printed in the error log. So, it's even more difficult to understand
> what's going on.
>
> One thing I'm wondering, but it's not strictly bound to this problem: I
> personally think that when you issue a "Clean" on the server, it would
> be reasonable for WTP to stop the server before doing the clean
> operation. In fact, it may happen that the clean fails because Tomcat
> itself is holding locks on JAR files and I can't think of any scenario
> in which cleaning without a previous stop of the server would be useful.
> What do you think about this?
>
> Regarding your proposal, I would prefer if you could provide a WebTools
> version 3.3.1 with enhanced logging for test purposes. In this way I may
> install it on my co-workers' systems, so we may reproduce the problem
> faster. In fact, I personally see these kind of publishing problems not
> so frequently, but other colleagues see them almost always, instead.
>
> If we are able to gather enough information to setup a simple test case
> that reproduces the problem or to identify the steps that may lead to
> reproduce the problem in our environment, I may then be able to debug
> (but please note that I never did that on an Eclipse project!)
>
> Thank you in advance, I really appreciate your support.
> Mauro