Created attachment 502598[details]
Screen shot of admin-console showing confusion when invoking the Test Connection operation on a datasource
In some cases, the result of invoking an operation can be confused with the result of the actual operation.
For example:
For a datasource resource, you can invoke the "Test Connection" operation. Once the operation has been submitted, the "Operation History" will indicate "Successful". This "Successful" message can easily be interrupted as the connection to the datasource was successful when in reality, the actual test of the datasource connection returned failed.
This is confusing to operational admins looking at this UI to test a connection. In fact, this would mislead them to look at the wrong section and proceed thinking the connection has been obtained when it's not, and could possibly cause serious production ramifications. The UI should be re-worked and be more intuitive and clear.

The fix for this should be to improve the description of the 'testConnection' operation in the as5 plugin descriptor as follows:
<operation name="testConnection" displayName="Test Connection" description="Test if a connection can be obtained - returns true if a connection was obtained, or false if not; NOTE: this operation will always return a status of Successful - the results of the operation must be inspected to see whether or not a connection was obtained">

(In reply to comment #3)
> Fixed in master - commit 5f9ddb7.
I do not see 5f9ddb7, did you mean e431028f?
Also, please note that this doesn't truly resolve this BZ. It resolves the example use-case provided but the BZ is more general then that. It seems that this confusion can exist with any operation that would have a boolean result. Instead of using the "Success" status on the operation status, maybe we should be looking at:
In Progress --> [Failed | Complete]
Or even going as far as displaying the operations result as the state. If a boolean, "True|False", if a simple object "<object>.toString()", if a complex object, "Object Name" or "See Result" link.
The idea is that we want to convey to the user, via the UI, that what they are looking at is not the result of the operation but a link that will take you to the result of the operation.

I think we should investigate whether a still better solution is possible for
the Test Connection case. In that particular instance we should investigate
simply making the operation return a failure status if the internal call to the
AS to test the connection fails.