Hi Rajith,
The changes we are proposing to the Java test profiles only relate to
the Java broker used during test runs, nothing else will change, and
as always changes should be tested against the C++ broker.
Going forward, the 30-40min test profiles will pretty much be a thing
of the past as far as humans are concerned, and we will leave those up
to the Jenkins box 99% of the time, all the profiles we'd expect
people to run will complete significantly faster.
The obvious way to improve up our testing situation is to stop using
system tests where unit tests are more appropriate, i.e the vast
majority of cases. A lot of our current system tests are of low
utility and would be better implemented as unit tests. We should
probably also look towards implementing a suite of system tests that
exercise common messaging use cases, and going forward attempt to
transition as much of the rest into well thought through and
specifically targeted unit tests.
Robbie
On 4 July 2011 15:39, Rajith Attapattu wrote:
> Just to add to it, any changes to the Java client should also be
> tested with the cpp test profile.
> Any changes to the client failover mechanism should also be tested
> with the cpp.cluster profile.
>
> I know it's a pain to run all these profiles, especially considering
> each run takes about 30-40 mins.
> But atm there is no better alternative. I think long term we need to
> evaluate our testing strategy to see if we can come up with something
> more efficient.
>
> Regards,
>
> Rajith
>
> On Mon, Jul 4, 2011 at 7:11 AM, Robert Godfrey wrote:
>> On 4 July 2011 12:44, Keith Wall wrote:
>>
>>> Robbie and I are presently working on QPID-2811 and QPID-2815. These Jiras
>>> will rationalise the transport mechanism in Qpid and to allow client/broker
>>> to be tested together within the same JVM regardless of transport.
>>>
>>> It seems to me that now is a good time to review the test-profiles. I see
>>> the following problems:
>>>
>>> 1) 'default' profile being 0-8 makes little sense as most of the project's
>>> development focus is on 0-10, and with the removal of the old .NET client,
>>> we have no client that defaults to 0-8.
>>> 2) The time taken to run some test profiles is excessive and this is a
>>> disincentive to developer good practice. For instance, running java-0.10
>>> on
>>> my machine takes 30mins. This is harming productivity.
>>>
>>> Current profiles:
>>>
>>> default.testprofile - tests 0-8 using in-VM, broker/client share JVM,
>>> Memory Message Store
>>> java.testprofile - tests 0-9-1, using tcp, separate JVMs, Memory Message
>>> Store
>>> java-derby.testprofile - tests 0-9-1, using tcp, separate JVMs, Derby
>>> Message Store
>>>
>>> java.0.10.testprofile - tests 0-10, using tcp, separate JVMs,, Memory
>>> Message Store
>>> java-derby.0.10.testprofile -- tests 0-10, using tcp, separate JVMs, Derby
>>> Message Store
>>>
>>> My proposal:
>>>
>>> Four profiles intended for regular developer use. These profiles will
>>> exploit the feature for client/broker to share the same VM. This saves the
>>> overhead of each testcase spawning a new broker as a separate process, and
>>> brings the length of time to run the profile down considerably, assisting
>>> developer productivity.
>>>
>>> (mms=memory message store, dby = Apache Derby)
>>>
>>> java-mms.0-10
>>> java-dby.0-10
>>>
>>> java-mms.0-9-1
>>> java-dby.0-9-1
>>>
>>> Running client/broker in separate JVMs remains valuable, but is probably
>>> not
>>> something most developers will want to do most of the time (because of the
>>> time overhead). The intention would be we'd schedule these to run on CI
>>> (using Jenkin's matrix feature) and let Jenkins do the heavy lifting.
>>>
>>> java-mms-spawn.0-10
>>> java-dby-spawn.0-10
>>>
>>> java-mms-spawn.0-9-1
>>> java-dby-spawn.0-9-1
>>>
>>> The idea of the default testprofile would disappear and it would become an
>>> error to run -Dprofile=default. If the developer omits -Dprofile from the
>>> Ant command line, the system will run java-mms.0-10, which I think will be
>>> useful fallback to most developers.
>>>
>>> I think we also need a "Java Test Howto" page on the website, explaining
>>> the
>>> purposes behind the profiles, the differences between system and non-system
>>> tests and giving advice on appropriate use of both.
>>>
>>> Any thoughts?
>>>
>>> Keith.
>>>
>>
>>
>> All sounds exceedingly sensible to me.
>>
>> -- Rob
>>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org