Some Mauve regressions builder missed

From:

Mark Wielaard

Subject:

Some Mauve regressions builder missed

Date:

Wed, 11 Jan 2006 21:19:49 +0100

Hi all,
Seems builder.classpath.org misses some regressions. builder can only
accurately report when a PASS turns into a FAIL with the exact same
message. It deliberately doesn't report new FAILs (since those could be
from newly added tests). Since we sometimes use different messages when
something PASSes and when something FAILs the auto-builder cannot
currently detect those regressions. A special case are tests that crash
a VM, then the message is replaced with CRASHED. Which is incorrectly
interpreted as a new FAIL, but is obviously a regression. A workaround
is to create a new test harness that ignores checkpoint names and
PASS/FAIL messages. Such a harness should just use the test classname as
message plus a counter. I'll implement that after 0.20.
So here are the Mauve tests with regressions that builder missed:
- gnu.testlet.java.math.BigDecimal.DiagBigDecimal
This is the assert issue in mprec.c.
- gnu.testlet.java.net.DatagramPacket.DatagramPacketOffset
- gnu.testlet.java.net.DatagramPacket.DatagramPacketReceive
- gnu.testlet.java.net.DatagramPacket.DatagramPacketReceive2
- gnu.testlet.java.net.DatagramSocket.DatagramSocketTest
- gnu.testlet.java.net.DatagramSocket.DatagramSocketTest2
- gnu.testlet.java.net.InetAddress.getAllByName
- gnu.testlet.java.net.MulticastSocket.MulticastSocketTest
- gnu.testlet.java.net.ServerSocket.ServerSocketTest
- gnu.testlet.java.net.Socket.SocketTest
These are mostly the same issue (also an assert).
I am working on a patch.
- gnu.testlet.javax.swing.text.BoxView.spans
I am unclear why the results changed between 0.19 an 0.20pre without
the builder noticing. Maybe we accidentally reset the tester.
Not investigated yet.
- gnu.testlet.javax.swing.JTextField.createDefaultModel
- gnu.testlet.javax.swing.JTextField.CopyPaste
- gnu.testlet.javax.swing.BoxLayout.simplehorizontal
- gnu.testlet.javax.swing.BoxLayout.simplevertical
- gnu.testlet.javax.swing.AbstractButton.setRolloverEnabled
- gnu.testlet.javax.swing.JEditorPane.ConstructorsAndTypes
Not investigated yet.
If people have time please do investigate the above Mauve tests so we
can have a nice regression free release Friday.
Cheers,
Mark