Hi!
Jean-loup has sent me various testcases for the regression suite.
I will certainly go through them, identify wrong behavior and try to
teach Pachi to avoid the same mistakes in the future. However, I'm a
little busy right now and there is a big prerequisite someone might
be willing to help with.
We need to figure out a good format for regression test suite.
I think the best way would be to store SGF files (e.g. in t-regress/
subdirectory) and use markups within the SGFs describing the wrong and
correct reply; then, it should not be hard to script a smarter SGF-GTP
convertor than sgf2gtp that could create GTP input for Pachi for actual
testing.
I think it would be best to store our markup within comment flags,
so that it is easy for regular users to compose testcases; if we use
tags common SGF programs (e.g. cgoban3) cannot deal with, it might be
more elegant semantically but creating regression tests should be easy.
In the markup, we should be able to say mainly "in this position
moves a,b,c are bad" and "in this position, we should play either
a,b,c". Some more advanced statements like "in this position, group x
should be evaluated clearly dead" might come useful later too.
Anyone willing to figure out the details? Or at least write a brief
summary of the approach taken by others (probably mainly GNUGo and
Fuego)?
--
Petr "Pasky" Baudis
UNIX is user friendly, it's just picky about who its friends are.