My suggestion is to try to make it platform-independent but it's
significantly harder than rebaselining the results for pixel tests. As you
point out, it defeats one of the benefits of reftests if we end up
introducing platform-specific reference files.
- Ryosuke
On Thu, Apr 12, 2012 at 1:46 PM, Jacob Goldstein <jacobg at adobe.com> wrote:
> Right, but when those differences come to light, should the goal be to
> adjust the ref test to make it platform-independent, or is having
> platform-specific ref tests acceptable? Doesn't that put us in the same
> situation as having platform-specific pixel tests?
>> From: Ryosuke Niwa <rniwa at webkit.org>
> Date: Thu, 12 Apr 2012 13:43:26 -0700
> To: Jacob Goldstein <jacobg at adobe.com>
> Cc: Ojan Vafai <ojan at chromium.org>, Dirk Pranke <dpranke at chromium.org>,
> WebKit Development <webkit-dev at lists.webkit.org>
>> Subject: Re: [webkit-dev] Handling failing reftests
>> Right but you wouldn't know platform-specific issues until you land them.
> e.g. rounding errors, subtle font differences in edge cases, etc...
>> On Thu, Apr 12, 2012 at 1:30 PM, Jacob Goldstein <jacobg at adobe.com> wrote:
>>> Isn't the goal of writing a ref test that it is not platform specific?
>>>> From: Ryosuke Niwa <rniwa at webkit.org>
>> Date: Thu, 12 Apr 2012 12:29:42 -0700
>> To: Ojan Vafai <ojan at chromium.org>
>> Cc: Dirk Pranke <dpranke at chromium.org>, WebKit Development <
>>webkit-dev at lists.webkit.org>
>> Subject: Re: [webkit-dev] Handling failing reftests
>>>> On Thu, Apr 12, 2012 at 11:53 AM, Ojan Vafai <ojan at chromium.org> wrote:
>>>>>> I agree that it's hard sometimes to construct reftests that work, but
>>> once you've done so, the cost on the project of maintaining the test is
>>> usually considerably lower than pixel tests.
>>>>>>> Not so sure. There are cases where we have these platform specific
>> failures for ref tests, and they're much harder to fix than rebaselining
>> pixel results.
>>>> - Ryosuke
>>>>>-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120412/310909d0/attachment.html>