Bug 73406

Summary: [NRWT] TestResultWriter should pass tolerance=0 to port.diff_image in case of reftests.
Product: WebKit Reporter: Hayato Ito <hayato>
Component: Tools / TestsAssignee: Hayato Ito <hayato>
Status: RESOLVED FIXED    
Severity: Normal CC: abarth, dglazkov, ojan, rniwa, webkit.review.bot
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
pass zero none

Hayato Ito
Reported 2011-11-30 00:27:39 PST
TestResultWriter should pass tolerance=0 explicitly to port.diff_image in case of reftests. Mac port uses '0.1' as default value of tolerance. In case of reftests, we should use tolerance=0 in the phase of writing test results in addition to the phase of comparing images. See https://bugs.webkit.org/show_bug.cgi?id=73381 also.
Attachments
pass zero (11.44 KB, patch)
2011-11-30 01:48 PST, Hayato Ito 		no flags
DetailsFormatted DiffDiff
View All Add attachment proposed patch, testcase, etc.
Ryosuke Niwa
Comment 1 2011-11-30 00:54:47 PST
This doesn't really help as ImageDiff itself has a default tolerance level.
Hayato Ito
Comment 2 2011-11-30 01:27:32 PST
Is there any ImageDiff which doesn't accept tolerance parameter? It seems that ImageDiff used by chromium port doesn't use tolerance parameter. I assumed its default is 'zero' though I've not investigated every ImageDiff program's behavior on every platform yet. (In reply to comment #1) > This doesn't really help as ImageDiff itself has a default tolerance level.
Hayato Ito
Comment 3 2011-11-30 01:29:46 PST
At least, passing tolerance=0 resolves mac port's issue and this doesn't affect other platforms because we don't pass tolerance parameter to ImageDiff on other platforms. Things should not become worse than now, I guess.
Hayato Ito
Comment 4 2011-11-30 01:48:23 PST
Created attachment 117143 [details] pass zero
Ryosuke Niwa
Comment 5 2011-11-30 10:07:29 PST
Comment on attachment 117143 [details] pass zero View in context: https://bugs.webkit.org/attachment.cgi?id=117143&action=review > Tools/Scripts/webkitpy/layout_tests/controllers/test_result_writer_unittest.py:54 > + driver_output1, driver_output2, failures) Wrong indentation.
WebKit Review Bot
Comment 6 2011-11-30 13:23:02 PST
Comment on attachment 117143 [details] pass zero Attachment 117143 [details] did not pass chromium-ews (chromium-xvfb): Output: http://queues.webkit.org/results/10696325
WebKit Review Bot
Comment 7 2011-12-01 06:03:57 PST
Comment on attachment 117143 [details] pass zero Rejecting attachment 117143 [details] from commit-queue. Failed to run "['/mnt/git/webkit-commit-queue/Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '-..." exit_code: 2 Last 500 characters of output: and returned 256 at Tools/Scripts/update-webkit-chromium line 109. Re-trying 'depot_tools/gclient sync --force --reset --delete_unversioned_trees' No such file or directory at /mnt/git/webkit-commit-queue/Tools/Scripts/webkitdirs.pm line 2020. Failed to run "['Tools/Scripts/build-webkit', '--release', '--chromium', '--update-chromium']" exit_code: 2 sync --force --reset --delete_unversioned_trees' No such file or directory at /mnt/git/webkit-commit-queue/Tools/Scripts/webkitdirs.pm line 2020. Full output: http://queues.webkit.org/results/10695236
Ryosuke Niwa
Comment 8 2011-12-01 19:57:25 PST
Is https://bugs.webkit.org/show_bug.cgi?id=55652 a duplicate of this bug?
WebKit Review Bot
Comment 9 2011-12-01 20:32:44 PST
Comment on attachment 117143 [details] pass zero Clearing flags on attachment: 117143 Committed r101739: <http://trac.webkit.org/changeset/101739>
WebKit Review Bot
Comment 10 2011-12-01 20:32:49 PST
All reviewed patches have been landed. Closing bug.
Hayato Ito
Comment 11 2011-12-04 18:53:42 PST
*** Bug 55652 has been marked as a duplicate of this bug. ***
Hayato Ito
Comment 12 2011-12-04 18:54:46 PST
Right. I marked it as duplicate of this bug. (In reply to comment #8) > Is https://bugs.webkit.org/show_bug.cgi?id=55652 a duplicate of this bug?
Note You need to log in before you can comment on or make changes to this bug.