Bug 50263
| Summary: | Exception in flaky test reporting code in commit-queue | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Eric Seidel (no email) <eric> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | RESOLVED DUPLICATE | ||
| Severity: | Normal | CC: | abarth |
| Priority: | P2 | ||
| Version: | 528+ (Nightly build) | ||
| Hardware: | PC | ||
| OS: | OS X 10.5 | ||
| Bug Depends on: | 51272 | ||
| Bug Blocks: | |||
Eric Seidel (no email)
Exception in flaky test reporting code in commit-queue
From the log:
=== BUILD AGGREGATE TARGET All OF PROJECT DumpRenderTree WITH CONFIGURATION Release ===
Check dependencies
** BUILD SUCCEEDED **
Compiling Java tests
make: Nothing to be done for `default'.
Running tests from /Projects/CommitQueue/LayoutTests
Testing 22075 test cases.
accessibility .....................................................................................
animations ...........................................................
animations/stop-animation-on-suspend.html -> failed
Exiting early after 1 failures. 144 tests run.
44.81s total testing time
143 test cases (99%) succeeded
1 test case (<1%) had incorrect layout
1 test case (<1%) had stderr output
Patch does not pass tests
Traceback (most recent call last):
File "/Projects/CommitQueue/WebKitTools/Scripts/webkitpy/tool/bot/queueengine.py", line 108, in run
if not self._delegate.process_work_item(work_item):
File "/Projects/CommitQueue/WebKitTools/Scripts/webkitpy/tool/commands/queues.py", line 265, in process_work_item
if task.run():
File "/Projects/CommitQueue/WebKitTools/Scripts/webkitpy/tool/bot/commitqueuetask.py", line 188, in run
if not self._test_patch():
File "/Projects/CommitQueue/WebKitTools/Scripts/webkitpy/tool/bot/commitqueuetask.py", line 172, in _test_patch
self._report_flaky_tests(first_failing_tests + second_failing_tests)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'list'
Exception while preparing queue Sleeping until 2010-11-30 12:57:20 (2 mins).
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Eric Seidel (no email)
This would happen any time the tests fail on the first run, but we fail to get results from that failure.
The code in question:
def _test_patch(self):
if self._patch.is_rollout():
return True
if self._test():
return True
first_failing_tests = self._failing_tests_from_last_run()
if self._test():
self._report_flaky_tests(first_failing_tests)
return True
second_failing_tests = self._failing_tests_from_last_run()
if first_failing_tests != second_failing_tests:
self._report_flaky_tests(first_failing_tests + second_failing_tests)
return False
The fix is straightforward. We just make "first_failing_tests != second_failing_tests" into "first_failing_tests and (first_failing_tests != second_failing_tests)". However we might also want to add some special logging to track this case, as the layout tests shoudl not fail w/o us being able to get the results from the run!
Eric Seidel (no email)
I expect this to be fixed by bug 51272.
*** This bug has been marked as a duplicate of bug 51272 ***