Bug 203294

Summary: [iOS 13] fast/forms/ios/accessory-bar-navigation.html is a flaky failure
Product: WebKit Reporter: Wenson Hsieh <wenson_hsieh>
Component: Tools / TestsAssignee: Wenson Hsieh <wenson_hsieh>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, bdakin, commit-queue, dino, ews-watchlist, megan_gardner, thorton, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Repro attempt #1
ews-watchlist: commit-queue-
Archive of layout-test-results from ews214 for win-future
none
x1500 w/ prefixes (1)
none
x1500 w/ prefixes (2)
none
x1500 w/ suffixes (1)
none
x1500 w/ suffixes (2)
none
Patch
none
Fix a ChangeLog typo none

Wenson Hsieh
Reported 2019-10-23 08:32:43 PDT
<rdar://problem/54520848>
Attachments
Repro attempt #1 (1.92 MB, patch)
2019-10-23 09:51 PDT, Wenson Hsieh 		ews-watchlist: commit-queue-
DetailsFormatted DiffDiff
Archive of layout-test-results from ews214 for win-future (13.80 MB, application/zip)
2019-10-24 06:56 PDT, EWS Watchlist 		no flags
Details
x1500 w/ prefixes (1) (6.65 MB, patch)
2019-10-24 09:28 PDT, Wenson Hsieh 		no flags
DetailsFormatted DiffDiff
x1500 w/ prefixes (2) (6.65 MB, patch)
2019-10-24 09:29 PDT, Wenson Hsieh 		no flags
DetailsFormatted DiffDiff
x1500 w/ suffixes (1) (6.65 MB, patch)
2019-10-24 09:31 PDT, Wenson Hsieh 		no flags
DetailsFormatted DiffDiff
x1500 w/ suffixes (2) (6.65 MB, patch)
2019-10-24 09:32 PDT, Wenson Hsieh 		no flags
DetailsFormatted DiffDiff
Patch (7.83 KB, patch)
2019-10-25 10:25 PDT, Wenson Hsieh 		no flags
DetailsFormatted DiffDiff
Fix a ChangeLog typo (15.20 KB, patch)
2019-10-25 14:19 PDT, Wenson Hsieh 		no flags
DetailsFormatted DiffDiff
Show Obsolete (8) View All Add attachment proposed patch, testcase, etc.
Wenson Hsieh
Comment 1 2019-10-23 09:51:02 PDT
Created attachment 381683 [details] Repro attempt #1
EWS Watchlist
Comment 2 2019-10-24 06:56:37 PDT Comment hidden (obsolete)
EWS Watchlist
Comment 3 2019-10-24 06:56:39 PDT Comment hidden (obsolete)
Wenson Hsieh
Comment 4 2019-10-24 08:44:43 PDT
It seems 500 runs wasn’t enough to reproduce the failure :(
Alexey Proskuryakov
Comment 5 2019-10-24 09:01:49 PDT
The test is a lot more flaky than that, so I don't think that it's about the number of iterations. Another possibility is that there is something in a preceding test that breaks this one. Also the results history suggests that this may be more reproducible with a physical device.
Wenson Hsieh
Comment 6 2019-10-24 09:21:19 PDT
(In reply to Alexey Proskuryakov from comment #5) > The test is a lot more flaky than that, so I don't think that it's about the > number of iterations. Another possibility is that there is something in a > preceding test that breaks this one. > > Also the results history suggests that this may be more reproducible with a > physical device. I am aware that this is failing pretty frequently on internal bots. However, I wasn’t able to glean any useful information from their output :/ The test group that’s run prior to this test is fast/forms/fieldset, and then fast/forms/ios/accessory-bar-navigation.html is the first test of the fast/forms/ios group. The first thing I tried was to keep on running fast/forms/fieldset and then fast/forms/ios/accessory-bar-navigation.html for many iterations, but this yielded no reproduction. I did discover, however, that the failure output in this test is identical to what would occur if a software keyboard were to be presented instead of the usual hardware keyboard. This is the strongest lead so far. I suspect that something (in another system framework, or maybe even another test running in a different simulator) is either somehow causing this simulator to think the hardware keyboard is not attached, or is somehow rendering the SPI we call to ensure that the hardware keyboard is attached (GSEventSetHardwareKeyboardAttached) ineffective. I tried to run this test (accessory-bar-navigation.html) between other tests in fast/forms/ios to try and capture this state — hence the random “xyz” prefixes in front of each cloned layout test. But it seems this didn’t work either.
Wenson Hsieh
Comment 7 2019-10-24 09:28:42 PDT Comment hidden (obsolete)
Wenson Hsieh
Comment 8 2019-10-24 09:29:28 PDT Comment hidden (obsolete)
Wenson Hsieh
Comment 9 2019-10-24 09:31:46 PDT Comment hidden (obsolete)
Wenson Hsieh
Comment 10 2019-10-24 09:32:32 PDT Comment hidden (obsolete)
Wenson Hsieh
Comment 11 2019-10-24 10:06:01 PDT
Alexey and I discovered a way to consistently reproduce this failure (🥳). The key is to run the test on a newly created iOS simulator; the first test run in this manner will use a software keyboard instead of a hardware keyboard. The following invocation works for me: ``` run-webkit-tests --ios-simulator --debug --no-retry-failures --no-show-results --no-sample-on-timeout fast/forms/ios/accessory-bar-navigation.html --child-processes 1 ``` Calling off the queued EWS jobs.
Wenson Hsieh
Comment 12 2019-10-25 10:25:15 PDT
Created attachment 381939 [details] Patch
WebKit Commit Bot
Comment 13 2019-10-25 14:00:24 PDT
The commit-queue encountered the following flaky tests while processing attachment 381939 [details]: The commit-queue is continuing to process your patch.
WebKit Commit Bot
Comment 14 2019-10-25 14:00:32 PDT
The commit-queue encountered the following flaky tests while processing attachment 381939 [details]: imported/w3c/web-platform-tests/IndexedDB/interleaved-cursors-small.html bug 203433 The commit-queue is continuing to process your patch.
Wenson Hsieh
Comment 15 2019-10-25 14:19:49 PDT
Created attachment 381965 [details] Fix a ChangeLog typo
Wenson Hsieh
Comment 16 2019-10-25 14:25:41 PDT
Committed: https://trac.webkit.org/changeset/251601/webkit.
Note You need to log in before you can comment on or make changes to this bug.