Bug 97063

Summary: Page loading performance tests should measure time in C++
Product: WebKit Reporter: Ryosuke Niwa <rniwa>
Component: Tools / TestsAssignee: Ryosuke Niwa <rniwa>
Status: RESOLVED WONTFIX    
Severity: Normal CC: abarth, dbates, dpranke, haraken, jamesr, mjs, morrita, ojan, pdr, webkit.review.bot, zoltan
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on:    
Bug Blocks: 77037    
Attachments:
Description Flags
Sample output
none
Change used to generate the output none

Ryosuke Niwa
Reported 2012-09-18 19:55:43 PDT
Right now page loading performance tests (PerformanceTests/PageLoad/; most notably all SVG perf. tests) and replay tests (PerformanceTests/Replay) measure the load time in Python. We should do this measurement in C++ code instead to reduce the variance.
Attachments
Sample output (23.32 KB, text/html)
2012-09-18 20:44 PDT, Ryosuke Niwa 		no flags
Details
Change used to generate the output (14.96 KB, patch)
2012-09-18 21:02 PDT, Ryosuke Niwa 		no flags
DetailsFormatted DiffDiff
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.
Ryosuke Niwa
Comment 1 2012-09-18 20:44:30 PDT
Created attachment 164652 [details] Sample output On my second thought, I'm not certain this is an overall improvement. The variance has increased after the change.
Hajime Morrita
Comment 2 2012-09-18 21:02:27 PDT
(In reply to comment #1) > Created an attachment (id=164652) [details] > Sample output > > On my second thought, I'm not certain this is an overall improvement. The variance has increased after the change. Just for curiosity, do you have any WIP patch for this?
Ryosuke Niwa
Comment 3 2012-09-18 21:02:54 PDT
Created attachment 164654 [details] Change used to generate the output
Hajime Morrita
Comment 4 2012-09-18 21:25:43 PDT
(In reply to comment #3) > Created an attachment (id=164654) [details] > Change used to generate the output Thanks for the patch. This needs DRT help from each port and records only the start and the end. If we can track some other timings, this might worth having. But if this is only for the start-end measurement, touching DRTs looks a bit overkill.
Ryosuke Niwa
Comment 5 2012-09-18 21:29:42 PDT
(In reply to comment #4) > This needs DRT help from each port and records only the start and the end. > If we can track some other timings, this might worth having. > But if this is only for the start-end measurement, touching DRTs looks a bit overkill. Right. I had initially thought this will reduce the variance for some tests (e.g. https://bugs.webkit.org/show_bug.cgi?id=97062) but it turned out that it only amplified as the total time has reduced. With this patch, the variance is ~150% on that test :( I'm inclined to say this change is probably not worth the effort.
Zoltan Horvath
Comment 6 2012-09-22 00:35:54 PDT
(In reply to comment #5) > (In reply to comment #4) > > This needs DRT help from each port and records only the start and the end. > > If we can track some other timings, this might worth having. > > But if this is only for the start-end measurement, touching DRTs looks a bit overkill. > > Right. I had initially thought this will reduce the variance for some tests (e.g. https://bugs.webkit.org/show_bug.cgi?id=97062) but it turned out that it only amplified as the total time has reduced. With this patch, the variance is ~150% on that test :( > > I'm inclined to say this change is probably not worth the effort. It should be more precise to measure from DRT and we should either modify DRT because of the memory measurements. Just because keeping the variance low by using python to the measurements doesn't seem a good idea to me.
Note You need to log in before you can comment on or make changes to this bug.