haikuwebkit/PerformanceTests/LaunchTime/new_tab.py

112 lines
4.0 KiB
Python
Raw Permalink Normal View History

Add benchmark for WebKit process launch times https://bugs.webkit.org/show_bug.cgi?id=186414 Patch by Ben Richards <benton_richards@apple.com> on 2018-07-19 Reviewed by Ryosuke Niwa. Added two benchmarks, one for measuring browser new tab launch time and one for browser startup time. * LaunchTime/.gitignore: Added. * LaunchTime/feedback_client.html: Added. Displays benchmark progress in browser * LaunchTime/feedback_server.py: Added. (FeedbackServer): Sends data to feedback_client via websocket (FeedbackServer.__init__): (FeedbackServer._create_app): (FeedbackServer._start_server): (FeedbackServer._send_all_messages): (FeedbackServer.start): (FeedbackServer.stop): (FeedbackServer.send_message): Send a message to the feedback_client (FeedbackServer.wait_until_client_has_loaded): Wait until the feedback_client has opened a websocket connection to the feedback_server (FeedbackServer.MainHandler): Handler factory to create handler that serves feedback_client.html (FeedbackServer.MainHandler.Handler): (FeedbackServer.MainHandler.Handler.get): (FeedbackServer.WSHandler): Handler factory to create handler that sends data to feedback client (FeedbackServer.WSHandler.Handler): (FeedbackServer.WSHandler.Handler.open): On websocket connection opened (FeedbackServer.WSHandler.Handler.on_close): On websocket connection closed * LaunchTime/launch_time.py: Added. (DefaultLaunchTimeHandler): Abstract HTTP request handler for launch time benchmarks (DefaultLaunchTimeHandler.get_test_page): Default test page to be overridden by benchmarks (DefaultLaunchTimeHandler.get_blank_page): (DefaultLaunchTimeHandler.on_receive_stop_time): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (DefaultLaunchTimeHandler.log_message): Suppresses HTTP logs from SimpleHTTPRequestHandler (LaunchTimeBenchmark): Abstract class which launch time benchmarks inherit from and override methods desired to customize (LaunchTimeBenchmark.__init__): (LaunchTimeBenchmark._parse_browser_bundle_path): Parser for bundle path option (LaunchTimeBenchmark._parse_args): (LaunchTimeBenchmark._run_server): Target function for main server thread (LaunchTimeBenchmark._setup_servers): (LaunchTimeBenchmark._clean_up): (LaunchTimeBenchmark._exit_due_to_exception): (LaunchTimeBenchmark._geometric_mean): (LaunchTimeBenchmark._standard_deviation): (LaunchTimeBenchmark._compute_results): Returns mean and std dev of list of results and pretty prints if should_print=True is specified (LaunchTimeBenchmark._wait_times): Mimic numpy.linspace (LaunchTimeBenchmark.open_tab): Open a browser tab with the html given by self.response_handler.get_test_page (LaunchTimeBenchmark.launch_browser): Open a broser to either the feedback client (if option is set) or a blank page (LaunchTimeBenchmark.quit_browser): (LaunchTimeBenchmark.quit_browser.quit_app): (LaunchTimeBenchmark.quit_browser.is_app_closed): (LaunchTimeBenchmark.close_tab): (LaunchTimeBenchmark.wait): (LaunchTimeBenchmark.log): Print to console and send to feedback client if --feedback-in-browser flag is used (LaunchTimeBenchmark.log_verbose): Only logs if --verbose flag is used (LaunchTimeBenchmark.run): (LaunchTimeBenchmark.group_init): Initialization done before each round of iterations (LaunchTimeBenchmark.run_iteration): (LaunchTimeBenchmark.initialize): Convenience method to be overriden by subclasses which is called at the end of __init__ (LaunchTimeBenchmark.will_parse_arguments): Called before argparse.parse_args to let subclasses add new command line arguments (LaunchTimeBenchmark.did_parse_arguments): Called after argparse.parse_args to let subclass initialize based on command line arguments * LaunchTime/new_tab.py: Added (NewTabBenchmark): (NewTabBenchmark._parse_wait_time): Parser for wait time option (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.group_init): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler): (NewTabBenchmark.ResponseHandler.Handler): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/startup.py: Added (StartupBenchmark): This benchmark measures browser startup time and initial page load time (StartupBenchmark.initialize): (StartupBenchmark.run_iteration): (StartupBenchmark.ResponseHandler): (StartupBenchmark.ResponseHandler.Handler): (StartupBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/thirdparty/__init__.py: Added. (AutoinstallImportHook): Auto installs tornado package for feedback server (AutoinstallImportHook.__init__): (AutoinstallImportHook._ensure_autoinstalled_dir_is_in_sys_path): (AutoinstallImportHook.find_module): (AutoinstallImportHook._install_tornado): (AutoinstallImportHook.greater_than_equal_to_version): (AutoinstallImportHook._install): (AutoinstallImportHook.get_latest_pypi_url): (AutoinstallImportHook.install_binary): (autoinstall_everything): (get_os_info): Canonical link: https://commits.webkit.org/203012@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-19 21:54:48 +00:00
#! /usr/bin/env python
import argparse
import time
from threading import Event
from launch_time import LaunchTimeBenchmark, DefaultLaunchTimeHandler
class NewTabBenchmark(LaunchTimeBenchmark):
def _parse_wait_time(self, string):
values = string.split(':')
start = None
end = None
try:
if len(values) == 2:
start = float(values[0])
end = float(values[1])
if start > end:
raise
elif len(values) == 1:
start = float(values[0])
end = start
else:
raise
except:
raise argparse.ArgumentTypeError(
"'" + string + "' is not a range of numbers. Expected form is N:M where N < M")
return start, end
def initialize(self):
self.benchmark_description = "Measure time to open a new tab for a given browser."
self.response_handler = NewTabBenchmark.ResponseHandler(self)
self.start_time = None
self.stop_time = None
self.stop_signal_was_received = Event()
Add ability to ignore process prewarming for launch time benchmark https://bugs.webkit.org/show_bug.cgi?id=188462 Patch by Ben Richards <benton_richards@apple.com> on 2018-08-10 Reviewed by Ryosuke Niwa. Added a flag to the new tab benchmark that will open a blank tab before the tab to be measured in order to ignore process prewarming. * LaunchTime/launch_time.py: (DefaultLaunchTimeHandler): (DefaultLaunchTimeHandler.on_receive_stop_time): Deleted. (DefaultLaunchTimeHandler.on_receive_stop_signal): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (LaunchTimeBenchmark): (LaunchTimeBenchmark._standard_deviation): Fixed divide by zero bug when '-n' is set to 1 (LaunchTimeBenchmark.open_tab): Added option to open a blank tab (LaunchTimeBenchmark.run): * LaunchTime/new_tab.py: (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (NewTabBenchmark): * LaunchTime/startup.py: (StartupBenchmark.ResponseHandler.Handler.get_test_page): (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (StartupBenchmark): Canonical link: https://commits.webkit.org/203571@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234764 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-08-10 18:23:50 +00:00
self.allow_prewarm = True
Add benchmark for WebKit process launch times https://bugs.webkit.org/show_bug.cgi?id=186414 Patch by Ben Richards <benton_richards@apple.com> on 2018-07-19 Reviewed by Ryosuke Niwa. Added two benchmarks, one for measuring browser new tab launch time and one for browser startup time. * LaunchTime/.gitignore: Added. * LaunchTime/feedback_client.html: Added. Displays benchmark progress in browser * LaunchTime/feedback_server.py: Added. (FeedbackServer): Sends data to feedback_client via websocket (FeedbackServer.__init__): (FeedbackServer._create_app): (FeedbackServer._start_server): (FeedbackServer._send_all_messages): (FeedbackServer.start): (FeedbackServer.stop): (FeedbackServer.send_message): Send a message to the feedback_client (FeedbackServer.wait_until_client_has_loaded): Wait until the feedback_client has opened a websocket connection to the feedback_server (FeedbackServer.MainHandler): Handler factory to create handler that serves feedback_client.html (FeedbackServer.MainHandler.Handler): (FeedbackServer.MainHandler.Handler.get): (FeedbackServer.WSHandler): Handler factory to create handler that sends data to feedback client (FeedbackServer.WSHandler.Handler): (FeedbackServer.WSHandler.Handler.open): On websocket connection opened (FeedbackServer.WSHandler.Handler.on_close): On websocket connection closed * LaunchTime/launch_time.py: Added. (DefaultLaunchTimeHandler): Abstract HTTP request handler for launch time benchmarks (DefaultLaunchTimeHandler.get_test_page): Default test page to be overridden by benchmarks (DefaultLaunchTimeHandler.get_blank_page): (DefaultLaunchTimeHandler.on_receive_stop_time): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (DefaultLaunchTimeHandler.log_message): Suppresses HTTP logs from SimpleHTTPRequestHandler (LaunchTimeBenchmark): Abstract class which launch time benchmarks inherit from and override methods desired to customize (LaunchTimeBenchmark.__init__): (LaunchTimeBenchmark._parse_browser_bundle_path): Parser for bundle path option (LaunchTimeBenchmark._parse_args): (LaunchTimeBenchmark._run_server): Target function for main server thread (LaunchTimeBenchmark._setup_servers): (LaunchTimeBenchmark._clean_up): (LaunchTimeBenchmark._exit_due_to_exception): (LaunchTimeBenchmark._geometric_mean): (LaunchTimeBenchmark._standard_deviation): (LaunchTimeBenchmark._compute_results): Returns mean and std dev of list of results and pretty prints if should_print=True is specified (LaunchTimeBenchmark._wait_times): Mimic numpy.linspace (LaunchTimeBenchmark.open_tab): Open a browser tab with the html given by self.response_handler.get_test_page (LaunchTimeBenchmark.launch_browser): Open a broser to either the feedback client (if option is set) or a blank page (LaunchTimeBenchmark.quit_browser): (LaunchTimeBenchmark.quit_browser.quit_app): (LaunchTimeBenchmark.quit_browser.is_app_closed): (LaunchTimeBenchmark.close_tab): (LaunchTimeBenchmark.wait): (LaunchTimeBenchmark.log): Print to console and send to feedback client if --feedback-in-browser flag is used (LaunchTimeBenchmark.log_verbose): Only logs if --verbose flag is used (LaunchTimeBenchmark.run): (LaunchTimeBenchmark.group_init): Initialization done before each round of iterations (LaunchTimeBenchmark.run_iteration): (LaunchTimeBenchmark.initialize): Convenience method to be overriden by subclasses which is called at the end of __init__ (LaunchTimeBenchmark.will_parse_arguments): Called before argparse.parse_args to let subclasses add new command line arguments (LaunchTimeBenchmark.did_parse_arguments): Called after argparse.parse_args to let subclass initialize based on command line arguments * LaunchTime/new_tab.py: Added (NewTabBenchmark): (NewTabBenchmark._parse_wait_time): Parser for wait time option (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.group_init): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler): (NewTabBenchmark.ResponseHandler.Handler): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/startup.py: Added (StartupBenchmark): This benchmark measures browser startup time and initial page load time (StartupBenchmark.initialize): (StartupBenchmark.run_iteration): (StartupBenchmark.ResponseHandler): (StartupBenchmark.ResponseHandler.Handler): (StartupBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/thirdparty/__init__.py: Added. (AutoinstallImportHook): Auto installs tornado package for feedback server (AutoinstallImportHook.__init__): (AutoinstallImportHook._ensure_autoinstalled_dir_is_in_sys_path): (AutoinstallImportHook.find_module): (AutoinstallImportHook._install_tornado): (AutoinstallImportHook.greater_than_equal_to_version): (AutoinstallImportHook._install): (AutoinstallImportHook.get_latest_pypi_url): (AutoinstallImportHook.install_binary): (autoinstall_everything): (get_os_info): Canonical link: https://commits.webkit.org/203012@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-19 21:54:48 +00:00
def run_iteration(self):
Add ability to ignore process prewarming for launch time benchmark https://bugs.webkit.org/show_bug.cgi?id=188462 Patch by Ben Richards <benton_richards@apple.com> on 2018-08-10 Reviewed by Ryosuke Niwa. Added a flag to the new tab benchmark that will open a blank tab before the tab to be measured in order to ignore process prewarming. * LaunchTime/launch_time.py: (DefaultLaunchTimeHandler): (DefaultLaunchTimeHandler.on_receive_stop_time): Deleted. (DefaultLaunchTimeHandler.on_receive_stop_signal): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (LaunchTimeBenchmark): (LaunchTimeBenchmark._standard_deviation): Fixed divide by zero bug when '-n' is set to 1 (LaunchTimeBenchmark.open_tab): Added option to open a blank tab (LaunchTimeBenchmark.run): * LaunchTime/new_tab.py: (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (NewTabBenchmark): * LaunchTime/startup.py: (StartupBenchmark.ResponseHandler.Handler.get_test_page): (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (StartupBenchmark): Canonical link: https://commits.webkit.org/203571@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234764 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-08-10 18:23:50 +00:00
tabs_to_open = 1 if self.allow_prewarm else 2
self.stop_time = None
for _ in range(tabs_to_open - 1):
self.open_tab(blank=True)
Add benchmark for WebKit process launch times https://bugs.webkit.org/show_bug.cgi?id=186414 Patch by Ben Richards <benton_richards@apple.com> on 2018-07-19 Reviewed by Ryosuke Niwa. Added two benchmarks, one for measuring browser new tab launch time and one for browser startup time. * LaunchTime/.gitignore: Added. * LaunchTime/feedback_client.html: Added. Displays benchmark progress in browser * LaunchTime/feedback_server.py: Added. (FeedbackServer): Sends data to feedback_client via websocket (FeedbackServer.__init__): (FeedbackServer._create_app): (FeedbackServer._start_server): (FeedbackServer._send_all_messages): (FeedbackServer.start): (FeedbackServer.stop): (FeedbackServer.send_message): Send a message to the feedback_client (FeedbackServer.wait_until_client_has_loaded): Wait until the feedback_client has opened a websocket connection to the feedback_server (FeedbackServer.MainHandler): Handler factory to create handler that serves feedback_client.html (FeedbackServer.MainHandler.Handler): (FeedbackServer.MainHandler.Handler.get): (FeedbackServer.WSHandler): Handler factory to create handler that sends data to feedback client (FeedbackServer.WSHandler.Handler): (FeedbackServer.WSHandler.Handler.open): On websocket connection opened (FeedbackServer.WSHandler.Handler.on_close): On websocket connection closed * LaunchTime/launch_time.py: Added. (DefaultLaunchTimeHandler): Abstract HTTP request handler for launch time benchmarks (DefaultLaunchTimeHandler.get_test_page): Default test page to be overridden by benchmarks (DefaultLaunchTimeHandler.get_blank_page): (DefaultLaunchTimeHandler.on_receive_stop_time): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (DefaultLaunchTimeHandler.log_message): Suppresses HTTP logs from SimpleHTTPRequestHandler (LaunchTimeBenchmark): Abstract class which launch time benchmarks inherit from and override methods desired to customize (LaunchTimeBenchmark.__init__): (LaunchTimeBenchmark._parse_browser_bundle_path): Parser for bundle path option (LaunchTimeBenchmark._parse_args): (LaunchTimeBenchmark._run_server): Target function for main server thread (LaunchTimeBenchmark._setup_servers): (LaunchTimeBenchmark._clean_up): (LaunchTimeBenchmark._exit_due_to_exception): (LaunchTimeBenchmark._geometric_mean): (LaunchTimeBenchmark._standard_deviation): (LaunchTimeBenchmark._compute_results): Returns mean and std dev of list of results and pretty prints if should_print=True is specified (LaunchTimeBenchmark._wait_times): Mimic numpy.linspace (LaunchTimeBenchmark.open_tab): Open a browser tab with the html given by self.response_handler.get_test_page (LaunchTimeBenchmark.launch_browser): Open a broser to either the feedback client (if option is set) or a blank page (LaunchTimeBenchmark.quit_browser): (LaunchTimeBenchmark.quit_browser.quit_app): (LaunchTimeBenchmark.quit_browser.is_app_closed): (LaunchTimeBenchmark.close_tab): (LaunchTimeBenchmark.wait): (LaunchTimeBenchmark.log): Print to console and send to feedback client if --feedback-in-browser flag is used (LaunchTimeBenchmark.log_verbose): Only logs if --verbose flag is used (LaunchTimeBenchmark.run): (LaunchTimeBenchmark.group_init): Initialization done before each round of iterations (LaunchTimeBenchmark.run_iteration): (LaunchTimeBenchmark.initialize): Convenience method to be overriden by subclasses which is called at the end of __init__ (LaunchTimeBenchmark.will_parse_arguments): Called before argparse.parse_args to let subclasses add new command line arguments (LaunchTimeBenchmark.did_parse_arguments): Called after argparse.parse_args to let subclass initialize based on command line arguments * LaunchTime/new_tab.py: Added (NewTabBenchmark): (NewTabBenchmark._parse_wait_time): Parser for wait time option (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.group_init): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler): (NewTabBenchmark.ResponseHandler.Handler): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/startup.py: Added (StartupBenchmark): This benchmark measures browser startup time and initial page load time (StartupBenchmark.initialize): (StartupBenchmark.run_iteration): (StartupBenchmark.ResponseHandler): (StartupBenchmark.ResponseHandler.Handler): (StartupBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/thirdparty/__init__.py: Added. (AutoinstallImportHook): Auto installs tornado package for feedback server (AutoinstallImportHook.__init__): (AutoinstallImportHook._ensure_autoinstalled_dir_is_in_sys_path): (AutoinstallImportHook.find_module): (AutoinstallImportHook._install_tornado): (AutoinstallImportHook.greater_than_equal_to_version): (AutoinstallImportHook._install): (AutoinstallImportHook.get_latest_pypi_url): (AutoinstallImportHook.install_binary): (autoinstall_everything): (get_os_info): Canonical link: https://commits.webkit.org/203012@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-19 21:54:48 +00:00
self.start_time = time.time() * 1000
self.open_tab()
while self.stop_time is None:
self.stop_signal_was_received.wait()
result = self.stop_time - self.start_time
self.stop_signal_was_received.clear()
Add ability to ignore process prewarming for launch time benchmark https://bugs.webkit.org/show_bug.cgi?id=188462 Patch by Ben Richards <benton_richards@apple.com> on 2018-08-10 Reviewed by Ryosuke Niwa. Added a flag to the new tab benchmark that will open a blank tab before the tab to be measured in order to ignore process prewarming. * LaunchTime/launch_time.py: (DefaultLaunchTimeHandler): (DefaultLaunchTimeHandler.on_receive_stop_time): Deleted. (DefaultLaunchTimeHandler.on_receive_stop_signal): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (LaunchTimeBenchmark): (LaunchTimeBenchmark._standard_deviation): Fixed divide by zero bug when '-n' is set to 1 (LaunchTimeBenchmark.open_tab): Added option to open a blank tab (LaunchTimeBenchmark.run): * LaunchTime/new_tab.py: (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (NewTabBenchmark): * LaunchTime/startup.py: (StartupBenchmark.ResponseHandler.Handler.get_test_page): (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (StartupBenchmark): Canonical link: https://commits.webkit.org/203571@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234764 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-08-10 18:23:50 +00:00
for _ in range(tabs_to_open):
self.close_tab()
Add benchmark for WebKit process launch times https://bugs.webkit.org/show_bug.cgi?id=186414 Patch by Ben Richards <benton_richards@apple.com> on 2018-07-19 Reviewed by Ryosuke Niwa. Added two benchmarks, one for measuring browser new tab launch time and one for browser startup time. * LaunchTime/.gitignore: Added. * LaunchTime/feedback_client.html: Added. Displays benchmark progress in browser * LaunchTime/feedback_server.py: Added. (FeedbackServer): Sends data to feedback_client via websocket (FeedbackServer.__init__): (FeedbackServer._create_app): (FeedbackServer._start_server): (FeedbackServer._send_all_messages): (FeedbackServer.start): (FeedbackServer.stop): (FeedbackServer.send_message): Send a message to the feedback_client (FeedbackServer.wait_until_client_has_loaded): Wait until the feedback_client has opened a websocket connection to the feedback_server (FeedbackServer.MainHandler): Handler factory to create handler that serves feedback_client.html (FeedbackServer.MainHandler.Handler): (FeedbackServer.MainHandler.Handler.get): (FeedbackServer.WSHandler): Handler factory to create handler that sends data to feedback client (FeedbackServer.WSHandler.Handler): (FeedbackServer.WSHandler.Handler.open): On websocket connection opened (FeedbackServer.WSHandler.Handler.on_close): On websocket connection closed * LaunchTime/launch_time.py: Added. (DefaultLaunchTimeHandler): Abstract HTTP request handler for launch time benchmarks (DefaultLaunchTimeHandler.get_test_page): Default test page to be overridden by benchmarks (DefaultLaunchTimeHandler.get_blank_page): (DefaultLaunchTimeHandler.on_receive_stop_time): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (DefaultLaunchTimeHandler.log_message): Suppresses HTTP logs from SimpleHTTPRequestHandler (LaunchTimeBenchmark): Abstract class which launch time benchmarks inherit from and override methods desired to customize (LaunchTimeBenchmark.__init__): (LaunchTimeBenchmark._parse_browser_bundle_path): Parser for bundle path option (LaunchTimeBenchmark._parse_args): (LaunchTimeBenchmark._run_server): Target function for main server thread (LaunchTimeBenchmark._setup_servers): (LaunchTimeBenchmark._clean_up): (LaunchTimeBenchmark._exit_due_to_exception): (LaunchTimeBenchmark._geometric_mean): (LaunchTimeBenchmark._standard_deviation): (LaunchTimeBenchmark._compute_results): Returns mean and std dev of list of results and pretty prints if should_print=True is specified (LaunchTimeBenchmark._wait_times): Mimic numpy.linspace (LaunchTimeBenchmark.open_tab): Open a browser tab with the html given by self.response_handler.get_test_page (LaunchTimeBenchmark.launch_browser): Open a broser to either the feedback client (if option is set) or a blank page (LaunchTimeBenchmark.quit_browser): (LaunchTimeBenchmark.quit_browser.quit_app): (LaunchTimeBenchmark.quit_browser.is_app_closed): (LaunchTimeBenchmark.close_tab): (LaunchTimeBenchmark.wait): (LaunchTimeBenchmark.log): Print to console and send to feedback client if --feedback-in-browser flag is used (LaunchTimeBenchmark.log_verbose): Only logs if --verbose flag is used (LaunchTimeBenchmark.run): (LaunchTimeBenchmark.group_init): Initialization done before each round of iterations (LaunchTimeBenchmark.run_iteration): (LaunchTimeBenchmark.initialize): Convenience method to be overriden by subclasses which is called at the end of __init__ (LaunchTimeBenchmark.will_parse_arguments): Called before argparse.parse_args to let subclasses add new command line arguments (LaunchTimeBenchmark.did_parse_arguments): Called after argparse.parse_args to let subclass initialize based on command line arguments * LaunchTime/new_tab.py: Added (NewTabBenchmark): (NewTabBenchmark._parse_wait_time): Parser for wait time option (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.group_init): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler): (NewTabBenchmark.ResponseHandler.Handler): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/startup.py: Added (StartupBenchmark): This benchmark measures browser startup time and initial page load time (StartupBenchmark.initialize): (StartupBenchmark.run_iteration): (StartupBenchmark.ResponseHandler): (StartupBenchmark.ResponseHandler.Handler): (StartupBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/thirdparty/__init__.py: Added. (AutoinstallImportHook): Auto installs tornado package for feedback server (AutoinstallImportHook.__init__): (AutoinstallImportHook._ensure_autoinstalled_dir_is_in_sys_path): (AutoinstallImportHook.find_module): (AutoinstallImportHook._install_tornado): (AutoinstallImportHook.greater_than_equal_to_version): (AutoinstallImportHook._install): (AutoinstallImportHook.get_latest_pypi_url): (AutoinstallImportHook.install_binary): (autoinstall_everything): (get_os_info): Canonical link: https://commits.webkit.org/203012@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-19 21:54:48 +00:00
return result
def group_init(self):
self.launch_browser()
def get_test_name(self):
return "NewTabBenchmark"
Add benchmark for WebKit process launch times https://bugs.webkit.org/show_bug.cgi?id=186414 Patch by Ben Richards <benton_richards@apple.com> on 2018-07-19 Reviewed by Ryosuke Niwa. Added two benchmarks, one for measuring browser new tab launch time and one for browser startup time. * LaunchTime/.gitignore: Added. * LaunchTime/feedback_client.html: Added. Displays benchmark progress in browser * LaunchTime/feedback_server.py: Added. (FeedbackServer): Sends data to feedback_client via websocket (FeedbackServer.__init__): (FeedbackServer._create_app): (FeedbackServer._start_server): (FeedbackServer._send_all_messages): (FeedbackServer.start): (FeedbackServer.stop): (FeedbackServer.send_message): Send a message to the feedback_client (FeedbackServer.wait_until_client_has_loaded): Wait until the feedback_client has opened a websocket connection to the feedback_server (FeedbackServer.MainHandler): Handler factory to create handler that serves feedback_client.html (FeedbackServer.MainHandler.Handler): (FeedbackServer.MainHandler.Handler.get): (FeedbackServer.WSHandler): Handler factory to create handler that sends data to feedback client (FeedbackServer.WSHandler.Handler): (FeedbackServer.WSHandler.Handler.open): On websocket connection opened (FeedbackServer.WSHandler.Handler.on_close): On websocket connection closed * LaunchTime/launch_time.py: Added. (DefaultLaunchTimeHandler): Abstract HTTP request handler for launch time benchmarks (DefaultLaunchTimeHandler.get_test_page): Default test page to be overridden by benchmarks (DefaultLaunchTimeHandler.get_blank_page): (DefaultLaunchTimeHandler.on_receive_stop_time): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (DefaultLaunchTimeHandler.log_message): Suppresses HTTP logs from SimpleHTTPRequestHandler (LaunchTimeBenchmark): Abstract class which launch time benchmarks inherit from and override methods desired to customize (LaunchTimeBenchmark.__init__): (LaunchTimeBenchmark._parse_browser_bundle_path): Parser for bundle path option (LaunchTimeBenchmark._parse_args): (LaunchTimeBenchmark._run_server): Target function for main server thread (LaunchTimeBenchmark._setup_servers): (LaunchTimeBenchmark._clean_up): (LaunchTimeBenchmark._exit_due_to_exception): (LaunchTimeBenchmark._geometric_mean): (LaunchTimeBenchmark._standard_deviation): (LaunchTimeBenchmark._compute_results): Returns mean and std dev of list of results and pretty prints if should_print=True is specified (LaunchTimeBenchmark._wait_times): Mimic numpy.linspace (LaunchTimeBenchmark.open_tab): Open a browser tab with the html given by self.response_handler.get_test_page (LaunchTimeBenchmark.launch_browser): Open a broser to either the feedback client (if option is set) or a blank page (LaunchTimeBenchmark.quit_browser): (LaunchTimeBenchmark.quit_browser.quit_app): (LaunchTimeBenchmark.quit_browser.is_app_closed): (LaunchTimeBenchmark.close_tab): (LaunchTimeBenchmark.wait): (LaunchTimeBenchmark.log): Print to console and send to feedback client if --feedback-in-browser flag is used (LaunchTimeBenchmark.log_verbose): Only logs if --verbose flag is used (LaunchTimeBenchmark.run): (LaunchTimeBenchmark.group_init): Initialization done before each round of iterations (LaunchTimeBenchmark.run_iteration): (LaunchTimeBenchmark.initialize): Convenience method to be overriden by subclasses which is called at the end of __init__ (LaunchTimeBenchmark.will_parse_arguments): Called before argparse.parse_args to let subclasses add new command line arguments (LaunchTimeBenchmark.did_parse_arguments): Called after argparse.parse_args to let subclass initialize based on command line arguments * LaunchTime/new_tab.py: Added (NewTabBenchmark): (NewTabBenchmark._parse_wait_time): Parser for wait time option (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.group_init): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler): (NewTabBenchmark.ResponseHandler.Handler): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/startup.py: Added (StartupBenchmark): This benchmark measures browser startup time and initial page load time (StartupBenchmark.initialize): (StartupBenchmark.run_iteration): (StartupBenchmark.ResponseHandler): (StartupBenchmark.ResponseHandler.Handler): (StartupBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/thirdparty/__init__.py: Added. (AutoinstallImportHook): Auto installs tornado package for feedback server (AutoinstallImportHook.__init__): (AutoinstallImportHook._ensure_autoinstalled_dir_is_in_sys_path): (AutoinstallImportHook.find_module): (AutoinstallImportHook._install_tornado): (AutoinstallImportHook.greater_than_equal_to_version): (AutoinstallImportHook._install): (AutoinstallImportHook.get_latest_pypi_url): (AutoinstallImportHook.install_binary): (autoinstall_everything): (get_os_info): Canonical link: https://commits.webkit.org/203012@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-19 21:54:48 +00:00
def will_parse_arguments(self):
self.argument_parser.add_argument('-g', '--groups', type=int,
help='number of groups of iterations to run (default: {})'.format(self.iteration_groups))
self.argument_parser.add_argument('-w', '--wait-time', type=self._parse_wait_time,
help='wait time to use between iterations or range to scan (format is "N" or "N:M" where N < M, default: {}:{})'.format(self.wait_time_low, self.wait_time_high))
Add ability to ignore process prewarming for launch time benchmark https://bugs.webkit.org/show_bug.cgi?id=188462 Patch by Ben Richards <benton_richards@apple.com> on 2018-08-10 Reviewed by Ryosuke Niwa. Added a flag to the new tab benchmark that will open a blank tab before the tab to be measured in order to ignore process prewarming. * LaunchTime/launch_time.py: (DefaultLaunchTimeHandler): (DefaultLaunchTimeHandler.on_receive_stop_time): Deleted. (DefaultLaunchTimeHandler.on_receive_stop_signal): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (LaunchTimeBenchmark): (LaunchTimeBenchmark._standard_deviation): Fixed divide by zero bug when '-n' is set to 1 (LaunchTimeBenchmark.open_tab): Added option to open a blank tab (LaunchTimeBenchmark.run): * LaunchTime/new_tab.py: (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (NewTabBenchmark): * LaunchTime/startup.py: (StartupBenchmark.ResponseHandler.Handler.get_test_page): (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (StartupBenchmark): Canonical link: https://commits.webkit.org/203571@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234764 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-08-10 18:23:50 +00:00
self.argument_parser.add_argument('--no-prewarm', action='store_true',
help='attempt to ignore process prewarming (will most likely raise standard deviation)')
Add benchmark for WebKit process launch times https://bugs.webkit.org/show_bug.cgi?id=186414 Patch by Ben Richards <benton_richards@apple.com> on 2018-07-19 Reviewed by Ryosuke Niwa. Added two benchmarks, one for measuring browser new tab launch time and one for browser startup time. * LaunchTime/.gitignore: Added. * LaunchTime/feedback_client.html: Added. Displays benchmark progress in browser * LaunchTime/feedback_server.py: Added. (FeedbackServer): Sends data to feedback_client via websocket (FeedbackServer.__init__): (FeedbackServer._create_app): (FeedbackServer._start_server): (FeedbackServer._send_all_messages): (FeedbackServer.start): (FeedbackServer.stop): (FeedbackServer.send_message): Send a message to the feedback_client (FeedbackServer.wait_until_client_has_loaded): Wait until the feedback_client has opened a websocket connection to the feedback_server (FeedbackServer.MainHandler): Handler factory to create handler that serves feedback_client.html (FeedbackServer.MainHandler.Handler): (FeedbackServer.MainHandler.Handler.get): (FeedbackServer.WSHandler): Handler factory to create handler that sends data to feedback client (FeedbackServer.WSHandler.Handler): (FeedbackServer.WSHandler.Handler.open): On websocket connection opened (FeedbackServer.WSHandler.Handler.on_close): On websocket connection closed * LaunchTime/launch_time.py: Added. (DefaultLaunchTimeHandler): Abstract HTTP request handler for launch time benchmarks (DefaultLaunchTimeHandler.get_test_page): Default test page to be overridden by benchmarks (DefaultLaunchTimeHandler.get_blank_page): (DefaultLaunchTimeHandler.on_receive_stop_time): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (DefaultLaunchTimeHandler.log_message): Suppresses HTTP logs from SimpleHTTPRequestHandler (LaunchTimeBenchmark): Abstract class which launch time benchmarks inherit from and override methods desired to customize (LaunchTimeBenchmark.__init__): (LaunchTimeBenchmark._parse_browser_bundle_path): Parser for bundle path option (LaunchTimeBenchmark._parse_args): (LaunchTimeBenchmark._run_server): Target function for main server thread (LaunchTimeBenchmark._setup_servers): (LaunchTimeBenchmark._clean_up): (LaunchTimeBenchmark._exit_due_to_exception): (LaunchTimeBenchmark._geometric_mean): (LaunchTimeBenchmark._standard_deviation): (LaunchTimeBenchmark._compute_results): Returns mean and std dev of list of results and pretty prints if should_print=True is specified (LaunchTimeBenchmark._wait_times): Mimic numpy.linspace (LaunchTimeBenchmark.open_tab): Open a browser tab with the html given by self.response_handler.get_test_page (LaunchTimeBenchmark.launch_browser): Open a broser to either the feedback client (if option is set) or a blank page (LaunchTimeBenchmark.quit_browser): (LaunchTimeBenchmark.quit_browser.quit_app): (LaunchTimeBenchmark.quit_browser.is_app_closed): (LaunchTimeBenchmark.close_tab): (LaunchTimeBenchmark.wait): (LaunchTimeBenchmark.log): Print to console and send to feedback client if --feedback-in-browser flag is used (LaunchTimeBenchmark.log_verbose): Only logs if --verbose flag is used (LaunchTimeBenchmark.run): (LaunchTimeBenchmark.group_init): Initialization done before each round of iterations (LaunchTimeBenchmark.run_iteration): (LaunchTimeBenchmark.initialize): Convenience method to be overriden by subclasses which is called at the end of __init__ (LaunchTimeBenchmark.will_parse_arguments): Called before argparse.parse_args to let subclasses add new command line arguments (LaunchTimeBenchmark.did_parse_arguments): Called after argparse.parse_args to let subclass initialize based on command line arguments * LaunchTime/new_tab.py: Added (NewTabBenchmark): (NewTabBenchmark._parse_wait_time): Parser for wait time option (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.group_init): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler): (NewTabBenchmark.ResponseHandler.Handler): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/startup.py: Added (StartupBenchmark): This benchmark measures browser startup time and initial page load time (StartupBenchmark.initialize): (StartupBenchmark.run_iteration): (StartupBenchmark.ResponseHandler): (StartupBenchmark.ResponseHandler.Handler): (StartupBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/thirdparty/__init__.py: Added. (AutoinstallImportHook): Auto installs tornado package for feedback server (AutoinstallImportHook.__init__): (AutoinstallImportHook._ensure_autoinstalled_dir_is_in_sys_path): (AutoinstallImportHook.find_module): (AutoinstallImportHook._install_tornado): (AutoinstallImportHook.greater_than_equal_to_version): (AutoinstallImportHook._install): (AutoinstallImportHook.get_latest_pypi_url): (AutoinstallImportHook.install_binary): (autoinstall_everything): (get_os_info): Canonical link: https://commits.webkit.org/203012@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-19 21:54:48 +00:00
def did_parse_arguments(self, args):
if args.groups:
self.iteration_groups = args.groups
if args.wait_time:
self.wait_time_low, self.wait_time_high = args.wait_time
Add ability to ignore process prewarming for launch time benchmark https://bugs.webkit.org/show_bug.cgi?id=188462 Patch by Ben Richards <benton_richards@apple.com> on 2018-08-10 Reviewed by Ryosuke Niwa. Added a flag to the new tab benchmark that will open a blank tab before the tab to be measured in order to ignore process prewarming. * LaunchTime/launch_time.py: (DefaultLaunchTimeHandler): (DefaultLaunchTimeHandler.on_receive_stop_time): Deleted. (DefaultLaunchTimeHandler.on_receive_stop_signal): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (LaunchTimeBenchmark): (LaunchTimeBenchmark._standard_deviation): Fixed divide by zero bug when '-n' is set to 1 (LaunchTimeBenchmark.open_tab): Added option to open a blank tab (LaunchTimeBenchmark.run): * LaunchTime/new_tab.py: (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (NewTabBenchmark): * LaunchTime/startup.py: (StartupBenchmark.ResponseHandler.Handler.get_test_page): (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (StartupBenchmark): Canonical link: https://commits.webkit.org/203571@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234764 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-08-10 18:23:50 +00:00
if args.no_prewarm:
self.allow_prewarm = False
Add benchmark for WebKit process launch times https://bugs.webkit.org/show_bug.cgi?id=186414 Patch by Ben Richards <benton_richards@apple.com> on 2018-07-19 Reviewed by Ryosuke Niwa. Added two benchmarks, one for measuring browser new tab launch time and one for browser startup time. * LaunchTime/.gitignore: Added. * LaunchTime/feedback_client.html: Added. Displays benchmark progress in browser * LaunchTime/feedback_server.py: Added. (FeedbackServer): Sends data to feedback_client via websocket (FeedbackServer.__init__): (FeedbackServer._create_app): (FeedbackServer._start_server): (FeedbackServer._send_all_messages): (FeedbackServer.start): (FeedbackServer.stop): (FeedbackServer.send_message): Send a message to the feedback_client (FeedbackServer.wait_until_client_has_loaded): Wait until the feedback_client has opened a websocket connection to the feedback_server (FeedbackServer.MainHandler): Handler factory to create handler that serves feedback_client.html (FeedbackServer.MainHandler.Handler): (FeedbackServer.MainHandler.Handler.get): (FeedbackServer.WSHandler): Handler factory to create handler that sends data to feedback client (FeedbackServer.WSHandler.Handler): (FeedbackServer.WSHandler.Handler.open): On websocket connection opened (FeedbackServer.WSHandler.Handler.on_close): On websocket connection closed * LaunchTime/launch_time.py: Added. (DefaultLaunchTimeHandler): Abstract HTTP request handler for launch time benchmarks (DefaultLaunchTimeHandler.get_test_page): Default test page to be overridden by benchmarks (DefaultLaunchTimeHandler.get_blank_page): (DefaultLaunchTimeHandler.on_receive_stop_time): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (DefaultLaunchTimeHandler.log_message): Suppresses HTTP logs from SimpleHTTPRequestHandler (LaunchTimeBenchmark): Abstract class which launch time benchmarks inherit from and override methods desired to customize (LaunchTimeBenchmark.__init__): (LaunchTimeBenchmark._parse_browser_bundle_path): Parser for bundle path option (LaunchTimeBenchmark._parse_args): (LaunchTimeBenchmark._run_server): Target function for main server thread (LaunchTimeBenchmark._setup_servers): (LaunchTimeBenchmark._clean_up): (LaunchTimeBenchmark._exit_due_to_exception): (LaunchTimeBenchmark._geometric_mean): (LaunchTimeBenchmark._standard_deviation): (LaunchTimeBenchmark._compute_results): Returns mean and std dev of list of results and pretty prints if should_print=True is specified (LaunchTimeBenchmark._wait_times): Mimic numpy.linspace (LaunchTimeBenchmark.open_tab): Open a browser tab with the html given by self.response_handler.get_test_page (LaunchTimeBenchmark.launch_browser): Open a broser to either the feedback client (if option is set) or a blank page (LaunchTimeBenchmark.quit_browser): (LaunchTimeBenchmark.quit_browser.quit_app): (LaunchTimeBenchmark.quit_browser.is_app_closed): (LaunchTimeBenchmark.close_tab): (LaunchTimeBenchmark.wait): (LaunchTimeBenchmark.log): Print to console and send to feedback client if --feedback-in-browser flag is used (LaunchTimeBenchmark.log_verbose): Only logs if --verbose flag is used (LaunchTimeBenchmark.run): (LaunchTimeBenchmark.group_init): Initialization done before each round of iterations (LaunchTimeBenchmark.run_iteration): (LaunchTimeBenchmark.initialize): Convenience method to be overriden by subclasses which is called at the end of __init__ (LaunchTimeBenchmark.will_parse_arguments): Called before argparse.parse_args to let subclasses add new command line arguments (LaunchTimeBenchmark.did_parse_arguments): Called after argparse.parse_args to let subclass initialize based on command line arguments * LaunchTime/new_tab.py: Added (NewTabBenchmark): (NewTabBenchmark._parse_wait_time): Parser for wait time option (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.group_init): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler): (NewTabBenchmark.ResponseHandler.Handler): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/startup.py: Added (StartupBenchmark): This benchmark measures browser startup time and initial page load time (StartupBenchmark.initialize): (StartupBenchmark.run_iteration): (StartupBenchmark.ResponseHandler): (StartupBenchmark.ResponseHandler.Handler): (StartupBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/thirdparty/__init__.py: Added. (AutoinstallImportHook): Auto installs tornado package for feedback server (AutoinstallImportHook.__init__): (AutoinstallImportHook._ensure_autoinstalled_dir_is_in_sys_path): (AutoinstallImportHook.find_module): (AutoinstallImportHook._install_tornado): (AutoinstallImportHook.greater_than_equal_to_version): (AutoinstallImportHook._install): (AutoinstallImportHook.get_latest_pypi_url): (AutoinstallImportHook.install_binary): (autoinstall_everything): (get_os_info): Canonical link: https://commits.webkit.org/203012@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-19 21:54:48 +00:00
@staticmethod
def ResponseHandler(new_tab_benchmark):
class Handler(DefaultLaunchTimeHandler):
def get_test_page(self):
return '''<!DOCTYPE html>
<html>
<head>
<title>New Tab Benchmark</title>
<meta http-equiv="Content-Type" content="text/html" />
<script>
function sendDone() {
Add ability to ignore process prewarming for launch time benchmark https://bugs.webkit.org/show_bug.cgi?id=188462 Patch by Ben Richards <benton_richards@apple.com> on 2018-08-10 Reviewed by Ryosuke Niwa. Added a flag to the new tab benchmark that will open a blank tab before the tab to be measured in order to ignore process prewarming. * LaunchTime/launch_time.py: (DefaultLaunchTimeHandler): (DefaultLaunchTimeHandler.on_receive_stop_time): Deleted. (DefaultLaunchTimeHandler.on_receive_stop_signal): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (LaunchTimeBenchmark): (LaunchTimeBenchmark._standard_deviation): Fixed divide by zero bug when '-n' is set to 1 (LaunchTimeBenchmark.open_tab): Added option to open a blank tab (LaunchTimeBenchmark.run): * LaunchTime/new_tab.py: (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (NewTabBenchmark): * LaunchTime/startup.py: (StartupBenchmark.ResponseHandler.Handler.get_test_page): (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (StartupBenchmark): Canonical link: https://commits.webkit.org/203571@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234764 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-08-10 18:23:50 +00:00
const time = performance.timing.navigationStart
const request = new XMLHttpRequest();
Add benchmark for WebKit process launch times https://bugs.webkit.org/show_bug.cgi?id=186414 Patch by Ben Richards <benton_richards@apple.com> on 2018-07-19 Reviewed by Ryosuke Niwa. Added two benchmarks, one for measuring browser new tab launch time and one for browser startup time. * LaunchTime/.gitignore: Added. * LaunchTime/feedback_client.html: Added. Displays benchmark progress in browser * LaunchTime/feedback_server.py: Added. (FeedbackServer): Sends data to feedback_client via websocket (FeedbackServer.__init__): (FeedbackServer._create_app): (FeedbackServer._start_server): (FeedbackServer._send_all_messages): (FeedbackServer.start): (FeedbackServer.stop): (FeedbackServer.send_message): Send a message to the feedback_client (FeedbackServer.wait_until_client_has_loaded): Wait until the feedback_client has opened a websocket connection to the feedback_server (FeedbackServer.MainHandler): Handler factory to create handler that serves feedback_client.html (FeedbackServer.MainHandler.Handler): (FeedbackServer.MainHandler.Handler.get): (FeedbackServer.WSHandler): Handler factory to create handler that sends data to feedback client (FeedbackServer.WSHandler.Handler): (FeedbackServer.WSHandler.Handler.open): On websocket connection opened (FeedbackServer.WSHandler.Handler.on_close): On websocket connection closed * LaunchTime/launch_time.py: Added. (DefaultLaunchTimeHandler): Abstract HTTP request handler for launch time benchmarks (DefaultLaunchTimeHandler.get_test_page): Default test page to be overridden by benchmarks (DefaultLaunchTimeHandler.get_blank_page): (DefaultLaunchTimeHandler.on_receive_stop_time): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (DefaultLaunchTimeHandler.log_message): Suppresses HTTP logs from SimpleHTTPRequestHandler (LaunchTimeBenchmark): Abstract class which launch time benchmarks inherit from and override methods desired to customize (LaunchTimeBenchmark.__init__): (LaunchTimeBenchmark._parse_browser_bundle_path): Parser for bundle path option (LaunchTimeBenchmark._parse_args): (LaunchTimeBenchmark._run_server): Target function for main server thread (LaunchTimeBenchmark._setup_servers): (LaunchTimeBenchmark._clean_up): (LaunchTimeBenchmark._exit_due_to_exception): (LaunchTimeBenchmark._geometric_mean): (LaunchTimeBenchmark._standard_deviation): (LaunchTimeBenchmark._compute_results): Returns mean and std dev of list of results and pretty prints if should_print=True is specified (LaunchTimeBenchmark._wait_times): Mimic numpy.linspace (LaunchTimeBenchmark.open_tab): Open a browser tab with the html given by self.response_handler.get_test_page (LaunchTimeBenchmark.launch_browser): Open a broser to either the feedback client (if option is set) or a blank page (LaunchTimeBenchmark.quit_browser): (LaunchTimeBenchmark.quit_browser.quit_app): (LaunchTimeBenchmark.quit_browser.is_app_closed): (LaunchTimeBenchmark.close_tab): (LaunchTimeBenchmark.wait): (LaunchTimeBenchmark.log): Print to console and send to feedback client if --feedback-in-browser flag is used (LaunchTimeBenchmark.log_verbose): Only logs if --verbose flag is used (LaunchTimeBenchmark.run): (LaunchTimeBenchmark.group_init): Initialization done before each round of iterations (LaunchTimeBenchmark.run_iteration): (LaunchTimeBenchmark.initialize): Convenience method to be overriden by subclasses which is called at the end of __init__ (LaunchTimeBenchmark.will_parse_arguments): Called before argparse.parse_args to let subclasses add new command line arguments (LaunchTimeBenchmark.did_parse_arguments): Called after argparse.parse_args to let subclass initialize based on command line arguments * LaunchTime/new_tab.py: Added (NewTabBenchmark): (NewTabBenchmark._parse_wait_time): Parser for wait time option (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.group_init): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler): (NewTabBenchmark.ResponseHandler.Handler): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/startup.py: Added (StartupBenchmark): This benchmark measures browser startup time and initial page load time (StartupBenchmark.initialize): (StartupBenchmark.run_iteration): (StartupBenchmark.ResponseHandler): (StartupBenchmark.ResponseHandler.Handler): (StartupBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/thirdparty/__init__.py: Added. (AutoinstallImportHook): Auto installs tornado package for feedback server (AutoinstallImportHook.__init__): (AutoinstallImportHook._ensure_autoinstalled_dir_is_in_sys_path): (AutoinstallImportHook.find_module): (AutoinstallImportHook._install_tornado): (AutoinstallImportHook.greater_than_equal_to_version): (AutoinstallImportHook._install): (AutoinstallImportHook.get_latest_pypi_url): (AutoinstallImportHook.install_binary): (autoinstall_everything): (get_os_info): Canonical link: https://commits.webkit.org/203012@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-19 21:54:48 +00:00
request.open("POST", "done", false);
request.setRequestHeader('Content-Type', 'application/json');
request.send(JSON.stringify(time));
}
window.onload = sendDone;
</script>
</head>
<body>
<h1>New Tab Benchmark</h1>
</body>
</html>
'''
Add ability to ignore process prewarming for launch time benchmark https://bugs.webkit.org/show_bug.cgi?id=188462 Patch by Ben Richards <benton_richards@apple.com> on 2018-08-10 Reviewed by Ryosuke Niwa. Added a flag to the new tab benchmark that will open a blank tab before the tab to be measured in order to ignore process prewarming. * LaunchTime/launch_time.py: (DefaultLaunchTimeHandler): (DefaultLaunchTimeHandler.on_receive_stop_time): Deleted. (DefaultLaunchTimeHandler.on_receive_stop_signal): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (LaunchTimeBenchmark): (LaunchTimeBenchmark._standard_deviation): Fixed divide by zero bug when '-n' is set to 1 (LaunchTimeBenchmark.open_tab): Added option to open a blank tab (LaunchTimeBenchmark.run): * LaunchTime/new_tab.py: (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (NewTabBenchmark): * LaunchTime/startup.py: (StartupBenchmark.ResponseHandler.Handler.get_test_page): (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted. (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_signal): (StartupBenchmark): Canonical link: https://commits.webkit.org/203571@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234764 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-08-10 18:23:50 +00:00
def on_receive_stop_signal(self, data):
new_tab_benchmark.stop_time = float(data)
Add benchmark for WebKit process launch times https://bugs.webkit.org/show_bug.cgi?id=186414 Patch by Ben Richards <benton_richards@apple.com> on 2018-07-19 Reviewed by Ryosuke Niwa. Added two benchmarks, one for measuring browser new tab launch time and one for browser startup time. * LaunchTime/.gitignore: Added. * LaunchTime/feedback_client.html: Added. Displays benchmark progress in browser * LaunchTime/feedback_server.py: Added. (FeedbackServer): Sends data to feedback_client via websocket (FeedbackServer.__init__): (FeedbackServer._create_app): (FeedbackServer._start_server): (FeedbackServer._send_all_messages): (FeedbackServer.start): (FeedbackServer.stop): (FeedbackServer.send_message): Send a message to the feedback_client (FeedbackServer.wait_until_client_has_loaded): Wait until the feedback_client has opened a websocket connection to the feedback_server (FeedbackServer.MainHandler): Handler factory to create handler that serves feedback_client.html (FeedbackServer.MainHandler.Handler): (FeedbackServer.MainHandler.Handler.get): (FeedbackServer.WSHandler): Handler factory to create handler that sends data to feedback client (FeedbackServer.WSHandler.Handler): (FeedbackServer.WSHandler.Handler.open): On websocket connection opened (FeedbackServer.WSHandler.Handler.on_close): On websocket connection closed * LaunchTime/launch_time.py: Added. (DefaultLaunchTimeHandler): Abstract HTTP request handler for launch time benchmarks (DefaultLaunchTimeHandler.get_test_page): Default test page to be overridden by benchmarks (DefaultLaunchTimeHandler.get_blank_page): (DefaultLaunchTimeHandler.on_receive_stop_time): (DefaultLaunchTimeHandler.do_HEAD): (DefaultLaunchTimeHandler.do_GET): (DefaultLaunchTimeHandler.do_POST): (DefaultLaunchTimeHandler.log_message): Suppresses HTTP logs from SimpleHTTPRequestHandler (LaunchTimeBenchmark): Abstract class which launch time benchmarks inherit from and override methods desired to customize (LaunchTimeBenchmark.__init__): (LaunchTimeBenchmark._parse_browser_bundle_path): Parser for bundle path option (LaunchTimeBenchmark._parse_args): (LaunchTimeBenchmark._run_server): Target function for main server thread (LaunchTimeBenchmark._setup_servers): (LaunchTimeBenchmark._clean_up): (LaunchTimeBenchmark._exit_due_to_exception): (LaunchTimeBenchmark._geometric_mean): (LaunchTimeBenchmark._standard_deviation): (LaunchTimeBenchmark._compute_results): Returns mean and std dev of list of results and pretty prints if should_print=True is specified (LaunchTimeBenchmark._wait_times): Mimic numpy.linspace (LaunchTimeBenchmark.open_tab): Open a browser tab with the html given by self.response_handler.get_test_page (LaunchTimeBenchmark.launch_browser): Open a broser to either the feedback client (if option is set) or a blank page (LaunchTimeBenchmark.quit_browser): (LaunchTimeBenchmark.quit_browser.quit_app): (LaunchTimeBenchmark.quit_browser.is_app_closed): (LaunchTimeBenchmark.close_tab): (LaunchTimeBenchmark.wait): (LaunchTimeBenchmark.log): Print to console and send to feedback client if --feedback-in-browser flag is used (LaunchTimeBenchmark.log_verbose): Only logs if --verbose flag is used (LaunchTimeBenchmark.run): (LaunchTimeBenchmark.group_init): Initialization done before each round of iterations (LaunchTimeBenchmark.run_iteration): (LaunchTimeBenchmark.initialize): Convenience method to be overriden by subclasses which is called at the end of __init__ (LaunchTimeBenchmark.will_parse_arguments): Called before argparse.parse_args to let subclasses add new command line arguments (LaunchTimeBenchmark.did_parse_arguments): Called after argparse.parse_args to let subclass initialize based on command line arguments * LaunchTime/new_tab.py: Added (NewTabBenchmark): (NewTabBenchmark._parse_wait_time): Parser for wait time option (NewTabBenchmark.initialize): (NewTabBenchmark.run_iteration): (NewTabBenchmark.group_init): (NewTabBenchmark.will_parse_arguments): (NewTabBenchmark.did_parse_arguments): (NewTabBenchmark.ResponseHandler): (NewTabBenchmark.ResponseHandler.Handler): (NewTabBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/startup.py: Added (StartupBenchmark): This benchmark measures browser startup time and initial page load time (StartupBenchmark.initialize): (StartupBenchmark.run_iteration): (StartupBenchmark.ResponseHandler): (StartupBenchmark.ResponseHandler.Handler): (StartupBenchmark.ResponseHandler.Handler.get_test_page): * LaunchTime/thirdparty/__init__.py: Added. (AutoinstallImportHook): Auto installs tornado package for feedback server (AutoinstallImportHook.__init__): (AutoinstallImportHook._ensure_autoinstalled_dir_is_in_sys_path): (AutoinstallImportHook.find_module): (AutoinstallImportHook._install_tornado): (AutoinstallImportHook.greater_than_equal_to_version): (AutoinstallImportHook._install): (AutoinstallImportHook.get_latest_pypi_url): (AutoinstallImportHook.install_binary): (autoinstall_everything): (get_os_info): Canonical link: https://commits.webkit.org/203012@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-19 21:54:48 +00:00
new_tab_benchmark.stop_signal_was_received.set()
return Handler
if __name__ == '__main__':
NewTabBenchmark().run()