haikuwebkit/Websites/perf.webkit.org/tools/bundle-v3-scripts.py

97 lines
4.2 KiB
Python
Raw Permalink Normal View History

Add v3 UI to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=152311 Reviewed by Chris Dumez. Add the third iteration of the perf dashboard UI. UI for viewing and modifying analysis tasks is coming soon. The v3 UI is focused on speed, and removes all third-party script dependencies including jQuery, d3, and Ember. Both the DOM-based UI and graphing are implemented manually. The entire app is structured using new component library implemented in components/base.js. Each component is an instance of a subclass of ComponentBase which owns a single DOM element. Each subclass may supply static methods named htmlTemplate and cssTemplate as the template for a component instance. ComponentBase automatically clones the templates inside the associated element (or its shadow root on the supported browsers). Each subclass must supply a method called "render()" which constructs and updates the DOM as needed. There is a special component called Page, which represents an entire page. Each Page is opened by PageRouter's "route()" function. Each subclass of Page supplies "open()" for initialization and "updateFromSerializedState()" for a hash URL transition. The key feature of the v3 UI is the split of time series into chunks called clusters (see r194120). On an internal instance of the dashboard, the v2 UI downloads 27MB of data whereas the same page loads only 3MB of data in the v3. The key logic for fetching time series in chunks is implemented by MeasurementSet in /v3/models/measurement-set.js. We first fetch the cached primary cluster (the cluster that contains the newest data) at: /data/measurement-set-<platform-id>-<metric-id>.json If that's outdated according to lastModified in manifest.json, then we immediately re-fetch the primary cluster at: /api/measurement-set/?platform=<platform-id>&metric=<metric-id> Once the up-to-date primary cluster is fetched, we fetch all "secondary" clusters. For each cluster being fetched, including the primary, we invoke registered callbacks. In addition, the v3 UI reduces the initial page load time by loading a single bundled JS file generated by tools/bundle-v3-scripts.py. index.html has a fallback to load all 44 JS files individually during development. * public/api/analysis-tasks.php: (fetch_and_push_bugs_to_tasks): Added the code to fetch start and end run times. This is necessary in V3 UI because no longer fetch the entire time series. See r194120 for the new measurement set JSON API. (format_task): Compute the category of an analysis task based on "result" value. This will be re-vamped once I add the UI for the analysis task page in v3. * public/include/json-header.php: (require_format): CamelCase the name. (require_match_one_of_values): Ditto. (validate_arguments): Renamed from require_existence_of and used in measurement-set.php landed in r194120. * public/v3: Added. * public/v3/components: Added. * public/v3/components/base.js: Added. (ComponentBase): The base component class. (ComponentBase.prototype.element): Returns the DOM element associated with the DOM element. (ComponentBase.prototype.content): Returns the shadow root if one exists and the associated element otherwise. (ComponentBase.prototype.render): To be implemented by a subclass. (ComponentBase.prototype.renderReplace): A helper function to "render" DOM contents. (ComponentBase.prototype._constructShadowTree): Called inside the constructor to instantiate the templates. (ComponentBase.prototype._recursivelyReplaceUnknownElementsByComponents): Instantiates components referred by its element name inside the instantiated content. (ComponentBase.isElementInViewport): A helper function. Returns true if the element is in the viewport and it has non-zero width and height. (ComponentBase.defineElement): Defines a custom element that can be automatically instantiated from htmlTemplate. (ComponentBase.createElement): A helper function to create DOM tree to be used in "render()" method. (ComponentBase._addContentToElement): A helper for "createElement". (ComponentBase.createLink): A helper function to create a hyperlink or another clickable element (via callback). (ComponentBase.createActionHandler): A helper function to create an event listener that prevents the default action and stops the event propagation. * public/v3/components/button-base.js: Added. * public/v3/components/chart-status-view.js: Added. (ChartStatusView): A component that reports the current status of time-series-chart. It's subclasses by ChartPaneStatusView to provide additional information in the charts page's panes. * public/v3/components/close-button.js: Added. (CloseButton): * public/v3/components/commit-log-viewer.js: Added. (CommitLogViewer): A component that lists commit revisions along with commit messages for a range of data points. * public/v3/components/interactive-time-series-chart.js: Added. (InteractiveTimeSeriesChart): A subclass of InteractiveTimeSeriesChart with interactivity (selection & indicator). Selection and indicator are mutually exclusive. * public/v3/components/pane-selector.js: Added. (PaneSelector): A component for selecting (platform, metric) pair to add in the charts page. * public/v3/components/spinner-icon.js: Added. * public/v3/components/time-series-chart.js: Added. (TimeSeriesChart): A canvas-based chart component without interactivity. It takes a source list and options as the constructor arguments. A source list is a list of measurement sets (measurement-set.js) with drawing options. This component fetches data via MeasurementSet.fetchBetween inside TimeSeriesChart.prototype.setDomain and progressively updates the charts as more data arrives. The canvas is updated on animation frame via rAF and all layout and rendering metrics are lazily computed in _layout. In addition, this component samples data before rendering the chart when there are more data points per pixel in _ensureSampledTimeSeries. * public/v3/index.html: Added. Loads bundled-scripts.js if it exists, or individual script files otherwise. * public/v3/instrumentation.js: Added. This class is used to gather runtime statistics of v3 UI. (It measures the performance of the perf dashboard UI). * public/v3/main.js: Added. Bootstraps the app. (main): (fetchManifest): * public/v3/models: Added. * public/v3/models/analysis-task.js: Added. * public/v3/models/bug-tracker.js: Added. * public/v3/models/bug.js: Added. * public/v3/models/builder.js: Added. * public/v3/models/commit-log.js: Added. * public/v3/models/data-model.js: Added. (DataModelObject): The base class for various data objects that correspond to database tables. It supplies static hash map to find entries by id as well as other keys. (LabeledObject): A subclass of DataModelObject with the capability to find an object via its name. * public/v3/models/measurement-cluster.js: Added. (MeasurementCluster): Represents a single cluster or a chunk of data in a measurement set. * public/v3/models/measurement-set.js: Added. (MeasurementSet): Represents a measurement set. (MeasurementSet.findSet): Returns the singleton set given (metric, platform). We use singleton to avoid issuing multiple HTTP requests for the same JSON when there are multiple TimeSeriesChart that show the same graph (e.g. on charts page with overview and main charts). (MeasurementSet.prototype.findClusters): Finds the list of clusters to fetch in a given time range. (MeasurementSet.prototype.fetchBetween): Fetch clusters for a given time range and calls callback whenever new data arrives. The number of callbacks depends on the how many clusters need to be newly fetched. (MeasurementSet.prototype._fetchSecondaryClusters): Fetches non-primary (non-latest) clusters. (MeasurementSet.prototype._fetch): Issues a HTTP request to fetch a cluster. (MeasurementSet.prototype._didFetchJSON): Called when a cluster is fetched. (MeasurementSet.prototype._failedToFetchJSON): Called when the fetching of a cluster has failed. (MeasurementSet.prototype._invokeCallbacks): Invokes callbacks upon an approval of a new cluster. (MeasurementSet.prototype._addFetchedCluster): Adds the newly fetched cluster in the order. (MeasurementSet.prototype.fetchedTimeSeries): Returns a time series that contains data from all clusters that have been fetched. (TimeSeries.prototype.findById): Additions to TimeSeries defined in /v2/data.js. (TimeSeries.prototype.dataBetweenPoints): Ditto. (TimeSeries.prototype.firstPoint): Ditto. * public/v3/models/metric.js: Added. * public/v3/models/platform.js: Added. * public/v3/models/repository.js: Added. * public/v3/models/test.js: Added. * public/v3/pages: Added. * public/v3/pages/analysis-category-page.js: Added. The "Analysis" page that lists the analysis tasks. * public/v3/pages/analysis-category-toolbar.js: Added. The toolbar to filter analysis tasks based on its category (unconfirmed, bisecting, identified, closed) and a keyword. * public/v3/pages/analysis-task-page.js: Added. Not implemented yet. It just has the hyperlink to the v2 UI. * public/v3/pages/chart-pane-status-view.js: Added. (ChartPaneStatusView): A subclass of ChartStatusView used in the charts page. In addition to the current value, comparison to baseline/target, it shows the list of repository revisions (e.g. WebKit revision, OS version). * public/v3/pages/chart-pane.js: Added. (ChartPane): A component a pane in the charts page. Each pane has the overview chart and the main chart. The zooming is synced across all panes in the charts page. * public/v3/pages/charts-page.js: Added. Charts page. * public/v3/pages/charts-toolbar.js: Added. The toolbar to set the number of days to show. This affects the overview chart's domain in each pane. * public/v3/pages/create-analysis-task-page.js: Added. (CreateAnalysisTaskPage): A page that gets shown momentarily while creating a new analysis task. * public/v3/pages/dashboard-page.js: Added. A dashboard page. * public/v3/pages/dashboard-toolbar.js: Added. Its toolbar with buttons to select the number of days to show. * public/v3/pages/domain-control-toolbar.js: Added. An abstract superclass of charts and dashboard toolbars. * public/v3/pages/heading.js: Added. A component for displaying the header and toolbar, if exists, on each page. * public/v3/pages/page-router.js: Added. This class is responsible for updating the URL hashes as well as opening and updating each page when the hash changes (via back/forward navigation). * public/v3/pages/page-with-charts.js: Added. An abstract subclass of page used by dashboards and charts page. Supplies helper functions for creating TimeSeriesChart options. * public/v3/pages/page-with-heading.js: Added. An abstract subclass of page that uses the heading component. * public/v3/pages/page.js: Added. The Page component. * public/v3/pages/toolbar.js: Added. An abstract toolbar component. * public/v3/remote.js: Added. (getJSON): Fetches JSON from the remote server. (getJSONWithStatus): Ditto. Rejects the response if the status is not "OK". (PrivilegedAPI.sendRequest): Posts a HTTP request to a privileged API in /privileged-api/. (PrivilegedAPI.requestCSRFToken): Creates a new CSRF token to request a privileged API post. * tools/bundle-v3-scripts.py: Added. (main): Bundles js files together and minifies them by jsmin.py for the v3 UI. Without this script, we're forced to download 44 JS files or making each JS file contain multiple classes. * tools/jsmin.py: Copied from WebInspector / JavaScriptCore code. Canonical link: https://commits.webkit.org/170434@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@194130 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-12-16 05:19:08 +00:00
#!/usr/bin/python
# Copyright (C) 2013, 2015, 2016 Apple Inc. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
# THE POSSIBILITY OF SUCH DAMAGE.
Add v3 UI to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=152311 Reviewed by Chris Dumez. Add the third iteration of the perf dashboard UI. UI for viewing and modifying analysis tasks is coming soon. The v3 UI is focused on speed, and removes all third-party script dependencies including jQuery, d3, and Ember. Both the DOM-based UI and graphing are implemented manually. The entire app is structured using new component library implemented in components/base.js. Each component is an instance of a subclass of ComponentBase which owns a single DOM element. Each subclass may supply static methods named htmlTemplate and cssTemplate as the template for a component instance. ComponentBase automatically clones the templates inside the associated element (or its shadow root on the supported browsers). Each subclass must supply a method called "render()" which constructs and updates the DOM as needed. There is a special component called Page, which represents an entire page. Each Page is opened by PageRouter's "route()" function. Each subclass of Page supplies "open()" for initialization and "updateFromSerializedState()" for a hash URL transition. The key feature of the v3 UI is the split of time series into chunks called clusters (see r194120). On an internal instance of the dashboard, the v2 UI downloads 27MB of data whereas the same page loads only 3MB of data in the v3. The key logic for fetching time series in chunks is implemented by MeasurementSet in /v3/models/measurement-set.js. We first fetch the cached primary cluster (the cluster that contains the newest data) at: /data/measurement-set-<platform-id>-<metric-id>.json If that's outdated according to lastModified in manifest.json, then we immediately re-fetch the primary cluster at: /api/measurement-set/?platform=<platform-id>&metric=<metric-id> Once the up-to-date primary cluster is fetched, we fetch all "secondary" clusters. For each cluster being fetched, including the primary, we invoke registered callbacks. In addition, the v3 UI reduces the initial page load time by loading a single bundled JS file generated by tools/bundle-v3-scripts.py. index.html has a fallback to load all 44 JS files individually during development. * public/api/analysis-tasks.php: (fetch_and_push_bugs_to_tasks): Added the code to fetch start and end run times. This is necessary in V3 UI because no longer fetch the entire time series. See r194120 for the new measurement set JSON API. (format_task): Compute the category of an analysis task based on "result" value. This will be re-vamped once I add the UI for the analysis task page in v3. * public/include/json-header.php: (require_format): CamelCase the name. (require_match_one_of_values): Ditto. (validate_arguments): Renamed from require_existence_of and used in measurement-set.php landed in r194120. * public/v3: Added. * public/v3/components: Added. * public/v3/components/base.js: Added. (ComponentBase): The base component class. (ComponentBase.prototype.element): Returns the DOM element associated with the DOM element. (ComponentBase.prototype.content): Returns the shadow root if one exists and the associated element otherwise. (ComponentBase.prototype.render): To be implemented by a subclass. (ComponentBase.prototype.renderReplace): A helper function to "render" DOM contents. (ComponentBase.prototype._constructShadowTree): Called inside the constructor to instantiate the templates. (ComponentBase.prototype._recursivelyReplaceUnknownElementsByComponents): Instantiates components referred by its element name inside the instantiated content. (ComponentBase.isElementInViewport): A helper function. Returns true if the element is in the viewport and it has non-zero width and height. (ComponentBase.defineElement): Defines a custom element that can be automatically instantiated from htmlTemplate. (ComponentBase.createElement): A helper function to create DOM tree to be used in "render()" method. (ComponentBase._addContentToElement): A helper for "createElement". (ComponentBase.createLink): A helper function to create a hyperlink or another clickable element (via callback). (ComponentBase.createActionHandler): A helper function to create an event listener that prevents the default action and stops the event propagation. * public/v3/components/button-base.js: Added. * public/v3/components/chart-status-view.js: Added. (ChartStatusView): A component that reports the current status of time-series-chart. It's subclasses by ChartPaneStatusView to provide additional information in the charts page's panes. * public/v3/components/close-button.js: Added. (CloseButton): * public/v3/components/commit-log-viewer.js: Added. (CommitLogViewer): A component that lists commit revisions along with commit messages for a range of data points. * public/v3/components/interactive-time-series-chart.js: Added. (InteractiveTimeSeriesChart): A subclass of InteractiveTimeSeriesChart with interactivity (selection & indicator). Selection and indicator are mutually exclusive. * public/v3/components/pane-selector.js: Added. (PaneSelector): A component for selecting (platform, metric) pair to add in the charts page. * public/v3/components/spinner-icon.js: Added. * public/v3/components/time-series-chart.js: Added. (TimeSeriesChart): A canvas-based chart component without interactivity. It takes a source list and options as the constructor arguments. A source list is a list of measurement sets (measurement-set.js) with drawing options. This component fetches data via MeasurementSet.fetchBetween inside TimeSeriesChart.prototype.setDomain and progressively updates the charts as more data arrives. The canvas is updated on animation frame via rAF and all layout and rendering metrics are lazily computed in _layout. In addition, this component samples data before rendering the chart when there are more data points per pixel in _ensureSampledTimeSeries. * public/v3/index.html: Added. Loads bundled-scripts.js if it exists, or individual script files otherwise. * public/v3/instrumentation.js: Added. This class is used to gather runtime statistics of v3 UI. (It measures the performance of the perf dashboard UI). * public/v3/main.js: Added. Bootstraps the app. (main): (fetchManifest): * public/v3/models: Added. * public/v3/models/analysis-task.js: Added. * public/v3/models/bug-tracker.js: Added. * public/v3/models/bug.js: Added. * public/v3/models/builder.js: Added. * public/v3/models/commit-log.js: Added. * public/v3/models/data-model.js: Added. (DataModelObject): The base class for various data objects that correspond to database tables. It supplies static hash map to find entries by id as well as other keys. (LabeledObject): A subclass of DataModelObject with the capability to find an object via its name. * public/v3/models/measurement-cluster.js: Added. (MeasurementCluster): Represents a single cluster or a chunk of data in a measurement set. * public/v3/models/measurement-set.js: Added. (MeasurementSet): Represents a measurement set. (MeasurementSet.findSet): Returns the singleton set given (metric, platform). We use singleton to avoid issuing multiple HTTP requests for the same JSON when there are multiple TimeSeriesChart that show the same graph (e.g. on charts page with overview and main charts). (MeasurementSet.prototype.findClusters): Finds the list of clusters to fetch in a given time range. (MeasurementSet.prototype.fetchBetween): Fetch clusters for a given time range and calls callback whenever new data arrives. The number of callbacks depends on the how many clusters need to be newly fetched. (MeasurementSet.prototype._fetchSecondaryClusters): Fetches non-primary (non-latest) clusters. (MeasurementSet.prototype._fetch): Issues a HTTP request to fetch a cluster. (MeasurementSet.prototype._didFetchJSON): Called when a cluster is fetched. (MeasurementSet.prototype._failedToFetchJSON): Called when the fetching of a cluster has failed. (MeasurementSet.prototype._invokeCallbacks): Invokes callbacks upon an approval of a new cluster. (MeasurementSet.prototype._addFetchedCluster): Adds the newly fetched cluster in the order. (MeasurementSet.prototype.fetchedTimeSeries): Returns a time series that contains data from all clusters that have been fetched. (TimeSeries.prototype.findById): Additions to TimeSeries defined in /v2/data.js. (TimeSeries.prototype.dataBetweenPoints): Ditto. (TimeSeries.prototype.firstPoint): Ditto. * public/v3/models/metric.js: Added. * public/v3/models/platform.js: Added. * public/v3/models/repository.js: Added. * public/v3/models/test.js: Added. * public/v3/pages: Added. * public/v3/pages/analysis-category-page.js: Added. The "Analysis" page that lists the analysis tasks. * public/v3/pages/analysis-category-toolbar.js: Added. The toolbar to filter analysis tasks based on its category (unconfirmed, bisecting, identified, closed) and a keyword. * public/v3/pages/analysis-task-page.js: Added. Not implemented yet. It just has the hyperlink to the v2 UI. * public/v3/pages/chart-pane-status-view.js: Added. (ChartPaneStatusView): A subclass of ChartStatusView used in the charts page. In addition to the current value, comparison to baseline/target, it shows the list of repository revisions (e.g. WebKit revision, OS version). * public/v3/pages/chart-pane.js: Added. (ChartPane): A component a pane in the charts page. Each pane has the overview chart and the main chart. The zooming is synced across all panes in the charts page. * public/v3/pages/charts-page.js: Added. Charts page. * public/v3/pages/charts-toolbar.js: Added. The toolbar to set the number of days to show. This affects the overview chart's domain in each pane. * public/v3/pages/create-analysis-task-page.js: Added. (CreateAnalysisTaskPage): A page that gets shown momentarily while creating a new analysis task. * public/v3/pages/dashboard-page.js: Added. A dashboard page. * public/v3/pages/dashboard-toolbar.js: Added. Its toolbar with buttons to select the number of days to show. * public/v3/pages/domain-control-toolbar.js: Added. An abstract superclass of charts and dashboard toolbars. * public/v3/pages/heading.js: Added. A component for displaying the header and toolbar, if exists, on each page. * public/v3/pages/page-router.js: Added. This class is responsible for updating the URL hashes as well as opening and updating each page when the hash changes (via back/forward navigation). * public/v3/pages/page-with-charts.js: Added. An abstract subclass of page used by dashboards and charts page. Supplies helper functions for creating TimeSeriesChart options. * public/v3/pages/page-with-heading.js: Added. An abstract subclass of page that uses the heading component. * public/v3/pages/page.js: Added. The Page component. * public/v3/pages/toolbar.js: Added. An abstract toolbar component. * public/v3/remote.js: Added. (getJSON): Fetches JSON from the remote server. (getJSONWithStatus): Ditto. Rejects the response if the status is not "OK". (PrivilegedAPI.sendRequest): Posts a HTTP request to a privileged API in /privileged-api/. (PrivilegedAPI.requestCSRFToken): Creates a new CSRF token to request a privileged API post. * tools/bundle-v3-scripts.py: Added. (main): Bundles js files together and minifies them by jsmin.py for the v3 UI. Without this script, we're forced to download 44 JS files or making each JS file contain multiple classes. * tools/jsmin.py: Copied from WebInspector / JavaScriptCore code. Canonical link: https://commits.webkit.org/170434@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@194130 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-12-16 05:19:08 +00:00
import os
import re
Add v3 UI to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=152311 Reviewed by Chris Dumez. Add the third iteration of the perf dashboard UI. UI for viewing and modifying analysis tasks is coming soon. The v3 UI is focused on speed, and removes all third-party script dependencies including jQuery, d3, and Ember. Both the DOM-based UI and graphing are implemented manually. The entire app is structured using new component library implemented in components/base.js. Each component is an instance of a subclass of ComponentBase which owns a single DOM element. Each subclass may supply static methods named htmlTemplate and cssTemplate as the template for a component instance. ComponentBase automatically clones the templates inside the associated element (or its shadow root on the supported browsers). Each subclass must supply a method called "render()" which constructs and updates the DOM as needed. There is a special component called Page, which represents an entire page. Each Page is opened by PageRouter's "route()" function. Each subclass of Page supplies "open()" for initialization and "updateFromSerializedState()" for a hash URL transition. The key feature of the v3 UI is the split of time series into chunks called clusters (see r194120). On an internal instance of the dashboard, the v2 UI downloads 27MB of data whereas the same page loads only 3MB of data in the v3. The key logic for fetching time series in chunks is implemented by MeasurementSet in /v3/models/measurement-set.js. We first fetch the cached primary cluster (the cluster that contains the newest data) at: /data/measurement-set-<platform-id>-<metric-id>.json If that's outdated according to lastModified in manifest.json, then we immediately re-fetch the primary cluster at: /api/measurement-set/?platform=<platform-id>&metric=<metric-id> Once the up-to-date primary cluster is fetched, we fetch all "secondary" clusters. For each cluster being fetched, including the primary, we invoke registered callbacks. In addition, the v3 UI reduces the initial page load time by loading a single bundled JS file generated by tools/bundle-v3-scripts.py. index.html has a fallback to load all 44 JS files individually during development. * public/api/analysis-tasks.php: (fetch_and_push_bugs_to_tasks): Added the code to fetch start and end run times. This is necessary in V3 UI because no longer fetch the entire time series. See r194120 for the new measurement set JSON API. (format_task): Compute the category of an analysis task based on "result" value. This will be re-vamped once I add the UI for the analysis task page in v3. * public/include/json-header.php: (require_format): CamelCase the name. (require_match_one_of_values): Ditto. (validate_arguments): Renamed from require_existence_of and used in measurement-set.php landed in r194120. * public/v3: Added. * public/v3/components: Added. * public/v3/components/base.js: Added. (ComponentBase): The base component class. (ComponentBase.prototype.element): Returns the DOM element associated with the DOM element. (ComponentBase.prototype.content): Returns the shadow root if one exists and the associated element otherwise. (ComponentBase.prototype.render): To be implemented by a subclass. (ComponentBase.prototype.renderReplace): A helper function to "render" DOM contents. (ComponentBase.prototype._constructShadowTree): Called inside the constructor to instantiate the templates. (ComponentBase.prototype._recursivelyReplaceUnknownElementsByComponents): Instantiates components referred by its element name inside the instantiated content. (ComponentBase.isElementInViewport): A helper function. Returns true if the element is in the viewport and it has non-zero width and height. (ComponentBase.defineElement): Defines a custom element that can be automatically instantiated from htmlTemplate. (ComponentBase.createElement): A helper function to create DOM tree to be used in "render()" method. (ComponentBase._addContentToElement): A helper for "createElement". (ComponentBase.createLink): A helper function to create a hyperlink or another clickable element (via callback). (ComponentBase.createActionHandler): A helper function to create an event listener that prevents the default action and stops the event propagation. * public/v3/components/button-base.js: Added. * public/v3/components/chart-status-view.js: Added. (ChartStatusView): A component that reports the current status of time-series-chart. It's subclasses by ChartPaneStatusView to provide additional information in the charts page's panes. * public/v3/components/close-button.js: Added. (CloseButton): * public/v3/components/commit-log-viewer.js: Added. (CommitLogViewer): A component that lists commit revisions along with commit messages for a range of data points. * public/v3/components/interactive-time-series-chart.js: Added. (InteractiveTimeSeriesChart): A subclass of InteractiveTimeSeriesChart with interactivity (selection & indicator). Selection and indicator are mutually exclusive. * public/v3/components/pane-selector.js: Added. (PaneSelector): A component for selecting (platform, metric) pair to add in the charts page. * public/v3/components/spinner-icon.js: Added. * public/v3/components/time-series-chart.js: Added. (TimeSeriesChart): A canvas-based chart component without interactivity. It takes a source list and options as the constructor arguments. A source list is a list of measurement sets (measurement-set.js) with drawing options. This component fetches data via MeasurementSet.fetchBetween inside TimeSeriesChart.prototype.setDomain and progressively updates the charts as more data arrives. The canvas is updated on animation frame via rAF and all layout and rendering metrics are lazily computed in _layout. In addition, this component samples data before rendering the chart when there are more data points per pixel in _ensureSampledTimeSeries. * public/v3/index.html: Added. Loads bundled-scripts.js if it exists, or individual script files otherwise. * public/v3/instrumentation.js: Added. This class is used to gather runtime statistics of v3 UI. (It measures the performance of the perf dashboard UI). * public/v3/main.js: Added. Bootstraps the app. (main): (fetchManifest): * public/v3/models: Added. * public/v3/models/analysis-task.js: Added. * public/v3/models/bug-tracker.js: Added. * public/v3/models/bug.js: Added. * public/v3/models/builder.js: Added. * public/v3/models/commit-log.js: Added. * public/v3/models/data-model.js: Added. (DataModelObject): The base class for various data objects that correspond to database tables. It supplies static hash map to find entries by id as well as other keys. (LabeledObject): A subclass of DataModelObject with the capability to find an object via its name. * public/v3/models/measurement-cluster.js: Added. (MeasurementCluster): Represents a single cluster or a chunk of data in a measurement set. * public/v3/models/measurement-set.js: Added. (MeasurementSet): Represents a measurement set. (MeasurementSet.findSet): Returns the singleton set given (metric, platform). We use singleton to avoid issuing multiple HTTP requests for the same JSON when there are multiple TimeSeriesChart that show the same graph (e.g. on charts page with overview and main charts). (MeasurementSet.prototype.findClusters): Finds the list of clusters to fetch in a given time range. (MeasurementSet.prototype.fetchBetween): Fetch clusters for a given time range and calls callback whenever new data arrives. The number of callbacks depends on the how many clusters need to be newly fetched. (MeasurementSet.prototype._fetchSecondaryClusters): Fetches non-primary (non-latest) clusters. (MeasurementSet.prototype._fetch): Issues a HTTP request to fetch a cluster. (MeasurementSet.prototype._didFetchJSON): Called when a cluster is fetched. (MeasurementSet.prototype._failedToFetchJSON): Called when the fetching of a cluster has failed. (MeasurementSet.prototype._invokeCallbacks): Invokes callbacks upon an approval of a new cluster. (MeasurementSet.prototype._addFetchedCluster): Adds the newly fetched cluster in the order. (MeasurementSet.prototype.fetchedTimeSeries): Returns a time series that contains data from all clusters that have been fetched. (TimeSeries.prototype.findById): Additions to TimeSeries defined in /v2/data.js. (TimeSeries.prototype.dataBetweenPoints): Ditto. (TimeSeries.prototype.firstPoint): Ditto. * public/v3/models/metric.js: Added. * public/v3/models/platform.js: Added. * public/v3/models/repository.js: Added. * public/v3/models/test.js: Added. * public/v3/pages: Added. * public/v3/pages/analysis-category-page.js: Added. The "Analysis" page that lists the analysis tasks. * public/v3/pages/analysis-category-toolbar.js: Added. The toolbar to filter analysis tasks based on its category (unconfirmed, bisecting, identified, closed) and a keyword. * public/v3/pages/analysis-task-page.js: Added. Not implemented yet. It just has the hyperlink to the v2 UI. * public/v3/pages/chart-pane-status-view.js: Added. (ChartPaneStatusView): A subclass of ChartStatusView used in the charts page. In addition to the current value, comparison to baseline/target, it shows the list of repository revisions (e.g. WebKit revision, OS version). * public/v3/pages/chart-pane.js: Added. (ChartPane): A component a pane in the charts page. Each pane has the overview chart and the main chart. The zooming is synced across all panes in the charts page. * public/v3/pages/charts-page.js: Added. Charts page. * public/v3/pages/charts-toolbar.js: Added. The toolbar to set the number of days to show. This affects the overview chart's domain in each pane. * public/v3/pages/create-analysis-task-page.js: Added. (CreateAnalysisTaskPage): A page that gets shown momentarily while creating a new analysis task. * public/v3/pages/dashboard-page.js: Added. A dashboard page. * public/v3/pages/dashboard-toolbar.js: Added. Its toolbar with buttons to select the number of days to show. * public/v3/pages/domain-control-toolbar.js: Added. An abstract superclass of charts and dashboard toolbars. * public/v3/pages/heading.js: Added. A component for displaying the header and toolbar, if exists, on each page. * public/v3/pages/page-router.js: Added. This class is responsible for updating the URL hashes as well as opening and updating each page when the hash changes (via back/forward navigation). * public/v3/pages/page-with-charts.js: Added. An abstract subclass of page used by dashboards and charts page. Supplies helper functions for creating TimeSeriesChart options. * public/v3/pages/page-with-heading.js: Added. An abstract subclass of page that uses the heading component. * public/v3/pages/page.js: Added. The Page component. * public/v3/pages/toolbar.js: Added. An abstract toolbar component. * public/v3/remote.js: Added. (getJSON): Fetches JSON from the remote server. (getJSONWithStatus): Ditto. Rejects the response if the status is not "OK". (PrivilegedAPI.sendRequest): Posts a HTTP request to a privileged API in /privileged-api/. (PrivilegedAPI.requestCSRFToken): Creates a new CSRF token to request a privileged API post. * tools/bundle-v3-scripts.py: Added. (main): Bundles js files together and minifies them by jsmin.py for the v3 UI. Without this script, we're forced to download 44 JS files or making each JS file contain multiple classes. * tools/jsmin.py: Copied from WebInspector / JavaScriptCore code. Canonical link: https://commits.webkit.org/170434@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@194130 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-12-16 05:19:08 +00:00
import subprocess
import sys
import xml.dom.minidom
def main(argv):
tools_dir = os.path.dirname(__file__)
public_v3_dir = os.path.abspath(os.path.join(tools_dir, '..', 'public', 'v3'))
bundled_script = ''
index_html = xml.dom.minidom.parse(os.path.join(public_v3_dir, 'index.html'))
for template in index_html.getElementsByTagName('template'):
if template.getAttribute('id') != 'unbundled-scripts':
continue
unbundled_scripts = template.getElementsByTagName('script')
for script in unbundled_scripts:
src = script.getAttribute('src')
Add v3 UI to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=152311 Reviewed by Chris Dumez. Add the third iteration of the perf dashboard UI. UI for viewing and modifying analysis tasks is coming soon. The v3 UI is focused on speed, and removes all third-party script dependencies including jQuery, d3, and Ember. Both the DOM-based UI and graphing are implemented manually. The entire app is structured using new component library implemented in components/base.js. Each component is an instance of a subclass of ComponentBase which owns a single DOM element. Each subclass may supply static methods named htmlTemplate and cssTemplate as the template for a component instance. ComponentBase automatically clones the templates inside the associated element (or its shadow root on the supported browsers). Each subclass must supply a method called "render()" which constructs and updates the DOM as needed. There is a special component called Page, which represents an entire page. Each Page is opened by PageRouter's "route()" function. Each subclass of Page supplies "open()" for initialization and "updateFromSerializedState()" for a hash URL transition. The key feature of the v3 UI is the split of time series into chunks called clusters (see r194120). On an internal instance of the dashboard, the v2 UI downloads 27MB of data whereas the same page loads only 3MB of data in the v3. The key logic for fetching time series in chunks is implemented by MeasurementSet in /v3/models/measurement-set.js. We first fetch the cached primary cluster (the cluster that contains the newest data) at: /data/measurement-set-<platform-id>-<metric-id>.json If that's outdated according to lastModified in manifest.json, then we immediately re-fetch the primary cluster at: /api/measurement-set/?platform=<platform-id>&metric=<metric-id> Once the up-to-date primary cluster is fetched, we fetch all "secondary" clusters. For each cluster being fetched, including the primary, we invoke registered callbacks. In addition, the v3 UI reduces the initial page load time by loading a single bundled JS file generated by tools/bundle-v3-scripts.py. index.html has a fallback to load all 44 JS files individually during development. * public/api/analysis-tasks.php: (fetch_and_push_bugs_to_tasks): Added the code to fetch start and end run times. This is necessary in V3 UI because no longer fetch the entire time series. See r194120 for the new measurement set JSON API. (format_task): Compute the category of an analysis task based on "result" value. This will be re-vamped once I add the UI for the analysis task page in v3. * public/include/json-header.php: (require_format): CamelCase the name. (require_match_one_of_values): Ditto. (validate_arguments): Renamed from require_existence_of and used in measurement-set.php landed in r194120. * public/v3: Added. * public/v3/components: Added. * public/v3/components/base.js: Added. (ComponentBase): The base component class. (ComponentBase.prototype.element): Returns the DOM element associated with the DOM element. (ComponentBase.prototype.content): Returns the shadow root if one exists and the associated element otherwise. (ComponentBase.prototype.render): To be implemented by a subclass. (ComponentBase.prototype.renderReplace): A helper function to "render" DOM contents. (ComponentBase.prototype._constructShadowTree): Called inside the constructor to instantiate the templates. (ComponentBase.prototype._recursivelyReplaceUnknownElementsByComponents): Instantiates components referred by its element name inside the instantiated content. (ComponentBase.isElementInViewport): A helper function. Returns true if the element is in the viewport and it has non-zero width and height. (ComponentBase.defineElement): Defines a custom element that can be automatically instantiated from htmlTemplate. (ComponentBase.createElement): A helper function to create DOM tree to be used in "render()" method. (ComponentBase._addContentToElement): A helper for "createElement". (ComponentBase.createLink): A helper function to create a hyperlink or another clickable element (via callback). (ComponentBase.createActionHandler): A helper function to create an event listener that prevents the default action and stops the event propagation. * public/v3/components/button-base.js: Added. * public/v3/components/chart-status-view.js: Added. (ChartStatusView): A component that reports the current status of time-series-chart. It's subclasses by ChartPaneStatusView to provide additional information in the charts page's panes. * public/v3/components/close-button.js: Added. (CloseButton): * public/v3/components/commit-log-viewer.js: Added. (CommitLogViewer): A component that lists commit revisions along with commit messages for a range of data points. * public/v3/components/interactive-time-series-chart.js: Added. (InteractiveTimeSeriesChart): A subclass of InteractiveTimeSeriesChart with interactivity (selection & indicator). Selection and indicator are mutually exclusive. * public/v3/components/pane-selector.js: Added. (PaneSelector): A component for selecting (platform, metric) pair to add in the charts page. * public/v3/components/spinner-icon.js: Added. * public/v3/components/time-series-chart.js: Added. (TimeSeriesChart): A canvas-based chart component without interactivity. It takes a source list and options as the constructor arguments. A source list is a list of measurement sets (measurement-set.js) with drawing options. This component fetches data via MeasurementSet.fetchBetween inside TimeSeriesChart.prototype.setDomain and progressively updates the charts as more data arrives. The canvas is updated on animation frame via rAF and all layout and rendering metrics are lazily computed in _layout. In addition, this component samples data before rendering the chart when there are more data points per pixel in _ensureSampledTimeSeries. * public/v3/index.html: Added. Loads bundled-scripts.js if it exists, or individual script files otherwise. * public/v3/instrumentation.js: Added. This class is used to gather runtime statistics of v3 UI. (It measures the performance of the perf dashboard UI). * public/v3/main.js: Added. Bootstraps the app. (main): (fetchManifest): * public/v3/models: Added. * public/v3/models/analysis-task.js: Added. * public/v3/models/bug-tracker.js: Added. * public/v3/models/bug.js: Added. * public/v3/models/builder.js: Added. * public/v3/models/commit-log.js: Added. * public/v3/models/data-model.js: Added. (DataModelObject): The base class for various data objects that correspond to database tables. It supplies static hash map to find entries by id as well as other keys. (LabeledObject): A subclass of DataModelObject with the capability to find an object via its name. * public/v3/models/measurement-cluster.js: Added. (MeasurementCluster): Represents a single cluster or a chunk of data in a measurement set. * public/v3/models/measurement-set.js: Added. (MeasurementSet): Represents a measurement set. (MeasurementSet.findSet): Returns the singleton set given (metric, platform). We use singleton to avoid issuing multiple HTTP requests for the same JSON when there are multiple TimeSeriesChart that show the same graph (e.g. on charts page with overview and main charts). (MeasurementSet.prototype.findClusters): Finds the list of clusters to fetch in a given time range. (MeasurementSet.prototype.fetchBetween): Fetch clusters for a given time range and calls callback whenever new data arrives. The number of callbacks depends on the how many clusters need to be newly fetched. (MeasurementSet.prototype._fetchSecondaryClusters): Fetches non-primary (non-latest) clusters. (MeasurementSet.prototype._fetch): Issues a HTTP request to fetch a cluster. (MeasurementSet.prototype._didFetchJSON): Called when a cluster is fetched. (MeasurementSet.prototype._failedToFetchJSON): Called when the fetching of a cluster has failed. (MeasurementSet.prototype._invokeCallbacks): Invokes callbacks upon an approval of a new cluster. (MeasurementSet.prototype._addFetchedCluster): Adds the newly fetched cluster in the order. (MeasurementSet.prototype.fetchedTimeSeries): Returns a time series that contains data from all clusters that have been fetched. (TimeSeries.prototype.findById): Additions to TimeSeries defined in /v2/data.js. (TimeSeries.prototype.dataBetweenPoints): Ditto. (TimeSeries.prototype.firstPoint): Ditto. * public/v3/models/metric.js: Added. * public/v3/models/platform.js: Added. * public/v3/models/repository.js: Added. * public/v3/models/test.js: Added. * public/v3/pages: Added. * public/v3/pages/analysis-category-page.js: Added. The "Analysis" page that lists the analysis tasks. * public/v3/pages/analysis-category-toolbar.js: Added. The toolbar to filter analysis tasks based on its category (unconfirmed, bisecting, identified, closed) and a keyword. * public/v3/pages/analysis-task-page.js: Added. Not implemented yet. It just has the hyperlink to the v2 UI. * public/v3/pages/chart-pane-status-view.js: Added. (ChartPaneStatusView): A subclass of ChartStatusView used in the charts page. In addition to the current value, comparison to baseline/target, it shows the list of repository revisions (e.g. WebKit revision, OS version). * public/v3/pages/chart-pane.js: Added. (ChartPane): A component a pane in the charts page. Each pane has the overview chart and the main chart. The zooming is synced across all panes in the charts page. * public/v3/pages/charts-page.js: Added. Charts page. * public/v3/pages/charts-toolbar.js: Added. The toolbar to set the number of days to show. This affects the overview chart's domain in each pane. * public/v3/pages/create-analysis-task-page.js: Added. (CreateAnalysisTaskPage): A page that gets shown momentarily while creating a new analysis task. * public/v3/pages/dashboard-page.js: Added. A dashboard page. * public/v3/pages/dashboard-toolbar.js: Added. Its toolbar with buttons to select the number of days to show. * public/v3/pages/domain-control-toolbar.js: Added. An abstract superclass of charts and dashboard toolbars. * public/v3/pages/heading.js: Added. A component for displaying the header and toolbar, if exists, on each page. * public/v3/pages/page-router.js: Added. This class is responsible for updating the URL hashes as well as opening and updating each page when the hash changes (via back/forward navigation). * public/v3/pages/page-with-charts.js: Added. An abstract subclass of page used by dashboards and charts page. Supplies helper functions for creating TimeSeriesChart options. * public/v3/pages/page-with-heading.js: Added. An abstract subclass of page that uses the heading component. * public/v3/pages/page.js: Added. The Page component. * public/v3/pages/toolbar.js: Added. An abstract toolbar component. * public/v3/remote.js: Added. (getJSON): Fetches JSON from the remote server. (getJSONWithStatus): Ditto. Rejects the response if the status is not "OK". (PrivilegedAPI.sendRequest): Posts a HTTP request to a privileged API in /privileged-api/. (PrivilegedAPI.requestCSRFToken): Creates a new CSRF token to request a privileged API post. * tools/bundle-v3-scripts.py: Added. (main): Bundles js files together and minifies them by jsmin.py for the v3 UI. Without this script, we're forced to download 44 JS files or making each JS file contain multiple classes. * tools/jsmin.py: Copied from WebInspector / JavaScriptCore code. Canonical link: https://commits.webkit.org/170434@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@194130 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-12-16 05:19:08 +00:00
with open(os.path.join(public_v3_dir, src)) as script_file:
script_content = script_file.read()
script_content = re.sub(r'([\"\'])use strict\1;', '', script_content)
script_content = re.sub(r'(?P<prefix>\n\s*static\s+(?P<type>\w+)Template\(\)\s*{[^`]*`)(?P<content>[^`]*)(?P<suffix>`;?\s*})',
compress_template, script_content, re.MULTILINE)
bundled_script += script_content
Add v3 UI to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=152311 Reviewed by Chris Dumez. Add the third iteration of the perf dashboard UI. UI for viewing and modifying analysis tasks is coming soon. The v3 UI is focused on speed, and removes all third-party script dependencies including jQuery, d3, and Ember. Both the DOM-based UI and graphing are implemented manually. The entire app is structured using new component library implemented in components/base.js. Each component is an instance of a subclass of ComponentBase which owns a single DOM element. Each subclass may supply static methods named htmlTemplate and cssTemplate as the template for a component instance. ComponentBase automatically clones the templates inside the associated element (or its shadow root on the supported browsers). Each subclass must supply a method called "render()" which constructs and updates the DOM as needed. There is a special component called Page, which represents an entire page. Each Page is opened by PageRouter's "route()" function. Each subclass of Page supplies "open()" for initialization and "updateFromSerializedState()" for a hash URL transition. The key feature of the v3 UI is the split of time series into chunks called clusters (see r194120). On an internal instance of the dashboard, the v2 UI downloads 27MB of data whereas the same page loads only 3MB of data in the v3. The key logic for fetching time series in chunks is implemented by MeasurementSet in /v3/models/measurement-set.js. We first fetch the cached primary cluster (the cluster that contains the newest data) at: /data/measurement-set-<platform-id>-<metric-id>.json If that's outdated according to lastModified in manifest.json, then we immediately re-fetch the primary cluster at: /api/measurement-set/?platform=<platform-id>&metric=<metric-id> Once the up-to-date primary cluster is fetched, we fetch all "secondary" clusters. For each cluster being fetched, including the primary, we invoke registered callbacks. In addition, the v3 UI reduces the initial page load time by loading a single bundled JS file generated by tools/bundle-v3-scripts.py. index.html has a fallback to load all 44 JS files individually during development. * public/api/analysis-tasks.php: (fetch_and_push_bugs_to_tasks): Added the code to fetch start and end run times. This is necessary in V3 UI because no longer fetch the entire time series. See r194120 for the new measurement set JSON API. (format_task): Compute the category of an analysis task based on "result" value. This will be re-vamped once I add the UI for the analysis task page in v3. * public/include/json-header.php: (require_format): CamelCase the name. (require_match_one_of_values): Ditto. (validate_arguments): Renamed from require_existence_of and used in measurement-set.php landed in r194120. * public/v3: Added. * public/v3/components: Added. * public/v3/components/base.js: Added. (ComponentBase): The base component class. (ComponentBase.prototype.element): Returns the DOM element associated with the DOM element. (ComponentBase.prototype.content): Returns the shadow root if one exists and the associated element otherwise. (ComponentBase.prototype.render): To be implemented by a subclass. (ComponentBase.prototype.renderReplace): A helper function to "render" DOM contents. (ComponentBase.prototype._constructShadowTree): Called inside the constructor to instantiate the templates. (ComponentBase.prototype._recursivelyReplaceUnknownElementsByComponents): Instantiates components referred by its element name inside the instantiated content. (ComponentBase.isElementInViewport): A helper function. Returns true if the element is in the viewport and it has non-zero width and height. (ComponentBase.defineElement): Defines a custom element that can be automatically instantiated from htmlTemplate. (ComponentBase.createElement): A helper function to create DOM tree to be used in "render()" method. (ComponentBase._addContentToElement): A helper for "createElement". (ComponentBase.createLink): A helper function to create a hyperlink or another clickable element (via callback). (ComponentBase.createActionHandler): A helper function to create an event listener that prevents the default action and stops the event propagation. * public/v3/components/button-base.js: Added. * public/v3/components/chart-status-view.js: Added. (ChartStatusView): A component that reports the current status of time-series-chart. It's subclasses by ChartPaneStatusView to provide additional information in the charts page's panes. * public/v3/components/close-button.js: Added. (CloseButton): * public/v3/components/commit-log-viewer.js: Added. (CommitLogViewer): A component that lists commit revisions along with commit messages for a range of data points. * public/v3/components/interactive-time-series-chart.js: Added. (InteractiveTimeSeriesChart): A subclass of InteractiveTimeSeriesChart with interactivity (selection & indicator). Selection and indicator are mutually exclusive. * public/v3/components/pane-selector.js: Added. (PaneSelector): A component for selecting (platform, metric) pair to add in the charts page. * public/v3/components/spinner-icon.js: Added. * public/v3/components/time-series-chart.js: Added. (TimeSeriesChart): A canvas-based chart component without interactivity. It takes a source list and options as the constructor arguments. A source list is a list of measurement sets (measurement-set.js) with drawing options. This component fetches data via MeasurementSet.fetchBetween inside TimeSeriesChart.prototype.setDomain and progressively updates the charts as more data arrives. The canvas is updated on animation frame via rAF and all layout and rendering metrics are lazily computed in _layout. In addition, this component samples data before rendering the chart when there are more data points per pixel in _ensureSampledTimeSeries. * public/v3/index.html: Added. Loads bundled-scripts.js if it exists, or individual script files otherwise. * public/v3/instrumentation.js: Added. This class is used to gather runtime statistics of v3 UI. (It measures the performance of the perf dashboard UI). * public/v3/main.js: Added. Bootstraps the app. (main): (fetchManifest): * public/v3/models: Added. * public/v3/models/analysis-task.js: Added. * public/v3/models/bug-tracker.js: Added. * public/v3/models/bug.js: Added. * public/v3/models/builder.js: Added. * public/v3/models/commit-log.js: Added. * public/v3/models/data-model.js: Added. (DataModelObject): The base class for various data objects that correspond to database tables. It supplies static hash map to find entries by id as well as other keys. (LabeledObject): A subclass of DataModelObject with the capability to find an object via its name. * public/v3/models/measurement-cluster.js: Added. (MeasurementCluster): Represents a single cluster or a chunk of data in a measurement set. * public/v3/models/measurement-set.js: Added. (MeasurementSet): Represents a measurement set. (MeasurementSet.findSet): Returns the singleton set given (metric, platform). We use singleton to avoid issuing multiple HTTP requests for the same JSON when there are multiple TimeSeriesChart that show the same graph (e.g. on charts page with overview and main charts). (MeasurementSet.prototype.findClusters): Finds the list of clusters to fetch in a given time range. (MeasurementSet.prototype.fetchBetween): Fetch clusters for a given time range and calls callback whenever new data arrives. The number of callbacks depends on the how many clusters need to be newly fetched. (MeasurementSet.prototype._fetchSecondaryClusters): Fetches non-primary (non-latest) clusters. (MeasurementSet.prototype._fetch): Issues a HTTP request to fetch a cluster. (MeasurementSet.prototype._didFetchJSON): Called when a cluster is fetched. (MeasurementSet.prototype._failedToFetchJSON): Called when the fetching of a cluster has failed. (MeasurementSet.prototype._invokeCallbacks): Invokes callbacks upon an approval of a new cluster. (MeasurementSet.prototype._addFetchedCluster): Adds the newly fetched cluster in the order. (MeasurementSet.prototype.fetchedTimeSeries): Returns a time series that contains data from all clusters that have been fetched. (TimeSeries.prototype.findById): Additions to TimeSeries defined in /v2/data.js. (TimeSeries.prototype.dataBetweenPoints): Ditto. (TimeSeries.prototype.firstPoint): Ditto. * public/v3/models/metric.js: Added. * public/v3/models/platform.js: Added. * public/v3/models/repository.js: Added. * public/v3/models/test.js: Added. * public/v3/pages: Added. * public/v3/pages/analysis-category-page.js: Added. The "Analysis" page that lists the analysis tasks. * public/v3/pages/analysis-category-toolbar.js: Added. The toolbar to filter analysis tasks based on its category (unconfirmed, bisecting, identified, closed) and a keyword. * public/v3/pages/analysis-task-page.js: Added. Not implemented yet. It just has the hyperlink to the v2 UI. * public/v3/pages/chart-pane-status-view.js: Added. (ChartPaneStatusView): A subclass of ChartStatusView used in the charts page. In addition to the current value, comparison to baseline/target, it shows the list of repository revisions (e.g. WebKit revision, OS version). * public/v3/pages/chart-pane.js: Added. (ChartPane): A component a pane in the charts page. Each pane has the overview chart and the main chart. The zooming is synced across all panes in the charts page. * public/v3/pages/charts-page.js: Added. Charts page. * public/v3/pages/charts-toolbar.js: Added. The toolbar to set the number of days to show. This affects the overview chart's domain in each pane. * public/v3/pages/create-analysis-task-page.js: Added. (CreateAnalysisTaskPage): A page that gets shown momentarily while creating a new analysis task. * public/v3/pages/dashboard-page.js: Added. A dashboard page. * public/v3/pages/dashboard-toolbar.js: Added. Its toolbar with buttons to select the number of days to show. * public/v3/pages/domain-control-toolbar.js: Added. An abstract superclass of charts and dashboard toolbars. * public/v3/pages/heading.js: Added. A component for displaying the header and toolbar, if exists, on each page. * public/v3/pages/page-router.js: Added. This class is responsible for updating the URL hashes as well as opening and updating each page when the hash changes (via back/forward navigation). * public/v3/pages/page-with-charts.js: Added. An abstract subclass of page used by dashboards and charts page. Supplies helper functions for creating TimeSeriesChart options. * public/v3/pages/page-with-heading.js: Added. An abstract subclass of page that uses the heading component. * public/v3/pages/page.js: Added. The Page component. * public/v3/pages/toolbar.js: Added. An abstract toolbar component. * public/v3/remote.js: Added. (getJSON): Fetches JSON from the remote server. (getJSONWithStatus): Ditto. Rejects the response if the status is not "OK". (PrivilegedAPI.sendRequest): Posts a HTTP request to a privileged API in /privileged-api/. (PrivilegedAPI.requestCSRFToken): Creates a new CSRF token to request a privileged API post. * tools/bundle-v3-scripts.py: Added. (main): Bundles js files together and minifies them by jsmin.py for the v3 UI. Without this script, we're forced to download 44 JS files or making each JS file contain multiple classes. * tools/jsmin.py: Copied from WebInspector / JavaScriptCore code. Canonical link: https://commits.webkit.org/170434@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@194130 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-12-16 05:19:08 +00:00
jsmin = subprocess.Popen(['python', os.path.join(tools_dir, 'jsmin.py')], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
minified_script = jsmin.communicate(input=bundled_script)[0]
new_size = float(len(minified_script))
old_size = float(len(bundled_script))
print '%d -> %d (%.1f%%)' % (old_size, new_size, new_size / old_size * 100)
with open(os.path.join(public_v3_dir, 'bundled-scripts.js'), 'w') as bundled_file:
bundled_file.write(minified_script)
def compress_template(match):
prefix = match.group('prefix')
content = match.group('content')
suffix = match.group('suffix')
if match.group('type') == 'css':
content = cssminify(content)
else:
content = re.sub(r'\s+', ' ', content)
return prefix + content + suffix
def cssminify(css):
rules = (
(r"\/\*.*?\*\/", ""), # delete comments
(r"\n", ""), # delete new lines
(r"\s+", " "), # change multiple spaces to one space
(r"\s?([;{},~>!])\s?", r"\1"), # delete space where it is not needed
(r":\s", ":"), # delete spaces after colons, but not before. E.g. do not break selectors "a :focus", "b :matches(...)", "c :not(...)" where the leading space is significant
(r"\s?([-+])(?:\s(?![0-9(])(?!var))", r"\1"), # delete whitespace around + and - when not followed by a number, paren, or var(). E.g. strip for selector "a + b" but not "calc(a + b)" which requires spaces.
(r";}", "}") # change ';}' to '}' because the semicolon is not needed
)
css = css.replace("\r\n", "\n")
for rule in rules:
css = re.compile(rule[0], re.MULTILINE | re.UNICODE | re.DOTALL).sub(rule[1], css)
return css
Add v3 UI to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=152311 Reviewed by Chris Dumez. Add the third iteration of the perf dashboard UI. UI for viewing and modifying analysis tasks is coming soon. The v3 UI is focused on speed, and removes all third-party script dependencies including jQuery, d3, and Ember. Both the DOM-based UI and graphing are implemented manually. The entire app is structured using new component library implemented in components/base.js. Each component is an instance of a subclass of ComponentBase which owns a single DOM element. Each subclass may supply static methods named htmlTemplate and cssTemplate as the template for a component instance. ComponentBase automatically clones the templates inside the associated element (or its shadow root on the supported browsers). Each subclass must supply a method called "render()" which constructs and updates the DOM as needed. There is a special component called Page, which represents an entire page. Each Page is opened by PageRouter's "route()" function. Each subclass of Page supplies "open()" for initialization and "updateFromSerializedState()" for a hash URL transition. The key feature of the v3 UI is the split of time series into chunks called clusters (see r194120). On an internal instance of the dashboard, the v2 UI downloads 27MB of data whereas the same page loads only 3MB of data in the v3. The key logic for fetching time series in chunks is implemented by MeasurementSet in /v3/models/measurement-set.js. We first fetch the cached primary cluster (the cluster that contains the newest data) at: /data/measurement-set-<platform-id>-<metric-id>.json If that's outdated according to lastModified in manifest.json, then we immediately re-fetch the primary cluster at: /api/measurement-set/?platform=<platform-id>&metric=<metric-id> Once the up-to-date primary cluster is fetched, we fetch all "secondary" clusters. For each cluster being fetched, including the primary, we invoke registered callbacks. In addition, the v3 UI reduces the initial page load time by loading a single bundled JS file generated by tools/bundle-v3-scripts.py. index.html has a fallback to load all 44 JS files individually during development. * public/api/analysis-tasks.php: (fetch_and_push_bugs_to_tasks): Added the code to fetch start and end run times. This is necessary in V3 UI because no longer fetch the entire time series. See r194120 for the new measurement set JSON API. (format_task): Compute the category of an analysis task based on "result" value. This will be re-vamped once I add the UI for the analysis task page in v3. * public/include/json-header.php: (require_format): CamelCase the name. (require_match_one_of_values): Ditto. (validate_arguments): Renamed from require_existence_of and used in measurement-set.php landed in r194120. * public/v3: Added. * public/v3/components: Added. * public/v3/components/base.js: Added. (ComponentBase): The base component class. (ComponentBase.prototype.element): Returns the DOM element associated with the DOM element. (ComponentBase.prototype.content): Returns the shadow root if one exists and the associated element otherwise. (ComponentBase.prototype.render): To be implemented by a subclass. (ComponentBase.prototype.renderReplace): A helper function to "render" DOM contents. (ComponentBase.prototype._constructShadowTree): Called inside the constructor to instantiate the templates. (ComponentBase.prototype._recursivelyReplaceUnknownElementsByComponents): Instantiates components referred by its element name inside the instantiated content. (ComponentBase.isElementInViewport): A helper function. Returns true if the element is in the viewport and it has non-zero width and height. (ComponentBase.defineElement): Defines a custom element that can be automatically instantiated from htmlTemplate. (ComponentBase.createElement): A helper function to create DOM tree to be used in "render()" method. (ComponentBase._addContentToElement): A helper for "createElement". (ComponentBase.createLink): A helper function to create a hyperlink or another clickable element (via callback). (ComponentBase.createActionHandler): A helper function to create an event listener that prevents the default action and stops the event propagation. * public/v3/components/button-base.js: Added. * public/v3/components/chart-status-view.js: Added. (ChartStatusView): A component that reports the current status of time-series-chart. It's subclasses by ChartPaneStatusView to provide additional information in the charts page's panes. * public/v3/components/close-button.js: Added. (CloseButton): * public/v3/components/commit-log-viewer.js: Added. (CommitLogViewer): A component that lists commit revisions along with commit messages for a range of data points. * public/v3/components/interactive-time-series-chart.js: Added. (InteractiveTimeSeriesChart): A subclass of InteractiveTimeSeriesChart with interactivity (selection & indicator). Selection and indicator are mutually exclusive. * public/v3/components/pane-selector.js: Added. (PaneSelector): A component for selecting (platform, metric) pair to add in the charts page. * public/v3/components/spinner-icon.js: Added. * public/v3/components/time-series-chart.js: Added. (TimeSeriesChart): A canvas-based chart component without interactivity. It takes a source list and options as the constructor arguments. A source list is a list of measurement sets (measurement-set.js) with drawing options. This component fetches data via MeasurementSet.fetchBetween inside TimeSeriesChart.prototype.setDomain and progressively updates the charts as more data arrives. The canvas is updated on animation frame via rAF and all layout and rendering metrics are lazily computed in _layout. In addition, this component samples data before rendering the chart when there are more data points per pixel in _ensureSampledTimeSeries. * public/v3/index.html: Added. Loads bundled-scripts.js if it exists, or individual script files otherwise. * public/v3/instrumentation.js: Added. This class is used to gather runtime statistics of v3 UI. (It measures the performance of the perf dashboard UI). * public/v3/main.js: Added. Bootstraps the app. (main): (fetchManifest): * public/v3/models: Added. * public/v3/models/analysis-task.js: Added. * public/v3/models/bug-tracker.js: Added. * public/v3/models/bug.js: Added. * public/v3/models/builder.js: Added. * public/v3/models/commit-log.js: Added. * public/v3/models/data-model.js: Added. (DataModelObject): The base class for various data objects that correspond to database tables. It supplies static hash map to find entries by id as well as other keys. (LabeledObject): A subclass of DataModelObject with the capability to find an object via its name. * public/v3/models/measurement-cluster.js: Added. (MeasurementCluster): Represents a single cluster or a chunk of data in a measurement set. * public/v3/models/measurement-set.js: Added. (MeasurementSet): Represents a measurement set. (MeasurementSet.findSet): Returns the singleton set given (metric, platform). We use singleton to avoid issuing multiple HTTP requests for the same JSON when there are multiple TimeSeriesChart that show the same graph (e.g. on charts page with overview and main charts). (MeasurementSet.prototype.findClusters): Finds the list of clusters to fetch in a given time range. (MeasurementSet.prototype.fetchBetween): Fetch clusters for a given time range and calls callback whenever new data arrives. The number of callbacks depends on the how many clusters need to be newly fetched. (MeasurementSet.prototype._fetchSecondaryClusters): Fetches non-primary (non-latest) clusters. (MeasurementSet.prototype._fetch): Issues a HTTP request to fetch a cluster. (MeasurementSet.prototype._didFetchJSON): Called when a cluster is fetched. (MeasurementSet.prototype._failedToFetchJSON): Called when the fetching of a cluster has failed. (MeasurementSet.prototype._invokeCallbacks): Invokes callbacks upon an approval of a new cluster. (MeasurementSet.prototype._addFetchedCluster): Adds the newly fetched cluster in the order. (MeasurementSet.prototype.fetchedTimeSeries): Returns a time series that contains data from all clusters that have been fetched. (TimeSeries.prototype.findById): Additions to TimeSeries defined in /v2/data.js. (TimeSeries.prototype.dataBetweenPoints): Ditto. (TimeSeries.prototype.firstPoint): Ditto. * public/v3/models/metric.js: Added. * public/v3/models/platform.js: Added. * public/v3/models/repository.js: Added. * public/v3/models/test.js: Added. * public/v3/pages: Added. * public/v3/pages/analysis-category-page.js: Added. The "Analysis" page that lists the analysis tasks. * public/v3/pages/analysis-category-toolbar.js: Added. The toolbar to filter analysis tasks based on its category (unconfirmed, bisecting, identified, closed) and a keyword. * public/v3/pages/analysis-task-page.js: Added. Not implemented yet. It just has the hyperlink to the v2 UI. * public/v3/pages/chart-pane-status-view.js: Added. (ChartPaneStatusView): A subclass of ChartStatusView used in the charts page. In addition to the current value, comparison to baseline/target, it shows the list of repository revisions (e.g. WebKit revision, OS version). * public/v3/pages/chart-pane.js: Added. (ChartPane): A component a pane in the charts page. Each pane has the overview chart and the main chart. The zooming is synced across all panes in the charts page. * public/v3/pages/charts-page.js: Added. Charts page. * public/v3/pages/charts-toolbar.js: Added. The toolbar to set the number of days to show. This affects the overview chart's domain in each pane. * public/v3/pages/create-analysis-task-page.js: Added. (CreateAnalysisTaskPage): A page that gets shown momentarily while creating a new analysis task. * public/v3/pages/dashboard-page.js: Added. A dashboard page. * public/v3/pages/dashboard-toolbar.js: Added. Its toolbar with buttons to select the number of days to show. * public/v3/pages/domain-control-toolbar.js: Added. An abstract superclass of charts and dashboard toolbars. * public/v3/pages/heading.js: Added. A component for displaying the header and toolbar, if exists, on each page. * public/v3/pages/page-router.js: Added. This class is responsible for updating the URL hashes as well as opening and updating each page when the hash changes (via back/forward navigation). * public/v3/pages/page-with-charts.js: Added. An abstract subclass of page used by dashboards and charts page. Supplies helper functions for creating TimeSeriesChart options. * public/v3/pages/page-with-heading.js: Added. An abstract subclass of page that uses the heading component. * public/v3/pages/page.js: Added. The Page component. * public/v3/pages/toolbar.js: Added. An abstract toolbar component. * public/v3/remote.js: Added. (getJSON): Fetches JSON from the remote server. (getJSONWithStatus): Ditto. Rejects the response if the status is not "OK". (PrivilegedAPI.sendRequest): Posts a HTTP request to a privileged API in /privileged-api/. (PrivilegedAPI.requestCSRFToken): Creates a new CSRF token to request a privileged API post. * tools/bundle-v3-scripts.py: Added. (main): Bundles js files together and minifies them by jsmin.py for the v3 UI. Without this script, we're forced to download 44 JS files or making each JS file contain multiple classes. * tools/jsmin.py: Copied from WebInspector / JavaScriptCore code. Canonical link: https://commits.webkit.org/170434@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@194130 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-12-16 05:19:08 +00:00
if __name__ == "__main__":
main(sys.argv)