2016-03-24 20:17:01 +00:00
|
|
|
DROP TABLE IF EXISTS run_iterations CASCADE;
|
|
|
|
DROP TABLE IF EXISTS test_runs CASCADE;
|
|
|
|
DROP TABLE IF EXISTS test_configurations CASCADE;
|
|
|
|
DROP TYPE IF EXISTS test_configuration_type CASCADE;
|
|
|
|
DROP TABLE IF EXISTS aggregators CASCADE;
|
|
|
|
DROP TABLE IF EXISTS builds CASCADE;
|
|
|
|
DROP TABLE IF EXISTS committers CASCADE;
|
|
|
|
DROP TABLE IF EXISTS commits CASCADE;
|
|
|
|
DROP TABLE IF EXISTS build_commits CASCADE;
|
2017-03-13 09:10:29 +00:00
|
|
|
DROP TABLE IF EXISTS commit_ownerships CASCADE;
|
2021-03-22 21:23:38 +00:00
|
|
|
DROP TABLE IF EXISTS build_workers CASCADE;
|
2016-03-24 20:17:01 +00:00
|
|
|
DROP TABLE IF EXISTS builders CASCADE;
|
|
|
|
DROP TABLE IF EXISTS repositories CASCADE;
|
|
|
|
DROP TABLE IF EXISTS platforms CASCADE;
|
2020-10-28 01:00:35 +00:00
|
|
|
DROP TABLE IF EXISTS platform_groups CASCADE;
|
2016-03-24 20:17:01 +00:00
|
|
|
DROP TABLE IF EXISTS test_metrics CASCADE;
|
|
|
|
DROP TABLE IF EXISTS tests CASCADE;
|
|
|
|
DROP TABLE IF EXISTS reports CASCADE;
|
|
|
|
DROP TABLE IF EXISTS tracker_repositories CASCADE;
|
|
|
|
DROP TABLE IF EXISTS bug_trackers CASCADE;
|
|
|
|
DROP TABLE IF EXISTS task_commits CASCADE;
|
|
|
|
DROP TABLE IF EXISTS analysis_tasks CASCADE;
|
|
|
|
DROP TABLE IF EXISTS analysis_strategies CASCADE;
|
|
|
|
DROP TYPE IF EXISTS analysis_task_result_type CASCADE;
|
|
|
|
DROP TABLE IF EXISTS build_triggerables CASCADE;
|
|
|
|
DROP TABLE IF EXISTS triggerable_configurations CASCADE;
|
Introduce the notion of repository groups to triggerables
https://bugs.webkit.org/show_bug.cgi?id=170228
Reviewed by Chris Dumez.
On some triggerable, it's desirable to specify multiple sets of repositories that are accepted.
For example, if a repository X transitioned from Subversion to Git, and if a triggerable accepted X and
some other repository Y, then it's desirable to two sets: (X-Subversion, Y) and (X-Git, Y) since neither
(X-Subversion, X-Git) nor (X-Subversion, X-Git, Y) makes sense as a set.
This patch introduces triggerable_repository_groups table to represent a set of repositories accepted by
a triggerable. It has many to one relationship to build_triggerables and triggerable_repositories in turn
now has many to one relationship to triggerable_repository_groups instead of build_triggerables.
Also make it possible to disable a triggerable e.g. a set of tests and platforms are no longer supported.
We don't want to delete the triggerable completely from the database since it would result in the associated
A/B testing results being purged, which is not desirale.
To migrate an existing database, run the following transaction:
```sql
BEGIN;
ALTER TABLE build_triggerables ADD COLUMN triggerable_disabled boolean NOT NULL DEFAULT FALSE;
CREATE TABLE triggerable_repository_groups (
repositorygroup_id serial PRIMARY KEY,
repositorygroup_triggerable integer REFERENCES build_triggerables NOT NULL,
repositorygroup_name varchar(256) NOT NULL,
repositorygroup_description varchar(256),
repositorygroup_accepts_roots boolean NOT NULL DEFAULT FALSE,
CONSTRAINT repository_group_name_must_be_unique_for_triggerable
UNIQUE(repositorygroup_triggerable, repositorygroup_name));
INSERT INTO triggerable_repository_groups (repositorygroup_triggerable, repositorygroup_name)
SELECT triggerable_id, 'default' FROM build_triggerables;
ALTER TABLE triggerable_repositories ADD COLUMN trigrepo_group integer REFERENCES triggerable_repository_groups;
UPDATE triggerable_repositories SET trigrepo_group = repositorygroup_id FROM triggerable_repository_groups
WHERE trigrepo_triggerable = repositorygroup_triggerable;
ALTER TABLE triggerable_repositories ALTER COLUMN trigrepo_group SET NOT NULL;
ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_triggerable;
ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_sub_roots;
END;
```
* init-database.sql:
* public/admin/triggerables.php: Use a custom column to make forms to add and configure repository groups.
(insert_triggerable_repositories): Added.
(generate_repository_list): Added.
(generate_repository_form): Added.
(generate_repository_checkboxes): Now generates checkboxes for a repository group instead of a triggerable.
* public/include/manifest-generator.php:
(fetch_triggerables): Fixed the bug that we were not filtering results with query in /api/triggerable.
Rewrote it to include an array of repository groups, which in turn contains an array of repositories along
with its name and a description, and a boolean indicating whether it accepts a custom root file or not.
The boolean will be used when we're adding the support for perf try bots. We will keep acceptedRepositories
since it's still used by detect-changes.js.
* public/v3/models/manifest.js:
(Manifest._didFetchManifest): Resolve repositoriy, test, and platform IDs to their respective objects.
* public/v3/models/triggerable.js:
(Triggerable):
(Triggerable.prototype.isDisabled): Added.
(Triggerable.prototype.repositoryGroups): Added.
(Triggerable.prototype.acceptsTest): Added.
(TriggerableRepositoryGroup): Added.
(TriggerableRepositoryGroup.prototype.description): Added.
(TriggerableRepositoryGroup.prototype.acceptsCustomRoots): Added.
(TriggerableRepositoryGroup.prototype.repositories): Added.
* public/v3/pages/analysis-task-page.js:
(AnalysisTaskPage.prototype._didFetchTask): Don't use a disabled triggerable.
* server-tests/api-manifest-tests.js: Updated a test case to test repository groups.
* tools/js/database.js:
(tableToPrefixMap): Added triggerable_repository_groups.
* tools/js/v3-models.js: Imported TriggerableRepositoryGroup from triggerable.js.
Canonical link: https://commits.webkit.org/187454@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214975 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-05 23:12:46 +00:00
|
|
|
DROP TABLE IF EXISTS triggerable_repository_groups CASCADE;
|
2016-03-24 20:17:01 +00:00
|
|
|
DROP TABLE IF EXISTS triggerable_repositories CASCADE;
|
2017-03-16 20:53:35 +00:00
|
|
|
DROP TABLE IF EXISTS uploaded_files CASCADE;
|
2016-03-24 20:17:01 +00:00
|
|
|
DROP TABLE IF EXISTS bugs CASCADE;
|
|
|
|
DROP TABLE IF EXISTS analysis_test_groups CASCADE;
|
2021-05-26 01:23:22 +00:00
|
|
|
DROP TYPE IF EXISTS analysis_test_group_repetition_type CASCADE;
|
2017-03-14 23:06:40 +00:00
|
|
|
DROP TABLE IF EXISTS commit_sets CASCADE;
|
2017-04-21 21:31:06 +00:00
|
|
|
DROP TABLE IF EXISTS commit_set_items CASCADE;
|
2016-03-24 20:17:01 +00:00
|
|
|
DROP TABLE IF EXISTS build_requests CASCADE;
|
|
|
|
DROP TYPE IF EXISTS build_request_status_type CASCADE;
|
2014-12-18 22:53:52 +00:00
|
|
|
|
2014-02-08 03:11:53 +00:00
|
|
|
|
2020-10-28 01:00:35 +00:00
|
|
|
CREATE TABLE platform_groups (
|
|
|
|
platformgroup_id serial PRIMARY KEY,
|
|
|
|
platformgroup_name varchar(64) NOT NULL,
|
|
|
|
CONSTRAINT platform_group_name_must_be_unique UNIQUE (platformgroup_name));
|
|
|
|
|
2014-02-08 03:11:53 +00:00
|
|
|
CREATE TABLE platforms (
|
|
|
|
platform_id serial PRIMARY KEY,
|
|
|
|
platform_name varchar(64) NOT NULL,
|
2020-10-28 01:00:35 +00:00
|
|
|
platform_group integer REFERENCES platform_groups DEFAULT NULL,
|
2014-02-08 03:11:53 +00:00
|
|
|
platform_hidden boolean NOT NULL DEFAULT FALSE);
|
|
|
|
|
|
|
|
CREATE TABLE repositories (
|
|
|
|
repository_id serial PRIMARY KEY,
|
2017-03-13 09:10:29 +00:00
|
|
|
repository_owner integer REFERENCES repositories ON DELETE CASCADE,
|
2014-02-08 03:11:53 +00:00
|
|
|
repository_name varchar(64) NOT NULL,
|
|
|
|
repository_url varchar(1024),
|
2017-03-13 09:10:29 +00:00
|
|
|
repository_blame_url varchar(1024));
|
|
|
|
|
|
|
|
CREATE UNIQUE INDEX repository_name_owner_unique_index ON repositories (repository_owner, repository_name)
|
|
|
|
WHERE repository_owner IS NOT NULL;
|
|
|
|
CREATE UNIQUE INDEX repository_name_unique_index ON repositories (repository_name)
|
|
|
|
WHERE repository_owner IS NULL;
|
2014-02-08 03:11:53 +00:00
|
|
|
|
|
|
|
CREATE TABLE bug_trackers (
|
|
|
|
tracker_id serial PRIMARY KEY,
|
|
|
|
tracker_name varchar(64) NOT NULL,
|
Perf dashboard should provide a way to associate bugs with a test run
https://bugs.webkit.org/show_bug.cgi?id=137857
Reviewed by Andreas Kling.
Added a "privileged" API, /privileged-api/associate-bug, to associate a bug with a test run.
/privileged-api/ is to be protected by an authentication mechanism such as DigestAuth over https by
the Apache configuration.
The Cross Site Request (CSRF) Forgery prevention for privileged APIs work as follows. When a user is
about to make a privileged API access, the front end code obtains a CSRF token generated by POST'ing
to privileged-api/generate-csrf-token; the page sets a randomly generated salt and an expiration time
via the cookie and returns a token computed from those two values as well as the remote username.
The font end code then POST's the request along with the returned token. The server side code verifies
that the specified token can be generated from the salt and the expiration time set in the cookie, and
the token hasn't expired.
* init-database.sql: Added bug_url to bug_trackers table, and added bugs table. Each bug tracker will
have zero or exactly one bug associated with a test run.
* public/admin/bug-trackers.php: Added the support for editing bug_url.
* public/api/runs.php:
(fetch_runs_for_config): Modified the query to fetch bugs associated with test_runs.
(parse_bugs_array): Added. Parses the aggregated bugs and creates a dictionary that maps a tracker id to
an associated bug if there is one.
(format_run): Calls parse_bugs_array.
* public/include/json-header.php: Added helper functions to deal for CSRF prevention.
(ensure_privileged_api_data): Added. Dies immediately if the request's method is not POST or doesn't
have a valid JSON payload.
(ensure_privileged_api_data_and_token): Ditto. Also checks that the CSRF prevention token is valid.
(compute_token): Computes a CSRF token using the REMOTE_USER (e.g. set via BasicAuth), the salt, and
the expiration time stored in the cookie.
(verify_token): Returns true iff the specified token matches what compute_token returns from the cookie.
* public/include/manifest.php:
(ManifestGenerator::bug_trackers): Include bug_url as bugUrl in the manifest. Also use tracker_id instead
of tracker_name as the key in the manifest. This requires changes to both v1 and v2 front end.
* public/index.html:
(Chart..showTooltipWithResults): Updated for the manifest format changed mentioned above.
* public/privileged-api/associate-bug.php: Added.
(main): Added. Associates or dissociates a bug with a test run inside a transaction. It prevent a CSRF
attack via ensure_privileged_api_data_and_token, which calls verify_token.
* public/privileged-api/generate-csrf-token.php: Added. Generates a CSRF token valid for one hour.
* public/v2/app.css:
(.disabled .icon-button:hover g): Used by the "bugs" icon when a range of points or no points are
selected in a chart.
* public/v2/app.js:
(App.PaneController.actions.toggleBugsPane): Added. Toggles the visibility of the bugs pane when exactly
one point is selected in the chart. Also hides the search pane when making the bugs pane visible since
they would overlap on each other if both of them are shown.
(App.PaneController.actions.associateBug): Makes a privileged API request to associate the specified bug
with the currently selected point (test run). Updates the bug information in "details" and colors of dots
in the charts to reflect new states. Because chart data objects aren't real Ember objects for performance
reasons, we have to use a dirty hack of modifying a dummy counter bugsChangeCount.
(App.PaneController.actions.toggleSearchPane): Renamed from toggleSearch. Also hides the bugs pane when
showing the search pane.
(App.PaneController.actions.rangeChanged): Takes all selected points as the second argument instead of
taking start and end points as the second and the third arguments so that _showDetails can enumerate all
bugs in the selected range.
(App.PaneController._detailsChanged): Added. Hide the bugs pane whenever a new point is selected.
Also update singlySelectedPoint, which is used by toggleBugsPane and associateBug.
(App.PaneController._currentItemChanged): Updated for the _showDetails change.
(App.PaneController._showDetails): Takes an array of selected points in place of old arguments.
Simplified the code to compute the revision information. Calls _updateBugs to format the associated bugs.
(App.PaneController._updateBugs): Sets details.bugTrackers to a dictionary that maps a bug tracker id to
a bug tracker proxy with an array of (bugNumber, bugUrl) pairs and also editedBugNumber, which is used by
the bugs pane to associate or dissociate a bug number, if exactly one point is selected.
(App.InteractiveChartComponent._updateDotsWithBugs): Added. Sets hasBugs class on dots as needed.
(App.InteractiveChartComponent._setCurrentSelection): Finds and passes all points in the selected range
to selectionChanged action instead of just finding the first and the last points.
* public/v2/chart-pane.css: Updated the style.
* public/v2/data.js:
(PrivilegedAPI): Added. A wrapper for privileged APIs' CSRF tokens.
(PrivilegedAPI.sendRequest): Makes a privileged API call. Fetches a new CSRF token if needed.
(PrivilegedAPI._generateTokenInServerIfNeeded): Makes a request to privileged-api/generate-csrf-token if
we haven't already obtained a CSRF token or if the token has already been expired.
(PrivilegedAPI._post): Makes a single POST request to /privileged-api/* with a JSON payload.
(Measurement.prototype.bugs): Added.
(Measurement.prototype.hasBugs): Returns true iff bugs has more than one bug number.
(Measurement.prototype.associateBug): Associates a bug with a test run via privileged-api/associate-bug.
* public/v2/index.html: Added the bugs pane. Also added a list of bugs associated with the current run in
the details.
* public/v2/manifest.js:
(App.BugTracker.bugUrl):
(App.BugTracker.newBugUrl): Added.
(App.BugTracker.repositories): Added. This was a missing back reference to repositories.
(App.MetricSerializer.normalizePayload): Now parses/loads the list of bug trackers from the manifest.
(App.Manifest.repositoriesWithReportedCommits): Now initialized to an empty array instead of null.
(App.Manifest.bugTrackers): Added.
(App.Manifest._fetchedManifest): Sets App.Manifest.bugTrackers. Also sorts the list of repositories by
their respective ids to make the ordering stable.
Canonical link: https://commits.webkit.org/155807@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-22 00:53:39 +00:00
|
|
|
tracker_bug_url varchar(1024),
|
2014-02-08 03:11:53 +00:00
|
|
|
tracker_new_bug_url varchar(1024));
|
|
|
|
|
|
|
|
CREATE TABLE tracker_repositories (
|
|
|
|
tracrepo_tracker integer NOT NULL REFERENCES bug_trackers ON DELETE CASCADE,
|
|
|
|
tracrepo_repository integer NOT NULL REFERENCES repositories ON DELETE CASCADE);
|
|
|
|
|
|
|
|
CREATE TABLE builders (
|
|
|
|
builder_id serial PRIMARY KEY,
|
2015-06-10 02:00:40 +00:00
|
|
|
builder_name varchar(256) NOT NULL UNIQUE,
|
2014-12-20 02:38:04 +00:00
|
|
|
builder_password_hash character(64),
|
2014-02-08 03:11:53 +00:00
|
|
|
builder_build_url varchar(1024));
|
|
|
|
|
2021-03-22 21:23:38 +00:00
|
|
|
CREATE TABLE build_workers (
|
|
|
|
worker_id serial PRIMARY KEY,
|
|
|
|
worker_name varchar(64) NOT NULL UNIQUE,
|
|
|
|
worker_password_hash character(64));
|
2014-12-20 02:38:04 +00:00
|
|
|
|
2014-02-08 03:11:53 +00:00
|
|
|
CREATE TABLE builds (
|
|
|
|
build_id serial PRIMARY KEY,
|
|
|
|
build_builder integer REFERENCES builders ON DELETE CASCADE,
|
2021-03-22 21:23:38 +00:00
|
|
|
build_worker integer REFERENCES build_workers ON DELETE CASCADE,
|
2019-10-24 22:00:37 +00:00
|
|
|
build_tag varchar(64) NOT NULL,
|
2014-02-08 03:11:53 +00:00
|
|
|
build_time timestamp NOT NULL,
|
|
|
|
build_latest_revision timestamp,
|
2019-10-24 22:00:37 +00:00
|
|
|
CONSTRAINT builder_build_time_tuple_must_be_unique UNIQUE(build_builder, build_tag, build_time));
|
2014-02-08 03:11:53 +00:00
|
|
|
CREATE INDEX build_builder_index ON builds(build_builder);
|
|
|
|
|
2014-12-18 22:53:52 +00:00
|
|
|
CREATE TABLE committers (
|
|
|
|
committer_id serial PRIMARY KEY,
|
|
|
|
committer_repository integer NOT NULL REFERENCES repositories ON DELETE CASCADE,
|
|
|
|
committer_account varchar(320) NOT NULL,
|
|
|
|
committer_name varchar(128),
|
|
|
|
CONSTRAINT committer_in_repository_must_be_unique UNIQUE(committer_repository, committer_account));
|
|
|
|
CREATE INDEX committer_account_index ON committers(committer_account);
|
|
|
|
CREATE INDEX committer_name_index ON committers(committer_name);
|
|
|
|
|
Perf dashboard should store commit logs
https://bugs.webkit.org/show_bug.cgi?id=137510
Reviewed by Darin Adler.
For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself.
This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs,
and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from
a subversion directory. The git version of this script will be added in a follow up patch.
In the new database schema, each revision in each repository is represented by exactly one row in "commits"
instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many
relationship via "build_commits" relations.
In order to migrate an existing instance of this application, run the following SQL commands:
BEGIN;
INSERT INTO commits (commit_repository, commit_revision, commit_time)
(SELECT DISTINCT ON (revision_repository, revision_value)
revision_repository, revision_value, revision_time FROM build_revisions);
INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id
FROM commits, build_revisions
WHERE commit_repository = revision_repository AND commit_revision = revision_value;
DROP TABLE build_revisions;
COMMIT;
The helper script to submit commit logs can be used as follows:
python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org
feeder-slave feeder-slave-password 60 "webkit-patch find-users"
The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds
to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and
"feeder-slave-password" as the builder name and the builder password respectively.
The last, optional, argument is the shell command to convert a subversion account to the corresponding username.
e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout.
* init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations.
* public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form
/api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows:
- Not specified - It returns every single commit for a given repository.
- Matches "oldest" - It returns the commit with the oldest timestamp.
- Matches "latest" - It returns the commit with the latest timestamp.
- Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php.
- Is entirely alphanumeric - It returns the commit whose revision matches the filter.
- Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list
of commits added via report-commits.php between two timestamps retrieved from commits whose revisions
match the two alphanumeric values specified. Because it retrieves commits based on their timestamps,
the list may contain commits that do not appear as neither hash's ancestor in git/mercurial.
(main):
(commit_from_revision):
(fetch_commits_between):
(format_commits):
* public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits.
See tests/api-report-commits.js for examples on how to use this API.
* public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of
"build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready
to optimize this and other JSON APIs.
* public/include/db.php:
(Database::select_or_insert_row):
(Database::update_or_insert_row): Added.
(Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then
insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false.
(Database::select_first_row):
(Database::select_last_row): Added.
(Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting
$order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order.
* public/include/report-processor.php:
(ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race
condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result
concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent.
* run-tests.js:
(pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js.
* tests/api-report-commits.js: Added.
* tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement.
* tools: Added.
* tools/pull-svn.py: Added. See above for how to use this script.
(main):
(determine_first_revision_to_fetch):
(fetch_revision_from_dasbhoard):
(fetch_commit_and_resolve_author):
(fetch_commit):
(textContent):
(resolve_author_name_from_email):
(submit_commits):
Canonical link: https://commits.webkit.org/155341@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
|
|
|
CREATE TABLE commits (
|
|
|
|
commit_id serial PRIMARY KEY,
|
|
|
|
commit_repository integer NOT NULL REFERENCES repositories ON DELETE CASCADE,
|
|
|
|
commit_revision varchar(64) NOT NULL,
|
2021-04-01 00:50:11 +00:00
|
|
|
commit_revision_identifier varchar(64) DEFAULT NULL,
|
2017-02-24 07:17:48 +00:00
|
|
|
commit_previous_commit integer REFERENCES commits ON DELETE CASCADE,
|
Perf dashboard should store commit logs
https://bugs.webkit.org/show_bug.cgi?id=137510
Reviewed by Darin Adler.
For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself.
This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs,
and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from
a subversion directory. The git version of this script will be added in a follow up patch.
In the new database schema, each revision in each repository is represented by exactly one row in "commits"
instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many
relationship via "build_commits" relations.
In order to migrate an existing instance of this application, run the following SQL commands:
BEGIN;
INSERT INTO commits (commit_repository, commit_revision, commit_time)
(SELECT DISTINCT ON (revision_repository, revision_value)
revision_repository, revision_value, revision_time FROM build_revisions);
INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id
FROM commits, build_revisions
WHERE commit_repository = revision_repository AND commit_revision = revision_value;
DROP TABLE build_revisions;
COMMIT;
The helper script to submit commit logs can be used as follows:
python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org
feeder-slave feeder-slave-password 60 "webkit-patch find-users"
The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds
to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and
"feeder-slave-password" as the builder name and the builder password respectively.
The last, optional, argument is the shell command to convert a subversion account to the corresponding username.
e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout.
* init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations.
* public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form
/api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows:
- Not specified - It returns every single commit for a given repository.
- Matches "oldest" - It returns the commit with the oldest timestamp.
- Matches "latest" - It returns the commit with the latest timestamp.
- Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php.
- Is entirely alphanumeric - It returns the commit whose revision matches the filter.
- Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list
of commits added via report-commits.php between two timestamps retrieved from commits whose revisions
match the two alphanumeric values specified. Because it retrieves commits based on their timestamps,
the list may contain commits that do not appear as neither hash's ancestor in git/mercurial.
(main):
(commit_from_revision):
(fetch_commits_between):
(format_commits):
* public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits.
See tests/api-report-commits.js for examples on how to use this API.
* public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of
"build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready
to optimize this and other JSON APIs.
* public/include/db.php:
(Database::select_or_insert_row):
(Database::update_or_insert_row): Added.
(Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then
insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false.
(Database::select_first_row):
(Database::select_last_row): Added.
(Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting
$order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order.
* public/include/report-processor.php:
(ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race
condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result
concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent.
* run-tests.js:
(pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js.
* tests/api-report-commits.js: Added.
* tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement.
* tools: Added.
* tools/pull-svn.py: Added. See above for how to use this script.
(main):
(determine_first_revision_to_fetch):
(fetch_revision_from_dasbhoard):
(fetch_commit_and_resolve_author):
(fetch_commit):
(textContent):
(resolve_author_name_from_email):
(submit_commits):
Canonical link: https://commits.webkit.org/155341@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
|
|
|
commit_time timestamp,
|
2021-07-21 11:35:17 +00:00
|
|
|
commit_order bigint,
|
2014-12-18 22:53:52 +00:00
|
|
|
commit_committer integer REFERENCES committers ON DELETE CASCADE,
|
Perf dashboard should store commit logs
https://bugs.webkit.org/show_bug.cgi?id=137510
Reviewed by Darin Adler.
For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself.
This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs,
and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from
a subversion directory. The git version of this script will be added in a follow up patch.
In the new database schema, each revision in each repository is represented by exactly one row in "commits"
instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many
relationship via "build_commits" relations.
In order to migrate an existing instance of this application, run the following SQL commands:
BEGIN;
INSERT INTO commits (commit_repository, commit_revision, commit_time)
(SELECT DISTINCT ON (revision_repository, revision_value)
revision_repository, revision_value, revision_time FROM build_revisions);
INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id
FROM commits, build_revisions
WHERE commit_repository = revision_repository AND commit_revision = revision_value;
DROP TABLE build_revisions;
COMMIT;
The helper script to submit commit logs can be used as follows:
python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org
feeder-slave feeder-slave-password 60 "webkit-patch find-users"
The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds
to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and
"feeder-slave-password" as the builder name and the builder password respectively.
The last, optional, argument is the shell command to convert a subversion account to the corresponding username.
e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout.
* init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations.
* public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form
/api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows:
- Not specified - It returns every single commit for a given repository.
- Matches "oldest" - It returns the commit with the oldest timestamp.
- Matches "latest" - It returns the commit with the latest timestamp.
- Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php.
- Is entirely alphanumeric - It returns the commit whose revision matches the filter.
- Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list
of commits added via report-commits.php between two timestamps retrieved from commits whose revisions
match the two alphanumeric values specified. Because it retrieves commits based on their timestamps,
the list may contain commits that do not appear as neither hash's ancestor in git/mercurial.
(main):
(commit_from_revision):
(fetch_commits_between):
(format_commits):
* public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits.
See tests/api-report-commits.js for examples on how to use this API.
* public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of
"build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready
to optimize this and other JSON APIs.
* public/include/db.php:
(Database::select_or_insert_row):
(Database::update_or_insert_row): Added.
(Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then
insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false.
(Database::select_first_row):
(Database::select_last_row): Added.
(Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting
$order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order.
* public/include/report-processor.php:
(ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race
condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result
concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent.
* run-tests.js:
(pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js.
* tests/api-report-commits.js: Added.
* tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement.
* tools: Added.
* tools/pull-svn.py: Added. See above for how to use this script.
(main):
(determine_first_revision_to_fetch):
(fetch_revision_from_dasbhoard):
(fetch_commit_and_resolve_author):
(fetch_commit):
(textContent):
(resolve_author_name_from_email):
(submit_commits):
Canonical link: https://commits.webkit.org/155341@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
|
|
|
commit_message text,
|
|
|
|
commit_reported boolean NOT NULL DEFAULT FALSE,
|
2019-01-17 05:56:18 +00:00
|
|
|
commit_testability varchar(128) DEFAULT NULL,
|
2021-04-01 00:50:11 +00:00
|
|
|
CONSTRAINT commit_in_repository_must_be_unique UNIQUE(commit_repository, commit_revision),
|
|
|
|
CONSTRAINT commit_string_identifier_in_repository_must_be_unique UNIQUE(commit_repository, commit_revision_identifier));
|
Perf dashboard should store commit logs
https://bugs.webkit.org/show_bug.cgi?id=137510
Reviewed by Darin Adler.
For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself.
This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs,
and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from
a subversion directory. The git version of this script will be added in a follow up patch.
In the new database schema, each revision in each repository is represented by exactly one row in "commits"
instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many
relationship via "build_commits" relations.
In order to migrate an existing instance of this application, run the following SQL commands:
BEGIN;
INSERT INTO commits (commit_repository, commit_revision, commit_time)
(SELECT DISTINCT ON (revision_repository, revision_value)
revision_repository, revision_value, revision_time FROM build_revisions);
INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id
FROM commits, build_revisions
WHERE commit_repository = revision_repository AND commit_revision = revision_value;
DROP TABLE build_revisions;
COMMIT;
The helper script to submit commit logs can be used as follows:
python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org
feeder-slave feeder-slave-password 60 "webkit-patch find-users"
The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds
to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and
"feeder-slave-password" as the builder name and the builder password respectively.
The last, optional, argument is the shell command to convert a subversion account to the corresponding username.
e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout.
* init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations.
* public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form
/api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows:
- Not specified - It returns every single commit for a given repository.
- Matches "oldest" - It returns the commit with the oldest timestamp.
- Matches "latest" - It returns the commit with the latest timestamp.
- Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php.
- Is entirely alphanumeric - It returns the commit whose revision matches the filter.
- Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list
of commits added via report-commits.php between two timestamps retrieved from commits whose revisions
match the two alphanumeric values specified. Because it retrieves commits based on their timestamps,
the list may contain commits that do not appear as neither hash's ancestor in git/mercurial.
(main):
(commit_from_revision):
(fetch_commits_between):
(format_commits):
* public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits.
See tests/api-report-commits.js for examples on how to use this API.
* public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of
"build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready
to optimize this and other JSON APIs.
* public/include/db.php:
(Database::select_or_insert_row):
(Database::update_or_insert_row): Added.
(Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then
insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false.
(Database::select_first_row):
(Database::select_last_row): Added.
(Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting
$order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order.
* public/include/report-processor.php:
(ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race
condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result
concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent.
* run-tests.js:
(pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js.
* tests/api-report-commits.js: Added.
* tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement.
* tools: Added.
* tools/pull-svn.py: Added. See above for how to use this script.
(main):
(determine_first_revision_to_fetch):
(fetch_revision_from_dasbhoard):
(fetch_commit_and_resolve_author):
(fetch_commit):
(textContent):
(resolve_author_name_from_email):
(submit_commits):
Canonical link: https://commits.webkit.org/155341@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
|
|
|
CREATE INDEX commit_time_index ON commits(commit_time);
|
2015-12-15 04:06:44 +00:00
|
|
|
CREATE INDEX commit_order_index ON commits(commit_order);
|
Perf dashboard should store commit logs
https://bugs.webkit.org/show_bug.cgi?id=137510
Reviewed by Darin Adler.
For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself.
This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs,
and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from
a subversion directory. The git version of this script will be added in a follow up patch.
In the new database schema, each revision in each repository is represented by exactly one row in "commits"
instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many
relationship via "build_commits" relations.
In order to migrate an existing instance of this application, run the following SQL commands:
BEGIN;
INSERT INTO commits (commit_repository, commit_revision, commit_time)
(SELECT DISTINCT ON (revision_repository, revision_value)
revision_repository, revision_value, revision_time FROM build_revisions);
INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id
FROM commits, build_revisions
WHERE commit_repository = revision_repository AND commit_revision = revision_value;
DROP TABLE build_revisions;
COMMIT;
The helper script to submit commit logs can be used as follows:
python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org
feeder-slave feeder-slave-password 60 "webkit-patch find-users"
The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds
to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and
"feeder-slave-password" as the builder name and the builder password respectively.
The last, optional, argument is the shell command to convert a subversion account to the corresponding username.
e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout.
* init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations.
* public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form
/api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows:
- Not specified - It returns every single commit for a given repository.
- Matches "oldest" - It returns the commit with the oldest timestamp.
- Matches "latest" - It returns the commit with the latest timestamp.
- Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php.
- Is entirely alphanumeric - It returns the commit whose revision matches the filter.
- Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list
of commits added via report-commits.php between two timestamps retrieved from commits whose revisions
match the two alphanumeric values specified. Because it retrieves commits based on their timestamps,
the list may contain commits that do not appear as neither hash's ancestor in git/mercurial.
(main):
(commit_from_revision):
(fetch_commits_between):
(format_commits):
* public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits.
See tests/api-report-commits.js for examples on how to use this API.
* public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of
"build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready
to optimize this and other JSON APIs.
* public/include/db.php:
(Database::select_or_insert_row):
(Database::update_or_insert_row): Added.
(Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then
insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false.
(Database::select_first_row):
(Database::select_last_row): Added.
(Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting
$order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order.
* public/include/report-processor.php:
(ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race
condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result
concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent.
* run-tests.js:
(pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js.
* tests/api-report-commits.js: Added.
* tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement.
* tools: Added.
* tools/pull-svn.py: Added. See above for how to use this script.
(main):
(determine_first_revision_to_fetch):
(fetch_revision_from_dasbhoard):
(fetch_commit_and_resolve_author):
(fetch_commit):
(textContent):
(resolve_author_name_from_email):
(submit_commits):
Canonical link: https://commits.webkit.org/155341@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
|
|
|
|
2017-03-13 09:10:29 +00:00
|
|
|
CREATE TABLE commit_ownerships (
|
|
|
|
commit_owner integer NOT NULL REFERENCES commits ON DELETE CASCADE,
|
|
|
|
commit_owned integer NOT NULL REFERENCES commits ON DELETE CASCADE,
|
|
|
|
PRIMARY KEY (commit_owner, commit_owned)
|
|
|
|
);
|
|
|
|
|
Perf dashboard should store commit logs
https://bugs.webkit.org/show_bug.cgi?id=137510
Reviewed by Darin Adler.
For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself.
This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs,
and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from
a subversion directory. The git version of this script will be added in a follow up patch.
In the new database schema, each revision in each repository is represented by exactly one row in "commits"
instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many
relationship via "build_commits" relations.
In order to migrate an existing instance of this application, run the following SQL commands:
BEGIN;
INSERT INTO commits (commit_repository, commit_revision, commit_time)
(SELECT DISTINCT ON (revision_repository, revision_value)
revision_repository, revision_value, revision_time FROM build_revisions);
INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id
FROM commits, build_revisions
WHERE commit_repository = revision_repository AND commit_revision = revision_value;
DROP TABLE build_revisions;
COMMIT;
The helper script to submit commit logs can be used as follows:
python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org
feeder-slave feeder-slave-password 60 "webkit-patch find-users"
The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds
to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and
"feeder-slave-password" as the builder name and the builder password respectively.
The last, optional, argument is the shell command to convert a subversion account to the corresponding username.
e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout.
* init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations.
* public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form
/api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows:
- Not specified - It returns every single commit for a given repository.
- Matches "oldest" - It returns the commit with the oldest timestamp.
- Matches "latest" - It returns the commit with the latest timestamp.
- Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php.
- Is entirely alphanumeric - It returns the commit whose revision matches the filter.
- Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list
of commits added via report-commits.php between two timestamps retrieved from commits whose revisions
match the two alphanumeric values specified. Because it retrieves commits based on their timestamps,
the list may contain commits that do not appear as neither hash's ancestor in git/mercurial.
(main):
(commit_from_revision):
(fetch_commits_between):
(format_commits):
* public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits.
See tests/api-report-commits.js for examples on how to use this API.
* public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of
"build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready
to optimize this and other JSON APIs.
* public/include/db.php:
(Database::select_or_insert_row):
(Database::update_or_insert_row): Added.
(Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then
insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false.
(Database::select_first_row):
(Database::select_last_row): Added.
(Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting
$order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order.
* public/include/report-processor.php:
(ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race
condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result
concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent.
* run-tests.js:
(pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js.
* tests/api-report-commits.js: Added.
* tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement.
* tools: Added.
* tools/pull-svn.py: Added. See above for how to use this script.
(main):
(determine_first_revision_to_fetch):
(fetch_revision_from_dasbhoard):
(fetch_commit_and_resolve_author):
(fetch_commit):
(textContent):
(resolve_author_name_from_email):
(submit_commits):
Canonical link: https://commits.webkit.org/155341@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
|
|
|
CREATE TABLE build_commits (
|
|
|
|
commit_build integer NOT NULL REFERENCES builds ON DELETE CASCADE,
|
2014-10-09 00:19:14 +00:00
|
|
|
build_commit integer NOT NULL REFERENCES commits ON DELETE CASCADE,
|
Perf dashboard should store commit logs
https://bugs.webkit.org/show_bug.cgi?id=137510
Reviewed by Darin Adler.
For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself.
This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs,
and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from
a subversion directory. The git version of this script will be added in a follow up patch.
In the new database schema, each revision in each repository is represented by exactly one row in "commits"
instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many
relationship via "build_commits" relations.
In order to migrate an existing instance of this application, run the following SQL commands:
BEGIN;
INSERT INTO commits (commit_repository, commit_revision, commit_time)
(SELECT DISTINCT ON (revision_repository, revision_value)
revision_repository, revision_value, revision_time FROM build_revisions);
INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id
FROM commits, build_revisions
WHERE commit_repository = revision_repository AND commit_revision = revision_value;
DROP TABLE build_revisions;
COMMIT;
The helper script to submit commit logs can be used as follows:
python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org
feeder-slave feeder-slave-password 60 "webkit-patch find-users"
The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds
to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and
"feeder-slave-password" as the builder name and the builder password respectively.
The last, optional, argument is the shell command to convert a subversion account to the corresponding username.
e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout.
* init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations.
* public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form
/api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows:
- Not specified - It returns every single commit for a given repository.
- Matches "oldest" - It returns the commit with the oldest timestamp.
- Matches "latest" - It returns the commit with the latest timestamp.
- Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php.
- Is entirely alphanumeric - It returns the commit whose revision matches the filter.
- Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list
of commits added via report-commits.php between two timestamps retrieved from commits whose revisions
match the two alphanumeric values specified. Because it retrieves commits based on their timestamps,
the list may contain commits that do not appear as neither hash's ancestor in git/mercurial.
(main):
(commit_from_revision):
(fetch_commits_between):
(format_commits):
* public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits.
See tests/api-report-commits.js for examples on how to use this API.
* public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of
"build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready
to optimize this and other JSON APIs.
* public/include/db.php:
(Database::select_or_insert_row):
(Database::update_or_insert_row): Added.
(Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then
insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false.
(Database::select_first_row):
(Database::select_last_row): Added.
(Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting
$order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order.
* public/include/report-processor.php:
(ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race
condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result
concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent.
* run-tests.js:
(pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js.
* tests/api-report-commits.js: Added.
* tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement.
* tools: Added.
* tools/pull-svn.py: Added. See above for how to use this script.
(main):
(determine_first_revision_to_fetch):
(fetch_revision_from_dasbhoard):
(fetch_commit_and_resolve_author):
(fetch_commit):
(textContent):
(resolve_author_name_from_email):
(submit_commits):
Canonical link: https://commits.webkit.org/155341@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
|
|
|
PRIMARY KEY (commit_build, build_commit));
|
2014-02-08 03:11:53 +00:00
|
|
|
|
|
|
|
CREATE TABLE aggregators (
|
|
|
|
aggregator_id serial PRIMARY KEY,
|
|
|
|
aggregator_name varchar(64),
|
|
|
|
aggregator_definition text);
|
|
|
|
|
|
|
|
CREATE TABLE tests (
|
|
|
|
test_id serial PRIMARY KEY,
|
|
|
|
test_name varchar(255) NOT NULL,
|
|
|
|
test_parent integer REFERENCES tests ON DELETE CASCADE,
|
|
|
|
test_url varchar(1024) DEFAULT NULL,
|
|
|
|
CONSTRAINT parent_test_must_be_unique UNIQUE(test_parent, test_name));
|
|
|
|
|
|
|
|
CREATE TABLE test_metrics (
|
|
|
|
metric_id serial PRIMARY KEY,
|
|
|
|
metric_test integer NOT NULL REFERENCES tests ON DELETE CASCADE,
|
|
|
|
metric_name varchar(64) NOT NULL,
|
|
|
|
metric_aggregator integer REFERENCES aggregators ON DELETE CASCADE);
|
|
|
|
|
|
|
|
CREATE TYPE test_configuration_type as ENUM ('current', 'baseline', 'target');
|
|
|
|
CREATE TABLE test_configurations (
|
|
|
|
config_id serial PRIMARY KEY,
|
|
|
|
config_metric integer NOT NULL REFERENCES test_metrics ON DELETE CASCADE,
|
|
|
|
config_platform integer NOT NULL REFERENCES platforms ON DELETE CASCADE,
|
|
|
|
config_type test_configuration_type NOT NULL,
|
|
|
|
config_is_in_dashboard boolean NOT NULL DEFAULT FALSE,
|
Loading the perf dashboard takes multiple seconds
https://bugs.webkit.org/show_bug.cgi?id=141860
Reviewed by Andreas Kling.
This patch introduces the caches of JSON files returned by /api/ in /data/ directory. It also records
the last time test_runs rows associated with the requested platforms and metrics are inserted, updated,
or removed in the caches as well as the manifest JSON files ("last modified time"). Because the manifest
is regenerated each time a new test result is reported, the front end can compare last modified time in
the manifest file with that in a /api/runs JSON cache to detect the stale-ness.
More concretely, the front end first optimistically fetches the JSON in /data/. If the cache doesn't exit
or the last modified time in the cache doesn't match with that in the manifest file, it would fetch it
again via /api/runs. In the case the cache did exist, we render the charts based on the cache meanwhile.
This dramatically reduces the perceived latency for the page load since charts are drawn immediately using
the cache and we would only re-render the charts as new up-to-date JSON comes in.
This patch also changes the format of runs JSONs by pushing the exiting properties into 'configurations'
and adding 'lastModified' and 'elapsedTime' at the top level.
* init-database.sql: Added config_runs_last_modified to test_configurations table as well as a trigger to
auto-update this column upon changes to test_runs table.
* public/admin/test-configurations.php:
(add_run): Regenerate the manifest file to invalidate the /api/runs JSON cache.
(delete_run): Ditto.
* public/api/runs.php:
(main): Fetch all columns of test_configurations table including config_runs_last_modified. Also generate
the cache in /data/ directory.
(RunsGenerator::__construct): Compute the last modified time for this (platform, metric) pair.
(RunsGenerator::results): Put the old content in 'configurations' property and include 'lastModified' and
'elapsedTime' properties. 'elapsedTime' is added for debugging purposes.
(RunsGenerator::add_runs):
(RunsGenerator::parse_revisions_array):
* public/include/db.php:
(CONFIG_DIR): Added.
(generate_data_file): Added based on ManifestGenerator::store.
(Database::to_js_time): Extracted from RunsGenerator::add_runs to share code.
* public/include/json-header.php:
(echo_success): Renamed from success_json. Return the serialized JSON instead of echo'ing it so that we can
generate caches in /api/runs/.
(exit_with_success):
* public/include/manifest.php:
(ManifestGenerator::generate): Added 'elapsedTime' property for the time taken to generate the manifest.
It seems like we're generating it in 200-300ms for now so that's good.
(ManifestGenerator::store): Uses generate_data_file.
(ManifestGenerator::platforms): Added 'lastModified' array to each platform entry. This array contains the
last modified time for each (platform, metric) pair.
* public/index.html:
(fetchTest): Updated per the format change in runs JSON.
* public/v2/app.js:
(App.Pane._fetch): Fetch the cached JSON first. Refetch the uncached version if instructed as such.
(App.Pane._updateChartData): Extracted from App.Pane._fetch.
(App.Pane._handleFetchErrors): Ditto.
* public/v2/data.js:
(RunsData.fetchRuns): Takes the fourth argument indicating whether we should fetch the cached version or not.
The cached JSON is located in /data/ with the same filename. When fetching a cached JSON results in 404,
fulfill the promise with null as the result instead of rejecting it. The only client of this function which
sets useCache to true is App.Manifest.fetchRunsWithPlatformAndMetric, and it handles this special case.
* public/v2/manifest.js:
(App.DateArrayTransform): Added. Handles the array of last modified dates in platform objects.
(App.Platform.lastModifiedTimeForMetric): Added. Returns the last modified date in the manifest JSON.
(App.Manifest.fetchRunsWithPlatformAndMetric): Takes "useCache" like RunsData.fetchRuns. Set shouldRefetch
to true if response is null (the cache didn't exit) or the cache is out-of-date.
(App.Manifest._formatFetchedData): Extracted from App.Manifest.fetchRunsWithPlatformAndMetric.
* run-tests.js:
(initializeDatabase): Avoid splitting function definitions in the middle.
* tests/api-report.js: Added tests to verify that reporting new test results updates the last modified time
in test_configurations.
Canonical link: https://commits.webkit.org/159898@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@180468 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-02-21 01:28:34 +00:00
|
|
|
config_runs_last_modified timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
|
2014-02-08 03:11:53 +00:00
|
|
|
CONSTRAINT configuration_must_be_unique UNIQUE(config_metric, config_platform, config_type));
|
|
|
|
CREATE INDEX config_platform_index ON test_configurations(config_platform);
|
|
|
|
|
|
|
|
CREATE TABLE test_runs (
|
|
|
|
run_id serial PRIMARY KEY,
|
|
|
|
run_config integer NOT NULL REFERENCES test_configurations ON DELETE CASCADE,
|
|
|
|
run_build integer NOT NULL REFERENCES builds ON DELETE CASCADE,
|
|
|
|
run_iteration_count_cache smallint,
|
|
|
|
run_mean_cache double precision,
|
|
|
|
run_sum_cache double precision,
|
|
|
|
run_square_sum_cache double precision,
|
2016-06-13 19:47:38 +00:00
|
|
|
run_marked_outlier boolean NOT NULL DEFAULT FALSE,
|
2014-02-08 03:11:53 +00:00
|
|
|
CONSTRAINT test_config_build_must_be_unique UNIQUE(run_config, run_build));
|
|
|
|
CREATE INDEX run_config_index ON test_runs(run_config);
|
|
|
|
CREATE INDEX run_build_index ON test_runs(run_build);
|
|
|
|
|
|
|
|
CREATE TABLE run_iterations (
|
|
|
|
iteration_run integer NOT NULL REFERENCES test_runs ON DELETE CASCADE,
|
|
|
|
iteration_order smallint NOT NULL CHECK(iteration_order >= 0),
|
|
|
|
iteration_group smallint CHECK(iteration_group >= 0),
|
|
|
|
iteration_value double precision,
|
|
|
|
iteration_relative_time float,
|
|
|
|
PRIMARY KEY (iteration_run, iteration_order));
|
|
|
|
|
Loading the perf dashboard takes multiple seconds
https://bugs.webkit.org/show_bug.cgi?id=141860
Reviewed by Andreas Kling.
This patch introduces the caches of JSON files returned by /api/ in /data/ directory. It also records
the last time test_runs rows associated with the requested platforms and metrics are inserted, updated,
or removed in the caches as well as the manifest JSON files ("last modified time"). Because the manifest
is regenerated each time a new test result is reported, the front end can compare last modified time in
the manifest file with that in a /api/runs JSON cache to detect the stale-ness.
More concretely, the front end first optimistically fetches the JSON in /data/. If the cache doesn't exit
or the last modified time in the cache doesn't match with that in the manifest file, it would fetch it
again via /api/runs. In the case the cache did exist, we render the charts based on the cache meanwhile.
This dramatically reduces the perceived latency for the page load since charts are drawn immediately using
the cache and we would only re-render the charts as new up-to-date JSON comes in.
This patch also changes the format of runs JSONs by pushing the exiting properties into 'configurations'
and adding 'lastModified' and 'elapsedTime' at the top level.
* init-database.sql: Added config_runs_last_modified to test_configurations table as well as a trigger to
auto-update this column upon changes to test_runs table.
* public/admin/test-configurations.php:
(add_run): Regenerate the manifest file to invalidate the /api/runs JSON cache.
(delete_run): Ditto.
* public/api/runs.php:
(main): Fetch all columns of test_configurations table including config_runs_last_modified. Also generate
the cache in /data/ directory.
(RunsGenerator::__construct): Compute the last modified time for this (platform, metric) pair.
(RunsGenerator::results): Put the old content in 'configurations' property and include 'lastModified' and
'elapsedTime' properties. 'elapsedTime' is added for debugging purposes.
(RunsGenerator::add_runs):
(RunsGenerator::parse_revisions_array):
* public/include/db.php:
(CONFIG_DIR): Added.
(generate_data_file): Added based on ManifestGenerator::store.
(Database::to_js_time): Extracted from RunsGenerator::add_runs to share code.
* public/include/json-header.php:
(echo_success): Renamed from success_json. Return the serialized JSON instead of echo'ing it so that we can
generate caches in /api/runs/.
(exit_with_success):
* public/include/manifest.php:
(ManifestGenerator::generate): Added 'elapsedTime' property for the time taken to generate the manifest.
It seems like we're generating it in 200-300ms for now so that's good.
(ManifestGenerator::store): Uses generate_data_file.
(ManifestGenerator::platforms): Added 'lastModified' array to each platform entry. This array contains the
last modified time for each (platform, metric) pair.
* public/index.html:
(fetchTest): Updated per the format change in runs JSON.
* public/v2/app.js:
(App.Pane._fetch): Fetch the cached JSON first. Refetch the uncached version if instructed as such.
(App.Pane._updateChartData): Extracted from App.Pane._fetch.
(App.Pane._handleFetchErrors): Ditto.
* public/v2/data.js:
(RunsData.fetchRuns): Takes the fourth argument indicating whether we should fetch the cached version or not.
The cached JSON is located in /data/ with the same filename. When fetching a cached JSON results in 404,
fulfill the promise with null as the result instead of rejecting it. The only client of this function which
sets useCache to true is App.Manifest.fetchRunsWithPlatformAndMetric, and it handles this special case.
* public/v2/manifest.js:
(App.DateArrayTransform): Added. Handles the array of last modified dates in platform objects.
(App.Platform.lastModifiedTimeForMetric): Added. Returns the last modified date in the manifest JSON.
(App.Manifest.fetchRunsWithPlatformAndMetric): Takes "useCache" like RunsData.fetchRuns. Set shouldRefetch
to true if response is null (the cache didn't exit) or the cache is out-of-date.
(App.Manifest._formatFetchedData): Extracted from App.Manifest.fetchRunsWithPlatformAndMetric.
* run-tests.js:
(initializeDatabase): Avoid splitting function definitions in the middle.
* tests/api-report.js: Added tests to verify that reporting new test results updates the last modified time
in test_configurations.
Canonical link: https://commits.webkit.org/159898@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@180468 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-02-21 01:28:34 +00:00
|
|
|
CREATE OR REPLACE FUNCTION update_config_last_modified() RETURNS TRIGGER AS $update_config_last_modified$
|
|
|
|
BEGIN
|
|
|
|
IF TG_OP != 'DELETE' THEN
|
|
|
|
UPDATE test_configurations SET config_runs_last_modified = (CURRENT_TIMESTAMP AT TIME ZONE 'UTC') WHERE config_id = NEW.run_config;
|
|
|
|
ELSE
|
|
|
|
UPDATE test_configurations SET config_runs_last_modified = (CURRENT_TIMESTAMP AT TIME ZONE 'UTC') WHERE config_id = OLD.run_config;
|
|
|
|
END IF;
|
|
|
|
RETURN NULL;
|
|
|
|
END;
|
|
|
|
$update_config_last_modified$ LANGUAGE plpgsql;
|
|
|
|
|
|
|
|
CREATE TRIGGER update_config_last_modified AFTER INSERT OR UPDATE OR DELETE ON test_runs
|
|
|
|
FOR EACH ROW EXECUTE PROCEDURE update_config_last_modified();
|
|
|
|
|
2014-02-08 03:11:53 +00:00
|
|
|
CREATE TABLE reports (
|
|
|
|
report_id serial PRIMARY KEY,
|
|
|
|
report_builder integer NOT NULL REFERENCES builders ON DELETE RESTRICT,
|
2021-03-22 21:23:38 +00:00
|
|
|
report_worker integer REFERENCES build_workers ON DELETE RESTRICT,
|
2019-10-24 22:00:37 +00:00
|
|
|
report_build_tag varchar(64),
|
2014-02-08 03:11:53 +00:00
|
|
|
report_build integer REFERENCES builds,
|
|
|
|
report_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
|
|
|
|
report_committed_at timestamp,
|
|
|
|
report_content text,
|
|
|
|
report_failure varchar(64),
|
|
|
|
report_failure_details text);
|
Perf dashboard should provide a way to associate bugs with a test run
https://bugs.webkit.org/show_bug.cgi?id=137857
Reviewed by Andreas Kling.
Added a "privileged" API, /privileged-api/associate-bug, to associate a bug with a test run.
/privileged-api/ is to be protected by an authentication mechanism such as DigestAuth over https by
the Apache configuration.
The Cross Site Request (CSRF) Forgery prevention for privileged APIs work as follows. When a user is
about to make a privileged API access, the front end code obtains a CSRF token generated by POST'ing
to privileged-api/generate-csrf-token; the page sets a randomly generated salt and an expiration time
via the cookie and returns a token computed from those two values as well as the remote username.
The font end code then POST's the request along with the returned token. The server side code verifies
that the specified token can be generated from the salt and the expiration time set in the cookie, and
the token hasn't expired.
* init-database.sql: Added bug_url to bug_trackers table, and added bugs table. Each bug tracker will
have zero or exactly one bug associated with a test run.
* public/admin/bug-trackers.php: Added the support for editing bug_url.
* public/api/runs.php:
(fetch_runs_for_config): Modified the query to fetch bugs associated with test_runs.
(parse_bugs_array): Added. Parses the aggregated bugs and creates a dictionary that maps a tracker id to
an associated bug if there is one.
(format_run): Calls parse_bugs_array.
* public/include/json-header.php: Added helper functions to deal for CSRF prevention.
(ensure_privileged_api_data): Added. Dies immediately if the request's method is not POST or doesn't
have a valid JSON payload.
(ensure_privileged_api_data_and_token): Ditto. Also checks that the CSRF prevention token is valid.
(compute_token): Computes a CSRF token using the REMOTE_USER (e.g. set via BasicAuth), the salt, and
the expiration time stored in the cookie.
(verify_token): Returns true iff the specified token matches what compute_token returns from the cookie.
* public/include/manifest.php:
(ManifestGenerator::bug_trackers): Include bug_url as bugUrl in the manifest. Also use tracker_id instead
of tracker_name as the key in the manifest. This requires changes to both v1 and v2 front end.
* public/index.html:
(Chart..showTooltipWithResults): Updated for the manifest format changed mentioned above.
* public/privileged-api/associate-bug.php: Added.
(main): Added. Associates or dissociates a bug with a test run inside a transaction. It prevent a CSRF
attack via ensure_privileged_api_data_and_token, which calls verify_token.
* public/privileged-api/generate-csrf-token.php: Added. Generates a CSRF token valid for one hour.
* public/v2/app.css:
(.disabled .icon-button:hover g): Used by the "bugs" icon when a range of points or no points are
selected in a chart.
* public/v2/app.js:
(App.PaneController.actions.toggleBugsPane): Added. Toggles the visibility of the bugs pane when exactly
one point is selected in the chart. Also hides the search pane when making the bugs pane visible since
they would overlap on each other if both of them are shown.
(App.PaneController.actions.associateBug): Makes a privileged API request to associate the specified bug
with the currently selected point (test run). Updates the bug information in "details" and colors of dots
in the charts to reflect new states. Because chart data objects aren't real Ember objects for performance
reasons, we have to use a dirty hack of modifying a dummy counter bugsChangeCount.
(App.PaneController.actions.toggleSearchPane): Renamed from toggleSearch. Also hides the bugs pane when
showing the search pane.
(App.PaneController.actions.rangeChanged): Takes all selected points as the second argument instead of
taking start and end points as the second and the third arguments so that _showDetails can enumerate all
bugs in the selected range.
(App.PaneController._detailsChanged): Added. Hide the bugs pane whenever a new point is selected.
Also update singlySelectedPoint, which is used by toggleBugsPane and associateBug.
(App.PaneController._currentItemChanged): Updated for the _showDetails change.
(App.PaneController._showDetails): Takes an array of selected points in place of old arguments.
Simplified the code to compute the revision information. Calls _updateBugs to format the associated bugs.
(App.PaneController._updateBugs): Sets details.bugTrackers to a dictionary that maps a bug tracker id to
a bug tracker proxy with an array of (bugNumber, bugUrl) pairs and also editedBugNumber, which is used by
the bugs pane to associate or dissociate a bug number, if exactly one point is selected.
(App.InteractiveChartComponent._updateDotsWithBugs): Added. Sets hasBugs class on dots as needed.
(App.InteractiveChartComponent._setCurrentSelection): Finds and passes all points in the selected range
to selectionChanged action instead of just finding the first and the last points.
* public/v2/chart-pane.css: Updated the style.
* public/v2/data.js:
(PrivilegedAPI): Added. A wrapper for privileged APIs' CSRF tokens.
(PrivilegedAPI.sendRequest): Makes a privileged API call. Fetches a new CSRF token if needed.
(PrivilegedAPI._generateTokenInServerIfNeeded): Makes a request to privileged-api/generate-csrf-token if
we haven't already obtained a CSRF token or if the token has already been expired.
(PrivilegedAPI._post): Makes a single POST request to /privileged-api/* with a JSON payload.
(Measurement.prototype.bugs): Added.
(Measurement.prototype.hasBugs): Returns true iff bugs has more than one bug number.
(Measurement.prototype.associateBug): Associates a bug with a test run via privileged-api/associate-bug.
* public/v2/index.html: Added the bugs pane. Also added a list of bugs associated with the current run in
the details.
* public/v2/manifest.js:
(App.BugTracker.bugUrl):
(App.BugTracker.newBugUrl): Added.
(App.BugTracker.repositories): Added. This was a missing back reference to repositories.
(App.MetricSerializer.normalizePayload): Now parses/loads the list of bug trackers from the manifest.
(App.Manifest.repositoriesWithReportedCommits): Now initialized to an empty array instead of null.
(App.Manifest.bugTrackers): Added.
(App.Manifest._fetchedManifest): Sets App.Manifest.bugTrackers. Also sorts the list of repositories by
their respective ids to make the ordering stable.
Canonical link: https://commits.webkit.org/155807@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-22 00:53:39 +00:00
|
|
|
|
Perf dashboard should automatically detect regressions
https://bugs.webkit.org/show_bug.cgi?id=141443
Reviewed by Anders Carlsson.
Added a node.js script detect-changes.js to detect potential regressions and progressions
on the graphs tracked on v2 dashboards.
* init-database.sql: Added analysis_strategies table and task_segmentation and task_test_range
columns to analysis_tasks to keep the segmentation and test range selection strategies used
to create an analysis task.
* public/api/analysis-tasks.php:
(format_task): Include task_segmentation and analysis_tasks in the results.
* public/include/json-header.php:
(remote_user_name): Returns null when the privileged API is authenticated as a slave instead
of a CSRF prevention token.
(should_authenticate_as_slave): Added.
(ensure_privileged_api_data_and_token_or_slave): Added. Authenticate as a slave if slaveName
and slavePassword are specified. Since detect-changes.js and other slaves are not susceptible
to a CSRF attack, we don't need to check a CSRF token.
* public/privileged-api/create-analysis-task.php:
(main): Use ensure_privileged_api_data_and_token_or_slave to let detect-changes.js create new
analysis task. Also add or find segmentation and test range selection strategies if specified.
* public/privileged-api/create-test-group.php:
(main): Use ensure_privileged_api_data_and_token_or_slave.
* public/privileged-api/generate-csrf-token.php:
* public/v2/app.js:
(App.Pane._computeMovingAverageAndOutliers): _executeStrategy has been moved to Statistics.
* public/v2/data.js: Export Measurement, RunsData, TimeSeries. Used in detect-changes.js.
(Array.prototype.find): Added a polyfill to be used in node.js.
(RunsData.fetchRuns):
(RunsData.pathForFetchingRuns): Extracted from fetchRuns. Used in detect-changes.js.
(RunsData.createRunsDataInResponse): Extracted from App.Manifest._formatFetchedData to use it
in detect-changes.js.
(RunsData.unitFromMetricName): Ditto.
(RunsData.isSmallerBetter): Ditto.
(RunsData.prototype._timeSeriesByTimeInternal): Added secondaryTime to sort points when commit
times are identical.
(TimeSeries): When commit times are identical, order points based on build time. This is needed
for when we trigger two builds at two different OS versions with the same WebKit revision since
OS versions don't change the commit times.
(TimeSeries.prototype.findPointByIndex): Added.
(TimeSeries.prototype.rawValues): Added.
* public/v2/js/statistics.js:
(Statistics.TestRangeSelectionStrategies.[0]): Use the 99% two-sided probability as claimed in the
description of this strategy instead of the default probability. Also fixed a bug that debugging
code was referring to non-existent variables.
(Statistics.executeStrategy): Moved from App.Pane (app.js).
* public/v2/manifest.js:
(App.Manifest._formatFetchedData): Various code has been extracted into RunsData in data.js to be
used in detect-changes.js.
* tools/detect-changes.js: Added. The script fetches the manifest JSON, analyzes each graph in
the v2 dashboards, and creates an analysis task for the latest regression or progression detected.
It also schedules an A/B testing if possible and notifies another server; e.g. to send an email.
(main): Loads the settings JSON specified in the argument.
(fetchManifestAndAnalyzeData): The main loop that periodically wakes up to do the analysis.
(mapInOrder): Executes callback sequentially (i.e. blocking) on each item in the array.
(configurationsForTesting): Finds every (platform, metric) pair to analyze in the v2 dashbaords,
and computes various values for when statistically significant changes are detected later.
(analyzeConfiguration): Finds potential regressions and progression in the last X days where X
is the specified maximum number of days using the specified strategies. Sort the resultant ranges
in chronological order and create a new analysis task for the very last change we detected. We'll
eventually create an analysis task for all detected changes since we're repeating the analysis in
fetchManifestAndAnalyzeData after some time.
(computeRangesForTesting): Fetch measured values and compute ranges to test using the specified
segmentation and test range selection strategies. Once ranges are found, find overlapping analysis
tasks as they need to be filtered out in analyzeConfiguration to avoid creating multiple analysis
tasks for the same range (e.g. humans may create one before the script gets to do it).
(createAnalysisTaskAndNotify): Create a new analysis task for the specified range, trigger an A/B
testing if available, and notify another server with a HTML message as specified.
(findStrategyByLabel):
(changeTypeForRange): A change is a regression if values are getting larger in a smaller-is-better
test or values are getting smaller in a larger-is-better test and vice versa.
(summarizeRange): Create a human readable string that summarizes the change detected. e.g.
"Potential 3.2% regression detected between 2015-04-20 12:00 and 17:00".
(formatTimeRange):
(getJSON):
(postJSON):
(postNotification): Recursively replaces $title and $massage in the specified JSON template.
(instantiateNotificationTemplate):
(fetchJSON):
Canonical link: https://commits.webkit.org/162105@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@183232 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-04-24 01:16:37 +00:00
|
|
|
CREATE TABLE analysis_strategies (
|
|
|
|
strategy_id serial PRIMARY KEY,
|
|
|
|
strategy_name varchar(64) NOT NULL);
|
|
|
|
|
2015-04-21 02:48:22 +00:00
|
|
|
CREATE TYPE analysis_task_result_type as ENUM ('progression', 'regression', 'unchanged', 'inconclusive');
|
2014-11-07 23:47:09 +00:00
|
|
|
CREATE TABLE analysis_tasks (
|
|
|
|
task_id serial PRIMARY KEY,
|
|
|
|
task_name varchar(256) NOT NULL,
|
|
|
|
task_author varchar(256),
|
Perf dashboard should automatically detect regressions
https://bugs.webkit.org/show_bug.cgi?id=141443
Reviewed by Anders Carlsson.
Added a node.js script detect-changes.js to detect potential regressions and progressions
on the graphs tracked on v2 dashboards.
* init-database.sql: Added analysis_strategies table and task_segmentation and task_test_range
columns to analysis_tasks to keep the segmentation and test range selection strategies used
to create an analysis task.
* public/api/analysis-tasks.php:
(format_task): Include task_segmentation and analysis_tasks in the results.
* public/include/json-header.php:
(remote_user_name): Returns null when the privileged API is authenticated as a slave instead
of a CSRF prevention token.
(should_authenticate_as_slave): Added.
(ensure_privileged_api_data_and_token_or_slave): Added. Authenticate as a slave if slaveName
and slavePassword are specified. Since detect-changes.js and other slaves are not susceptible
to a CSRF attack, we don't need to check a CSRF token.
* public/privileged-api/create-analysis-task.php:
(main): Use ensure_privileged_api_data_and_token_or_slave to let detect-changes.js create new
analysis task. Also add or find segmentation and test range selection strategies if specified.
* public/privileged-api/create-test-group.php:
(main): Use ensure_privileged_api_data_and_token_or_slave.
* public/privileged-api/generate-csrf-token.php:
* public/v2/app.js:
(App.Pane._computeMovingAverageAndOutliers): _executeStrategy has been moved to Statistics.
* public/v2/data.js: Export Measurement, RunsData, TimeSeries. Used in detect-changes.js.
(Array.prototype.find): Added a polyfill to be used in node.js.
(RunsData.fetchRuns):
(RunsData.pathForFetchingRuns): Extracted from fetchRuns. Used in detect-changes.js.
(RunsData.createRunsDataInResponse): Extracted from App.Manifest._formatFetchedData to use it
in detect-changes.js.
(RunsData.unitFromMetricName): Ditto.
(RunsData.isSmallerBetter): Ditto.
(RunsData.prototype._timeSeriesByTimeInternal): Added secondaryTime to sort points when commit
times are identical.
(TimeSeries): When commit times are identical, order points based on build time. This is needed
for when we trigger two builds at two different OS versions with the same WebKit revision since
OS versions don't change the commit times.
(TimeSeries.prototype.findPointByIndex): Added.
(TimeSeries.prototype.rawValues): Added.
* public/v2/js/statistics.js:
(Statistics.TestRangeSelectionStrategies.[0]): Use the 99% two-sided probability as claimed in the
description of this strategy instead of the default probability. Also fixed a bug that debugging
code was referring to non-existent variables.
(Statistics.executeStrategy): Moved from App.Pane (app.js).
* public/v2/manifest.js:
(App.Manifest._formatFetchedData): Various code has been extracted into RunsData in data.js to be
used in detect-changes.js.
* tools/detect-changes.js: Added. The script fetches the manifest JSON, analyzes each graph in
the v2 dashboards, and creates an analysis task for the latest regression or progression detected.
It also schedules an A/B testing if possible and notifies another server; e.g. to send an email.
(main): Loads the settings JSON specified in the argument.
(fetchManifestAndAnalyzeData): The main loop that periodically wakes up to do the analysis.
(mapInOrder): Executes callback sequentially (i.e. blocking) on each item in the array.
(configurationsForTesting): Finds every (platform, metric) pair to analyze in the v2 dashbaords,
and computes various values for when statistically significant changes are detected later.
(analyzeConfiguration): Finds potential regressions and progression in the last X days where X
is the specified maximum number of days using the specified strategies. Sort the resultant ranges
in chronological order and create a new analysis task for the very last change we detected. We'll
eventually create an analysis task for all detected changes since we're repeating the analysis in
fetchManifestAndAnalyzeData after some time.
(computeRangesForTesting): Fetch measured values and compute ranges to test using the specified
segmentation and test range selection strategies. Once ranges are found, find overlapping analysis
tasks as they need to be filtered out in analyzeConfiguration to avoid creating multiple analysis
tasks for the same range (e.g. humans may create one before the script gets to do it).
(createAnalysisTaskAndNotify): Create a new analysis task for the specified range, trigger an A/B
testing if available, and notify another server with a HTML message as specified.
(findStrategyByLabel):
(changeTypeForRange): A change is a regression if values are getting larger in a smaller-is-better
test or values are getting smaller in a larger-is-better test and vice versa.
(summarizeRange): Create a human readable string that summarizes the change detected. e.g.
"Potential 3.2% regression detected between 2015-04-20 12:00 and 17:00".
(formatTimeRange):
(getJSON):
(postJSON):
(postNotification): Recursively replaces $title and $massage in the specified JSON template.
(instantiateNotificationTemplate):
(fetchJSON):
Canonical link: https://commits.webkit.org/162105@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@183232 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-04-24 01:16:37 +00:00
|
|
|
task_segmentation integer REFERENCES analysis_strategies,
|
|
|
|
task_test_range integer REFERENCES analysis_strategies,
|
2014-11-07 23:47:09 +00:00
|
|
|
task_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
|
Add the UI for scheduling a A/B testing with a custom root
https://bugs.webkit.org/show_bug.cgi?id=170622
Reviewed by Anders Carlsson.
This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch
would update the syncing script to schedule such an A/B testing job to a buildbot instance.
* ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace
the file path for the backup.
* init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column
to store the root file in commit_set_relationships.
* public/api/build-requests.php:
(main): Include the uploaded files.
* public/api/commits.php:
(main): Added the support for querying the latest commits for a given platform. This is used in a new page
to create a custom analysis task to autofill the latest revisions for a given platform.
* public/api/test-groups.php:
(main): Include the uploaded files.
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id.
(BuildRequestsFetcher::uploaded_files): Added.
(BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in
each commit set, and inserting its meta data in the list of uplaoded files.
* public/include/commit-log-fetcher.php:
(CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally,
we should be finding the latest commit for a given platform, but this is very slow so instead find the commit
of the latest build for a given platform.
* public/privileged-api/create-test-group.php:
(main): Added the support for creating an analysis task along with a group.
(commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists
and include it in commit_set_relationships. Because commits and upload files are stored in a different column
in commit_set_relationships, this function now stores the information for each row of commit_set_relationships
except the commit set ID, which is unknown until the set is created, instead of a commit ID.
(ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format
is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here.
* public/privileged-api/upload-file.php:
(main): Fixed a typo. Also added one more error check.
* public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform,
and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with
custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class.
(CustomAnalysisTaskConfigurator):
(CustomAnalysisTaskConfigurator.prototype.tests): Added.
(CustomAnalysisTaskConfigurator.prototype.platform): Added.
(CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both
have been configured by the user.
(CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added.
(CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the
"comparison" configuration.
(CustomAnalysisTaskConfigurator.prototype.render): Added.
(CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests
that can be scheduled by a triggerable.
(CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms
that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only
lets the user select a single test but the intent is to allow multiple tests to be selected in the near future.
(CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select
an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to
render both the list of tests and platforms.
(CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of
tests and platforms. Returns an error when some tests belong to another triggearalbe.
(CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when
the current triggerable has changed. We try to use the repository group of the same name if there is any, and
defaults to the first repository group if there is none. This allows the set of repositories to be specified to
more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two
distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the
other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to
specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform
to a Mac platform.
(CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that
TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set
if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical.
(CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions
and the csutom roots the user had specified.
(CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions
and custom roots for "baseline" and "comparison".
(CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added.
(CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added.
(CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions
and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are
shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional.
(CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added.
(CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify
a revision for a given repository. Autofills it with the latest commit for the currently selected platform if
the user had not modified the field by the time the revisions are fetched.
(CustomAnalysisTaskConfigurator.htmlTemplate): Added.
(CustomAnalysisTaskConfigurator.cssTemplate): Added.
* public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline"
or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a
file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash.
(InstantFileUploader):
(InstantFileUploader.prototype.hasFileToUpload): Added.
(InstantFileUploader.prototype.uploadedFiles): Added.
(InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison"
configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically
mirror the newly uploaded custom root to "comparision" configuration.
(InstantFileUploader.prototype.didConstructShadowTree): Added.
(InstantFileUploader.prototype.render): Added.
(InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files.
(InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with
a progress bar.
(InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file.
(InstantFileUploader.prototype._formatUploadError): Added.
(InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on
the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and
start uploading the file if there isn't one.
(InstantFileUploader.prototype._removeUploadedFile): Added.
(InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to
the list of uploaded files.
(InstantFileUploader.htmlTemplate): Added.
(InstantFileUploader.cssTemplate): Added.
* public/v3/index.html:
* public/v3/models/analysis-task.js:
(AnalysisTask): Made platform and metric optional as it is now.
(AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric.
(AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task.
(AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks.
(AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric
or a platform instead of silently skipping them.
* public/v3/models/build-request.js:
(BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests.
* public/v3/models/commit-log.js:
(CommitLog.fetchLatestCommitForPlatform): Added.
* public/v3/models/commit-set.js:
(CommitSet): Added this._customRoots.
(CommitSet.prototype.customRoots): Returns this._customRoots.
(CommitSet.prototype.equals): Returns false when the set of custom roots are not equal.
(CommitSet.areCustomRootsEqual): Added.
(CustomCommitSet):
(CustomCommitSet.prototype.equals): Added.
(CustomCommitSet.prototype.customRoots): Added.
(CustomCommitSet.prototype.addCustomRoot): Added.
* public/v3/models/manifest.js:
(Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit.
This allows a file size check in the client size instead of uploading it to the server and receiving an error.
* public/v3/models/metric.js:
(Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles.
* public/v3/models/test-group.js:
(TestGroup.prototype.createWithTask): Added.
(TestGroup.prototype.createAndRefetchTestGroups):
(TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups.
(TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups.
* public/v3/models/triggerable.js:
(Triggerable.triggerablePlatformsForTests): Added.
(Triggerable.sortByNamePreferringSmallerRepositories): Added.
* public/v3/models/uploaded-file.js:
(UploadedFile.prototype.createdAt): Added.
(UploadedFile.prototype.filename): Added.
(UploadedFile.prototype.author): Added.
(UploadedFile.prototype.size): Added.
(UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit.
(UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected
promise instead of a resolved promise with null.
* public/v3/pages/analysis-category-page.js:
(AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and
metric being null for some analysis tasks.
* public/v3/pages/analysis-task-page.js:
(AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks.
(AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks.
(AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks
since it's not associated with exactly one pair.
* public/v3/pages/chart-pane-status-view.js:
(ChartPaneStatusView.prototype._renderBuildRevisionTable):
(ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime.
* public/v3/pages/chart-pane.js:
(ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in
CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range.
* public/v3/pages/create-analysis-task-page.js:
(CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create
a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots.
(CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is
the old behavior of this page.
(CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added.
(CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added.
(CreateAnalysisTaskPage.prototype.render):
(CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select
element to specify the numebr of iterations when a message is set.
(CreateAnalysisTaskPage.htmlTemplate):
(CreateAnalysisTaskPage.cssTemplate):
* public/v3/pages/page-router.js:
(PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed.
* server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of
uploaded files associated with build requests.
* server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task
and a test group with custom roots.
* server-tests/resources/mock-data.js:
(MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table.
* tools/js/remote.js: Include global.FormData from form-data.js.
* unit-tests/build-request-tests.js:
(sampleBuildRequestData): Updated the mock response.
* unit-tests/buildbot-syncer-tests.js:
(createSampleBuildRequest): Ditto.
* unit-tests/test-groups-tests.js:
(sampleTestGroup): Ditto.
Canonical link: https://commits.webkit.org/187631@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
|
|
|
task_platform integer REFERENCES platforms,
|
|
|
|
task_metric integer REFERENCES test_metrics,
|
2014-11-07 23:47:09 +00:00
|
|
|
task_start_run integer REFERENCES test_runs,
|
2016-01-05 22:49:04 +00:00
|
|
|
task_start_run_time timestamp,
|
2014-11-07 23:47:09 +00:00
|
|
|
task_end_run integer REFERENCES test_runs,
|
2016-01-05 22:49:04 +00:00
|
|
|
task_end_run_time timestamp,
|
2015-04-21 02:48:22 +00:00
|
|
|
task_result analysis_task_result_type,
|
|
|
|
task_needed boolean,
|
2014-11-22 04:25:19 +00:00
|
|
|
CONSTRAINT analysis_task_should_be_unique_for_range UNIQUE(task_start_run, task_end_run),
|
Add the UI for scheduling a A/B testing with a custom root
https://bugs.webkit.org/show_bug.cgi?id=170622
Reviewed by Anders Carlsson.
This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch
would update the syncing script to schedule such an A/B testing job to a buildbot instance.
* ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace
the file path for the backup.
* init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column
to store the root file in commit_set_relationships.
* public/api/build-requests.php:
(main): Include the uploaded files.
* public/api/commits.php:
(main): Added the support for querying the latest commits for a given platform. This is used in a new page
to create a custom analysis task to autofill the latest revisions for a given platform.
* public/api/test-groups.php:
(main): Include the uploaded files.
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id.
(BuildRequestsFetcher::uploaded_files): Added.
(BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in
each commit set, and inserting its meta data in the list of uplaoded files.
* public/include/commit-log-fetcher.php:
(CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally,
we should be finding the latest commit for a given platform, but this is very slow so instead find the commit
of the latest build for a given platform.
* public/privileged-api/create-test-group.php:
(main): Added the support for creating an analysis task along with a group.
(commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists
and include it in commit_set_relationships. Because commits and upload files are stored in a different column
in commit_set_relationships, this function now stores the information for each row of commit_set_relationships
except the commit set ID, which is unknown until the set is created, instead of a commit ID.
(ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format
is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here.
* public/privileged-api/upload-file.php:
(main): Fixed a typo. Also added one more error check.
* public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform,
and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with
custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class.
(CustomAnalysisTaskConfigurator):
(CustomAnalysisTaskConfigurator.prototype.tests): Added.
(CustomAnalysisTaskConfigurator.prototype.platform): Added.
(CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both
have been configured by the user.
(CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added.
(CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the
"comparison" configuration.
(CustomAnalysisTaskConfigurator.prototype.render): Added.
(CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests
that can be scheduled by a triggerable.
(CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms
that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only
lets the user select a single test but the intent is to allow multiple tests to be selected in the near future.
(CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select
an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to
render both the list of tests and platforms.
(CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of
tests and platforms. Returns an error when some tests belong to another triggearalbe.
(CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when
the current triggerable has changed. We try to use the repository group of the same name if there is any, and
defaults to the first repository group if there is none. This allows the set of repositories to be specified to
more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two
distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the
other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to
specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform
to a Mac platform.
(CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that
TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set
if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical.
(CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions
and the csutom roots the user had specified.
(CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions
and custom roots for "baseline" and "comparison".
(CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added.
(CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added.
(CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions
and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are
shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional.
(CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added.
(CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify
a revision for a given repository. Autofills it with the latest commit for the currently selected platform if
the user had not modified the field by the time the revisions are fetched.
(CustomAnalysisTaskConfigurator.htmlTemplate): Added.
(CustomAnalysisTaskConfigurator.cssTemplate): Added.
* public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline"
or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a
file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash.
(InstantFileUploader):
(InstantFileUploader.prototype.hasFileToUpload): Added.
(InstantFileUploader.prototype.uploadedFiles): Added.
(InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison"
configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically
mirror the newly uploaded custom root to "comparision" configuration.
(InstantFileUploader.prototype.didConstructShadowTree): Added.
(InstantFileUploader.prototype.render): Added.
(InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files.
(InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with
a progress bar.
(InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file.
(InstantFileUploader.prototype._formatUploadError): Added.
(InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on
the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and
start uploading the file if there isn't one.
(InstantFileUploader.prototype._removeUploadedFile): Added.
(InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to
the list of uploaded files.
(InstantFileUploader.htmlTemplate): Added.
(InstantFileUploader.cssTemplate): Added.
* public/v3/index.html:
* public/v3/models/analysis-task.js:
(AnalysisTask): Made platform and metric optional as it is now.
(AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric.
(AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task.
(AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks.
(AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric
or a platform instead of silently skipping them.
* public/v3/models/build-request.js:
(BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests.
* public/v3/models/commit-log.js:
(CommitLog.fetchLatestCommitForPlatform): Added.
* public/v3/models/commit-set.js:
(CommitSet): Added this._customRoots.
(CommitSet.prototype.customRoots): Returns this._customRoots.
(CommitSet.prototype.equals): Returns false when the set of custom roots are not equal.
(CommitSet.areCustomRootsEqual): Added.
(CustomCommitSet):
(CustomCommitSet.prototype.equals): Added.
(CustomCommitSet.prototype.customRoots): Added.
(CustomCommitSet.prototype.addCustomRoot): Added.
* public/v3/models/manifest.js:
(Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit.
This allows a file size check in the client size instead of uploading it to the server and receiving an error.
* public/v3/models/metric.js:
(Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles.
* public/v3/models/test-group.js:
(TestGroup.prototype.createWithTask): Added.
(TestGroup.prototype.createAndRefetchTestGroups):
(TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups.
(TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups.
* public/v3/models/triggerable.js:
(Triggerable.triggerablePlatformsForTests): Added.
(Triggerable.sortByNamePreferringSmallerRepositories): Added.
* public/v3/models/uploaded-file.js:
(UploadedFile.prototype.createdAt): Added.
(UploadedFile.prototype.filename): Added.
(UploadedFile.prototype.author): Added.
(UploadedFile.prototype.size): Added.
(UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit.
(UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected
promise instead of a resolved promise with null.
* public/v3/pages/analysis-category-page.js:
(AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and
metric being null for some analysis tasks.
* public/v3/pages/analysis-task-page.js:
(AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks.
(AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks.
(AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks
since it's not associated with exactly one pair.
* public/v3/pages/chart-pane-status-view.js:
(ChartPaneStatusView.prototype._renderBuildRevisionTable):
(ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime.
* public/v3/pages/chart-pane.js:
(ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in
CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range.
* public/v3/pages/create-analysis-task-page.js:
(CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create
a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots.
(CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is
the old behavior of this page.
(CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added.
(CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added.
(CreateAnalysisTaskPage.prototype.render):
(CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select
element to specify the numebr of iterations when a message is set.
(CreateAnalysisTaskPage.htmlTemplate):
(CreateAnalysisTaskPage.cssTemplate):
* public/v3/pages/page-router.js:
(PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed.
* server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of
uploaded files associated with build requests.
* server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task
and a test group with custom roots.
* server-tests/resources/mock-data.js:
(MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table.
* tools/js/remote.js: Include global.FormData from form-data.js.
* unit-tests/build-request-tests.js:
(sampleBuildRequestData): Updated the mock response.
* unit-tests/buildbot-syncer-tests.js:
(createSampleBuildRequest): Ditto.
* unit-tests/test-groups-tests.js:
(sampleTestGroup): Ditto.
Canonical link: https://commits.webkit.org/187631@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
|
|
|
CONSTRAINT analysis_task_must_be_associated_with_run_or_be_custom
|
|
|
|
CHECK ((task_start_run IS NULL AND task_start_run_time IS NULL
|
|
|
|
AND task_end_run IS NULL AND task_end_run_time IS NULL
|
|
|
|
AND task_platform IS NULL AND task_metric IS NULL)
|
|
|
|
OR (task_start_run IS NOT NULL AND task_start_run_time IS NOT NULL
|
|
|
|
AND task_end_run IS NOT NULL AND task_end_run_time IS NOT NULL
|
|
|
|
AND task_platform IS NOT NULL AND task_metric IS NOT NULL)));
|
2014-11-20 23:25:07 +00:00
|
|
|
|
2016-03-16 07:02:28 +00:00
|
|
|
CREATE TABLE task_commits (
|
|
|
|
taskcommit_task integer NOT NULL REFERENCES analysis_tasks ON DELETE CASCADE,
|
|
|
|
taskcommit_commit integer NOT NULL REFERENCES commits ON DELETE CASCADE,
|
2016-03-17 20:48:28 +00:00
|
|
|
taskcommit_is_fix boolean NOT NULL,
|
2016-03-16 07:02:28 +00:00
|
|
|
CONSTRAINT task_commit_must_be_unique UNIQUE(taskcommit_task, taskcommit_commit));
|
|
|
|
|
2014-11-20 23:25:07 +00:00
|
|
|
CREATE TABLE bugs (
|
|
|
|
bug_id serial PRIMARY KEY,
|
|
|
|
bug_task integer REFERENCES analysis_tasks NOT NULL,
|
|
|
|
bug_tracker integer REFERENCES bug_trackers NOT NULL,
|
2015-05-20 21:52:35 +00:00
|
|
|
bug_number integer NOT NULL);
|
2014-11-07 23:47:09 +00:00
|
|
|
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
CREATE TABLE build_triggerables (
|
|
|
|
triggerable_id serial PRIMARY KEY,
|
Introduce the notion of repository groups to triggerables
https://bugs.webkit.org/show_bug.cgi?id=170228
Reviewed by Chris Dumez.
On some triggerable, it's desirable to specify multiple sets of repositories that are accepted.
For example, if a repository X transitioned from Subversion to Git, and if a triggerable accepted X and
some other repository Y, then it's desirable to two sets: (X-Subversion, Y) and (X-Git, Y) since neither
(X-Subversion, X-Git) nor (X-Subversion, X-Git, Y) makes sense as a set.
This patch introduces triggerable_repository_groups table to represent a set of repositories accepted by
a triggerable. It has many to one relationship to build_triggerables and triggerable_repositories in turn
now has many to one relationship to triggerable_repository_groups instead of build_triggerables.
Also make it possible to disable a triggerable e.g. a set of tests and platforms are no longer supported.
We don't want to delete the triggerable completely from the database since it would result in the associated
A/B testing results being purged, which is not desirale.
To migrate an existing database, run the following transaction:
```sql
BEGIN;
ALTER TABLE build_triggerables ADD COLUMN triggerable_disabled boolean NOT NULL DEFAULT FALSE;
CREATE TABLE triggerable_repository_groups (
repositorygroup_id serial PRIMARY KEY,
repositorygroup_triggerable integer REFERENCES build_triggerables NOT NULL,
repositorygroup_name varchar(256) NOT NULL,
repositorygroup_description varchar(256),
repositorygroup_accepts_roots boolean NOT NULL DEFAULT FALSE,
CONSTRAINT repository_group_name_must_be_unique_for_triggerable
UNIQUE(repositorygroup_triggerable, repositorygroup_name));
INSERT INTO triggerable_repository_groups (repositorygroup_triggerable, repositorygroup_name)
SELECT triggerable_id, 'default' FROM build_triggerables;
ALTER TABLE triggerable_repositories ADD COLUMN trigrepo_group integer REFERENCES triggerable_repository_groups;
UPDATE triggerable_repositories SET trigrepo_group = repositorygroup_id FROM triggerable_repository_groups
WHERE trigrepo_triggerable = repositorygroup_triggerable;
ALTER TABLE triggerable_repositories ALTER COLUMN trigrepo_group SET NOT NULL;
ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_triggerable;
ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_sub_roots;
END;
```
* init-database.sql:
* public/admin/triggerables.php: Use a custom column to make forms to add and configure repository groups.
(insert_triggerable_repositories): Added.
(generate_repository_list): Added.
(generate_repository_form): Added.
(generate_repository_checkboxes): Now generates checkboxes for a repository group instead of a triggerable.
* public/include/manifest-generator.php:
(fetch_triggerables): Fixed the bug that we were not filtering results with query in /api/triggerable.
Rewrote it to include an array of repository groups, which in turn contains an array of repositories along
with its name and a description, and a boolean indicating whether it accepts a custom root file or not.
The boolean will be used when we're adding the support for perf try bots. We will keep acceptedRepositories
since it's still used by detect-changes.js.
* public/v3/models/manifest.js:
(Manifest._didFetchManifest): Resolve repositoriy, test, and platform IDs to their respective objects.
* public/v3/models/triggerable.js:
(Triggerable):
(Triggerable.prototype.isDisabled): Added.
(Triggerable.prototype.repositoryGroups): Added.
(Triggerable.prototype.acceptsTest): Added.
(TriggerableRepositoryGroup): Added.
(TriggerableRepositoryGroup.prototype.description): Added.
(TriggerableRepositoryGroup.prototype.acceptsCustomRoots): Added.
(TriggerableRepositoryGroup.prototype.repositories): Added.
* public/v3/pages/analysis-task-page.js:
(AnalysisTaskPage.prototype._didFetchTask): Don't use a disabled triggerable.
* server-tests/api-manifest-tests.js: Updated a test case to test repository groups.
* tools/js/database.js:
(tableToPrefixMap): Added triggerable_repository_groups.
* tools/js/v3-models.js: Imported TriggerableRepositoryGroup from triggerable.js.
Canonical link: https://commits.webkit.org/187454@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214975 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-05 23:12:46 +00:00
|
|
|
triggerable_name varchar(64) NOT NULL UNIQUE,
|
|
|
|
triggerable_disabled boolean NOT NULL DEFAULT FALSE);
|
|
|
|
|
|
|
|
CREATE TABLE triggerable_repository_groups (
|
|
|
|
repositorygroup_id serial PRIMARY KEY,
|
|
|
|
repositorygroup_triggerable integer REFERENCES build_triggerables NOT NULL,
|
|
|
|
repositorygroup_name varchar(256) NOT NULL,
|
|
|
|
repositorygroup_description varchar(256),
|
|
|
|
repositorygroup_accepts_roots boolean NOT NULL DEFAULT FALSE,
|
2018-04-16 06:58:36 +00:00
|
|
|
repositorygroup_hidden boolean NOT NULL DEFAULT FALSE,
|
Introduce the notion of repository groups to triggerables
https://bugs.webkit.org/show_bug.cgi?id=170228
Reviewed by Chris Dumez.
On some triggerable, it's desirable to specify multiple sets of repositories that are accepted.
For example, if a repository X transitioned from Subversion to Git, and if a triggerable accepted X and
some other repository Y, then it's desirable to two sets: (X-Subversion, Y) and (X-Git, Y) since neither
(X-Subversion, X-Git) nor (X-Subversion, X-Git, Y) makes sense as a set.
This patch introduces triggerable_repository_groups table to represent a set of repositories accepted by
a triggerable. It has many to one relationship to build_triggerables and triggerable_repositories in turn
now has many to one relationship to triggerable_repository_groups instead of build_triggerables.
Also make it possible to disable a triggerable e.g. a set of tests and platforms are no longer supported.
We don't want to delete the triggerable completely from the database since it would result in the associated
A/B testing results being purged, which is not desirale.
To migrate an existing database, run the following transaction:
```sql
BEGIN;
ALTER TABLE build_triggerables ADD COLUMN triggerable_disabled boolean NOT NULL DEFAULT FALSE;
CREATE TABLE triggerable_repository_groups (
repositorygroup_id serial PRIMARY KEY,
repositorygroup_triggerable integer REFERENCES build_triggerables NOT NULL,
repositorygroup_name varchar(256) NOT NULL,
repositorygroup_description varchar(256),
repositorygroup_accepts_roots boolean NOT NULL DEFAULT FALSE,
CONSTRAINT repository_group_name_must_be_unique_for_triggerable
UNIQUE(repositorygroup_triggerable, repositorygroup_name));
INSERT INTO triggerable_repository_groups (repositorygroup_triggerable, repositorygroup_name)
SELECT triggerable_id, 'default' FROM build_triggerables;
ALTER TABLE triggerable_repositories ADD COLUMN trigrepo_group integer REFERENCES triggerable_repository_groups;
UPDATE triggerable_repositories SET trigrepo_group = repositorygroup_id FROM triggerable_repository_groups
WHERE trigrepo_triggerable = repositorygroup_triggerable;
ALTER TABLE triggerable_repositories ALTER COLUMN trigrepo_group SET NOT NULL;
ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_triggerable;
ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_sub_roots;
END;
```
* init-database.sql:
* public/admin/triggerables.php: Use a custom column to make forms to add and configure repository groups.
(insert_triggerable_repositories): Added.
(generate_repository_list): Added.
(generate_repository_form): Added.
(generate_repository_checkboxes): Now generates checkboxes for a repository group instead of a triggerable.
* public/include/manifest-generator.php:
(fetch_triggerables): Fixed the bug that we were not filtering results with query in /api/triggerable.
Rewrote it to include an array of repository groups, which in turn contains an array of repositories along
with its name and a description, and a boolean indicating whether it accepts a custom root file or not.
The boolean will be used when we're adding the support for perf try bots. We will keep acceptedRepositories
since it's still used by detect-changes.js.
* public/v3/models/manifest.js:
(Manifest._didFetchManifest): Resolve repositoriy, test, and platform IDs to their respective objects.
* public/v3/models/triggerable.js:
(Triggerable):
(Triggerable.prototype.isDisabled): Added.
(Triggerable.prototype.repositoryGroups): Added.
(Triggerable.prototype.acceptsTest): Added.
(TriggerableRepositoryGroup): Added.
(TriggerableRepositoryGroup.prototype.description): Added.
(TriggerableRepositoryGroup.prototype.acceptsCustomRoots): Added.
(TriggerableRepositoryGroup.prototype.repositories): Added.
* public/v3/pages/analysis-task-page.js:
(AnalysisTaskPage.prototype._didFetchTask): Don't use a disabled triggerable.
* server-tests/api-manifest-tests.js: Updated a test case to test repository groups.
* tools/js/database.js:
(tableToPrefixMap): Added triggerable_repository_groups.
* tools/js/v3-models.js: Imported TriggerableRepositoryGroup from triggerable.js.
Canonical link: https://commits.webkit.org/187454@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214975 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-05 23:12:46 +00:00
|
|
|
CONSTRAINT repository_group_name_must_be_unique_for_triggerable UNIQUE(repositorygroup_triggerable, repositorygroup_name));
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
|
|
|
|
CREATE TABLE triggerable_repositories (
|
|
|
|
trigrepo_repository integer REFERENCES repositories NOT NULL,
|
Each build request should be associated with a repository group
https://bugs.webkit.org/show_bug.cgi?id=170528
Rubber-stamped by Chris Dumez.
Make the buildbot syncing script use the concept of repository groups so that each repository group can post
a different set of properties to buildbot. In order to do this, we associate each build request with
a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via
/api/update-triggerable just the same way the set of the supported platform, test pairs are updated.
Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name
to either a string value or a repository name enclosed in < and >:
```js
"repositoryGroups": {
"webkit-svn": {
"repositories": ["WebKit", "macOS"],
"properties": {"os": "<macOS>", "wk": "<WebKit>"}
}
}
```
With this, removed the support for specifying a repository to use in generic dictionary of properties via
a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of
repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in
build requests.
After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group.
Run the appropriate database queries to set the repository group on each build request. Because of this change,
this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests.
Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same
test group if the next build request to be scheduled does not specify a repository group.
* init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for
repository and group pair in triggerable_repositories table.
* public/api/update-triggerable.php:
(main): Validate and insert repository groups.
(validate_configurations): Extracted from main.
(validate_repository_groups): Added.
* public/v3/models/repository.js:
(Repository.findTopLevelByName): Added.
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response.
* public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given
triggerable for a list of repositories.
(RepositoryGroupFinder): Added.
(RepositoryGroupFinder::__construct): Added.
(RepositoryGroupFinder::find_by_repositories): Added.
(RepositoryGroupFinder::populate_map): Added.
* public/privileged-api/create-test-group.php:
(main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains
"set", the list of commit IDs, and "repository_group", the repository group identified for each commit set.
Use that to set the repository group in each new build request.
(commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group.
(ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its
argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]}
* public/v3/models/build-request.js:
(BuildRequest):
(BuildRequest.prototype.triggerable): Added.
(BuildRequest.prototype.repositoryGroup): Added.
(BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group.
* public/v3/models/triggerable.js:
(Triggerable.prototype.name): Added.
(Triggerable.prototype.acceptedRepositories): Deleted.
(TriggerableRepositoryGroup):
(TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group
* server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request.
* server-tests/api-manifest-tests.js: Added assertions for the repository groups.
* server-tests/api-report-tests.js:
(.emptyReport):
(.reportWithTwoLevelsOfAggregations):
* server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with
a triggerable.
(.updateWithOSXRepositoryGroup):
(.mapRepositoriesByGroup):
* server-tests/privileged-api-create-test-group-tests.js:
(addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups
in existing test cases, and added a test case for creating a test group with two different repository groups.
* server-tests/resources/mock-data.js:
(MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps.
(MockData.emptyTriggeragbleId): Added.
(MockData.macosRepositoryId): Added.
(MockData.webkitRepositoryId): Added.
(MockData.gitWebkitRepositoryId): Added.
(MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some
tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks.
(MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js.
(MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js.
(MockData.addAnotherMockTestGroup): Cleanup.
(MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes.
(MockData.mockTestSyncConfigWithTwoBuilders): Ditto.
* server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test
updating the set of repository groups in addition to the set of test, platform pairs.
(.refetchManifest): Added.
* tools/js/buildbot-syncer.js:
(BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument,
and buildRequestArgument as the third argument.
(BuildbotSyncer.prototype.repositoryGroups): Added.
(BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find.
Also added an assertion that the build request is associated with a repository group.
(BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to
specify a revision in favor of explicity listing each property and repository name in a repository group.
(BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties
shared across syncers, the name of properties which specifies the build slave name and build request ID. These
values are not stored as top-level properties and superseded by the concept of repository groups.
(BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups.
(BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of
properties or buildRequestArgument (often inherited from shared).
(BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of
allowed proeprties in each configuration now that they're specified as top-level properties.
* tools/js/buildbot-triggerable.js:
(BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups.
(BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is
not included in the list of valid build requests.
(BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes
those that lack a repository group set.
(BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds
the next build request to be scheduled for the test group.
(BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from
_scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of
a test group information since syncOnce now calls _nextRequestInGroup to find the next build request.
* tools/js/v3-models.js:
* unit-tests/build-request-tests.js: Fixed the test name.
* unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests
for parsing repository groups.
(sampleiOSConfig): Updated the mock configuration per code changes.
(sampleiOSConfigWithExpansions): Ditto.
(smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration.
Various test cases have been updated to reflect this.
(createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock
configurations. The git hash was there to test "rootOptions", which this patch removed.
(samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding"
which, again, this patch removed.
(sampleInProgressBuild): Ditto.
(sampleFinishedBuild): Ditto.
* unit-tests/resources/mock-v3-models.js:
(MockModels.inject): Added ock repository groups so that existing tests will continue to function.
Canonical link: https://commits.webkit.org/187490@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
|
|
|
trigrepo_group integer REFERENCES triggerable_repository_groups NOT NULL,
|
Add the support for scheduling a A/B testing with a patch.
https://bugs.webkit.org/show_bug.cgi?id=171209
Reviewed by Chris Dumez.
Added the support for creating a custom test group with a patch applied.
First, each repository in a repository group has a boolean indicating whether a given repository can have
a patch applied or not. When any configuration in a test group contains a patch, we create build requests
without a test specified in order to "build" those patches. These build requests have negative order numbers
to differentiate them from regular build requests. We can't simply build ones with patches since there could
be differences in SDK, build options, etc... when patches are applied.
The JSON format for commit sets returned by /api/build-requests have been changed from using an array of
commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now
uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with
a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch
instead of a revision string to specify commit sets.
Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to
an options dictionary instead of an array of repositories names. For now, the only supported option is
acceptsPatch but will be extended when we add the support for rolling back system components.
e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"]
On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method
allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element
with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root"
instead of the generic label like "choose a file".
* init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given
repository can have a patch applied or not. Made request_test optional in build_requests for when a build
request is created to build patches. Such a build request have a negative request_order. Updated the related
constraints accordingly.
* public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch
applied in each repository group. Only show the repositories in the repository group for this purpose since
there is no way to accept a patch on a repository without it being a part of the group.
(generate_repository_form): Now takes the markup for checkboxes instead of generating one itself.
(generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is
shown when the repository ID exists as a key in this array, and is checked when its value is true. The new
capability to skip repositories not in the array is used to hide repositories not associated with the group
in the list of checkboxes to indicate a repository accepts a patch.
* public/api/update-triggerable.php:
(main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch
boolean for each repository in the group if set in the update.
(validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which
contains an array of repository IDs to find the existing repository group that matches the set via
RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary
specifying repository and acceptsPatch.
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as
"commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID
and the patch file's ID respectively as "revisionItems".
(BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to
add either a patch file or a custom root file in the list of uploaded files in the result.
* public/include/manifest-generator.php:
(fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries
with "repository" and "acceptsPatch" as keys.
* public/include/repository-group-finder.php:
(RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group
allows a patch on a repository. Used in /privileged-api/create-test-group.
(RepositoryGroupFinder::accepts_patch): Added.
(RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group.
* public/privileged-api/create-test-group.php:
(main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create
build requests to "build" patches if there is any patch file specified.
(commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify
a revision and a patch if any instead of just a revision string for each repository. Also validate that each
repository is allowed to have a patch once the repository group has been found for the set of repositories.
(ensure_commit_sets):
* public/v3/components/custom-analysis-task-configurator.js:
(CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of
configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to
_customRootUploaders for clarity.
(CustomAnalysisTaskConfigurator.prototype.setCommitSets):
(CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added.
(CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree):
(CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for
patches. We only allow a single patch per repository.
(CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories
which can have patches in the current repository group.
(CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's
now that each row can have a patch file uploader.
* public/v3/components/instant-file-uploader.js:
(InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button
in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click
event of the button to let the user pick a file.
(InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files.
(InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when
the newly added button element is clicked to open the browser's file picker.
(InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file
and there is already some file being uploaded in this instance.
(InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes
from the default slot content.
* public/v3/models/build-request.js:
(BuildRequest): Made the test optional.
(BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch.
(BuildRequest.prototype.isTest): Returns true if this is a build request for running tests.
(BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's
constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by
an array of dictionaries with commit and patch properties.
* public/v3/models/commit-set.js:
(CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of
the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData.
(CommitSet.prototype.commitForRepository):
(CommitSet.prototype.revisionForRepository):
(CommitSet.prototype.patchForRepository): Added.
(CommitSet.prototype.latestCommitTime): Modernized the code.
(CommitSet.prototype.equals): Modernized the code. Also added the check for patches.
(MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map.
(CustomCommitSet.prototype.setRevisionForRepository):
(CustomCommitSet.prototype.equals): Added the check for patches.
(CustomCommitSet.prototype.revisionForRepository):
(CustomCommitSet.prototype.patchForRepository): Added.
* public/v3/models/manifest.js:
(Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries
with commit and patch properties.
* public/v3/models/repository.js:
(Repository.prototype.ownerId): Renamed from owner for clarity.
* public/v3/models/test-group.js:
(TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not
necessary anymore with LazilyEvaluatedFunction.
(TestGroup.prototype.addBuildRequest):
(TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to
build patches.
(TestGroup.prototype.platform): Ditto.
(TestGroup.prototype._lastRequest): Added.
(TestGroup.prototype._orderedBuildRequests): Added.
(TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to
build patches).
(TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily.
(TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets.
(TestGroup.prototype.requestsForCommitSet):
(TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on
_commitSetToLabel since requestedSets is always of the length two at the moment.
(TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set.
* public/v3/models/triggerable.js:
(TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use
sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName.
(TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches.
(TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added.
* server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit
IDs by an array of dictionaries with commit and patch properties.
* server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId.
* server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId,
and added a test case for updating whether a given repository group allows custom roots as well as patches
on repositories via /api/update-triggerable.
(.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change.
(.refetchManifest): Added.
* server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format
change. Also added a test for creating a test group with a duplicate name, which is expected to fail with
DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed
in the matching repository group.
(.addTriggerableAndCreateTask): Updated per the format change.
* server-tests/resources/mock-data.js:
(MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file.
The new test case in api-update-triggerable.js requires this.
(MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change.
(MockData.mockTestSyncConfigWithTwoBuilders): Ditto.
* server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations,
and added assertions about custom roots and patches in the test case for updateTriggerables.
* tools/js/buildbot-syncer.js:
(BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group
to make it more user friendly. Now each repository group uses a dictionary of repository names to its options
in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID
and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects.
* tools/js/buildbot-triggerable.js:
(BuildbotTriggerable.prototype.updateTriggerable):
* unit-tests/build-request-tests.js:
(sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests.
* unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple
of new test cases for the syncing script's configuration validation.
(sampleiOSConfig):
(smallConfiguration):
(createSampleBuildRequest):
* unit-tests/resources/mock-v3-models.js:
(MockModels.inject): Updated per the repository group format change.
* unit-tests/test-groups-tests.js:
(sampleTestGroup): Updated per the commit sets format change in /api/build-requests.
Canonical link: https://commits.webkit.org/188378@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
|
|
|
trigrepo_accepts_patch boolean NOT NULL DEFAULT FALSE,
|
Each build request should be associated with a repository group
https://bugs.webkit.org/show_bug.cgi?id=170528
Rubber-stamped by Chris Dumez.
Make the buildbot syncing script use the concept of repository groups so that each repository group can post
a different set of properties to buildbot. In order to do this, we associate each build request with
a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via
/api/update-triggerable just the same way the set of the supported platform, test pairs are updated.
Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name
to either a string value or a repository name enclosed in < and >:
```js
"repositoryGroups": {
"webkit-svn": {
"repositories": ["WebKit", "macOS"],
"properties": {"os": "<macOS>", "wk": "<WebKit>"}
}
}
```
With this, removed the support for specifying a repository to use in generic dictionary of properties via
a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of
repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in
build requests.
After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group.
Run the appropriate database queries to set the repository group on each build request. Because of this change,
this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests.
Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same
test group if the next build request to be scheduled does not specify a repository group.
* init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for
repository and group pair in triggerable_repositories table.
* public/api/update-triggerable.php:
(main): Validate and insert repository groups.
(validate_configurations): Extracted from main.
(validate_repository_groups): Added.
* public/v3/models/repository.js:
(Repository.findTopLevelByName): Added.
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response.
* public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given
triggerable for a list of repositories.
(RepositoryGroupFinder): Added.
(RepositoryGroupFinder::__construct): Added.
(RepositoryGroupFinder::find_by_repositories): Added.
(RepositoryGroupFinder::populate_map): Added.
* public/privileged-api/create-test-group.php:
(main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains
"set", the list of commit IDs, and "repository_group", the repository group identified for each commit set.
Use that to set the repository group in each new build request.
(commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group.
(ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its
argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]}
* public/v3/models/build-request.js:
(BuildRequest):
(BuildRequest.prototype.triggerable): Added.
(BuildRequest.prototype.repositoryGroup): Added.
(BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group.
* public/v3/models/triggerable.js:
(Triggerable.prototype.name): Added.
(Triggerable.prototype.acceptedRepositories): Deleted.
(TriggerableRepositoryGroup):
(TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group
* server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request.
* server-tests/api-manifest-tests.js: Added assertions for the repository groups.
* server-tests/api-report-tests.js:
(.emptyReport):
(.reportWithTwoLevelsOfAggregations):
* server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with
a triggerable.
(.updateWithOSXRepositoryGroup):
(.mapRepositoriesByGroup):
* server-tests/privileged-api-create-test-group-tests.js:
(addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups
in existing test cases, and added a test case for creating a test group with two different repository groups.
* server-tests/resources/mock-data.js:
(MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps.
(MockData.emptyTriggeragbleId): Added.
(MockData.macosRepositoryId): Added.
(MockData.webkitRepositoryId): Added.
(MockData.gitWebkitRepositoryId): Added.
(MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some
tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks.
(MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js.
(MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js.
(MockData.addAnotherMockTestGroup): Cleanup.
(MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes.
(MockData.mockTestSyncConfigWithTwoBuilders): Ditto.
* server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test
updating the set of repository groups in addition to the set of test, platform pairs.
(.refetchManifest): Added.
* tools/js/buildbot-syncer.js:
(BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument,
and buildRequestArgument as the third argument.
(BuildbotSyncer.prototype.repositoryGroups): Added.
(BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find.
Also added an assertion that the build request is associated with a repository group.
(BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to
specify a revision in favor of explicity listing each property and repository name in a repository group.
(BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties
shared across syncers, the name of properties which specifies the build slave name and build request ID. These
values are not stored as top-level properties and superseded by the concept of repository groups.
(BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups.
(BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of
properties or buildRequestArgument (often inherited from shared).
(BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of
allowed proeprties in each configuration now that they're specified as top-level properties.
* tools/js/buildbot-triggerable.js:
(BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups.
(BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is
not included in the list of valid build requests.
(BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes
those that lack a repository group set.
(BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds
the next build request to be scheduled for the test group.
(BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from
_scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of
a test group information since syncOnce now calls _nextRequestInGroup to find the next build request.
* tools/js/v3-models.js:
* unit-tests/build-request-tests.js: Fixed the test name.
* unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests
for parsing repository groups.
(sampleiOSConfig): Updated the mock configuration per code changes.
(sampleiOSConfigWithExpansions): Ditto.
(smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration.
Various test cases have been updated to reflect this.
(createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock
configurations. The git hash was there to test "rootOptions", which this patch removed.
(samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding"
which, again, this patch removed.
(sampleInProgressBuild): Ditto.
(sampleFinishedBuild): Ditto.
* unit-tests/resources/mock-v3-models.js:
(MockModels.inject): Added ock repository groups so that existing tests will continue to function.
Canonical link: https://commits.webkit.org/187490@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
|
|
|
CONSTRAINT repository_must_be_unique_for_repository_group UNIQUE(trigrepo_repository, trigrepo_group));
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
|
|
|
|
CREATE TABLE triggerable_configurations (
|
|
|
|
trigconfig_test integer REFERENCES tests NOT NULL,
|
|
|
|
trigconfig_platform integer REFERENCES platforms NOT NULL,
|
|
|
|
trigconfig_triggerable integer REFERENCES build_triggerables NOT NULL,
|
|
|
|
CONSTRAINT triggerable_must_be_unique_for_test_and_platform UNIQUE(trigconfig_test, trigconfig_platform));
|
|
|
|
|
2017-03-16 20:53:35 +00:00
|
|
|
CREATE TABLE uploaded_files (
|
|
|
|
file_id serial PRIMARY KEY,
|
|
|
|
file_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
|
|
|
|
file_deleted_at timestamp,
|
|
|
|
file_mime varchar(64),
|
|
|
|
file_filename varchar(1024) NOT NULL,
|
|
|
|
file_extension varchar(16),
|
|
|
|
file_author varchar(256),
|
|
|
|
file_size bigint NOT NULL,
|
|
|
|
file_sha256 char(64) NOT NULL);
|
|
|
|
CREATE INDEX file_author_index ON uploaded_files(file_author);
|
|
|
|
CREATE UNIQUE INDEX file_sha256_index ON uploaded_files(file_sha256) WHERE file_deleted_at is NULL;
|
|
|
|
|
2021-05-26 01:23:22 +00:00
|
|
|
CREATE TYPE analysis_test_group_repetition_type as ENUM ('alternating', 'sequential');
|
2014-11-07 23:47:09 +00:00
|
|
|
CREATE TABLE analysis_test_groups (
|
|
|
|
testgroup_id serial PRIMARY KEY,
|
2016-02-19 04:25:12 +00:00
|
|
|
testgroup_task integer REFERENCES analysis_tasks NOT NULL,
|
2014-11-07 23:47:09 +00:00
|
|
|
testgroup_name varchar(256),
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
testgroup_author varchar(256),
|
|
|
|
testgroup_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
|
2016-02-19 04:18:27 +00:00
|
|
|
testgroup_hidden boolean NOT NULL DEFAULT FALSE,
|
2018-06-08 03:44:28 +00:00
|
|
|
testgroup_needs_notification boolean NOT NULL DEFAULT FALSE,
|
|
|
|
testgroup_notification_sent_at timestamp DEFAULT NULL,
|
2018-10-05 00:17:49 +00:00
|
|
|
testgroup_initial_repetition_count integer NOT NULL,
|
2021-05-26 01:23:22 +00:00
|
|
|
testgroup_repetition_type analysis_test_group_repetition_type NOT NULL DEFAULT 'alternating',
|
2018-10-05 00:17:49 +00:00
|
|
|
testgroup_may_need_more_requests boolean DEFAULT FALSE,
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
CONSTRAINT testgroup_name_must_be_unique_for_each_task UNIQUE(testgroup_task, testgroup_name));
|
2014-11-07 23:47:09 +00:00
|
|
|
CREATE INDEX testgroup_task_index ON analysis_test_groups(testgroup_task);
|
|
|
|
|
2017-03-14 23:06:40 +00:00
|
|
|
CREATE TABLE commit_sets (
|
|
|
|
commitset_id serial PRIMARY KEY);
|
2014-11-07 23:47:09 +00:00
|
|
|
|
2017-04-21 21:31:06 +00:00
|
|
|
CREATE TABLE commit_set_items (
|
2017-03-14 23:06:40 +00:00
|
|
|
commitset_set integer REFERENCES commit_sets NOT NULL,
|
Add the UI for scheduling a A/B testing with a custom root
https://bugs.webkit.org/show_bug.cgi?id=170622
Reviewed by Anders Carlsson.
This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch
would update the syncing script to schedule such an A/B testing job to a buildbot instance.
* ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace
the file path for the backup.
* init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column
to store the root file in commit_set_relationships.
* public/api/build-requests.php:
(main): Include the uploaded files.
* public/api/commits.php:
(main): Added the support for querying the latest commits for a given platform. This is used in a new page
to create a custom analysis task to autofill the latest revisions for a given platform.
* public/api/test-groups.php:
(main): Include the uploaded files.
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id.
(BuildRequestsFetcher::uploaded_files): Added.
(BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in
each commit set, and inserting its meta data in the list of uplaoded files.
* public/include/commit-log-fetcher.php:
(CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally,
we should be finding the latest commit for a given platform, but this is very slow so instead find the commit
of the latest build for a given platform.
* public/privileged-api/create-test-group.php:
(main): Added the support for creating an analysis task along with a group.
(commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists
and include it in commit_set_relationships. Because commits and upload files are stored in a different column
in commit_set_relationships, this function now stores the information for each row of commit_set_relationships
except the commit set ID, which is unknown until the set is created, instead of a commit ID.
(ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format
is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here.
* public/privileged-api/upload-file.php:
(main): Fixed a typo. Also added one more error check.
* public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform,
and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with
custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class.
(CustomAnalysisTaskConfigurator):
(CustomAnalysisTaskConfigurator.prototype.tests): Added.
(CustomAnalysisTaskConfigurator.prototype.platform): Added.
(CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both
have been configured by the user.
(CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added.
(CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the
"comparison" configuration.
(CustomAnalysisTaskConfigurator.prototype.render): Added.
(CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests
that can be scheduled by a triggerable.
(CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms
that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only
lets the user select a single test but the intent is to allow multiple tests to be selected in the near future.
(CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select
an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to
render both the list of tests and platforms.
(CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of
tests and platforms. Returns an error when some tests belong to another triggearalbe.
(CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when
the current triggerable has changed. We try to use the repository group of the same name if there is any, and
defaults to the first repository group if there is none. This allows the set of repositories to be specified to
more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two
distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the
other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to
specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform
to a Mac platform.
(CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that
TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set
if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical.
(CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions
and the csutom roots the user had specified.
(CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions
and custom roots for "baseline" and "comparison".
(CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added.
(CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added.
(CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions
and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are
shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional.
(CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added.
(CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify
a revision for a given repository. Autofills it with the latest commit for the currently selected platform if
the user had not modified the field by the time the revisions are fetched.
(CustomAnalysisTaskConfigurator.htmlTemplate): Added.
(CustomAnalysisTaskConfigurator.cssTemplate): Added.
* public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline"
or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a
file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash.
(InstantFileUploader):
(InstantFileUploader.prototype.hasFileToUpload): Added.
(InstantFileUploader.prototype.uploadedFiles): Added.
(InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison"
configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically
mirror the newly uploaded custom root to "comparision" configuration.
(InstantFileUploader.prototype.didConstructShadowTree): Added.
(InstantFileUploader.prototype.render): Added.
(InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files.
(InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with
a progress bar.
(InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file.
(InstantFileUploader.prototype._formatUploadError): Added.
(InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on
the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and
start uploading the file if there isn't one.
(InstantFileUploader.prototype._removeUploadedFile): Added.
(InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to
the list of uploaded files.
(InstantFileUploader.htmlTemplate): Added.
(InstantFileUploader.cssTemplate): Added.
* public/v3/index.html:
* public/v3/models/analysis-task.js:
(AnalysisTask): Made platform and metric optional as it is now.
(AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric.
(AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task.
(AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks.
(AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric
or a platform instead of silently skipping them.
* public/v3/models/build-request.js:
(BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests.
* public/v3/models/commit-log.js:
(CommitLog.fetchLatestCommitForPlatform): Added.
* public/v3/models/commit-set.js:
(CommitSet): Added this._customRoots.
(CommitSet.prototype.customRoots): Returns this._customRoots.
(CommitSet.prototype.equals): Returns false when the set of custom roots are not equal.
(CommitSet.areCustomRootsEqual): Added.
(CustomCommitSet):
(CustomCommitSet.prototype.equals): Added.
(CustomCommitSet.prototype.customRoots): Added.
(CustomCommitSet.prototype.addCustomRoot): Added.
* public/v3/models/manifest.js:
(Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit.
This allows a file size check in the client size instead of uploading it to the server and receiving an error.
* public/v3/models/metric.js:
(Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles.
* public/v3/models/test-group.js:
(TestGroup.prototype.createWithTask): Added.
(TestGroup.prototype.createAndRefetchTestGroups):
(TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups.
(TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups.
* public/v3/models/triggerable.js:
(Triggerable.triggerablePlatformsForTests): Added.
(Triggerable.sortByNamePreferringSmallerRepositories): Added.
* public/v3/models/uploaded-file.js:
(UploadedFile.prototype.createdAt): Added.
(UploadedFile.prototype.filename): Added.
(UploadedFile.prototype.author): Added.
(UploadedFile.prototype.size): Added.
(UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit.
(UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected
promise instead of a resolved promise with null.
* public/v3/pages/analysis-category-page.js:
(AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and
metric being null for some analysis tasks.
* public/v3/pages/analysis-task-page.js:
(AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks.
(AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks.
(AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks
since it's not associated with exactly one pair.
* public/v3/pages/chart-pane-status-view.js:
(ChartPaneStatusView.prototype._renderBuildRevisionTable):
(ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime.
* public/v3/pages/chart-pane.js:
(ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in
CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range.
* public/v3/pages/create-analysis-task-page.js:
(CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create
a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots.
(CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is
the old behavior of this page.
(CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added.
(CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added.
(CreateAnalysisTaskPage.prototype.render):
(CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select
element to specify the numebr of iterations when a message is set.
(CreateAnalysisTaskPage.htmlTemplate):
(CreateAnalysisTaskPage.cssTemplate):
* public/v3/pages/page-router.js:
(PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed.
* server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of
uploaded files associated with build requests.
* server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task
and a test group with custom roots.
* server-tests/resources/mock-data.js:
(MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table.
* tools/js/remote.js: Include global.FormData from form-data.js.
* unit-tests/build-request-tests.js:
(sampleBuildRequestData): Updated the mock response.
* unit-tests/buildbot-syncer-tests.js:
(createSampleBuildRequest): Ditto.
* unit-tests/test-groups-tests.js:
(sampleTestGroup): Ditto.
Canonical link: https://commits.webkit.org/187631@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
|
|
|
commitset_commit integer REFERENCES commits,
|
2017-09-19 20:14:45 +00:00
|
|
|
commitset_commit_owner integer REFERENCES commits DEFAULT NULL,
|
2017-04-21 21:31:06 +00:00
|
|
|
commitset_patch_file integer REFERENCES uploaded_files,
|
Add the UI for scheduling a A/B testing with a custom root
https://bugs.webkit.org/show_bug.cgi?id=170622
Reviewed by Anders Carlsson.
This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch
would update the syncing script to schedule such an A/B testing job to a buildbot instance.
* ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace
the file path for the backup.
* init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column
to store the root file in commit_set_relationships.
* public/api/build-requests.php:
(main): Include the uploaded files.
* public/api/commits.php:
(main): Added the support for querying the latest commits for a given platform. This is used in a new page
to create a custom analysis task to autofill the latest revisions for a given platform.
* public/api/test-groups.php:
(main): Include the uploaded files.
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id.
(BuildRequestsFetcher::uploaded_files): Added.
(BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in
each commit set, and inserting its meta data in the list of uplaoded files.
* public/include/commit-log-fetcher.php:
(CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally,
we should be finding the latest commit for a given platform, but this is very slow so instead find the commit
of the latest build for a given platform.
* public/privileged-api/create-test-group.php:
(main): Added the support for creating an analysis task along with a group.
(commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists
and include it in commit_set_relationships. Because commits and upload files are stored in a different column
in commit_set_relationships, this function now stores the information for each row of commit_set_relationships
except the commit set ID, which is unknown until the set is created, instead of a commit ID.
(ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format
is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here.
* public/privileged-api/upload-file.php:
(main): Fixed a typo. Also added one more error check.
* public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform,
and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with
custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class.
(CustomAnalysisTaskConfigurator):
(CustomAnalysisTaskConfigurator.prototype.tests): Added.
(CustomAnalysisTaskConfigurator.prototype.platform): Added.
(CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both
have been configured by the user.
(CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added.
(CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the
"comparison" configuration.
(CustomAnalysisTaskConfigurator.prototype.render): Added.
(CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests
that can be scheduled by a triggerable.
(CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms
that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only
lets the user select a single test but the intent is to allow multiple tests to be selected in the near future.
(CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select
an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to
render both the list of tests and platforms.
(CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of
tests and platforms. Returns an error when some tests belong to another triggearalbe.
(CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when
the current triggerable has changed. We try to use the repository group of the same name if there is any, and
defaults to the first repository group if there is none. This allows the set of repositories to be specified to
more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two
distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the
other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to
specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform
to a Mac platform.
(CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that
TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set
if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical.
(CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions
and the csutom roots the user had specified.
(CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions
and custom roots for "baseline" and "comparison".
(CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added.
(CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added.
(CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions
and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are
shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional.
(CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added.
(CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify
a revision for a given repository. Autofills it with the latest commit for the currently selected platform if
the user had not modified the field by the time the revisions are fetched.
(CustomAnalysisTaskConfigurator.htmlTemplate): Added.
(CustomAnalysisTaskConfigurator.cssTemplate): Added.
* public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline"
or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a
file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash.
(InstantFileUploader):
(InstantFileUploader.prototype.hasFileToUpload): Added.
(InstantFileUploader.prototype.uploadedFiles): Added.
(InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison"
configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically
mirror the newly uploaded custom root to "comparision" configuration.
(InstantFileUploader.prototype.didConstructShadowTree): Added.
(InstantFileUploader.prototype.render): Added.
(InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files.
(InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with
a progress bar.
(InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file.
(InstantFileUploader.prototype._formatUploadError): Added.
(InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on
the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and
start uploading the file if there isn't one.
(InstantFileUploader.prototype._removeUploadedFile): Added.
(InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to
the list of uploaded files.
(InstantFileUploader.htmlTemplate): Added.
(InstantFileUploader.cssTemplate): Added.
* public/v3/index.html:
* public/v3/models/analysis-task.js:
(AnalysisTask): Made platform and metric optional as it is now.
(AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric.
(AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task.
(AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks.
(AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric
or a platform instead of silently skipping them.
* public/v3/models/build-request.js:
(BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests.
* public/v3/models/commit-log.js:
(CommitLog.fetchLatestCommitForPlatform): Added.
* public/v3/models/commit-set.js:
(CommitSet): Added this._customRoots.
(CommitSet.prototype.customRoots): Returns this._customRoots.
(CommitSet.prototype.equals): Returns false when the set of custom roots are not equal.
(CommitSet.areCustomRootsEqual): Added.
(CustomCommitSet):
(CustomCommitSet.prototype.equals): Added.
(CustomCommitSet.prototype.customRoots): Added.
(CustomCommitSet.prototype.addCustomRoot): Added.
* public/v3/models/manifest.js:
(Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit.
This allows a file size check in the client size instead of uploading it to the server and receiving an error.
* public/v3/models/metric.js:
(Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles.
* public/v3/models/test-group.js:
(TestGroup.prototype.createWithTask): Added.
(TestGroup.prototype.createAndRefetchTestGroups):
(TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups.
(TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups.
* public/v3/models/triggerable.js:
(Triggerable.triggerablePlatformsForTests): Added.
(Triggerable.sortByNamePreferringSmallerRepositories): Added.
* public/v3/models/uploaded-file.js:
(UploadedFile.prototype.createdAt): Added.
(UploadedFile.prototype.filename): Added.
(UploadedFile.prototype.author): Added.
(UploadedFile.prototype.size): Added.
(UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit.
(UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected
promise instead of a resolved promise with null.
* public/v3/pages/analysis-category-page.js:
(AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and
metric being null for some analysis tasks.
* public/v3/pages/analysis-task-page.js:
(AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks.
(AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks.
(AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks
since it's not associated with exactly one pair.
* public/v3/pages/chart-pane-status-view.js:
(ChartPaneStatusView.prototype._renderBuildRevisionTable):
(ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime.
* public/v3/pages/chart-pane.js:
(ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in
CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range.
* public/v3/pages/create-analysis-task-page.js:
(CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create
a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots.
(CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is
the old behavior of this page.
(CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added.
(CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added.
(CreateAnalysisTaskPage.prototype.render):
(CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select
element to specify the numebr of iterations when a message is set.
(CreateAnalysisTaskPage.htmlTemplate):
(CreateAnalysisTaskPage.cssTemplate):
* public/v3/pages/page-router.js:
(PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed.
* server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of
uploaded files associated with build requests.
* server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task
and a test group with custom roots.
* server-tests/resources/mock-data.js:
(MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table.
* tools/js/remote.js: Include global.FormData from form-data.js.
* unit-tests/build-request-tests.js:
(sampleBuildRequestData): Updated the mock response.
* unit-tests/buildbot-syncer-tests.js:
(createSampleBuildRequest): Ditto.
* unit-tests/test-groups-tests.js:
(sampleTestGroup): Ditto.
Canonical link: https://commits.webkit.org/187631@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
|
|
|
commitset_root_file integer REFERENCES uploaded_files,
|
2017-09-19 20:14:45 +00:00
|
|
|
commitset_requires_build boolean DEFAULT FALSE,
|
2017-04-21 21:31:06 +00:00
|
|
|
CONSTRAINT commitset_must_have_commit_or_root CHECK (commitset_commit IS NOT NULL OR commitset_root_file IS NOT NULL),
|
2017-09-19 20:14:45 +00:00
|
|
|
CONSTRAINT commitset_with_patch_must_have_commit CHECK (commitset_patch_file IS NULL OR commitset_commit IS NOT NULL),
|
|
|
|
CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE),
|
|
|
|
CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE));
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
|
2016-02-19 04:18:27 +00:00
|
|
|
CREATE TYPE build_request_status_type as ENUM ('pending', 'scheduled', 'running', 'failed', 'completed', 'canceled');
|
2014-11-07 23:47:09 +00:00
|
|
|
CREATE TABLE build_requests (
|
|
|
|
request_id serial PRIMARY KEY,
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
request_triggerable integer REFERENCES build_triggerables NOT NULL,
|
Each build request should be associated with a repository group
https://bugs.webkit.org/show_bug.cgi?id=170528
Rubber-stamped by Chris Dumez.
Make the buildbot syncing script use the concept of repository groups so that each repository group can post
a different set of properties to buildbot. In order to do this, we associate each build request with
a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via
/api/update-triggerable just the same way the set of the supported platform, test pairs are updated.
Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name
to either a string value or a repository name enclosed in < and >:
```js
"repositoryGroups": {
"webkit-svn": {
"repositories": ["WebKit", "macOS"],
"properties": {"os": "<macOS>", "wk": "<WebKit>"}
}
}
```
With this, removed the support for specifying a repository to use in generic dictionary of properties via
a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of
repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in
build requests.
After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group.
Run the appropriate database queries to set the repository group on each build request. Because of this change,
this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests.
Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same
test group if the next build request to be scheduled does not specify a repository group.
* init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for
repository and group pair in triggerable_repositories table.
* public/api/update-triggerable.php:
(main): Validate and insert repository groups.
(validate_configurations): Extracted from main.
(validate_repository_groups): Added.
* public/v3/models/repository.js:
(Repository.findTopLevelByName): Added.
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response.
* public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given
triggerable for a list of repositories.
(RepositoryGroupFinder): Added.
(RepositoryGroupFinder::__construct): Added.
(RepositoryGroupFinder::find_by_repositories): Added.
(RepositoryGroupFinder::populate_map): Added.
* public/privileged-api/create-test-group.php:
(main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains
"set", the list of commit IDs, and "repository_group", the repository group identified for each commit set.
Use that to set the repository group in each new build request.
(commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group.
(ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its
argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]}
* public/v3/models/build-request.js:
(BuildRequest):
(BuildRequest.prototype.triggerable): Added.
(BuildRequest.prototype.repositoryGroup): Added.
(BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group.
* public/v3/models/triggerable.js:
(Triggerable.prototype.name): Added.
(Triggerable.prototype.acceptedRepositories): Deleted.
(TriggerableRepositoryGroup):
(TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group
* server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request.
* server-tests/api-manifest-tests.js: Added assertions for the repository groups.
* server-tests/api-report-tests.js:
(.emptyReport):
(.reportWithTwoLevelsOfAggregations):
* server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with
a triggerable.
(.updateWithOSXRepositoryGroup):
(.mapRepositoriesByGroup):
* server-tests/privileged-api-create-test-group-tests.js:
(addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups
in existing test cases, and added a test case for creating a test group with two different repository groups.
* server-tests/resources/mock-data.js:
(MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps.
(MockData.emptyTriggeragbleId): Added.
(MockData.macosRepositoryId): Added.
(MockData.webkitRepositoryId): Added.
(MockData.gitWebkitRepositoryId): Added.
(MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some
tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks.
(MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js.
(MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js.
(MockData.addAnotherMockTestGroup): Cleanup.
(MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes.
(MockData.mockTestSyncConfigWithTwoBuilders): Ditto.
* server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test
updating the set of repository groups in addition to the set of test, platform pairs.
(.refetchManifest): Added.
* tools/js/buildbot-syncer.js:
(BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument,
and buildRequestArgument as the third argument.
(BuildbotSyncer.prototype.repositoryGroups): Added.
(BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find.
Also added an assertion that the build request is associated with a repository group.
(BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to
specify a revision in favor of explicity listing each property and repository name in a repository group.
(BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties
shared across syncers, the name of properties which specifies the build slave name and build request ID. These
values are not stored as top-level properties and superseded by the concept of repository groups.
(BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups.
(BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of
properties or buildRequestArgument (often inherited from shared).
(BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of
allowed proeprties in each configuration now that they're specified as top-level properties.
* tools/js/buildbot-triggerable.js:
(BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups.
(BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is
not included in the list of valid build requests.
(BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes
those that lack a repository group set.
(BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds
the next build request to be scheduled for the test group.
(BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from
_scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of
a test group information since syncOnce now calls _nextRequestInGroup to find the next build request.
* tools/js/v3-models.js:
* unit-tests/build-request-tests.js: Fixed the test name.
* unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests
for parsing repository groups.
(sampleiOSConfig): Updated the mock configuration per code changes.
(sampleiOSConfigWithExpansions): Ditto.
(smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration.
Various test cases have been updated to reflect this.
(createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock
configurations. The git hash was there to test "rootOptions", which this patch removed.
(samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding"
which, again, this patch removed.
(sampleInProgressBuild): Ditto.
(sampleFinishedBuild): Ditto.
* unit-tests/resources/mock-v3-models.js:
(MockModels.inject): Added ock repository groups so that existing tests will continue to function.
Canonical link: https://commits.webkit.org/187490@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
|
|
|
request_repository_group integer REFERENCES triggerable_repository_groups,
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
request_platform integer REFERENCES platforms NOT NULL,
|
Add the support for scheduling a A/B testing with a patch.
https://bugs.webkit.org/show_bug.cgi?id=171209
Reviewed by Chris Dumez.
Added the support for creating a custom test group with a patch applied.
First, each repository in a repository group has a boolean indicating whether a given repository can have
a patch applied or not. When any configuration in a test group contains a patch, we create build requests
without a test specified in order to "build" those patches. These build requests have negative order numbers
to differentiate them from regular build requests. We can't simply build ones with patches since there could
be differences in SDK, build options, etc... when patches are applied.
The JSON format for commit sets returned by /api/build-requests have been changed from using an array of
commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now
uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with
a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch
instead of a revision string to specify commit sets.
Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to
an options dictionary instead of an array of repositories names. For now, the only supported option is
acceptsPatch but will be extended when we add the support for rolling back system components.
e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"]
On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method
allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element
with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root"
instead of the generic label like "choose a file".
* init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given
repository can have a patch applied or not. Made request_test optional in build_requests for when a build
request is created to build patches. Such a build request have a negative request_order. Updated the related
constraints accordingly.
* public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch
applied in each repository group. Only show the repositories in the repository group for this purpose since
there is no way to accept a patch on a repository without it being a part of the group.
(generate_repository_form): Now takes the markup for checkboxes instead of generating one itself.
(generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is
shown when the repository ID exists as a key in this array, and is checked when its value is true. The new
capability to skip repositories not in the array is used to hide repositories not associated with the group
in the list of checkboxes to indicate a repository accepts a patch.
* public/api/update-triggerable.php:
(main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch
boolean for each repository in the group if set in the update.
(validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which
contains an array of repository IDs to find the existing repository group that matches the set via
RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary
specifying repository and acceptsPatch.
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as
"commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID
and the patch file's ID respectively as "revisionItems".
(BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to
add either a patch file or a custom root file in the list of uploaded files in the result.
* public/include/manifest-generator.php:
(fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries
with "repository" and "acceptsPatch" as keys.
* public/include/repository-group-finder.php:
(RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group
allows a patch on a repository. Used in /privileged-api/create-test-group.
(RepositoryGroupFinder::accepts_patch): Added.
(RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group.
* public/privileged-api/create-test-group.php:
(main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create
build requests to "build" patches if there is any patch file specified.
(commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify
a revision and a patch if any instead of just a revision string for each repository. Also validate that each
repository is allowed to have a patch once the repository group has been found for the set of repositories.
(ensure_commit_sets):
* public/v3/components/custom-analysis-task-configurator.js:
(CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of
configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to
_customRootUploaders for clarity.
(CustomAnalysisTaskConfigurator.prototype.setCommitSets):
(CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added.
(CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree):
(CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for
patches. We only allow a single patch per repository.
(CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories
which can have patches in the current repository group.
(CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's
now that each row can have a patch file uploader.
* public/v3/components/instant-file-uploader.js:
(InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button
in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click
event of the button to let the user pick a file.
(InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files.
(InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when
the newly added button element is clicked to open the browser's file picker.
(InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file
and there is already some file being uploaded in this instance.
(InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes
from the default slot content.
* public/v3/models/build-request.js:
(BuildRequest): Made the test optional.
(BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch.
(BuildRequest.prototype.isTest): Returns true if this is a build request for running tests.
(BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's
constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by
an array of dictionaries with commit and patch properties.
* public/v3/models/commit-set.js:
(CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of
the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData.
(CommitSet.prototype.commitForRepository):
(CommitSet.prototype.revisionForRepository):
(CommitSet.prototype.patchForRepository): Added.
(CommitSet.prototype.latestCommitTime): Modernized the code.
(CommitSet.prototype.equals): Modernized the code. Also added the check for patches.
(MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map.
(CustomCommitSet.prototype.setRevisionForRepository):
(CustomCommitSet.prototype.equals): Added the check for patches.
(CustomCommitSet.prototype.revisionForRepository):
(CustomCommitSet.prototype.patchForRepository): Added.
* public/v3/models/manifest.js:
(Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries
with commit and patch properties.
* public/v3/models/repository.js:
(Repository.prototype.ownerId): Renamed from owner for clarity.
* public/v3/models/test-group.js:
(TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not
necessary anymore with LazilyEvaluatedFunction.
(TestGroup.prototype.addBuildRequest):
(TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to
build patches.
(TestGroup.prototype.platform): Ditto.
(TestGroup.prototype._lastRequest): Added.
(TestGroup.prototype._orderedBuildRequests): Added.
(TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to
build patches).
(TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily.
(TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets.
(TestGroup.prototype.requestsForCommitSet):
(TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on
_commitSetToLabel since requestedSets is always of the length two at the moment.
(TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set.
* public/v3/models/triggerable.js:
(TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use
sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName.
(TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches.
(TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added.
* server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit
IDs by an array of dictionaries with commit and patch properties.
* server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId.
* server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId,
and added a test case for updating whether a given repository group allows custom roots as well as patches
on repositories via /api/update-triggerable.
(.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change.
(.refetchManifest): Added.
* server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format
change. Also added a test for creating a test group with a duplicate name, which is expected to fail with
DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed
in the matching repository group.
(.addTriggerableAndCreateTask): Updated per the format change.
* server-tests/resources/mock-data.js:
(MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file.
The new test case in api-update-triggerable.js requires this.
(MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change.
(MockData.mockTestSyncConfigWithTwoBuilders): Ditto.
* server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations,
and added assertions about custom roots and patches in the test case for updateTriggerables.
* tools/js/buildbot-syncer.js:
(BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group
to make it more user friendly. Now each repository group uses a dictionary of repository names to its options
in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID
and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects.
* tools/js/buildbot-triggerable.js:
(BuildbotTriggerable.prototype.updateTriggerable):
* unit-tests/build-request-tests.js:
(sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests.
* unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple
of new test cases for the syncing script's configuration validation.
(sampleiOSConfig):
(smallConfiguration):
(createSampleBuildRequest):
* unit-tests/resources/mock-v3-models.js:
(MockModels.inject): Updated per the repository group format change.
* unit-tests/test-groups-tests.js:
(sampleTestGroup): Updated per the commit sets format change in /api/build-requests.
Canonical link: https://commits.webkit.org/188378@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
|
|
|
request_test integer REFERENCES tests,
|
2016-02-19 04:25:12 +00:00
|
|
|
request_group integer REFERENCES analysis_test_groups NOT NULL,
|
2014-11-07 23:47:09 +00:00
|
|
|
request_order integer NOT NULL,
|
2017-03-14 23:06:40 +00:00
|
|
|
request_commit_set integer REFERENCES commit_sets NOT NULL,
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
request_status build_request_status_type NOT NULL DEFAULT 'pending',
|
|
|
|
request_url varchar(1024),
|
2014-11-07 23:47:09 +00:00
|
|
|
request_build integer REFERENCES builds,
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
request_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
|
Add the support for scheduling a A/B testing with a patch.
https://bugs.webkit.org/show_bug.cgi?id=171209
Reviewed by Chris Dumez.
Added the support for creating a custom test group with a patch applied.
First, each repository in a repository group has a boolean indicating whether a given repository can have
a patch applied or not. When any configuration in a test group contains a patch, we create build requests
without a test specified in order to "build" those patches. These build requests have negative order numbers
to differentiate them from regular build requests. We can't simply build ones with patches since there could
be differences in SDK, build options, etc... when patches are applied.
The JSON format for commit sets returned by /api/build-requests have been changed from using an array of
commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now
uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with
a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch
instead of a revision string to specify commit sets.
Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to
an options dictionary instead of an array of repositories names. For now, the only supported option is
acceptsPatch but will be extended when we add the support for rolling back system components.
e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"]
On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method
allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element
with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root"
instead of the generic label like "choose a file".
* init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given
repository can have a patch applied or not. Made request_test optional in build_requests for when a build
request is created to build patches. Such a build request have a negative request_order. Updated the related
constraints accordingly.
* public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch
applied in each repository group. Only show the repositories in the repository group for this purpose since
there is no way to accept a patch on a repository without it being a part of the group.
(generate_repository_form): Now takes the markup for checkboxes instead of generating one itself.
(generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is
shown when the repository ID exists as a key in this array, and is checked when its value is true. The new
capability to skip repositories not in the array is used to hide repositories not associated with the group
in the list of checkboxes to indicate a repository accepts a patch.
* public/api/update-triggerable.php:
(main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch
boolean for each repository in the group if set in the update.
(validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which
contains an array of repository IDs to find the existing repository group that matches the set via
RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary
specifying repository and acceptsPatch.
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as
"commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID
and the patch file's ID respectively as "revisionItems".
(BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to
add either a patch file or a custom root file in the list of uploaded files in the result.
* public/include/manifest-generator.php:
(fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries
with "repository" and "acceptsPatch" as keys.
* public/include/repository-group-finder.php:
(RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group
allows a patch on a repository. Used in /privileged-api/create-test-group.
(RepositoryGroupFinder::accepts_patch): Added.
(RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group.
* public/privileged-api/create-test-group.php:
(main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create
build requests to "build" patches if there is any patch file specified.
(commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify
a revision and a patch if any instead of just a revision string for each repository. Also validate that each
repository is allowed to have a patch once the repository group has been found for the set of repositories.
(ensure_commit_sets):
* public/v3/components/custom-analysis-task-configurator.js:
(CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of
configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to
_customRootUploaders for clarity.
(CustomAnalysisTaskConfigurator.prototype.setCommitSets):
(CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added.
(CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree):
(CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for
patches. We only allow a single patch per repository.
(CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed.
(CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories
which can have patches in the current repository group.
(CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's
now that each row can have a patch file uploader.
* public/v3/components/instant-file-uploader.js:
(InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button
in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click
event of the button to let the user pick a file.
(InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files.
(InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when
the newly added button element is clicked to open the browser's file picker.
(InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file
and there is already some file being uploaded in this instance.
(InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes
from the default slot content.
* public/v3/models/build-request.js:
(BuildRequest): Made the test optional.
(BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch.
(BuildRequest.prototype.isTest): Returns true if this is a build request for running tests.
(BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's
constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by
an array of dictionaries with commit and patch properties.
* public/v3/models/commit-set.js:
(CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of
the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData.
(CommitSet.prototype.commitForRepository):
(CommitSet.prototype.revisionForRepository):
(CommitSet.prototype.patchForRepository): Added.
(CommitSet.prototype.latestCommitTime): Modernized the code.
(CommitSet.prototype.equals): Modernized the code. Also added the check for patches.
(MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map.
(CustomCommitSet.prototype.setRevisionForRepository):
(CustomCommitSet.prototype.equals): Added the check for patches.
(CustomCommitSet.prototype.revisionForRepository):
(CustomCommitSet.prototype.patchForRepository): Added.
* public/v3/models/manifest.js:
(Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries
with commit and patch properties.
* public/v3/models/repository.js:
(Repository.prototype.ownerId): Renamed from owner for clarity.
* public/v3/models/test-group.js:
(TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not
necessary anymore with LazilyEvaluatedFunction.
(TestGroup.prototype.addBuildRequest):
(TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to
build patches.
(TestGroup.prototype.platform): Ditto.
(TestGroup.prototype._lastRequest): Added.
(TestGroup.prototype._orderedBuildRequests): Added.
(TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to
build patches).
(TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily.
(TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets.
(TestGroup.prototype.requestsForCommitSet):
(TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on
_commitSetToLabel since requestedSets is always of the length two at the moment.
(TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set.
* public/v3/models/triggerable.js:
(TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use
sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName.
(TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches.
(TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added.
* server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit
IDs by an array of dictionaries with commit and patch properties.
* server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId.
* server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId,
and added a test case for updating whether a given repository group allows custom roots as well as patches
on repositories via /api/update-triggerable.
(.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change.
(.refetchManifest): Added.
* server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format
change. Also added a test for creating a test group with a duplicate name, which is expected to fail with
DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed
in the matching repository group.
(.addTriggerableAndCreateTask): Updated per the format change.
* server-tests/resources/mock-data.js:
(MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file.
The new test case in api-update-triggerable.js requires this.
(MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change.
(MockData.mockTestSyncConfigWithTwoBuilders): Ditto.
* server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations,
and added assertions about custom roots and patches in the test case for updateTriggerables.
* tools/js/buildbot-syncer.js:
(BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group
to make it more user friendly. Now each repository group uses a dictionary of repository names to its options
in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID
and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects.
* tools/js/buildbot-triggerable.js:
(BuildbotTriggerable.prototype.updateTriggerable):
* unit-tests/build-request-tests.js:
(sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests.
* unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple
of new test cases for the syncing script's configuration validation.
(sampleiOSConfig):
(smallConfiguration):
(createSampleBuildRequest):
* unit-tests/resources/mock-v3-models.js:
(MockModels.inject): Updated per the repository group format change.
* unit-tests/test-groups-tests.js:
(sampleTestGroup): Updated per the commit sets format change in /api/build-requests.
Canonical link: https://commits.webkit.org/188378@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
|
|
|
CONSTRAINT build_request_order_must_be_unique_in_group UNIQUE(request_group, request_order),
|
|
|
|
CONSTRAINT build_request_order_must_be_positive_for_testing
|
|
|
|
CHECK ((request_test IS NOT NULL AND request_order >= 0) OR (request_test IS NULL AND request_order < 0)));
|
2017-02-24 07:17:48 +00:00
|
|
|
CREATE INDEX build_request_triggerable ON build_requests(request_triggerable);
|
Perf dashboard should have the ability to post A/B testing builds
https://bugs.webkit.org/show_bug.cgi?id=140317
Rubber-stamped by Simon Fraser.
This patch adds the support for triggering A/B testing from the perf dashboard.
We add a few new tables to the database. "build_triggerables", which represents a set of builders
that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set
of repositories for which an arbitrary revision can be specified for A/B testing.
"triggerable_configurations" specifies a triggerable available on a given test on a given platform.
"roots" table which specifies the revision used in a given root set in each repository.
* init-database.sql: Added "build_triggerables", "triggerable_repositories",
"triggerable_configurations", and "roots" tables. Added references to "build_triggerables",
"platforms", and "tests" tables as well as columns to store status, status url, and creation time
to build_requests table. Also made each test group's name unique in a given analysis task as it
would be confusing to have multiple test groups of the same name.
* public/admin/tests.php: Added the UI and the code to associate a test with a triggerable.
* public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories
for which a specific revision can be set in an A/B testing on a given triggerable.
* public/api/build-requests.php: Added. Returns the list of open build requests on a specified
triggerable. Also updates the status' and the status urls of specified build requests when
buildRequestUpdates is provided in the raw POST data.
(main):
* public/api/runs.php:
(fetch_runs_for_config): Don't include results associated with a build request, meaning they are
results of an A/B testing.
* public/api/test-groups.php:
(main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back.
* public/api/triggerables.php: Added.
(main): Returns a list of triggerables or a triggerable associated with a given analysis task.
* public/include/admin-header.php:
* public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php.
(BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests
and root sets used in those requests.D
(BuildRequestsFetcher::__construct):
(BuildRequestsFetcher::fetch_for_task):
(BuildRequestsFetcher::fetch_for_group):
(BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable):
(BuildRequestsFetcher::has_results):
(BuildRequestsFetcher::results):
(BuildRequestsFetcher::results_with_resolved_ids):
(BuildRequestsFetcher::results_internal):
(BuildRequestsFetcher::root_sets):
(BuildRequestsFetcher::fetch_roots_for_set):
* public/include/db.php:
(Database::prefixed_column_names): Don't return "$prefix_" when there are no columns.
(Database::insert_row): Support taking an empty array for values. This is useful in "root_sets"
table since it only has the primary key, id, column.
(Database::select_or_insert_row):
(Database::update_or_insert_row):
(Database::update_row): Added.
(Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row
should be inserted when no row matches the specified criteria. This is used while updating
build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting
new build requests in that API.
(Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty.
This is used in public/api/triggerables.php when no analysis task is specified.
* public/include/json-header.php:
(find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the
triggerable associated with the closest ancestor of the test. Since issuing a new query for each
ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually
find the closest ancestor with a triggerable.
* public/include/report-processor.php:
(ReportProcessor::process):
(ReportProcessor::resolve_build_id): Associate a build request with the newly created build
if jobId or buildRequest is specified.
* public/include/test-name-resolver.php:
(TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that
test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change.
(TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on
a given platform.
* public/include/test-path-resolver.php: Added.
(TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire
"tests" table to do this since there could be arbitrary number of ancestors for a given test.
This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables
in order to compute full test metric names.
(TestPathResolver::__construct):
(TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to
the highest (a test without a parent).
(TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from
the highest ancestor to the test itself.
(TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map.
* public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups.
(main):
(commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions
for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding
set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will
result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]].
* public/v2/analysis.js:
(App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache.
(App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily.
(App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group.
(App.TestGroup.create): Added. Creates a new A/B testing group.
(App.Triggerable): Added.
(App.TriggerableAdapter): Added.
(App.TriggerableAdapter.buildURL): Added.
(App.BuildRequest.testGroup): Renamed from group.
(App.BuildRequest.orderLabel): Added. One-based index to be used in labels.
(App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used
in this build request.
(App.BuildRequest.status): Added.
(App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status.
(App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused.
* public/v2/app.js:
(App.AnalysisTaskController.testGroups): Added.
(App.AnalysisTaskController.possibleRepetitionCounts): Added.
(App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property
but an observer that updates "roots" property. Filter out the repositories that are not accepted
by the associated triggerable as they will be ignored.
(App.AnalysisTaskController.actions.createTestGroup): Added.
* public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action.
* tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back
the status of those builds to the perf dashboard. A similar script can be written to support
other continuous builds systems.
(main): Fetches the list of pending builds as well as currently running or completed builds from
a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule
a single new build on each builder with no pending builds, and marks the set of open build requests
that have been scheduled to run on the buildbot but not found in the first step as stale.
(load_config): Loads a JSON that contains the configurations for each builder. e.g.
[
{
"platform": "mac-mavericks",
"test": ["Parser", "html5-full-render.html"],
"builder": "Trunk Syrah Production Perf AB Tests",
"arguments": {
"forcescheduler": "force-mac-mavericks-release-perf",
"webkit_revision": "$WebKit",
"jobid": "$buildRequest"
}
}
]
(find_request_updates): Return a list of build request status updates to make based on the pending
builds as well as in-progress and completed builds on each builder on the buildbot. When a build is
completed, we use the special status "failedIfNotCompleted" which results in "failed" status only
if the build request had not been completed. This is necessary because a failed build will not
report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure.
(update_and_fetch_build_requests): Submit the build request status updates and retrieve the list
of open requests the perf dashboard has.
(find_stale_request_updates): Compute the list of build requests that have been scheduled on the
buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot
or human canceling a build may trigger such a state.
(schedule_request): Schedules a build with the arguments specified in the configuration JSON after
replacing repository names with their revisions and buildRequest with the build request id.
(config_for_request): Finds a builder for the test and the platform of a build request.
(fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth.
(property_value_from_build): Returns the value of a specific property in a buildbot build.
(request_id_from_build): Returns the build request id of a given buildbot build if there is one.
Canonical link: https://commits.webkit.org/158312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
|
|
|
CREATE INDEX build_request_build ON build_requests(request_build);
|