haikuwebkit/Websites/perf.webkit.org/init-database.sql

336 lines
16 KiB
MySQL
Raw Permalink Normal View History

Add mocha server tests for /api/build-requests https://bugs.webkit.org/show_bug.cgi?id=155831 Reviewed by Chris Dumez. Added the new mocha.js based server-tests for /api/build-requests. The new harness automatically: - starts a new Apache instance - switches the database during testing via setting an environmental variable - backups and restores public/data directory during testing As a result, developer no longer has to manually setup Apache, edit config.json manually to use a testing database, or run /api/manifest.php to re-generate the manifest file after testing. This patch also makes ID resolution optional on /api/build-requests so that v3 model based syncing scripts can re-use the same code as the v3 UI to process the JSON. tools/sync-with-buildbot.py has been modified to use this option (useLegacyIdResolution). * config.json: Added configurations for the test httpd server. * init-database.sql: Don't error when tables and types don't exist (when database is empty). * public/api/build-requests.php: (main): Made the ID resolution optional with useLegacyIdResolution. Also removed "updates" from the results JSON since it's never used. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_roots_for_set_if_needed): Fixed the bug that we would include the same commit multiple times for each root set. * public/include/db.php: (config): If present, use ORG_WEBKIT_PERF_CONFIG_PATH instead of Websites/perf.webkit.org/config.json. * server-tests: Added. * server-tests/api-build-requests-tests.js: Added. Tests for /api/build-requests. (.addMockData): * server-tests/resources: Added. * server-tests/resources/test-server.conf: Added. Apache configuration file for testing. * server-tests/resources/test-server.js: Added. (TestSever): Added. (TestSever.prototype.start): Added. (TestSever.prototype.stop): Added. (TestSever.prototype.remoteAPI): Added. Configures RemoteAPI to be used with the test sever. (TestSever.prototype.database): Added. Returns Database configured to use the test database. (TestSever.prototype._constructTestConfig): Creates config.json for testing. The file is generated by _start and db.php's config() reads it from the environmental variable: ORG_WEBKIT_PERF_CONFIG_PATH. (TestSever.prototype._ensureDataDirectory): Renames public/data to public/original-data if exists, and creates a new empty public/data. (TestSever.prototype._restoreDataDirectory): Deletes public/data and renames public/original-data back to public/data. (TestSever.prototype._ensureTestDatabase): Drops the test database if exists and creates a new one. (TestSever.prototype.initDatabase): Run init-database.sql to start each test with a consistent state. (TestSever.prototype._executePgsqlCommand): Executes a postgres command line tool such as psql. (TestSever.prototype._determinePgsqlDirectory): Finds the directory that contains psql. (TestSever.prototype._startApache): Starts an Apache instance for testing. (TestSever.prototype._stopApache): Stops the Apache instance for testing. (TestSever.prototype._waitForPid): Waits for the Apache pid file to appear or disappear. (before): Start the test server at the beginning. (beforeEach): Re-initialize all tables before each test. (after): Stop the test server at the end. * tools/js/config.js: (Config.prototype.path): (Config.prototype.serverRoot): Added. The path to Websites/perf.webkit.org/public/. (Config.prototype.pathFromRoot): Added. Resolves a path from Websites/perf.webkit.org. * tools/js/database.js: (Database): Now optionally takes the database name to use a different database during testing. (Database.prototype.connect): (Database.prototype.query): Added. (Database.prototype.insert): Added. (tableToPrefixMap): Maps table name to its prefix. Used by Database.insert. * tools/js/remote.js: Added. (RemoteAPI): Added. This is node.js equivalent of RemoteAPI in public/v3/remote.js. (RemoteAPI.prototype.configure): Added. (RemoteAPI.prototype.fetchJSON): Added. (RemoteAPI.prototype.fetchJSONWithStatus): Added. (RemoteAPI.prototype.sendHttpRequest): Added. * tools/sync-with-buildbot.py: (main): Use useLegacyIdResolution as this script relies on the legacy behavior. * unit-tests/checkconfig.js: pg was never directly used in this test. Canonical link: https://commits.webkit.org/173950@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@198642 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-03-24 20:17:01 +00:00
DROP TABLE IF EXISTS run_iterations CASCADE;
DROP TABLE IF EXISTS test_runs CASCADE;
DROP TABLE IF EXISTS test_configurations CASCADE;
DROP TYPE IF EXISTS test_configuration_type CASCADE;
DROP TABLE IF EXISTS aggregators CASCADE;
DROP TABLE IF EXISTS builds CASCADE;
DROP TABLE IF EXISTS committers CASCADE;
DROP TABLE IF EXISTS commits CASCADE;
DROP TABLE IF EXISTS build_commits CASCADE;
Add the ability to report a commit with sub-commits. https://bugs.webkit.org/show_bug.cgi?id=168962 Reviewed by Ryosuke Niwa. Introduce 'commit_ownerships' which records ownership between commits. On existing production server, run ``` CREATE TABLE commit_ownerships ( commit_owner integer NOT NULL REFERENCES commits ON DELETE CASCADE, commit_ownee integer NOT NULL REFERENCES commits ON DELETE CASCADE, PRIMARY KEY (commit_owner, commit_ownee) ); ALTER TABLE repositories RENAME repository_parent TO repository_owner; ALTER TABLE repositories DROP repository_name_must_be_unique; CREATE UNIQUE INDEX repository_name_owner_unique_index ON repositories (repository_owner, repository_name) WHERE repository_owner IS NOT NULL; CREATE UNIQUE INDEX repository_name_unique_index ON repositories (repository_name) WHERE repository_owner IS NULL; ``` to update database. Add unit-tests to cover this change. * init-database.sql: * public/api/report-commits.php: * public/include/commit-log-fetcher.php: * public/include/db.php: * public/include/manifest-generator.php: * public/include/report-processor.php: * public/v3/models/repository.js: (Repository): (Repository.prototype.owner): * server-tests/admin-reprocess-report-tests.js: (addBuilderForReport.simpleReportWithRevisions.0.then): (then): * server-tests/api-manifest.js: (then): * server-tests/api-report-commits-tests.js: (addSlaveForReport.sameRepositoryNameInSubCommitAndMajorCommit.then): (then): (addSlaveForReport.systemVersionCommitWithSubcommits.then): (addSlaveForReport.multipleSystemVersionCommitsWithSubcommits.then): (addSlaveForReport.systemVersionCommitWithEmptySubcommits.then): (addSlaveForReport.systemVersionCommitAndSubcommitWithTimestamp.then): * tools/js/database.js: Canonical link: https://commits.webkit.org/186535@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213788 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-13 09:10:29 +00:00
DROP TABLE IF EXISTS commit_ownerships CASCADE;
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
DROP TABLE IF EXISTS build_workers CASCADE;
Add mocha server tests for /api/build-requests https://bugs.webkit.org/show_bug.cgi?id=155831 Reviewed by Chris Dumez. Added the new mocha.js based server-tests for /api/build-requests. The new harness automatically: - starts a new Apache instance - switches the database during testing via setting an environmental variable - backups and restores public/data directory during testing As a result, developer no longer has to manually setup Apache, edit config.json manually to use a testing database, or run /api/manifest.php to re-generate the manifest file after testing. This patch also makes ID resolution optional on /api/build-requests so that v3 model based syncing scripts can re-use the same code as the v3 UI to process the JSON. tools/sync-with-buildbot.py has been modified to use this option (useLegacyIdResolution). * config.json: Added configurations for the test httpd server. * init-database.sql: Don't error when tables and types don't exist (when database is empty). * public/api/build-requests.php: (main): Made the ID resolution optional with useLegacyIdResolution. Also removed "updates" from the results JSON since it's never used. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_roots_for_set_if_needed): Fixed the bug that we would include the same commit multiple times for each root set. * public/include/db.php: (config): If present, use ORG_WEBKIT_PERF_CONFIG_PATH instead of Websites/perf.webkit.org/config.json. * server-tests: Added. * server-tests/api-build-requests-tests.js: Added. Tests for /api/build-requests. (.addMockData): * server-tests/resources: Added. * server-tests/resources/test-server.conf: Added. Apache configuration file for testing. * server-tests/resources/test-server.js: Added. (TestSever): Added. (TestSever.prototype.start): Added. (TestSever.prototype.stop): Added. (TestSever.prototype.remoteAPI): Added. Configures RemoteAPI to be used with the test sever. (TestSever.prototype.database): Added. Returns Database configured to use the test database. (TestSever.prototype._constructTestConfig): Creates config.json for testing. The file is generated by _start and db.php's config() reads it from the environmental variable: ORG_WEBKIT_PERF_CONFIG_PATH. (TestSever.prototype._ensureDataDirectory): Renames public/data to public/original-data if exists, and creates a new empty public/data. (TestSever.prototype._restoreDataDirectory): Deletes public/data and renames public/original-data back to public/data. (TestSever.prototype._ensureTestDatabase): Drops the test database if exists and creates a new one. (TestSever.prototype.initDatabase): Run init-database.sql to start each test with a consistent state. (TestSever.prototype._executePgsqlCommand): Executes a postgres command line tool such as psql. (TestSever.prototype._determinePgsqlDirectory): Finds the directory that contains psql. (TestSever.prototype._startApache): Starts an Apache instance for testing. (TestSever.prototype._stopApache): Stops the Apache instance for testing. (TestSever.prototype._waitForPid): Waits for the Apache pid file to appear or disappear. (before): Start the test server at the beginning. (beforeEach): Re-initialize all tables before each test. (after): Stop the test server at the end. * tools/js/config.js: (Config.prototype.path): (Config.prototype.serverRoot): Added. The path to Websites/perf.webkit.org/public/. (Config.prototype.pathFromRoot): Added. Resolves a path from Websites/perf.webkit.org. * tools/js/database.js: (Database): Now optionally takes the database name to use a different database during testing. (Database.prototype.connect): (Database.prototype.query): Added. (Database.prototype.insert): Added. (tableToPrefixMap): Maps table name to its prefix. Used by Database.insert. * tools/js/remote.js: Added. (RemoteAPI): Added. This is node.js equivalent of RemoteAPI in public/v3/remote.js. (RemoteAPI.prototype.configure): Added. (RemoteAPI.prototype.fetchJSON): Added. (RemoteAPI.prototype.fetchJSONWithStatus): Added. (RemoteAPI.prototype.sendHttpRequest): Added. * tools/sync-with-buildbot.py: (main): Use useLegacyIdResolution as this script relies on the legacy behavior. * unit-tests/checkconfig.js: pg was never directly used in this test. Canonical link: https://commits.webkit.org/173950@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@198642 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-03-24 20:17:01 +00:00
DROP TABLE IF EXISTS builders CASCADE;
DROP TABLE IF EXISTS repositories CASCADE;
DROP TABLE IF EXISTS platforms CASCADE;
Refactor 'platforms' table to contain group information. https://bugs.webkit.org/show_bug.cgi?id=193132 Reviewed by Ryosuke Niwa. Group 'platforms' so that bisecting can use commit sets from other platforms which are in the same platform group. This will help to enhance the bisecting capability to higher granularity. Added admin page to manage platform groups. Updated platform admin page to allow update platform group. SQL query to update existing database: ''' BEGIN; CREATE TABLE platform_groups ( platformgroup_id serial PRIMARY KEY, platformgroup_name varchar(64) NOT NULL, CONSTRAINT platform_group_name_must_be_unique UNIQUE (platformgroup_name)); ALTER TABLE platforms ADD COLUMN platform_group integer REFERENCES platform_groups DEFAULT NULL; END; ''' * init-database.sql: Added 'platform_group' column to 'platforms' table. Added 'platform_groups' table. * migrate-database.sql: Updated migration script accordingly. * public/admin/platform-groups.php: Added a page to manage platform groups. * public/admin/platforms.php: Added support to update platform group in admin page. * public/include/admin-header.php: Added link to page that maneges platform groups. * public/include/manifest-generator.php: Added 'platformGroup' in manifest. * public/v3/index.html: Included 'platform-groups.js'. * public/v3/models/analysis-task.js: (AnalysisTask.prototype.async.commitSetsFromTestGroupsAndMeasurementSet): Included measurement commit sets from other platforms in the same platform group. (AnalysisTask.prototype._commitSetForOtherPlatformsInSameGroup): Helper function to find measurment commit set for other platforms in same platform group. * public/v3/models/manifest.js: Added step to build 'PlatformGroup' instances. * public/v3/models/platform-group.js: Added 'PlatformGroup' to represent entries in 'platform_groups' table. (PlatformGroup): (PlatformGroup.prototype.addPlatform): (PlatformGroup.prototype.platforms): * public/v3/models/platform.js: Added itself to 'PlatformGroup' in constructor if it belongs to a group. (Platform.prototype.group): Return the platform group of current platform. * public/v3/models/time-series.js: Added helper function to show view between a given time range. (TimeSeries.prototype.viewBetweenTime): * server-tests/api-manifest-tests.js: Updated unit test to also test platform group initialization. * tools/js/database.js: Added prefix for 'platform_groups'. * tools/js/v3-models.js: Added import for platform group. Canonical link: https://commits.webkit.org/230971@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@269083 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-10-28 01:00:35 +00:00
DROP TABLE IF EXISTS platform_groups CASCADE;
Add mocha server tests for /api/build-requests https://bugs.webkit.org/show_bug.cgi?id=155831 Reviewed by Chris Dumez. Added the new mocha.js based server-tests for /api/build-requests. The new harness automatically: - starts a new Apache instance - switches the database during testing via setting an environmental variable - backups and restores public/data directory during testing As a result, developer no longer has to manually setup Apache, edit config.json manually to use a testing database, or run /api/manifest.php to re-generate the manifest file after testing. This patch also makes ID resolution optional on /api/build-requests so that v3 model based syncing scripts can re-use the same code as the v3 UI to process the JSON. tools/sync-with-buildbot.py has been modified to use this option (useLegacyIdResolution). * config.json: Added configurations for the test httpd server. * init-database.sql: Don't error when tables and types don't exist (when database is empty). * public/api/build-requests.php: (main): Made the ID resolution optional with useLegacyIdResolution. Also removed "updates" from the results JSON since it's never used. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_roots_for_set_if_needed): Fixed the bug that we would include the same commit multiple times for each root set. * public/include/db.php: (config): If present, use ORG_WEBKIT_PERF_CONFIG_PATH instead of Websites/perf.webkit.org/config.json. * server-tests: Added. * server-tests/api-build-requests-tests.js: Added. Tests for /api/build-requests. (.addMockData): * server-tests/resources: Added. * server-tests/resources/test-server.conf: Added. Apache configuration file for testing. * server-tests/resources/test-server.js: Added. (TestSever): Added. (TestSever.prototype.start): Added. (TestSever.prototype.stop): Added. (TestSever.prototype.remoteAPI): Added. Configures RemoteAPI to be used with the test sever. (TestSever.prototype.database): Added. Returns Database configured to use the test database. (TestSever.prototype._constructTestConfig): Creates config.json for testing. The file is generated by _start and db.php's config() reads it from the environmental variable: ORG_WEBKIT_PERF_CONFIG_PATH. (TestSever.prototype._ensureDataDirectory): Renames public/data to public/original-data if exists, and creates a new empty public/data. (TestSever.prototype._restoreDataDirectory): Deletes public/data and renames public/original-data back to public/data. (TestSever.prototype._ensureTestDatabase): Drops the test database if exists and creates a new one. (TestSever.prototype.initDatabase): Run init-database.sql to start each test with a consistent state. (TestSever.prototype._executePgsqlCommand): Executes a postgres command line tool such as psql. (TestSever.prototype._determinePgsqlDirectory): Finds the directory that contains psql. (TestSever.prototype._startApache): Starts an Apache instance for testing. (TestSever.prototype._stopApache): Stops the Apache instance for testing. (TestSever.prototype._waitForPid): Waits for the Apache pid file to appear or disappear. (before): Start the test server at the beginning. (beforeEach): Re-initialize all tables before each test. (after): Stop the test server at the end. * tools/js/config.js: (Config.prototype.path): (Config.prototype.serverRoot): Added. The path to Websites/perf.webkit.org/public/. (Config.prototype.pathFromRoot): Added. Resolves a path from Websites/perf.webkit.org. * tools/js/database.js: (Database): Now optionally takes the database name to use a different database during testing. (Database.prototype.connect): (Database.prototype.query): Added. (Database.prototype.insert): Added. (tableToPrefixMap): Maps table name to its prefix. Used by Database.insert. * tools/js/remote.js: Added. (RemoteAPI): Added. This is node.js equivalent of RemoteAPI in public/v3/remote.js. (RemoteAPI.prototype.configure): Added. (RemoteAPI.prototype.fetchJSON): Added. (RemoteAPI.prototype.fetchJSONWithStatus): Added. (RemoteAPI.prototype.sendHttpRequest): Added. * tools/sync-with-buildbot.py: (main): Use useLegacyIdResolution as this script relies on the legacy behavior. * unit-tests/checkconfig.js: pg was never directly used in this test. Canonical link: https://commits.webkit.org/173950@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@198642 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-03-24 20:17:01 +00:00
DROP TABLE IF EXISTS test_metrics CASCADE;
DROP TABLE IF EXISTS tests CASCADE;
DROP TABLE IF EXISTS reports CASCADE;
DROP TABLE IF EXISTS tracker_repositories CASCADE;
DROP TABLE IF EXISTS bug_trackers CASCADE;
DROP TABLE IF EXISTS task_commits CASCADE;
DROP TABLE IF EXISTS analysis_tasks CASCADE;
DROP TABLE IF EXISTS analysis_strategies CASCADE;
DROP TYPE IF EXISTS analysis_task_result_type CASCADE;
DROP TABLE IF EXISTS build_triggerables CASCADE;
DROP TABLE IF EXISTS triggerable_configurations CASCADE;
Introduce the notion of repository groups to triggerables https://bugs.webkit.org/show_bug.cgi?id=170228 Reviewed by Chris Dumez. On some triggerable, it's desirable to specify multiple sets of repositories that are accepted. For example, if a repository X transitioned from Subversion to Git, and if a triggerable accepted X and some other repository Y, then it's desirable to two sets: (X-Subversion, Y) and (X-Git, Y) since neither (X-Subversion, X-Git) nor (X-Subversion, X-Git, Y) makes sense as a set. This patch introduces triggerable_repository_groups table to represent a set of repositories accepted by a triggerable. It has many to one relationship to build_triggerables and triggerable_repositories in turn now has many to one relationship to triggerable_repository_groups instead of build_triggerables. Also make it possible to disable a triggerable e.g. a set of tests and platforms are no longer supported. We don't want to delete the triggerable completely from the database since it would result in the associated A/B testing results being purged, which is not desirale. To migrate an existing database, run the following transaction: ```sql BEGIN; ALTER TABLE build_triggerables ADD COLUMN triggerable_disabled boolean NOT NULL DEFAULT FALSE; CREATE TABLE triggerable_repository_groups ( repositorygroup_id serial PRIMARY KEY, repositorygroup_triggerable integer REFERENCES build_triggerables NOT NULL, repositorygroup_name varchar(256) NOT NULL, repositorygroup_description varchar(256), repositorygroup_accepts_roots boolean NOT NULL DEFAULT FALSE, CONSTRAINT repository_group_name_must_be_unique_for_triggerable UNIQUE(repositorygroup_triggerable, repositorygroup_name)); INSERT INTO triggerable_repository_groups (repositorygroup_triggerable, repositorygroup_name) SELECT triggerable_id, 'default' FROM build_triggerables; ALTER TABLE triggerable_repositories ADD COLUMN trigrepo_group integer REFERENCES triggerable_repository_groups; UPDATE triggerable_repositories SET trigrepo_group = repositorygroup_id FROM triggerable_repository_groups WHERE trigrepo_triggerable = repositorygroup_triggerable; ALTER TABLE triggerable_repositories ALTER COLUMN trigrepo_group SET NOT NULL; ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_triggerable; ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_sub_roots; END; ``` * init-database.sql: * public/admin/triggerables.php: Use a custom column to make forms to add and configure repository groups. (insert_triggerable_repositories): Added. (generate_repository_list): Added. (generate_repository_form): Added. (generate_repository_checkboxes): Now generates checkboxes for a repository group instead of a triggerable. * public/include/manifest-generator.php: (fetch_triggerables): Fixed the bug that we were not filtering results with query in /api/triggerable. Rewrote it to include an array of repository groups, which in turn contains an array of repositories along with its name and a description, and a boolean indicating whether it accepts a custom root file or not. The boolean will be used when we're adding the support for perf try bots. We will keep acceptedRepositories since it's still used by detect-changes.js. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Resolve repositoriy, test, and platform IDs to their respective objects. * public/v3/models/triggerable.js: (Triggerable): (Triggerable.prototype.isDisabled): Added. (Triggerable.prototype.repositoryGroups): Added. (Triggerable.prototype.acceptsTest): Added. (TriggerableRepositoryGroup): Added. (TriggerableRepositoryGroup.prototype.description): Added. (TriggerableRepositoryGroup.prototype.acceptsCustomRoots): Added. (TriggerableRepositoryGroup.prototype.repositories): Added. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't use a disabled triggerable. * server-tests/api-manifest-tests.js: Updated a test case to test repository groups. * tools/js/database.js: (tableToPrefixMap): Added triggerable_repository_groups. * tools/js/v3-models.js: Imported TriggerableRepositoryGroup from triggerable.js. Canonical link: https://commits.webkit.org/187454@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214975 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-05 23:12:46 +00:00
DROP TABLE IF EXISTS triggerable_repository_groups CASCADE;
Add mocha server tests for /api/build-requests https://bugs.webkit.org/show_bug.cgi?id=155831 Reviewed by Chris Dumez. Added the new mocha.js based server-tests for /api/build-requests. The new harness automatically: - starts a new Apache instance - switches the database during testing via setting an environmental variable - backups and restores public/data directory during testing As a result, developer no longer has to manually setup Apache, edit config.json manually to use a testing database, or run /api/manifest.php to re-generate the manifest file after testing. This patch also makes ID resolution optional on /api/build-requests so that v3 model based syncing scripts can re-use the same code as the v3 UI to process the JSON. tools/sync-with-buildbot.py has been modified to use this option (useLegacyIdResolution). * config.json: Added configurations for the test httpd server. * init-database.sql: Don't error when tables and types don't exist (when database is empty). * public/api/build-requests.php: (main): Made the ID resolution optional with useLegacyIdResolution. Also removed "updates" from the results JSON since it's never used. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_roots_for_set_if_needed): Fixed the bug that we would include the same commit multiple times for each root set. * public/include/db.php: (config): If present, use ORG_WEBKIT_PERF_CONFIG_PATH instead of Websites/perf.webkit.org/config.json. * server-tests: Added. * server-tests/api-build-requests-tests.js: Added. Tests for /api/build-requests. (.addMockData): * server-tests/resources: Added. * server-tests/resources/test-server.conf: Added. Apache configuration file for testing. * server-tests/resources/test-server.js: Added. (TestSever): Added. (TestSever.prototype.start): Added. (TestSever.prototype.stop): Added. (TestSever.prototype.remoteAPI): Added. Configures RemoteAPI to be used with the test sever. (TestSever.prototype.database): Added. Returns Database configured to use the test database. (TestSever.prototype._constructTestConfig): Creates config.json for testing. The file is generated by _start and db.php's config() reads it from the environmental variable: ORG_WEBKIT_PERF_CONFIG_PATH. (TestSever.prototype._ensureDataDirectory): Renames public/data to public/original-data if exists, and creates a new empty public/data. (TestSever.prototype._restoreDataDirectory): Deletes public/data and renames public/original-data back to public/data. (TestSever.prototype._ensureTestDatabase): Drops the test database if exists and creates a new one. (TestSever.prototype.initDatabase): Run init-database.sql to start each test with a consistent state. (TestSever.prototype._executePgsqlCommand): Executes a postgres command line tool such as psql. (TestSever.prototype._determinePgsqlDirectory): Finds the directory that contains psql. (TestSever.prototype._startApache): Starts an Apache instance for testing. (TestSever.prototype._stopApache): Stops the Apache instance for testing. (TestSever.prototype._waitForPid): Waits for the Apache pid file to appear or disappear. (before): Start the test server at the beginning. (beforeEach): Re-initialize all tables before each test. (after): Stop the test server at the end. * tools/js/config.js: (Config.prototype.path): (Config.prototype.serverRoot): Added. The path to Websites/perf.webkit.org/public/. (Config.prototype.pathFromRoot): Added. Resolves a path from Websites/perf.webkit.org. * tools/js/database.js: (Database): Now optionally takes the database name to use a different database during testing. (Database.prototype.connect): (Database.prototype.query): Added. (Database.prototype.insert): Added. (tableToPrefixMap): Maps table name to its prefix. Used by Database.insert. * tools/js/remote.js: Added. (RemoteAPI): Added. This is node.js equivalent of RemoteAPI in public/v3/remote.js. (RemoteAPI.prototype.configure): Added. (RemoteAPI.prototype.fetchJSON): Added. (RemoteAPI.prototype.fetchJSONWithStatus): Added. (RemoteAPI.prototype.sendHttpRequest): Added. * tools/sync-with-buildbot.py: (main): Use useLegacyIdResolution as this script relies on the legacy behavior. * unit-tests/checkconfig.js: pg was never directly used in this test. Canonical link: https://commits.webkit.org/173950@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@198642 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-03-24 20:17:01 +00:00
DROP TABLE IF EXISTS triggerable_repositories CASCADE;
Add the file uploading capability to the perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=169737 Reviewed by Chris Dumez. Added /privileged-api/upload-file to upload a file, and /api/uploaded-file/ to download the file and retrieve its meta data based on its SHA256. We treat two files with the identical SHA256 as identical since anyone who can upload a file using this mechanism can execute arbitrary code in our bots anyway. This is important for avoiding uploading a large darwinup roots multiple times to the server, saving both user's time/bandwidth and server's disk space. * config.json: Added uploadDirectory, uploadFileLimitInMB, and uploadUserQuotaInMB as options. * init-database.sql: Added uploaded_files table. * public/api/uploaded-file.php: Added. (main): /api/uploaded-file/N would download uploaded_file with id=N. /api/uploaded-file/?sha256=X would return the meta data for uploaded_file with sha256=X. (stream_file_content): Streams the file content in 64KB chunks. We support Range & If-Range HTTP request headers so that browsers can pause and resume downloading of a large root file. (parse_range_header): Parses Range HTTP request header. * public/include/json-header.php: (remote_user_name): Use the default argument of NULL. * public/include/manifest-generator.php: (ManifestGenerator::generate): Include the maximum upload size in the manifest file to let the frontend code preemptively check the file size before attempting to submit a file. * public/include/uploaded-file-helpers.php: Added. (format_uploaded_file): (uploaded_file_path_for_row): * public/privileged-api/upload-file-form.html: Added. For debugging purposes. (fetchCSRFfToken): (upload): * public/privileged-api/upload-file.php: Added. (main): (query_total_file_size): (create_uploaded_file_from_form_data): * public/shared/common-remote.js: (CommonRemoteAPI.prototype.postFormData): Added. (CommonRemoteAPI.prototype.postFormDataWithStatus): Added. (CommonRemoteAPI.prototype.sendHttpRequestWithFormData): Added. (CommonRemoteAPI.prototype._asJSON): Throw an exception instead of calling a non-existent reject. * public/v3/models/uploaded-file.js: Added. (UploadedFile): Added. (UploadedFile.uploadFile): Added. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Added. Finds the file with the same SHA256 in the server to avoid uploading a large custom root multiple times. (UploadedFile._computeSHA256Hash): Added. * public/v3/privileged-api.js: (PrivilegedAPI.prototype.sendRequest): Added the options dictionary as a third argument. For now, only support useFormData boolean. * public/v3/remote.js: (BrowserRemoteAPI.prototype.sendHttpRequestWithFormData): Added. * server-tests/api-manifest.js: Updated per the inclusion of fileUploadSizeLimit in the manifest. * server-tests/api-uploaded-file.js: Added. * server-tests/privileged-api-upload-file-tests.js: Added. * server-tests/resources/temporary-file.js: Added. (TemporaryFile): Added. A helper class for creating a temporary file to upload. (TemporaryFile.makeTemporaryFileOfSizeInMB): (TemporaryFile.makeTemporaryFile): (TemporaryFile.inject): * server-tests/resources/test-server.conf: Set upload_max_filesize and post_max_size for testing. * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): Use uploadFileLimitInMB and uploadUserQuotaInMB of 2MB and 5MB. (TestServer.prototype._ensureDataDirectory): Create a directory to store uploaded files inside the data directory. In a production server, we can place it outside ServerRoot / DocumentRoot. (TestServer.prototype.cleanDataDirectory): Delete the aforementioned directory as needed. * tools/js/database.js: (tableToPrefixMap): Added uploaded_files. * tools/js/remote.js: (NodeRemoteAPI.prototype.sendHttpRequest): Added a dictionary to specify request headers and a callback to process the response as arguments. Fixed the bug that any 2xx code other than 200 was resulting in a rejected promise. Also include the response headers in the result for tests. Finally, when content is a function, call that instead of writing the content since FormData requires a custom logic. (NodeRemoteAPI.prototype.sendHttpRequestWithFormData): Added. * tools/js/v3-models.js: Include uploaded-file.js. * tools/run-tests.py: (main): Add form-data as a new dependency. Canonical link: https://commits.webkit.org/186730@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214065 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-16 20:53:35 +00:00
DROP TABLE IF EXISTS uploaded_files CASCADE;
Add mocha server tests for /api/build-requests https://bugs.webkit.org/show_bug.cgi?id=155831 Reviewed by Chris Dumez. Added the new mocha.js based server-tests for /api/build-requests. The new harness automatically: - starts a new Apache instance - switches the database during testing via setting an environmental variable - backups and restores public/data directory during testing As a result, developer no longer has to manually setup Apache, edit config.json manually to use a testing database, or run /api/manifest.php to re-generate the manifest file after testing. This patch also makes ID resolution optional on /api/build-requests so that v3 model based syncing scripts can re-use the same code as the v3 UI to process the JSON. tools/sync-with-buildbot.py has been modified to use this option (useLegacyIdResolution). * config.json: Added configurations for the test httpd server. * init-database.sql: Don't error when tables and types don't exist (when database is empty). * public/api/build-requests.php: (main): Made the ID resolution optional with useLegacyIdResolution. Also removed "updates" from the results JSON since it's never used. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_roots_for_set_if_needed): Fixed the bug that we would include the same commit multiple times for each root set. * public/include/db.php: (config): If present, use ORG_WEBKIT_PERF_CONFIG_PATH instead of Websites/perf.webkit.org/config.json. * server-tests: Added. * server-tests/api-build-requests-tests.js: Added. Tests for /api/build-requests. (.addMockData): * server-tests/resources: Added. * server-tests/resources/test-server.conf: Added. Apache configuration file for testing. * server-tests/resources/test-server.js: Added. (TestSever): Added. (TestSever.prototype.start): Added. (TestSever.prototype.stop): Added. (TestSever.prototype.remoteAPI): Added. Configures RemoteAPI to be used with the test sever. (TestSever.prototype.database): Added. Returns Database configured to use the test database. (TestSever.prototype._constructTestConfig): Creates config.json for testing. The file is generated by _start and db.php's config() reads it from the environmental variable: ORG_WEBKIT_PERF_CONFIG_PATH. (TestSever.prototype._ensureDataDirectory): Renames public/data to public/original-data if exists, and creates a new empty public/data. (TestSever.prototype._restoreDataDirectory): Deletes public/data and renames public/original-data back to public/data. (TestSever.prototype._ensureTestDatabase): Drops the test database if exists and creates a new one. (TestSever.prototype.initDatabase): Run init-database.sql to start each test with a consistent state. (TestSever.prototype._executePgsqlCommand): Executes a postgres command line tool such as psql. (TestSever.prototype._determinePgsqlDirectory): Finds the directory that contains psql. (TestSever.prototype._startApache): Starts an Apache instance for testing. (TestSever.prototype._stopApache): Stops the Apache instance for testing. (TestSever.prototype._waitForPid): Waits for the Apache pid file to appear or disappear. (before): Start the test server at the beginning. (beforeEach): Re-initialize all tables before each test. (after): Stop the test server at the end. * tools/js/config.js: (Config.prototype.path): (Config.prototype.serverRoot): Added. The path to Websites/perf.webkit.org/public/. (Config.prototype.pathFromRoot): Added. Resolves a path from Websites/perf.webkit.org. * tools/js/database.js: (Database): Now optionally takes the database name to use a different database during testing. (Database.prototype.connect): (Database.prototype.query): Added. (Database.prototype.insert): Added. (tableToPrefixMap): Maps table name to its prefix. Used by Database.insert. * tools/js/remote.js: Added. (RemoteAPI): Added. This is node.js equivalent of RemoteAPI in public/v3/remote.js. (RemoteAPI.prototype.configure): Added. (RemoteAPI.prototype.fetchJSON): Added. (RemoteAPI.prototype.fetchJSONWithStatus): Added. (RemoteAPI.prototype.sendHttpRequest): Added. * tools/sync-with-buildbot.py: (main): Use useLegacyIdResolution as this script relies on the legacy behavior. * unit-tests/checkconfig.js: pg was never directly used in this test. Canonical link: https://commits.webkit.org/173950@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@198642 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-03-24 20:17:01 +00:00
DROP TABLE IF EXISTS bugs CASCADE;
DROP TABLE IF EXISTS analysis_test_groups CASCADE;
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
DROP TYPE IF EXISTS analysis_test_group_repetition_type CASCADE;
Rename RootSet to CommitSet https://bugs.webkit.org/show_bug.cgi?id=169580 Rubber-stamped by Chris Dumez. Renamed root_sets to commit_sets and roots to commit_set_relationships in the database schema, and renamed related classes in public/v3/ and tools accordingly. RootSet, MeasurementRootSet, and CustomRootSet are respectively renamed to CommitSet, MeasurementCommitSet, and CustomCommitSet. In order to migrate the database, run: ``` BEGIN; ALTER TABLE root_sets RENAME TO commit_sets; ALTER TABLE commit_sets RENAME COLUMN rootset_id TO commitset_id; ALTER TABLE roots RENAME TO commit_set_relationships; ALTER TABLE commit_set_relationships RENAME COLUMN root_set TO commitset_set; ALTER TABLE commit_set_relationships RENAME COLUMN root_commit TO commitset_commit; ALTER TABLE build_requests RENAME COLUMN request_root_set TO request_commit_set; END; ``` * browser-tests/index.html: * init-database.sql: * public/api/build-requests.php: (main): * public/api/test-groups.php: (main): (format_test_group): * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::commit_sets): Renamed from root_sets. (BuildRequestsFetcher::commits): Renamed from roots. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Renamed from fetch_roots_for_set_if_needed. * public/privileged-api/create-test-group.php: (main): (ensure_commit_sets): Renamed from commit_sets_from_root_sets. * public/v3/components/analysis-results-viewer.js: (AnalysisResultsViewer.prototype.buildRowGroups): (AnalysisResultsViewer.prototype._collectCommitSetsInTestGroups): Renamed from _collectRootSetsInTestGroups. (AnalysisResultsViewer.prototype._buildRowsForPointsAndTestGroups): (AnalysisResultsViewer.prototype._buildRowsForPointsAndTestGroups): (AnalysisResultsViewer.CommitSetInTestGroup): Renamed from RootSetInTestGroup. (AnalysisResultsViewer.CommitSetInTestGroup.prototype.constructor): (AnalysisResultsViewer.CommitSetInTestGroup.prototype.commitSet): Renamed from rootSet. (AnalysisResultsViewer.CommitSetInTestGroup.prototype.succeedingCommitSet): Renamed from succeedingRootSet. (AnalysisResultsViewer.TestGroupStackingBlock.prototype.constructor): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.addRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.isComplete): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.startRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.endRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype._computeTestGroupStatus): * public/v3/components/chart-revision-range.js: (ChartRevisionRange.prototype._revisionForPoint): (ChartRevisionRange.prototype._computeRevisionList): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.constructor): (CustomizableTestGroupForm.prototype.setCommitSetMap): Renamed from setRootSetMap. (CustomizableTestGroupForm.prototype._submitted): (CustomizableTestGroupForm.prototype._computeCommitSetMap): Renamed from _computeRootSetMap. (CustomizableTestGroupForm.prototype.render): Renamed from render. (CustomizableTestGroupForm.prototype._constructRevisionRadioButtons): * public/v3/components/results-table.js: (ResultsTable.prototype.render): (ResultsTable.prototype._createRevisionListCells): (ResultsTable.prototype._computeRepositoryList): (ResultsTableRow.prototype.constructor): (ResultsTableRow.prototype.commitSet): Renamed from rootSet. * public/v3/components/test-group-results-table.js: (TestGroupResultsTable.prototype.buildRowGroups): * public/v3/index.html: * public/v3/models/build-request.js: (BuildRequest.prototype.constructor): (BuildRequest.prototype.updateSingleton): (BuildRequest.prototype.commitSet): Renamed from rootSet. (BuildRequest.constructBuildRequestsFromData): * public/v3/models/commit-set.js: Renamed from public/v3/models/root-set.js. (CommitSet): Renamed from RootSet. (CommitSet.containsMultipleCommitsForRepository): (MeasurementCommitSet): Renamed from MeasurementRootSet. (MeasurementCommitSet.prototype.namedStaticMap): (MeasurementCommitSet.prototype.ensureNamedStaticMap): (MeasurementCommitSet.namedStaticMap): (MeasurementCommitSet.ensureNamedStaticMap): (MeasurementCommitSet.ensureSingleton): (CustomCommitSet): Renamed from CustomRootSet. * public/v3/models/measurement-adaptor.js: (MeasurementAdaptor.prototype.applyTo): * public/v3/models/test-group.js: (TestGroup.prototype.constructor): (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.repetitionCount): (TestGroup.prototype.requestedCommitSets): Renamed from requestedRootSets. (TestGroup.prototype.requestsForCommitSet): Renamed from requestsForRootSet. (TestGroup.prototype.labelForCommitSet): Renamed from labelForRootSet. (TestGroup.prototype.didSetResult): (TestGroup.prototype.compareTestResults): (TestGroup.prototype._valuesForCommitSet): Renamed from _valuesForRootSet. (TestGroup.prototype.createAndRefetchTestGroups): * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype.render): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype._createNewTestGroupFromChart): (AnalysisTaskPage.prototype._createNewTestGroupFromViewer): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): * server-tests/api-build-requests-tests.js: * server-tests/resources/mock-data.js: (MockData.resetV3Models): (MockData.addMockData): (MockData.addAnotherMockTestGroup): * tools/detect-changes.js: (createAnalysisTaskAndNotify): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype._propertiesForBuildRequest): (BuildbotSyncer.prototype._revisionSetFromCommitSetWithExclusionList): * tools/js/database.js: (tableToPrefixMap): * tools/js/v3-models.js: * tools/sync-buildbot.js: (syncLoop): * tools/sync-with-buildbot.py: Deleted. No longer used. * unit-tests/analysis-task-tests.js: * unit-tests/build-request-tests.js: (sampleBuildRequestData): * unit-tests/buildbot-syncer-tests.js: (sampleCommitSetData): * unit-tests/measurement-adaptor-tests.js: * unit-tests/measurement-set-tests.js: * unit-tests/resources/mock-v3-models.js: (MockModels.inject): * unit-tests/test-groups-tests.js: (sampleTestGroup): Canonical link: https://commits.webkit.org/186635@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213952 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-14 23:06:40 +00:00
DROP TABLE IF EXISTS commit_sets CASCADE;
DROP TABLE IF EXISTS commit_set_items CASCADE;
Add mocha server tests for /api/build-requests https://bugs.webkit.org/show_bug.cgi?id=155831 Reviewed by Chris Dumez. Added the new mocha.js based server-tests for /api/build-requests. The new harness automatically: - starts a new Apache instance - switches the database during testing via setting an environmental variable - backups and restores public/data directory during testing As a result, developer no longer has to manually setup Apache, edit config.json manually to use a testing database, or run /api/manifest.php to re-generate the manifest file after testing. This patch also makes ID resolution optional on /api/build-requests so that v3 model based syncing scripts can re-use the same code as the v3 UI to process the JSON. tools/sync-with-buildbot.py has been modified to use this option (useLegacyIdResolution). * config.json: Added configurations for the test httpd server. * init-database.sql: Don't error when tables and types don't exist (when database is empty). * public/api/build-requests.php: (main): Made the ID resolution optional with useLegacyIdResolution. Also removed "updates" from the results JSON since it's never used. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_roots_for_set_if_needed): Fixed the bug that we would include the same commit multiple times for each root set. * public/include/db.php: (config): If present, use ORG_WEBKIT_PERF_CONFIG_PATH instead of Websites/perf.webkit.org/config.json. * server-tests: Added. * server-tests/api-build-requests-tests.js: Added. Tests for /api/build-requests. (.addMockData): * server-tests/resources: Added. * server-tests/resources/test-server.conf: Added. Apache configuration file for testing. * server-tests/resources/test-server.js: Added. (TestSever): Added. (TestSever.prototype.start): Added. (TestSever.prototype.stop): Added. (TestSever.prototype.remoteAPI): Added. Configures RemoteAPI to be used with the test sever. (TestSever.prototype.database): Added. Returns Database configured to use the test database. (TestSever.prototype._constructTestConfig): Creates config.json for testing. The file is generated by _start and db.php's config() reads it from the environmental variable: ORG_WEBKIT_PERF_CONFIG_PATH. (TestSever.prototype._ensureDataDirectory): Renames public/data to public/original-data if exists, and creates a new empty public/data. (TestSever.prototype._restoreDataDirectory): Deletes public/data and renames public/original-data back to public/data. (TestSever.prototype._ensureTestDatabase): Drops the test database if exists and creates a new one. (TestSever.prototype.initDatabase): Run init-database.sql to start each test with a consistent state. (TestSever.prototype._executePgsqlCommand): Executes a postgres command line tool such as psql. (TestSever.prototype._determinePgsqlDirectory): Finds the directory that contains psql. (TestSever.prototype._startApache): Starts an Apache instance for testing. (TestSever.prototype._stopApache): Stops the Apache instance for testing. (TestSever.prototype._waitForPid): Waits for the Apache pid file to appear or disappear. (before): Start the test server at the beginning. (beforeEach): Re-initialize all tables before each test. (after): Stop the test server at the end. * tools/js/config.js: (Config.prototype.path): (Config.prototype.serverRoot): Added. The path to Websites/perf.webkit.org/public/. (Config.prototype.pathFromRoot): Added. Resolves a path from Websites/perf.webkit.org. * tools/js/database.js: (Database): Now optionally takes the database name to use a different database during testing. (Database.prototype.connect): (Database.prototype.query): Added. (Database.prototype.insert): Added. (tableToPrefixMap): Maps table name to its prefix. Used by Database.insert. * tools/js/remote.js: Added. (RemoteAPI): Added. This is node.js equivalent of RemoteAPI in public/v3/remote.js. (RemoteAPI.prototype.configure): Added. (RemoteAPI.prototype.fetchJSON): Added. (RemoteAPI.prototype.fetchJSONWithStatus): Added. (RemoteAPI.prototype.sendHttpRequest): Added. * tools/sync-with-buildbot.py: (main): Use useLegacyIdResolution as this script relies on the legacy behavior. * unit-tests/checkconfig.js: pg was never directly used in this test. Canonical link: https://commits.webkit.org/173950@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@198642 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-03-24 20:17:01 +00:00
DROP TABLE IF EXISTS build_requests CASCADE;
DROP TYPE IF EXISTS build_request_status_type CASCADE;
New perf dashboard should not duplicate author information in each commit https://bugs.webkit.org/show_bug.cgi?id=139756 Reviewed by Darin Adler. Instead of each commit having author name and email, make it reference a newly added committers table. Also replace "email" by "account" since some repositories don't use emails as account names. This improves the keyword search performance in commits.php since LIKE now runs on committers table, which only contains as many rows as there are accounts in each repository, instead of commits table which contains every commit ever happened in each repository. To migrate an existing database into match the new schema, run: BEGIN; INSERT INTO committers (committer_repository, committer_name, committer_email) (SELECT DISTINCT commit_repository, commit_author_name, commit_author_email FROM commits WHERE commit_author_email IS NOT NULL); ALTER TABLE commits ADD COLUMN commit_committer integer REFERENCES committers ON DELETE CASCADE; UPDATE commits SET commit_committer = committer_id FROM committers WHERE commit_repository = committer_repository AND commit_author_email = committer_email; ALTER TABLE commits DROP COLUMN commit_author_name CASCADE; ALTER TABLE commits DROP COLUMN commit_author_email CASCADE; COMMIT; * init-database.sql: Added committers table, and replaced author columns in commits table by a foreign reference to committers. Also added the missing drop table statements. * public/api/commits.php: (main): Fetch the corresponding committers row for a single commit. Also wrap a single commit by an array here instead of doing it in format_commit. (fetch_commits_between): Updated queries to JOIN commits with committers. (format_commit): Takes both commit and committers rows. Also don't wrap the result in an array as that is now done in main. * public/api/report-commits.php: (main): Store the reported committer information or update the existing entry if there is one. * tests/admin-regenerate-manifest.js: Fixed tests. * tests/api-report-commits.js: Ditto. Also added a test for updating an existing committer entry. * tools/pull-svn.py: Renamed email to account. (main): (fetch_commit_and_resolve_author): (fetch_commit): (resolve_author_name_from_account): (resolve_author_name_from_email): Deleted. Canonical link: https://commits.webkit.org/157706@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@177530 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-12-18 22:53:52 +00:00
Refactor 'platforms' table to contain group information. https://bugs.webkit.org/show_bug.cgi?id=193132 Reviewed by Ryosuke Niwa. Group 'platforms' so that bisecting can use commit sets from other platforms which are in the same platform group. This will help to enhance the bisecting capability to higher granularity. Added admin page to manage platform groups. Updated platform admin page to allow update platform group. SQL query to update existing database: ''' BEGIN; CREATE TABLE platform_groups ( platformgroup_id serial PRIMARY KEY, platformgroup_name varchar(64) NOT NULL, CONSTRAINT platform_group_name_must_be_unique UNIQUE (platformgroup_name)); ALTER TABLE platforms ADD COLUMN platform_group integer REFERENCES platform_groups DEFAULT NULL; END; ''' * init-database.sql: Added 'platform_group' column to 'platforms' table. Added 'platform_groups' table. * migrate-database.sql: Updated migration script accordingly. * public/admin/platform-groups.php: Added a page to manage platform groups. * public/admin/platforms.php: Added support to update platform group in admin page. * public/include/admin-header.php: Added link to page that maneges platform groups. * public/include/manifest-generator.php: Added 'platformGroup' in manifest. * public/v3/index.html: Included 'platform-groups.js'. * public/v3/models/analysis-task.js: (AnalysisTask.prototype.async.commitSetsFromTestGroupsAndMeasurementSet): Included measurement commit sets from other platforms in the same platform group. (AnalysisTask.prototype._commitSetForOtherPlatformsInSameGroup): Helper function to find measurment commit set for other platforms in same platform group. * public/v3/models/manifest.js: Added step to build 'PlatformGroup' instances. * public/v3/models/platform-group.js: Added 'PlatformGroup' to represent entries in 'platform_groups' table. (PlatformGroup): (PlatformGroup.prototype.addPlatform): (PlatformGroup.prototype.platforms): * public/v3/models/platform.js: Added itself to 'PlatformGroup' in constructor if it belongs to a group. (Platform.prototype.group): Return the platform group of current platform. * public/v3/models/time-series.js: Added helper function to show view between a given time range. (TimeSeries.prototype.viewBetweenTime): * server-tests/api-manifest-tests.js: Updated unit test to also test platform group initialization. * tools/js/database.js: Added prefix for 'platform_groups'. * tools/js/v3-models.js: Added import for platform group. Canonical link: https://commits.webkit.org/230971@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@269083 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-10-28 01:00:35 +00:00
CREATE TABLE platform_groups (
platformgroup_id serial PRIMARY KEY,
platformgroup_name varchar(64) NOT NULL,
CONSTRAINT platform_group_name_must_be_unique UNIQUE (platformgroup_name));
CREATE TABLE platforms (
platform_id serial PRIMARY KEY,
platform_name varchar(64) NOT NULL,
Refactor 'platforms' table to contain group information. https://bugs.webkit.org/show_bug.cgi?id=193132 Reviewed by Ryosuke Niwa. Group 'platforms' so that bisecting can use commit sets from other platforms which are in the same platform group. This will help to enhance the bisecting capability to higher granularity. Added admin page to manage platform groups. Updated platform admin page to allow update platform group. SQL query to update existing database: ''' BEGIN; CREATE TABLE platform_groups ( platformgroup_id serial PRIMARY KEY, platformgroup_name varchar(64) NOT NULL, CONSTRAINT platform_group_name_must_be_unique UNIQUE (platformgroup_name)); ALTER TABLE platforms ADD COLUMN platform_group integer REFERENCES platform_groups DEFAULT NULL; END; ''' * init-database.sql: Added 'platform_group' column to 'platforms' table. Added 'platform_groups' table. * migrate-database.sql: Updated migration script accordingly. * public/admin/platform-groups.php: Added a page to manage platform groups. * public/admin/platforms.php: Added support to update platform group in admin page. * public/include/admin-header.php: Added link to page that maneges platform groups. * public/include/manifest-generator.php: Added 'platformGroup' in manifest. * public/v3/index.html: Included 'platform-groups.js'. * public/v3/models/analysis-task.js: (AnalysisTask.prototype.async.commitSetsFromTestGroupsAndMeasurementSet): Included measurement commit sets from other platforms in the same platform group. (AnalysisTask.prototype._commitSetForOtherPlatformsInSameGroup): Helper function to find measurment commit set for other platforms in same platform group. * public/v3/models/manifest.js: Added step to build 'PlatformGroup' instances. * public/v3/models/platform-group.js: Added 'PlatformGroup' to represent entries in 'platform_groups' table. (PlatformGroup): (PlatformGroup.prototype.addPlatform): (PlatformGroup.prototype.platforms): * public/v3/models/platform.js: Added itself to 'PlatformGroup' in constructor if it belongs to a group. (Platform.prototype.group): Return the platform group of current platform. * public/v3/models/time-series.js: Added helper function to show view between a given time range. (TimeSeries.prototype.viewBetweenTime): * server-tests/api-manifest-tests.js: Updated unit test to also test platform group initialization. * tools/js/database.js: Added prefix for 'platform_groups'. * tools/js/v3-models.js: Added import for platform group. Canonical link: https://commits.webkit.org/230971@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@269083 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-10-28 01:00:35 +00:00
platform_group integer REFERENCES platform_groups DEFAULT NULL,
platform_hidden boolean NOT NULL DEFAULT FALSE);
CREATE TABLE repositories (
repository_id serial PRIMARY KEY,
Add the ability to report a commit with sub-commits. https://bugs.webkit.org/show_bug.cgi?id=168962 Reviewed by Ryosuke Niwa. Introduce 'commit_ownerships' which records ownership between commits. On existing production server, run ``` CREATE TABLE commit_ownerships ( commit_owner integer NOT NULL REFERENCES commits ON DELETE CASCADE, commit_ownee integer NOT NULL REFERENCES commits ON DELETE CASCADE, PRIMARY KEY (commit_owner, commit_ownee) ); ALTER TABLE repositories RENAME repository_parent TO repository_owner; ALTER TABLE repositories DROP repository_name_must_be_unique; CREATE UNIQUE INDEX repository_name_owner_unique_index ON repositories (repository_owner, repository_name) WHERE repository_owner IS NOT NULL; CREATE UNIQUE INDEX repository_name_unique_index ON repositories (repository_name) WHERE repository_owner IS NULL; ``` to update database. Add unit-tests to cover this change. * init-database.sql: * public/api/report-commits.php: * public/include/commit-log-fetcher.php: * public/include/db.php: * public/include/manifest-generator.php: * public/include/report-processor.php: * public/v3/models/repository.js: (Repository): (Repository.prototype.owner): * server-tests/admin-reprocess-report-tests.js: (addBuilderForReport.simpleReportWithRevisions.0.then): (then): * server-tests/api-manifest.js: (then): * server-tests/api-report-commits-tests.js: (addSlaveForReport.sameRepositoryNameInSubCommitAndMajorCommit.then): (then): (addSlaveForReport.systemVersionCommitWithSubcommits.then): (addSlaveForReport.multipleSystemVersionCommitsWithSubcommits.then): (addSlaveForReport.systemVersionCommitWithEmptySubcommits.then): (addSlaveForReport.systemVersionCommitAndSubcommitWithTimestamp.then): * tools/js/database.js: Canonical link: https://commits.webkit.org/186535@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213788 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-13 09:10:29 +00:00
repository_owner integer REFERENCES repositories ON DELETE CASCADE,
repository_name varchar(64) NOT NULL,
repository_url varchar(1024),
Add the ability to report a commit with sub-commits. https://bugs.webkit.org/show_bug.cgi?id=168962 Reviewed by Ryosuke Niwa. Introduce 'commit_ownerships' which records ownership between commits. On existing production server, run ``` CREATE TABLE commit_ownerships ( commit_owner integer NOT NULL REFERENCES commits ON DELETE CASCADE, commit_ownee integer NOT NULL REFERENCES commits ON DELETE CASCADE, PRIMARY KEY (commit_owner, commit_ownee) ); ALTER TABLE repositories RENAME repository_parent TO repository_owner; ALTER TABLE repositories DROP repository_name_must_be_unique; CREATE UNIQUE INDEX repository_name_owner_unique_index ON repositories (repository_owner, repository_name) WHERE repository_owner IS NOT NULL; CREATE UNIQUE INDEX repository_name_unique_index ON repositories (repository_name) WHERE repository_owner IS NULL; ``` to update database. Add unit-tests to cover this change. * init-database.sql: * public/api/report-commits.php: * public/include/commit-log-fetcher.php: * public/include/db.php: * public/include/manifest-generator.php: * public/include/report-processor.php: * public/v3/models/repository.js: (Repository): (Repository.prototype.owner): * server-tests/admin-reprocess-report-tests.js: (addBuilderForReport.simpleReportWithRevisions.0.then): (then): * server-tests/api-manifest.js: (then): * server-tests/api-report-commits-tests.js: (addSlaveForReport.sameRepositoryNameInSubCommitAndMajorCommit.then): (then): (addSlaveForReport.systemVersionCommitWithSubcommits.then): (addSlaveForReport.multipleSystemVersionCommitsWithSubcommits.then): (addSlaveForReport.systemVersionCommitWithEmptySubcommits.then): (addSlaveForReport.systemVersionCommitAndSubcommitWithTimestamp.then): * tools/js/database.js: Canonical link: https://commits.webkit.org/186535@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213788 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-13 09:10:29 +00:00
repository_blame_url varchar(1024));
CREATE UNIQUE INDEX repository_name_owner_unique_index ON repositories (repository_owner, repository_name)
WHERE repository_owner IS NOT NULL;
CREATE UNIQUE INDEX repository_name_unique_index ON repositories (repository_name)
WHERE repository_owner IS NULL;
CREATE TABLE bug_trackers (
tracker_id serial PRIMARY KEY,
tracker_name varchar(64) NOT NULL,
Perf dashboard should provide a way to associate bugs with a test run https://bugs.webkit.org/show_bug.cgi?id=137857 Reviewed by Andreas Kling. Added a "privileged" API, /privileged-api/associate-bug, to associate a bug with a test run. /privileged-api/ is to be protected by an authentication mechanism such as DigestAuth over https by the Apache configuration. The Cross Site Request (CSRF) Forgery prevention for privileged APIs work as follows. When a user is about to make a privileged API access, the front end code obtains a CSRF token generated by POST'ing to privileged-api/generate-csrf-token; the page sets a randomly generated salt and an expiration time via the cookie and returns a token computed from those two values as well as the remote username. The font end code then POST's the request along with the returned token. The server side code verifies that the specified token can be generated from the salt and the expiration time set in the cookie, and the token hasn't expired. * init-database.sql: Added bug_url to bug_trackers table, and added bugs table. Each bug tracker will have zero or exactly one bug associated with a test run. * public/admin/bug-trackers.php: Added the support for editing bug_url. * public/api/runs.php: (fetch_runs_for_config): Modified the query to fetch bugs associated with test_runs. (parse_bugs_array): Added. Parses the aggregated bugs and creates a dictionary that maps a tracker id to an associated bug if there is one. (format_run): Calls parse_bugs_array. * public/include/json-header.php: Added helper functions to deal for CSRF prevention. (ensure_privileged_api_data): Added. Dies immediately if the request's method is not POST or doesn't have a valid JSON payload. (ensure_privileged_api_data_and_token): Ditto. Also checks that the CSRF prevention token is valid. (compute_token): Computes a CSRF token using the REMOTE_USER (e.g. set via BasicAuth), the salt, and the expiration time stored in the cookie. (verify_token): Returns true iff the specified token matches what compute_token returns from the cookie. * public/include/manifest.php: (ManifestGenerator::bug_trackers): Include bug_url as bugUrl in the manifest. Also use tracker_id instead of tracker_name as the key in the manifest. This requires changes to both v1 and v2 front end. * public/index.html: (Chart..showTooltipWithResults): Updated for the manifest format changed mentioned above. * public/privileged-api/associate-bug.php: Added. (main): Added. Associates or dissociates a bug with a test run inside a transaction. It prevent a CSRF attack via ensure_privileged_api_data_and_token, which calls verify_token. * public/privileged-api/generate-csrf-token.php: Added. Generates a CSRF token valid for one hour. * public/v2/app.css: (.disabled .icon-button:hover g): Used by the "bugs" icon when a range of points or no points are selected in a chart. * public/v2/app.js: (App.PaneController.actions.toggleBugsPane): Added. Toggles the visibility of the bugs pane when exactly one point is selected in the chart. Also hides the search pane when making the bugs pane visible since they would overlap on each other if both of them are shown. (App.PaneController.actions.associateBug): Makes a privileged API request to associate the specified bug with the currently selected point (test run). Updates the bug information in "details" and colors of dots in the charts to reflect new states. Because chart data objects aren't real Ember objects for performance reasons, we have to use a dirty hack of modifying a dummy counter bugsChangeCount. (App.PaneController.actions.toggleSearchPane): Renamed from toggleSearch. Also hides the bugs pane when showing the search pane. (App.PaneController.actions.rangeChanged): Takes all selected points as the second argument instead of taking start and end points as the second and the third arguments so that _showDetails can enumerate all bugs in the selected range. (App.PaneController._detailsChanged): Added. Hide the bugs pane whenever a new point is selected. Also update singlySelectedPoint, which is used by toggleBugsPane and associateBug. (App.PaneController._currentItemChanged): Updated for the _showDetails change. (App.PaneController._showDetails): Takes an array of selected points in place of old arguments. Simplified the code to compute the revision information. Calls _updateBugs to format the associated bugs. (App.PaneController._updateBugs): Sets details.bugTrackers to a dictionary that maps a bug tracker id to a bug tracker proxy with an array of (bugNumber, bugUrl) pairs and also editedBugNumber, which is used by the bugs pane to associate or dissociate a bug number, if exactly one point is selected. (App.InteractiveChartComponent._updateDotsWithBugs): Added. Sets hasBugs class on dots as needed. (App.InteractiveChartComponent._setCurrentSelection): Finds and passes all points in the selected range to selectionChanged action instead of just finding the first and the last points. * public/v2/chart-pane.css: Updated the style. * public/v2/data.js: (PrivilegedAPI): Added. A wrapper for privileged APIs' CSRF tokens. (PrivilegedAPI.sendRequest): Makes a privileged API call. Fetches a new CSRF token if needed. (PrivilegedAPI._generateTokenInServerIfNeeded): Makes a request to privileged-api/generate-csrf-token if we haven't already obtained a CSRF token or if the token has already been expired. (PrivilegedAPI._post): Makes a single POST request to /privileged-api/* with a JSON payload. (Measurement.prototype.bugs): Added. (Measurement.prototype.hasBugs): Returns true iff bugs has more than one bug number. (Measurement.prototype.associateBug): Associates a bug with a test run via privileged-api/associate-bug. * public/v2/index.html: Added the bugs pane. Also added a list of bugs associated with the current run in the details. * public/v2/manifest.js: (App.BugTracker.bugUrl): (App.BugTracker.newBugUrl): Added. (App.BugTracker.repositories): Added. This was a missing back reference to repositories. (App.MetricSerializer.normalizePayload): Now parses/loads the list of bug trackers from the manifest. (App.Manifest.repositoriesWithReportedCommits): Now initialized to an empty array instead of null. (App.Manifest.bugTrackers): Added. (App.Manifest._fetchedManifest): Sets App.Manifest.bugTrackers. Also sorts the list of repositories by their respective ids to make the ordering stable. Canonical link: https://commits.webkit.org/155807@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-22 00:53:39 +00:00
tracker_bug_url varchar(1024),
tracker_new_bug_url varchar(1024));
CREATE TABLE tracker_repositories (
tracrepo_tracker integer NOT NULL REFERENCES bug_trackers ON DELETE CASCADE,
tracrepo_repository integer NOT NULL REFERENCES repositories ON DELETE CASCADE);
CREATE TABLE builders (
builder_id serial PRIMARY KEY,
builder_name varchar(256) NOT NULL UNIQUE,
Perf dashboard should support authentication via a slave password https://bugs.webkit.org/show_bug.cgi?id=139837 Reviewed by Andreas Kling. For historical reasons, perf dashboard conflated builders and build slaves. As a result we ended up having to add multiple builders with the same password when a single build slave is shared among them. This patch introduces the concept of build_slave into the perf dashboard to end this madness. * init-database.sql: Added build_slave table as well as references to it in builds and reports. * public/admin/build-slaves.php: Added. * public/admin/builders.php: Added the support for updating passwords. * public/include/admin-header.php: (update_field): Takes an extra argument when a new value needs to be supplied by the caller instead of being retrieved from $_POST. (AdministrativePage::render_table): Use array_get to retrieve a value out of the database row since the raw may not exist (e.g. new_password). (AdministrativePage::render_form_to_add): Added the support for post_insertion. Don't render the form control here when this flag evaluates to TRUE. * public/include/report-processor.php: (ReportProcessor::process): Added the logic to authenticate with slaveName and slavePassword if those values are present in the report. In addition, try authenticating builderName with slavePassword if builderPassword is not specified. When neither password is specified, exit with BuilderNotFound. Also insert the slave or the builder whichever is missing after we've successfully authenticated. (ReportProcessor::construct_build_data): Takes a builder ID and an optional slave ID instead of a builder row. (ReportProcessor::store_report): Store the slave ID with the report. (ReportProcessor::resolve_build_id): Exit with MismatchingBuildSlave when the slave associated with the matching build is different from what's being reported. * tests/api-report.js: Added a bunch of tests to test the new features of /api/report. (.addSlave): Added. Canonical link: https://commits.webkit.org/157776@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@177614 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-12-20 02:38:04 +00:00
builder_password_hash character(64),
builder_build_url varchar(1024));
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
CREATE TABLE build_workers (
worker_id serial PRIMARY KEY,
worker_name varchar(64) NOT NULL UNIQUE,
worker_password_hash character(64));
Perf dashboard should support authentication via a slave password https://bugs.webkit.org/show_bug.cgi?id=139837 Reviewed by Andreas Kling. For historical reasons, perf dashboard conflated builders and build slaves. As a result we ended up having to add multiple builders with the same password when a single build slave is shared among them. This patch introduces the concept of build_slave into the perf dashboard to end this madness. * init-database.sql: Added build_slave table as well as references to it in builds and reports. * public/admin/build-slaves.php: Added. * public/admin/builders.php: Added the support for updating passwords. * public/include/admin-header.php: (update_field): Takes an extra argument when a new value needs to be supplied by the caller instead of being retrieved from $_POST. (AdministrativePage::render_table): Use array_get to retrieve a value out of the database row since the raw may not exist (e.g. new_password). (AdministrativePage::render_form_to_add): Added the support for post_insertion. Don't render the form control here when this flag evaluates to TRUE. * public/include/report-processor.php: (ReportProcessor::process): Added the logic to authenticate with slaveName and slavePassword if those values are present in the report. In addition, try authenticating builderName with slavePassword if builderPassword is not specified. When neither password is specified, exit with BuilderNotFound. Also insert the slave or the builder whichever is missing after we've successfully authenticated. (ReportProcessor::construct_build_data): Takes a builder ID and an optional slave ID instead of a builder row. (ReportProcessor::store_report): Store the slave ID with the report. (ReportProcessor::resolve_build_id): Exit with MismatchingBuildSlave when the slave associated with the matching build is different from what's being reported. * tests/api-report.js: Added a bunch of tests to test the new features of /api/report. (.addSlave): Added. Canonical link: https://commits.webkit.org/157776@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@177614 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-12-20 02:38:04 +00:00
CREATE TABLE builds (
build_id serial PRIMARY KEY,
build_builder integer REFERENCES builders ON DELETE CASCADE,
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
build_worker integer REFERENCES build_workers ON DELETE CASCADE,
Refactor "builds" table "build_number" row to "build_tag" to fit more generic use cases. https://bugs.webkit.org/show_bug.cgi?id=203270 Reviewed by Ryosuke Niwa. Rename "build_number" to "build_tag" to fit more generic use cases. Add support to be backward compatible to accept reports contain either "buildNumber" or "buildTag". * browser-tests/index.html: * init-database.sql: * migrate-database.sql: Updated migration sql for this change. * public/admin/test-configurations.php: * public/api/measurement-set.php: * public/api/runs.php: * public/api/upload-root.php: Added support for both "buildNumber" and "buildTag". * public/include/report-processor.php: Added support for both "buildNumber" and "buildTag". * public/v3/models/builder.js: (Builder.prototype.urlForBuild): (Builder): (Build): (Build.prototype.buildTag): (Build.prototype.label): (Build.prototype.url): (Build.prototype.buildNumber): Deleted. * public/v3/models/measurement-adaptor.js: (MeasurementAdaptor): (MeasurementAdaptor.prototype.applyTo.return.build): * public/v3/pages/chart-pane-status-view.js: * public/v3/pages/test-freshness-page.js: (TestFreshnessPage.prototype._renderTooltip): * server-tests/admin-platforms-tests.js: (reportsForDifferentPlatforms): * server-tests/admin-reprocess-report-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-manifest-tests.js: * server-tests/api-measurement-set-tests.js: * server-tests/api-report-tests.js: Added unit tests for testing "buildNumber" and "buildTag". (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptySlaveReport): (makeReport): (reportWithSameSubtestName): * server-tests/api-upload-root-tests.js: Added unit tests for testing "buildNumber" and "buildTag". * server-tests/privileged-api-add-build-requests-tests.js: * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async.createAnalysisTask): * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.sampleBuildData): (MockData.finishedBuildData): * server-tests/tools-sync-buildbot-integration-tests.js: * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildTag): (BuildbotBuildEntry.prototype.url): (BuildbotSyncer.prototype.urlForBuildTag): (BuildbotBuildEntry.prototype.buildNumber): Deleted. (BuildbotSyncer.prototype.urlForBuildNumber): Deleted. * unit-tests/analysis-task-tests.js: (measurementCluster): * unit-tests/buildbot-syncer-tests.js: * unit-tests/measurement-adaptor-tests.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/measurement-set-tests.js: * unit-tests/resources/mock-v3-models.js: Canonical link: https://commits.webkit.org/216795@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@251564 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-10-24 22:00:37 +00:00
build_tag varchar(64) NOT NULL,
build_time timestamp NOT NULL,
build_latest_revision timestamp,
Refactor "builds" table "build_number" row to "build_tag" to fit more generic use cases. https://bugs.webkit.org/show_bug.cgi?id=203270 Reviewed by Ryosuke Niwa. Rename "build_number" to "build_tag" to fit more generic use cases. Add support to be backward compatible to accept reports contain either "buildNumber" or "buildTag". * browser-tests/index.html: * init-database.sql: * migrate-database.sql: Updated migration sql for this change. * public/admin/test-configurations.php: * public/api/measurement-set.php: * public/api/runs.php: * public/api/upload-root.php: Added support for both "buildNumber" and "buildTag". * public/include/report-processor.php: Added support for both "buildNumber" and "buildTag". * public/v3/models/builder.js: (Builder.prototype.urlForBuild): (Builder): (Build): (Build.prototype.buildTag): (Build.prototype.label): (Build.prototype.url): (Build.prototype.buildNumber): Deleted. * public/v3/models/measurement-adaptor.js: (MeasurementAdaptor): (MeasurementAdaptor.prototype.applyTo.return.build): * public/v3/pages/chart-pane-status-view.js: * public/v3/pages/test-freshness-page.js: (TestFreshnessPage.prototype._renderTooltip): * server-tests/admin-platforms-tests.js: (reportsForDifferentPlatforms): * server-tests/admin-reprocess-report-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-manifest-tests.js: * server-tests/api-measurement-set-tests.js: * server-tests/api-report-tests.js: Added unit tests for testing "buildNumber" and "buildTag". (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptySlaveReport): (makeReport): (reportWithSameSubtestName): * server-tests/api-upload-root-tests.js: Added unit tests for testing "buildNumber" and "buildTag". * server-tests/privileged-api-add-build-requests-tests.js: * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async.createAnalysisTask): * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.sampleBuildData): (MockData.finishedBuildData): * server-tests/tools-sync-buildbot-integration-tests.js: * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildTag): (BuildbotBuildEntry.prototype.url): (BuildbotSyncer.prototype.urlForBuildTag): (BuildbotBuildEntry.prototype.buildNumber): Deleted. (BuildbotSyncer.prototype.urlForBuildNumber): Deleted. * unit-tests/analysis-task-tests.js: (measurementCluster): * unit-tests/buildbot-syncer-tests.js: * unit-tests/measurement-adaptor-tests.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/measurement-set-tests.js: * unit-tests/resources/mock-v3-models.js: Canonical link: https://commits.webkit.org/216795@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@251564 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-10-24 22:00:37 +00:00
CONSTRAINT builder_build_time_tuple_must_be_unique UNIQUE(build_builder, build_tag, build_time));
CREATE INDEX build_builder_index ON builds(build_builder);
New perf dashboard should not duplicate author information in each commit https://bugs.webkit.org/show_bug.cgi?id=139756 Reviewed by Darin Adler. Instead of each commit having author name and email, make it reference a newly added committers table. Also replace "email" by "account" since some repositories don't use emails as account names. This improves the keyword search performance in commits.php since LIKE now runs on committers table, which only contains as many rows as there are accounts in each repository, instead of commits table which contains every commit ever happened in each repository. To migrate an existing database into match the new schema, run: BEGIN; INSERT INTO committers (committer_repository, committer_name, committer_email) (SELECT DISTINCT commit_repository, commit_author_name, commit_author_email FROM commits WHERE commit_author_email IS NOT NULL); ALTER TABLE commits ADD COLUMN commit_committer integer REFERENCES committers ON DELETE CASCADE; UPDATE commits SET commit_committer = committer_id FROM committers WHERE commit_repository = committer_repository AND commit_author_email = committer_email; ALTER TABLE commits DROP COLUMN commit_author_name CASCADE; ALTER TABLE commits DROP COLUMN commit_author_email CASCADE; COMMIT; * init-database.sql: Added committers table, and replaced author columns in commits table by a foreign reference to committers. Also added the missing drop table statements. * public/api/commits.php: (main): Fetch the corresponding committers row for a single commit. Also wrap a single commit by an array here instead of doing it in format_commit. (fetch_commits_between): Updated queries to JOIN commits with committers. (format_commit): Takes both commit and committers rows. Also don't wrap the result in an array as that is now done in main. * public/api/report-commits.php: (main): Store the reported committer information or update the existing entry if there is one. * tests/admin-regenerate-manifest.js: Fixed tests. * tests/api-report-commits.js: Ditto. Also added a test for updating an existing committer entry. * tools/pull-svn.py: Renamed email to account. (main): (fetch_commit_and_resolve_author): (fetch_commit): (resolve_author_name_from_account): (resolve_author_name_from_email): Deleted. Canonical link: https://commits.webkit.org/157706@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@177530 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-12-18 22:53:52 +00:00
CREATE TABLE committers (
committer_id serial PRIMARY KEY,
committer_repository integer NOT NULL REFERENCES repositories ON DELETE CASCADE,
committer_account varchar(320) NOT NULL,
committer_name varchar(128),
CONSTRAINT committer_in_repository_must_be_unique UNIQUE(committer_repository, committer_account));
CREATE INDEX committer_account_index ON committers(committer_account);
CREATE INDEX committer_name_index ON committers(committer_name);
Perf dashboard should store commit logs https://bugs.webkit.org/show_bug.cgi?id=137510 Reviewed by Darin Adler. For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself. This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs, and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from a subversion directory. The git version of this script will be added in a follow up patch. In the new database schema, each revision in each repository is represented by exactly one row in "commits" instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many relationship via "build_commits" relations. In order to migrate an existing instance of this application, run the following SQL commands: BEGIN; INSERT INTO commits (commit_repository, commit_revision, commit_time) (SELECT DISTINCT ON (revision_repository, revision_value) revision_repository, revision_value, revision_time FROM build_revisions); INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id FROM commits, build_revisions WHERE commit_repository = revision_repository AND commit_revision = revision_value; DROP TABLE build_revisions; COMMIT; The helper script to submit commit logs can be used as follows: python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org feeder-slave feeder-slave-password 60 "webkit-patch find-users" The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and "feeder-slave-password" as the builder name and the builder password respectively. The last, optional, argument is the shell command to convert a subversion account to the corresponding username. e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout. * init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations. * public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form /api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows: - Not specified - It returns every single commit for a given repository. - Matches "oldest" - It returns the commit with the oldest timestamp. - Matches "latest" - It returns the commit with the latest timestamp. - Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php. - Is entirely alphanumeric - It returns the commit whose revision matches the filter. - Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list of commits added via report-commits.php between two timestamps retrieved from commits whose revisions match the two alphanumeric values specified. Because it retrieves commits based on their timestamps, the list may contain commits that do not appear as neither hash's ancestor in git/mercurial. (main): (commit_from_revision): (fetch_commits_between): (format_commits): * public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits. See tests/api-report-commits.js for examples on how to use this API. * public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of "build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready to optimize this and other JSON APIs. * public/include/db.php: (Database::select_or_insert_row): (Database::update_or_insert_row): Added. (Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false. (Database::select_first_row): (Database::select_last_row): Added. (Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting $order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order. * public/include/report-processor.php: (ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent. * run-tests.js: (pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js. * tests/api-report-commits.js: Added. * tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement. * tools: Added. * tools/pull-svn.py: Added. See above for how to use this script. (main): (determine_first_revision_to_fetch): (fetch_revision_from_dasbhoard): (fetch_commit_and_resolve_author): (fetch_commit): (textContent): (resolve_author_name_from_email): (submit_commits): Canonical link: https://commits.webkit.org/155341@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
CREATE TABLE commits (
commit_id serial PRIMARY KEY,
commit_repository integer NOT NULL REFERENCES repositories ON DELETE CASCADE,
commit_revision varchar(64) NOT NULL,
commit_revision_identifier varchar(64) DEFAULT NULL,
commit_previous_commit integer REFERENCES commits ON DELETE CASCADE,
Perf dashboard should store commit logs https://bugs.webkit.org/show_bug.cgi?id=137510 Reviewed by Darin Adler. For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself. This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs, and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from a subversion directory. The git version of this script will be added in a follow up patch. In the new database schema, each revision in each repository is represented by exactly one row in "commits" instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many relationship via "build_commits" relations. In order to migrate an existing instance of this application, run the following SQL commands: BEGIN; INSERT INTO commits (commit_repository, commit_revision, commit_time) (SELECT DISTINCT ON (revision_repository, revision_value) revision_repository, revision_value, revision_time FROM build_revisions); INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id FROM commits, build_revisions WHERE commit_repository = revision_repository AND commit_revision = revision_value; DROP TABLE build_revisions; COMMIT; The helper script to submit commit logs can be used as follows: python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org feeder-slave feeder-slave-password 60 "webkit-patch find-users" The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and "feeder-slave-password" as the builder name and the builder password respectively. The last, optional, argument is the shell command to convert a subversion account to the corresponding username. e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout. * init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations. * public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form /api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows: - Not specified - It returns every single commit for a given repository. - Matches "oldest" - It returns the commit with the oldest timestamp. - Matches "latest" - It returns the commit with the latest timestamp. - Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php. - Is entirely alphanumeric - It returns the commit whose revision matches the filter. - Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list of commits added via report-commits.php between two timestamps retrieved from commits whose revisions match the two alphanumeric values specified. Because it retrieves commits based on their timestamps, the list may contain commits that do not appear as neither hash's ancestor in git/mercurial. (main): (commit_from_revision): (fetch_commits_between): (format_commits): * public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits. See tests/api-report-commits.js for examples on how to use this API. * public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of "build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready to optimize this and other JSON APIs. * public/include/db.php: (Database::select_or_insert_row): (Database::update_or_insert_row): Added. (Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false. (Database::select_first_row): (Database::select_last_row): Added. (Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting $order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order. * public/include/report-processor.php: (ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent. * run-tests.js: (pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js. * tests/api-report-commits.js: Added. * tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement. * tools: Added. * tools/pull-svn.py: Added. See above for how to use this script. (main): (determine_first_revision_to_fetch): (fetch_revision_from_dasbhoard): (fetch_commit_and_resolve_author): (fetch_commit): (textContent): (resolve_author_name_from_email): (submit_commits): Canonical link: https://commits.webkit.org/155341@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
commit_time timestamp,
commit_order bigint,
New perf dashboard should not duplicate author information in each commit https://bugs.webkit.org/show_bug.cgi?id=139756 Reviewed by Darin Adler. Instead of each commit having author name and email, make it reference a newly added committers table. Also replace "email" by "account" since some repositories don't use emails as account names. This improves the keyword search performance in commits.php since LIKE now runs on committers table, which only contains as many rows as there are accounts in each repository, instead of commits table which contains every commit ever happened in each repository. To migrate an existing database into match the new schema, run: BEGIN; INSERT INTO committers (committer_repository, committer_name, committer_email) (SELECT DISTINCT commit_repository, commit_author_name, commit_author_email FROM commits WHERE commit_author_email IS NOT NULL); ALTER TABLE commits ADD COLUMN commit_committer integer REFERENCES committers ON DELETE CASCADE; UPDATE commits SET commit_committer = committer_id FROM committers WHERE commit_repository = committer_repository AND commit_author_email = committer_email; ALTER TABLE commits DROP COLUMN commit_author_name CASCADE; ALTER TABLE commits DROP COLUMN commit_author_email CASCADE; COMMIT; * init-database.sql: Added committers table, and replaced author columns in commits table by a foreign reference to committers. Also added the missing drop table statements. * public/api/commits.php: (main): Fetch the corresponding committers row for a single commit. Also wrap a single commit by an array here instead of doing it in format_commit. (fetch_commits_between): Updated queries to JOIN commits with committers. (format_commit): Takes both commit and committers rows. Also don't wrap the result in an array as that is now done in main. * public/api/report-commits.php: (main): Store the reported committer information or update the existing entry if there is one. * tests/admin-regenerate-manifest.js: Fixed tests. * tests/api-report-commits.js: Ditto. Also added a test for updating an existing committer entry. * tools/pull-svn.py: Renamed email to account. (main): (fetch_commit_and_resolve_author): (fetch_commit): (resolve_author_name_from_account): (resolve_author_name_from_email): Deleted. Canonical link: https://commits.webkit.org/157706@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@177530 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-12-18 22:53:52 +00:00
commit_committer integer REFERENCES committers ON DELETE CASCADE,
Perf dashboard should store commit logs https://bugs.webkit.org/show_bug.cgi?id=137510 Reviewed by Darin Adler. For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself. This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs, and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from a subversion directory. The git version of this script will be added in a follow up patch. In the new database schema, each revision in each repository is represented by exactly one row in "commits" instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many relationship via "build_commits" relations. In order to migrate an existing instance of this application, run the following SQL commands: BEGIN; INSERT INTO commits (commit_repository, commit_revision, commit_time) (SELECT DISTINCT ON (revision_repository, revision_value) revision_repository, revision_value, revision_time FROM build_revisions); INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id FROM commits, build_revisions WHERE commit_repository = revision_repository AND commit_revision = revision_value; DROP TABLE build_revisions; COMMIT; The helper script to submit commit logs can be used as follows: python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org feeder-slave feeder-slave-password 60 "webkit-patch find-users" The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and "feeder-slave-password" as the builder name and the builder password respectively. The last, optional, argument is the shell command to convert a subversion account to the corresponding username. e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout. * init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations. * public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form /api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows: - Not specified - It returns every single commit for a given repository. - Matches "oldest" - It returns the commit with the oldest timestamp. - Matches "latest" - It returns the commit with the latest timestamp. - Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php. - Is entirely alphanumeric - It returns the commit whose revision matches the filter. - Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list of commits added via report-commits.php between two timestamps retrieved from commits whose revisions match the two alphanumeric values specified. Because it retrieves commits based on their timestamps, the list may contain commits that do not appear as neither hash's ancestor in git/mercurial. (main): (commit_from_revision): (fetch_commits_between): (format_commits): * public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits. See tests/api-report-commits.js for examples on how to use this API. * public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of "build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready to optimize this and other JSON APIs. * public/include/db.php: (Database::select_or_insert_row): (Database::update_or_insert_row): Added. (Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false. (Database::select_first_row): (Database::select_last_row): Added. (Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting $order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order. * public/include/report-processor.php: (ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent. * run-tests.js: (pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js. * tests/api-report-commits.js: Added. * tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement. * tools: Added. * tools/pull-svn.py: Added. See above for how to use this script. (main): (determine_first_revision_to_fetch): (fetch_revision_from_dasbhoard): (fetch_commit_and_resolve_author): (fetch_commit): (textContent): (resolve_author_name_from_email): (submit_commits): Canonical link: https://commits.webkit.org/155341@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
commit_message text,
commit_reported boolean NOT NULL DEFAULT FALSE,
Add UI in analysis task page to show commit testability information. https://bugs.webkit.org/show_bug.cgi?id=192972 Reviewed by Ryosuke Niwa. Add UI in custom analysis task configuration and customizable test group form to show testability information. Fix a bug in 'CustomAnalysisTaskConfigurator._updateCommitSetMap' that 'currentComparison' is incorrectly set. SQL to update existing database: ALTER TABLE commits ADD COLUMN IF NOT EXISTS commit_testability varchar(128) DEFAULT NULL; * browser-tests/custom-analysis-task-configurator-tests.js: Added a unit test for the bug in 'CustomAnalysisTaskConfigurator._updateCommitSetMap'. Added a unit test to make sure 'CustomAnalysisTaskConfigurator' still works when commit fetching never returns. * browser-tests/index.html: Imported ''custom-analysis-task-configurator-tests.js'. * init-database.sql: Increase 'commit_testability' field length from 64 characters to 128. * public/v3/components/custom-analysis-task-configurator.js: Added UI to show testability information. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype._didUpdateSelectedPlatforms): Should reset related field for corresponding repositories that user does not specify revision. (CustomAnalysisTaskConfigurator.prototype._updateMapFromSpecifiedRevisionsForConfiguration): A helper function to update '_specifiedCommits' and '_invalidRevisionsByConfiguration' per '_specifiedRevisions'. (CustomAnalysisTaskConfigurator.prototype.render): (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Fixed a bug that 'currentComparison' is incorrectly set. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): (CustomAnalysisTaskConfigurator.prototype.async._fetchCommitsForConfiguration): (CustomAnalysisTaskConfigurator.prototype.async._resolveRevision): (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): (CustomAnalysisTaskConfigurator.prototype._buildTestabilityList): (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): (CustomAnalysisTaskConfigurator.cssTemplate): * public/v3/components/customizable-test-group-form.js: Added UI to show testability information. (CustomizableTestGroupForm.prototype._renderCustomRevisionTable): (CustomizableTestGroupForm.prototype._constructTestabilityRows.): (CustomizableTestGroupForm.prototype._constructTestabilityRows): (CustomizableTestGroupForm.prototype._constructRevisionRadioButtons): Changing either revision editor or radio button should trigger a re-render as testability information for updated revision may change. (CustomizableTestGroupForm.cssTemplate): * public/v3/models/commit-set.js: (IntermediateCommitSet.prototype.commitsWithTestability): Renamed from 'commitsWithTestabilityWarnings'. (IntermediateCommitSet.prototype.commitsWithTestabilityWarnings): Deleted. Canonical link: https://commits.webkit.org/208032@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@240104 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-01-17 05:56:18 +00:00
commit_testability varchar(128) DEFAULT NULL,
CONSTRAINT commit_in_repository_must_be_unique UNIQUE(commit_repository, commit_revision),
CONSTRAINT commit_string_identifier_in_repository_must_be_unique UNIQUE(commit_repository, commit_revision_identifier));
Perf dashboard should store commit logs https://bugs.webkit.org/show_bug.cgi?id=137510 Reviewed by Darin Adler. For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself. This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs, and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from a subversion directory. The git version of this script will be added in a follow up patch. In the new database schema, each revision in each repository is represented by exactly one row in "commits" instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many relationship via "build_commits" relations. In order to migrate an existing instance of this application, run the following SQL commands: BEGIN; INSERT INTO commits (commit_repository, commit_revision, commit_time) (SELECT DISTINCT ON (revision_repository, revision_value) revision_repository, revision_value, revision_time FROM build_revisions); INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id FROM commits, build_revisions WHERE commit_repository = revision_repository AND commit_revision = revision_value; DROP TABLE build_revisions; COMMIT; The helper script to submit commit logs can be used as follows: python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org feeder-slave feeder-slave-password 60 "webkit-patch find-users" The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and "feeder-slave-password" as the builder name and the builder password respectively. The last, optional, argument is the shell command to convert a subversion account to the corresponding username. e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout. * init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations. * public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form /api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows: - Not specified - It returns every single commit for a given repository. - Matches "oldest" - It returns the commit with the oldest timestamp. - Matches "latest" - It returns the commit with the latest timestamp. - Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php. - Is entirely alphanumeric - It returns the commit whose revision matches the filter. - Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list of commits added via report-commits.php between two timestamps retrieved from commits whose revisions match the two alphanumeric values specified. Because it retrieves commits based on their timestamps, the list may contain commits that do not appear as neither hash's ancestor in git/mercurial. (main): (commit_from_revision): (fetch_commits_between): (format_commits): * public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits. See tests/api-report-commits.js for examples on how to use this API. * public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of "build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready to optimize this and other JSON APIs. * public/include/db.php: (Database::select_or_insert_row): (Database::update_or_insert_row): Added. (Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false. (Database::select_first_row): (Database::select_last_row): Added. (Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting $order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order. * public/include/report-processor.php: (ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent. * run-tests.js: (pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js. * tests/api-report-commits.js: Added. * tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement. * tools: Added. * tools/pull-svn.py: Added. See above for how to use this script. (main): (determine_first_revision_to_fetch): (fetch_revision_from_dasbhoard): (fetch_commit_and_resolve_author): (fetch_commit): (textContent): (resolve_author_name_from_email): (submit_commits): Canonical link: https://commits.webkit.org/155341@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
CREATE INDEX commit_time_index ON commits(commit_time);
CREATE INDEX commit_order_index ON commits(commit_order);
Perf dashboard should store commit logs https://bugs.webkit.org/show_bug.cgi?id=137510 Reviewed by Darin Adler. For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself. This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs, and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from a subversion directory. The git version of this script will be added in a follow up patch. In the new database schema, each revision in each repository is represented by exactly one row in "commits" instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many relationship via "build_commits" relations. In order to migrate an existing instance of this application, run the following SQL commands: BEGIN; INSERT INTO commits (commit_repository, commit_revision, commit_time) (SELECT DISTINCT ON (revision_repository, revision_value) revision_repository, revision_value, revision_time FROM build_revisions); INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id FROM commits, build_revisions WHERE commit_repository = revision_repository AND commit_revision = revision_value; DROP TABLE build_revisions; COMMIT; The helper script to submit commit logs can be used as follows: python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org feeder-slave feeder-slave-password 60 "webkit-patch find-users" The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and "feeder-slave-password" as the builder name and the builder password respectively. The last, optional, argument is the shell command to convert a subversion account to the corresponding username. e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout. * init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations. * public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form /api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows: - Not specified - It returns every single commit for a given repository. - Matches "oldest" - It returns the commit with the oldest timestamp. - Matches "latest" - It returns the commit with the latest timestamp. - Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php. - Is entirely alphanumeric - It returns the commit whose revision matches the filter. - Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list of commits added via report-commits.php between two timestamps retrieved from commits whose revisions match the two alphanumeric values specified. Because it retrieves commits based on their timestamps, the list may contain commits that do not appear as neither hash's ancestor in git/mercurial. (main): (commit_from_revision): (fetch_commits_between): (format_commits): * public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits. See tests/api-report-commits.js for examples on how to use this API. * public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of "build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready to optimize this and other JSON APIs. * public/include/db.php: (Database::select_or_insert_row): (Database::update_or_insert_row): Added. (Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false. (Database::select_first_row): (Database::select_last_row): Added. (Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting $order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order. * public/include/report-processor.php: (ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent. * run-tests.js: (pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js. * tests/api-report-commits.js: Added. * tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement. * tools: Added. * tools/pull-svn.py: Added. See above for how to use this script. (main): (determine_first_revision_to_fetch): (fetch_revision_from_dasbhoard): (fetch_commit_and_resolve_author): (fetch_commit): (textContent): (resolve_author_name_from_email): (submit_commits): Canonical link: https://commits.webkit.org/155341@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
Add the ability to report a commit with sub-commits. https://bugs.webkit.org/show_bug.cgi?id=168962 Reviewed by Ryosuke Niwa. Introduce 'commit_ownerships' which records ownership between commits. On existing production server, run ``` CREATE TABLE commit_ownerships ( commit_owner integer NOT NULL REFERENCES commits ON DELETE CASCADE, commit_ownee integer NOT NULL REFERENCES commits ON DELETE CASCADE, PRIMARY KEY (commit_owner, commit_ownee) ); ALTER TABLE repositories RENAME repository_parent TO repository_owner; ALTER TABLE repositories DROP repository_name_must_be_unique; CREATE UNIQUE INDEX repository_name_owner_unique_index ON repositories (repository_owner, repository_name) WHERE repository_owner IS NOT NULL; CREATE UNIQUE INDEX repository_name_unique_index ON repositories (repository_name) WHERE repository_owner IS NULL; ``` to update database. Add unit-tests to cover this change. * init-database.sql: * public/api/report-commits.php: * public/include/commit-log-fetcher.php: * public/include/db.php: * public/include/manifest-generator.php: * public/include/report-processor.php: * public/v3/models/repository.js: (Repository): (Repository.prototype.owner): * server-tests/admin-reprocess-report-tests.js: (addBuilderForReport.simpleReportWithRevisions.0.then): (then): * server-tests/api-manifest.js: (then): * server-tests/api-report-commits-tests.js: (addSlaveForReport.sameRepositoryNameInSubCommitAndMajorCommit.then): (then): (addSlaveForReport.systemVersionCommitWithSubcommits.then): (addSlaveForReport.multipleSystemVersionCommitsWithSubcommits.then): (addSlaveForReport.systemVersionCommitWithEmptySubcommits.then): (addSlaveForReport.systemVersionCommitAndSubcommitWithTimestamp.then): * tools/js/database.js: Canonical link: https://commits.webkit.org/186535@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213788 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-13 09:10:29 +00:00
CREATE TABLE commit_ownerships (
commit_owner integer NOT NULL REFERENCES commits ON DELETE CASCADE,
commit_owned integer NOT NULL REFERENCES commits ON DELETE CASCADE,
PRIMARY KEY (commit_owner, commit_owned)
);
Perf dashboard should store commit logs https://bugs.webkit.org/show_bug.cgi?id=137510 Reviewed by Darin Adler. For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself. This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs, and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from a subversion directory. The git version of this script will be added in a follow up patch. In the new database schema, each revision in each repository is represented by exactly one row in "commits" instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many relationship via "build_commits" relations. In order to migrate an existing instance of this application, run the following SQL commands: BEGIN; INSERT INTO commits (commit_repository, commit_revision, commit_time) (SELECT DISTINCT ON (revision_repository, revision_value) revision_repository, revision_value, revision_time FROM build_revisions); INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id FROM commits, build_revisions WHERE commit_repository = revision_repository AND commit_revision = revision_value; DROP TABLE build_revisions; COMMIT; The helper script to submit commit logs can be used as follows: python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org feeder-slave feeder-slave-password 60 "webkit-patch find-users" The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and "feeder-slave-password" as the builder name and the builder password respectively. The last, optional, argument is the shell command to convert a subversion account to the corresponding username. e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout. * init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations. * public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form /api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows: - Not specified - It returns every single commit for a given repository. - Matches "oldest" - It returns the commit with the oldest timestamp. - Matches "latest" - It returns the commit with the latest timestamp. - Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php. - Is entirely alphanumeric - It returns the commit whose revision matches the filter. - Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list of commits added via report-commits.php between two timestamps retrieved from commits whose revisions match the two alphanumeric values specified. Because it retrieves commits based on their timestamps, the list may contain commits that do not appear as neither hash's ancestor in git/mercurial. (main): (commit_from_revision): (fetch_commits_between): (format_commits): * public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits. See tests/api-report-commits.js for examples on how to use this API. * public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of "build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready to optimize this and other JSON APIs. * public/include/db.php: (Database::select_or_insert_row): (Database::update_or_insert_row): Added. (Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false. (Database::select_first_row): (Database::select_last_row): Added. (Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting $order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order. * public/include/report-processor.php: (ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent. * run-tests.js: (pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js. * tests/api-report-commits.js: Added. * tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement. * tools: Added. * tools/pull-svn.py: Added. See above for how to use this script. (main): (determine_first_revision_to_fetch): (fetch_revision_from_dasbhoard): (fetch_commit_and_resolve_author): (fetch_commit): (textContent): (resolve_author_name_from_email): (submit_commits): Canonical link: https://commits.webkit.org/155341@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
CREATE TABLE build_commits (
commit_build integer NOT NULL REFERENCES builds ON DELETE CASCADE,
build_commit integer NOT NULL REFERENCES commits ON DELETE CASCADE,
Perf dashboard should store commit logs https://bugs.webkit.org/show_bug.cgi?id=137510 Reviewed by Darin Adler. For the v2 version of the perf dashboard, we would like to be able to see commit logs in the dashboard itself. This patch replaces "build_revisions" table with "commits" and "build_commits" relations to store commit logs, and add JSON APIs to report and retrieve them. It also adds a tools/pull-svn.py to pull commit logs from a subversion directory. The git version of this script will be added in a follow up patch. In the new database schema, each revision in each repository is represented by exactly one row in "commits" instead of one row for each build in "build_revisions". "commits" and "builds" now have a proper many-to-many relationship via "build_commits" relations. In order to migrate an existing instance of this application, run the following SQL commands: BEGIN; INSERT INTO commits (commit_repository, commit_revision, commit_time) (SELECT DISTINCT ON (revision_repository, revision_value) revision_repository, revision_value, revision_time FROM build_revisions); INSERT INTO build_commits (commit_build, build_commit) SELECT revision_build, commit_id FROM commits, build_revisions WHERE commit_repository = revision_repository AND commit_revision = revision_value; DROP TABLE build_revisions; COMMIT; The helper script to submit commit logs can be used as follows: python ./tools/pull-svn.py "WebKit" https://svn.webkit.org/repository/webkit/ https://perf.webkit.org feeder-slave feeder-slave-password 60 "webkit-patch find-users" The above command will pull the subversion server at https://svn.webkit.org/repository/webkit/ every 60 seconds to retrieve at most 10 commits, and submits the results to https://perf.webkit.org using "feeder-slave" and "feeder-slave-password" as the builder name and the builder password respectively. The last, optional, argument is the shell command to convert a subversion account to the corresponding username. e.g. "webkit-patch find-users rniwa@webkit.org" yields "Ryosuke Niwa" <rniwa@webkit.org> in the stdout. * init-database.sql: Replaced "build_revisions" relation with "commits" and "build_commits" relations. * public/api/commits.php: Added. Retrieves a list of commits based on arguments in its path of the form /api/commits/<repository-name>/<filter>. The behavior of this API depends on <filter> as follows: - Not specified - It returns every single commit for a given repository. - Matches "oldest" - It returns the commit with the oldest timestamp. - Matches "latest" - It returns the commit with the latest timestamp. - Matches "last-reported" - It returns the commit with the latest timestamp added via report-commits.php. - Is entirely alphanumeric - It returns the commit whose revision matches the filter. - Is of the form <alphanumeric>:<alphanumeric> or <alphanumeric>-<alphanumeric> - It retrieves the list of commits added via report-commits.php between two timestamps retrieved from commits whose revisions match the two alphanumeric values specified. Because it retrieves commits based on their timestamps, the list may contain commits that do not appear as neither hash's ancestor in git/mercurial. (main): (commit_from_revision): (fetch_commits_between): (format_commits): * public/api/report-commits.php: Added. A JSON API to report new subversion, git, or mercurial commits. See tests/api-report-commits.js for examples on how to use this API. * public/api/runs.php: Updated the query to use "commit_builds" and "commits" relations instead of "build_revisions". Regrettably, the new query is 20% slower but I'm going to wait until the new UI is ready to optimize this and other JSON APIs. * public/include/db.php: (Database::select_or_insert_row): (Database::update_or_insert_row): Added. (Database::_select_update_or_insert_row): Extracted from select_or_insert_row. Try to update first and then insert if the update fails for update_or_insert_row. Preserves the old behavior when $should_update is false. (Database::select_first_row): (Database::select_last_row): Added. (Database::select_first_or_last_row): Extracted from select_first_row. Fixed a bug that we were asserting $order_by to be not alphanumeric/underscore. Retrieve the last row instead of the first if $descending_order. * public/include/report-processor.php: (ReportProcessor::resolve_build_id): Store commits instead of build_revisions. We don't worry about the race condition for adding "build_commits" rows since we shouldn't have a single tester submitting the same result concurrently. Even if it happened, it will only result in a PHP error and the database will stay consistent. * run-tests.js: (pathToTests): Don't call path.resolve with "undefined" testName; It throws an exception in the latest node.js. * tests/api-report-commits.js: Added. * tests/api-report.js: Fixed a test per build_revisions to build_commits/commits replacement. * tools: Added. * tools/pull-svn.py: Added. See above for how to use this script. (main): (determine_first_revision_to_fetch): (fetch_revision_from_dasbhoard): (fetch_commit_and_resolve_author): (fetch_commit): (textContent): (resolve_author_name_from_email): (submit_commits): Canonical link: https://commits.webkit.org/155341@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@174459 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-08 17:20:32 +00:00
PRIMARY KEY (commit_build, build_commit));
CREATE TABLE aggregators (
aggregator_id serial PRIMARY KEY,
aggregator_name varchar(64),
aggregator_definition text);
CREATE TABLE tests (
test_id serial PRIMARY KEY,
test_name varchar(255) NOT NULL,
test_parent integer REFERENCES tests ON DELETE CASCADE,
test_url varchar(1024) DEFAULT NULL,
CONSTRAINT parent_test_must_be_unique UNIQUE(test_parent, test_name));
CREATE TABLE test_metrics (
metric_id serial PRIMARY KEY,
metric_test integer NOT NULL REFERENCES tests ON DELETE CASCADE,
metric_name varchar(64) NOT NULL,
metric_aggregator integer REFERENCES aggregators ON DELETE CASCADE);
CREATE TYPE test_configuration_type as ENUM ('current', 'baseline', 'target');
CREATE TABLE test_configurations (
config_id serial PRIMARY KEY,
config_metric integer NOT NULL REFERENCES test_metrics ON DELETE CASCADE,
config_platform integer NOT NULL REFERENCES platforms ON DELETE CASCADE,
config_type test_configuration_type NOT NULL,
config_is_in_dashboard boolean NOT NULL DEFAULT FALSE,
Loading the perf dashboard takes multiple seconds https://bugs.webkit.org/show_bug.cgi?id=141860 Reviewed by Andreas Kling. This patch introduces the caches of JSON files returned by /api/ in /data/ directory. It also records the last time test_runs rows associated with the requested platforms and metrics are inserted, updated, or removed in the caches as well as the manifest JSON files ("last modified time"). Because the manifest is regenerated each time a new test result is reported, the front end can compare last modified time in the manifest file with that in a /api/runs JSON cache to detect the stale-ness. More concretely, the front end first optimistically fetches the JSON in /data/. If the cache doesn't exit or the last modified time in the cache doesn't match with that in the manifest file, it would fetch it again via /api/runs. In the case the cache did exist, we render the charts based on the cache meanwhile. This dramatically reduces the perceived latency for the page load since charts are drawn immediately using the cache and we would only re-render the charts as new up-to-date JSON comes in. This patch also changes the format of runs JSONs by pushing the exiting properties into 'configurations' and adding 'lastModified' and 'elapsedTime' at the top level. * init-database.sql: Added config_runs_last_modified to test_configurations table as well as a trigger to auto-update this column upon changes to test_runs table. * public/admin/test-configurations.php: (add_run): Regenerate the manifest file to invalidate the /api/runs JSON cache. (delete_run): Ditto. * public/api/runs.php: (main): Fetch all columns of test_configurations table including config_runs_last_modified. Also generate the cache in /data/ directory. (RunsGenerator::__construct): Compute the last modified time for this (platform, metric) pair. (RunsGenerator::results): Put the old content in 'configurations' property and include 'lastModified' and 'elapsedTime' properties. 'elapsedTime' is added for debugging purposes. (RunsGenerator::add_runs): (RunsGenerator::parse_revisions_array): * public/include/db.php: (CONFIG_DIR): Added. (generate_data_file): Added based on ManifestGenerator::store. (Database::to_js_time): Extracted from RunsGenerator::add_runs to share code. * public/include/json-header.php: (echo_success): Renamed from success_json. Return the serialized JSON instead of echo'ing it so that we can generate caches in /api/runs/. (exit_with_success): * public/include/manifest.php: (ManifestGenerator::generate): Added 'elapsedTime' property for the time taken to generate the manifest. It seems like we're generating it in 200-300ms for now so that's good. (ManifestGenerator::store): Uses generate_data_file. (ManifestGenerator::platforms): Added 'lastModified' array to each platform entry. This array contains the last modified time for each (platform, metric) pair. * public/index.html: (fetchTest): Updated per the format change in runs JSON. * public/v2/app.js: (App.Pane._fetch): Fetch the cached JSON first. Refetch the uncached version if instructed as such. (App.Pane._updateChartData): Extracted from App.Pane._fetch. (App.Pane._handleFetchErrors): Ditto. * public/v2/data.js: (RunsData.fetchRuns): Takes the fourth argument indicating whether we should fetch the cached version or not. The cached JSON is located in /data/ with the same filename. When fetching a cached JSON results in 404, fulfill the promise with null as the result instead of rejecting it. The only client of this function which sets useCache to true is App.Manifest.fetchRunsWithPlatformAndMetric, and it handles this special case. * public/v2/manifest.js: (App.DateArrayTransform): Added. Handles the array of last modified dates in platform objects. (App.Platform.lastModifiedTimeForMetric): Added. Returns the last modified date in the manifest JSON. (App.Manifest.fetchRunsWithPlatformAndMetric): Takes "useCache" like RunsData.fetchRuns. Set shouldRefetch to true if response is null (the cache didn't exit) or the cache is out-of-date. (App.Manifest._formatFetchedData): Extracted from App.Manifest.fetchRunsWithPlatformAndMetric. * run-tests.js: (initializeDatabase): Avoid splitting function definitions in the middle. * tests/api-report.js: Added tests to verify that reporting new test results updates the last modified time in test_configurations. Canonical link: https://commits.webkit.org/159898@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@180468 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-02-21 01:28:34 +00:00
config_runs_last_modified timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
CONSTRAINT configuration_must_be_unique UNIQUE(config_metric, config_platform, config_type));
CREATE INDEX config_platform_index ON test_configurations(config_platform);
CREATE TABLE test_runs (
run_id serial PRIMARY KEY,
run_config integer NOT NULL REFERENCES test_configurations ON DELETE CASCADE,
run_build integer NOT NULL REFERENCES builds ON DELETE CASCADE,
run_iteration_count_cache smallint,
run_mean_cache double precision,
run_sum_cache double precision,
run_square_sum_cache double precision,
Invalid token error when trying to create an A/B analysis for a range https://bugs.webkit.org/show_bug.cgi?id=158679 Reviewed by Chris Dumez. The problem in this particular case was due to another website overriding cookies for our subdomain. Make PrivilegedAPI robust against its token becoming invalid in general to fix the bug since the cookie is only available under /privileged-api/ and the v3 UI can't access it for security reasons. This patch factors out PrivilegedAPI out of remote.js so that it can be tested separately in server tests as well as unit tests even though RemoteAPI itself is implemented differently in each case. * init-database.sql: Added a forgotten default value "false" to run_marked_outlier. * public/v3/index.html: * public/v3/privileged-api.js: Added. Extracted out of public/v3/remote.js. (PrivilegedAPI.sendRequest): Fixed the bug. When the initial request fails with "InvalidToken" error, re-generate the token and re-issue the request. (PrivilegedAPI.requestCSRFToken): * public/v3/remote.js: (RemoteAPI.postJSON): Added to match tools/js/remote.js. (RemoteAPI.postJSONWithStatus): Ditto. (PrivilegedAPI): Moved to privileged-api.js. * server-tests/api-measurement-set-tests.js: Removed the unused require for crypto. * server-tests/privileged-api-upate-run-status.js: Added tests for /privileged-api/update-run-status. * server-tests/resources/test-server.js: (TestServer.prototype.inject): Clear the cookies as well as tokens in PrivilegedAPI. * tools/js/remote.js: (RemoteAPI): Added the support for PrivilegedAPI by making cookie set by the server persist. (RemoteAPI.prototype.clearCookies): Added for tests. (RemoteAPI.prototype.postJSON): Make sure sendHttpRequest always sends a valid JSON. (RemoteAPI.prototype.postJSONWithStatus): Added since this API is used PrivilegedAPI. (RemoteAPI.prototype.sendHttpRequest): Retain the cookie set by the server and send it back in each request. * tools/js/v3-models.js: * unit-tests/privileged-api-tests.js: Added unit tests for PrivilegedAPI. * unit-tests/resources/mock-remote-api.js: (MockRemoteAPI.postJSON): Added for unit testing. (MockRemoteAPI.postJSONWithStatus): Ditto. Canonical link: https://commits.webkit.org/176780@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@202001 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-06-13 19:47:38 +00:00
run_marked_outlier boolean NOT NULL DEFAULT FALSE,
CONSTRAINT test_config_build_must_be_unique UNIQUE(run_config, run_build));
CREATE INDEX run_config_index ON test_runs(run_config);
CREATE INDEX run_build_index ON test_runs(run_build);
CREATE TABLE run_iterations (
iteration_run integer NOT NULL REFERENCES test_runs ON DELETE CASCADE,
iteration_order smallint NOT NULL CHECK(iteration_order >= 0),
iteration_group smallint CHECK(iteration_group >= 0),
iteration_value double precision,
iteration_relative_time float,
PRIMARY KEY (iteration_run, iteration_order));
Loading the perf dashboard takes multiple seconds https://bugs.webkit.org/show_bug.cgi?id=141860 Reviewed by Andreas Kling. This patch introduces the caches of JSON files returned by /api/ in /data/ directory. It also records the last time test_runs rows associated with the requested platforms and metrics are inserted, updated, or removed in the caches as well as the manifest JSON files ("last modified time"). Because the manifest is regenerated each time a new test result is reported, the front end can compare last modified time in the manifest file with that in a /api/runs JSON cache to detect the stale-ness. More concretely, the front end first optimistically fetches the JSON in /data/. If the cache doesn't exit or the last modified time in the cache doesn't match with that in the manifest file, it would fetch it again via /api/runs. In the case the cache did exist, we render the charts based on the cache meanwhile. This dramatically reduces the perceived latency for the page load since charts are drawn immediately using the cache and we would only re-render the charts as new up-to-date JSON comes in. This patch also changes the format of runs JSONs by pushing the exiting properties into 'configurations' and adding 'lastModified' and 'elapsedTime' at the top level. * init-database.sql: Added config_runs_last_modified to test_configurations table as well as a trigger to auto-update this column upon changes to test_runs table. * public/admin/test-configurations.php: (add_run): Regenerate the manifest file to invalidate the /api/runs JSON cache. (delete_run): Ditto. * public/api/runs.php: (main): Fetch all columns of test_configurations table including config_runs_last_modified. Also generate the cache in /data/ directory. (RunsGenerator::__construct): Compute the last modified time for this (platform, metric) pair. (RunsGenerator::results): Put the old content in 'configurations' property and include 'lastModified' and 'elapsedTime' properties. 'elapsedTime' is added for debugging purposes. (RunsGenerator::add_runs): (RunsGenerator::parse_revisions_array): * public/include/db.php: (CONFIG_DIR): Added. (generate_data_file): Added based on ManifestGenerator::store. (Database::to_js_time): Extracted from RunsGenerator::add_runs to share code. * public/include/json-header.php: (echo_success): Renamed from success_json. Return the serialized JSON instead of echo'ing it so that we can generate caches in /api/runs/. (exit_with_success): * public/include/manifest.php: (ManifestGenerator::generate): Added 'elapsedTime' property for the time taken to generate the manifest. It seems like we're generating it in 200-300ms for now so that's good. (ManifestGenerator::store): Uses generate_data_file. (ManifestGenerator::platforms): Added 'lastModified' array to each platform entry. This array contains the last modified time for each (platform, metric) pair. * public/index.html: (fetchTest): Updated per the format change in runs JSON. * public/v2/app.js: (App.Pane._fetch): Fetch the cached JSON first. Refetch the uncached version if instructed as such. (App.Pane._updateChartData): Extracted from App.Pane._fetch. (App.Pane._handleFetchErrors): Ditto. * public/v2/data.js: (RunsData.fetchRuns): Takes the fourth argument indicating whether we should fetch the cached version or not. The cached JSON is located in /data/ with the same filename. When fetching a cached JSON results in 404, fulfill the promise with null as the result instead of rejecting it. The only client of this function which sets useCache to true is App.Manifest.fetchRunsWithPlatformAndMetric, and it handles this special case. * public/v2/manifest.js: (App.DateArrayTransform): Added. Handles the array of last modified dates in platform objects. (App.Platform.lastModifiedTimeForMetric): Added. Returns the last modified date in the manifest JSON. (App.Manifest.fetchRunsWithPlatformAndMetric): Takes "useCache" like RunsData.fetchRuns. Set shouldRefetch to true if response is null (the cache didn't exit) or the cache is out-of-date. (App.Manifest._formatFetchedData): Extracted from App.Manifest.fetchRunsWithPlatformAndMetric. * run-tests.js: (initializeDatabase): Avoid splitting function definitions in the middle. * tests/api-report.js: Added tests to verify that reporting new test results updates the last modified time in test_configurations. Canonical link: https://commits.webkit.org/159898@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@180468 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-02-21 01:28:34 +00:00
CREATE OR REPLACE FUNCTION update_config_last_modified() RETURNS TRIGGER AS $update_config_last_modified$
BEGIN
IF TG_OP != 'DELETE' THEN
UPDATE test_configurations SET config_runs_last_modified = (CURRENT_TIMESTAMP AT TIME ZONE 'UTC') WHERE config_id = NEW.run_config;
ELSE
UPDATE test_configurations SET config_runs_last_modified = (CURRENT_TIMESTAMP AT TIME ZONE 'UTC') WHERE config_id = OLD.run_config;
END IF;
RETURN NULL;
END;
$update_config_last_modified$ LANGUAGE plpgsql;
CREATE TRIGGER update_config_last_modified AFTER INSERT OR UPDATE OR DELETE ON test_runs
FOR EACH ROW EXECUTE PROCEDURE update_config_last_modified();
CREATE TABLE reports (
report_id serial PRIMARY KEY,
report_builder integer NOT NULL REFERENCES builders ON DELETE RESTRICT,
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
report_worker integer REFERENCES build_workers ON DELETE RESTRICT,
Refactor "builds" table "build_number" row to "build_tag" to fit more generic use cases. https://bugs.webkit.org/show_bug.cgi?id=203270 Reviewed by Ryosuke Niwa. Rename "build_number" to "build_tag" to fit more generic use cases. Add support to be backward compatible to accept reports contain either "buildNumber" or "buildTag". * browser-tests/index.html: * init-database.sql: * migrate-database.sql: Updated migration sql for this change. * public/admin/test-configurations.php: * public/api/measurement-set.php: * public/api/runs.php: * public/api/upload-root.php: Added support for both "buildNumber" and "buildTag". * public/include/report-processor.php: Added support for both "buildNumber" and "buildTag". * public/v3/models/builder.js: (Builder.prototype.urlForBuild): (Builder): (Build): (Build.prototype.buildTag): (Build.prototype.label): (Build.prototype.url): (Build.prototype.buildNumber): Deleted. * public/v3/models/measurement-adaptor.js: (MeasurementAdaptor): (MeasurementAdaptor.prototype.applyTo.return.build): * public/v3/pages/chart-pane-status-view.js: * public/v3/pages/test-freshness-page.js: (TestFreshnessPage.prototype._renderTooltip): * server-tests/admin-platforms-tests.js: (reportsForDifferentPlatforms): * server-tests/admin-reprocess-report-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-manifest-tests.js: * server-tests/api-measurement-set-tests.js: * server-tests/api-report-tests.js: Added unit tests for testing "buildNumber" and "buildTag". (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptySlaveReport): (makeReport): (reportWithSameSubtestName): * server-tests/api-upload-root-tests.js: Added unit tests for testing "buildNumber" and "buildTag". * server-tests/privileged-api-add-build-requests-tests.js: * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async.createAnalysisTask): * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.sampleBuildData): (MockData.finishedBuildData): * server-tests/tools-sync-buildbot-integration-tests.js: * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildTag): (BuildbotBuildEntry.prototype.url): (BuildbotSyncer.prototype.urlForBuildTag): (BuildbotBuildEntry.prototype.buildNumber): Deleted. (BuildbotSyncer.prototype.urlForBuildNumber): Deleted. * unit-tests/analysis-task-tests.js: (measurementCluster): * unit-tests/buildbot-syncer-tests.js: * unit-tests/measurement-adaptor-tests.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/measurement-set-tests.js: * unit-tests/resources/mock-v3-models.js: Canonical link: https://commits.webkit.org/216795@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@251564 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-10-24 22:00:37 +00:00
report_build_tag varchar(64),
report_build integer REFERENCES builds,
report_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
report_committed_at timestamp,
report_content text,
report_failure varchar(64),
report_failure_details text);
Perf dashboard should provide a way to associate bugs with a test run https://bugs.webkit.org/show_bug.cgi?id=137857 Reviewed by Andreas Kling. Added a "privileged" API, /privileged-api/associate-bug, to associate a bug with a test run. /privileged-api/ is to be protected by an authentication mechanism such as DigestAuth over https by the Apache configuration. The Cross Site Request (CSRF) Forgery prevention for privileged APIs work as follows. When a user is about to make a privileged API access, the front end code obtains a CSRF token generated by POST'ing to privileged-api/generate-csrf-token; the page sets a randomly generated salt and an expiration time via the cookie and returns a token computed from those two values as well as the remote username. The font end code then POST's the request along with the returned token. The server side code verifies that the specified token can be generated from the salt and the expiration time set in the cookie, and the token hasn't expired. * init-database.sql: Added bug_url to bug_trackers table, and added bugs table. Each bug tracker will have zero or exactly one bug associated with a test run. * public/admin/bug-trackers.php: Added the support for editing bug_url. * public/api/runs.php: (fetch_runs_for_config): Modified the query to fetch bugs associated with test_runs. (parse_bugs_array): Added. Parses the aggregated bugs and creates a dictionary that maps a tracker id to an associated bug if there is one. (format_run): Calls parse_bugs_array. * public/include/json-header.php: Added helper functions to deal for CSRF prevention. (ensure_privileged_api_data): Added. Dies immediately if the request's method is not POST or doesn't have a valid JSON payload. (ensure_privileged_api_data_and_token): Ditto. Also checks that the CSRF prevention token is valid. (compute_token): Computes a CSRF token using the REMOTE_USER (e.g. set via BasicAuth), the salt, and the expiration time stored in the cookie. (verify_token): Returns true iff the specified token matches what compute_token returns from the cookie. * public/include/manifest.php: (ManifestGenerator::bug_trackers): Include bug_url as bugUrl in the manifest. Also use tracker_id instead of tracker_name as the key in the manifest. This requires changes to both v1 and v2 front end. * public/index.html: (Chart..showTooltipWithResults): Updated for the manifest format changed mentioned above. * public/privileged-api/associate-bug.php: Added. (main): Added. Associates or dissociates a bug with a test run inside a transaction. It prevent a CSRF attack via ensure_privileged_api_data_and_token, which calls verify_token. * public/privileged-api/generate-csrf-token.php: Added. Generates a CSRF token valid for one hour. * public/v2/app.css: (.disabled .icon-button:hover g): Used by the "bugs" icon when a range of points or no points are selected in a chart. * public/v2/app.js: (App.PaneController.actions.toggleBugsPane): Added. Toggles the visibility of the bugs pane when exactly one point is selected in the chart. Also hides the search pane when making the bugs pane visible since they would overlap on each other if both of them are shown. (App.PaneController.actions.associateBug): Makes a privileged API request to associate the specified bug with the currently selected point (test run). Updates the bug information in "details" and colors of dots in the charts to reflect new states. Because chart data objects aren't real Ember objects for performance reasons, we have to use a dirty hack of modifying a dummy counter bugsChangeCount. (App.PaneController.actions.toggleSearchPane): Renamed from toggleSearch. Also hides the bugs pane when showing the search pane. (App.PaneController.actions.rangeChanged): Takes all selected points as the second argument instead of taking start and end points as the second and the third arguments so that _showDetails can enumerate all bugs in the selected range. (App.PaneController._detailsChanged): Added. Hide the bugs pane whenever a new point is selected. Also update singlySelectedPoint, which is used by toggleBugsPane and associateBug. (App.PaneController._currentItemChanged): Updated for the _showDetails change. (App.PaneController._showDetails): Takes an array of selected points in place of old arguments. Simplified the code to compute the revision information. Calls _updateBugs to format the associated bugs. (App.PaneController._updateBugs): Sets details.bugTrackers to a dictionary that maps a bug tracker id to a bug tracker proxy with an array of (bugNumber, bugUrl) pairs and also editedBugNumber, which is used by the bugs pane to associate or dissociate a bug number, if exactly one point is selected. (App.InteractiveChartComponent._updateDotsWithBugs): Added. Sets hasBugs class on dots as needed. (App.InteractiveChartComponent._setCurrentSelection): Finds and passes all points in the selected range to selectionChanged action instead of just finding the first and the last points. * public/v2/chart-pane.css: Updated the style. * public/v2/data.js: (PrivilegedAPI): Added. A wrapper for privileged APIs' CSRF tokens. (PrivilegedAPI.sendRequest): Makes a privileged API call. Fetches a new CSRF token if needed. (PrivilegedAPI._generateTokenInServerIfNeeded): Makes a request to privileged-api/generate-csrf-token if we haven't already obtained a CSRF token or if the token has already been expired. (PrivilegedAPI._post): Makes a single POST request to /privileged-api/* with a JSON payload. (Measurement.prototype.bugs): Added. (Measurement.prototype.hasBugs): Returns true iff bugs has more than one bug number. (Measurement.prototype.associateBug): Associates a bug with a test run via privileged-api/associate-bug. * public/v2/index.html: Added the bugs pane. Also added a list of bugs associated with the current run in the details. * public/v2/manifest.js: (App.BugTracker.bugUrl): (App.BugTracker.newBugUrl): Added. (App.BugTracker.repositories): Added. This was a missing back reference to repositories. (App.MetricSerializer.normalizePayload): Now parses/loads the list of bug trackers from the manifest. (App.Manifest.repositoriesWithReportedCommits): Now initialized to an empty array instead of null. (App.Manifest.bugTrackers): Added. (App.Manifest._fetchedManifest): Sets App.Manifest.bugTrackers. Also sorts the list of repositories by their respective ids to make the ordering stable. Canonical link: https://commits.webkit.org/155807@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175006 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-10-22 00:53:39 +00:00
Perf dashboard should automatically detect regressions https://bugs.webkit.org/show_bug.cgi?id=141443 Reviewed by Anders Carlsson. Added a node.js script detect-changes.js to detect potential regressions and progressions on the graphs tracked on v2 dashboards. * init-database.sql: Added analysis_strategies table and task_segmentation and task_test_range columns to analysis_tasks to keep the segmentation and test range selection strategies used to create an analysis task. * public/api/analysis-tasks.php: (format_task): Include task_segmentation and analysis_tasks in the results. * public/include/json-header.php: (remote_user_name): Returns null when the privileged API is authenticated as a slave instead of a CSRF prevention token. (should_authenticate_as_slave): Added. (ensure_privileged_api_data_and_token_or_slave): Added. Authenticate as a slave if slaveName and slavePassword are specified. Since detect-changes.js and other slaves are not susceptible to a CSRF attack, we don't need to check a CSRF token. * public/privileged-api/create-analysis-task.php: (main): Use ensure_privileged_api_data_and_token_or_slave to let detect-changes.js create new analysis task. Also add or find segmentation and test range selection strategies if specified. * public/privileged-api/create-test-group.php: (main): Use ensure_privileged_api_data_and_token_or_slave. * public/privileged-api/generate-csrf-token.php: * public/v2/app.js: (App.Pane._computeMovingAverageAndOutliers): _executeStrategy has been moved to Statistics. * public/v2/data.js: Export Measurement, RunsData, TimeSeries. Used in detect-changes.js. (Array.prototype.find): Added a polyfill to be used in node.js. (RunsData.fetchRuns): (RunsData.pathForFetchingRuns): Extracted from fetchRuns. Used in detect-changes.js. (RunsData.createRunsDataInResponse): Extracted from App.Manifest._formatFetchedData to use it in detect-changes.js. (RunsData.unitFromMetricName): Ditto. (RunsData.isSmallerBetter): Ditto. (RunsData.prototype._timeSeriesByTimeInternal): Added secondaryTime to sort points when commit times are identical. (TimeSeries): When commit times are identical, order points based on build time. This is needed for when we trigger two builds at two different OS versions with the same WebKit revision since OS versions don't change the commit times. (TimeSeries.prototype.findPointByIndex): Added. (TimeSeries.prototype.rawValues): Added. * public/v2/js/statistics.js: (Statistics.TestRangeSelectionStrategies.[0]): Use the 99% two-sided probability as claimed in the description of this strategy instead of the default probability. Also fixed a bug that debugging code was referring to non-existent variables. (Statistics.executeStrategy): Moved from App.Pane (app.js). * public/v2/manifest.js: (App.Manifest._formatFetchedData): Various code has been extracted into RunsData in data.js to be used in detect-changes.js. * tools/detect-changes.js: Added. The script fetches the manifest JSON, analyzes each graph in the v2 dashboards, and creates an analysis task for the latest regression or progression detected. It also schedules an A/B testing if possible and notifies another server; e.g. to send an email. (main): Loads the settings JSON specified in the argument. (fetchManifestAndAnalyzeData): The main loop that periodically wakes up to do the analysis. (mapInOrder): Executes callback sequentially (i.e. blocking) on each item in the array. (configurationsForTesting): Finds every (platform, metric) pair to analyze in the v2 dashbaords, and computes various values for when statistically significant changes are detected later. (analyzeConfiguration): Finds potential regressions and progression in the last X days where X is the specified maximum number of days using the specified strategies. Sort the resultant ranges in chronological order and create a new analysis task for the very last change we detected. We'll eventually create an analysis task for all detected changes since we're repeating the analysis in fetchManifestAndAnalyzeData after some time. (computeRangesForTesting): Fetch measured values and compute ranges to test using the specified segmentation and test range selection strategies. Once ranges are found, find overlapping analysis tasks as they need to be filtered out in analyzeConfiguration to avoid creating multiple analysis tasks for the same range (e.g. humans may create one before the script gets to do it). (createAnalysisTaskAndNotify): Create a new analysis task for the specified range, trigger an A/B testing if available, and notify another server with a HTML message as specified. (findStrategyByLabel): (changeTypeForRange): A change is a regression if values are getting larger in a smaller-is-better test or values are getting smaller in a larger-is-better test and vice versa. (summarizeRange): Create a human readable string that summarizes the change detected. e.g. "Potential 3.2% regression detected between 2015-04-20 12:00 and 17:00". (formatTimeRange): (getJSON): (postJSON): (postNotification): Recursively replaces $title and $massage in the specified JSON template. (instantiateNotificationTemplate): (fetchJSON): Canonical link: https://commits.webkit.org/162105@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@183232 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-04-24 01:16:37 +00:00
CREATE TABLE analysis_strategies (
strategy_id serial PRIMARY KEY,
strategy_name varchar(64) NOT NULL);
Perf dashboard should have UI to set status on analysis tasks https://bugs.webkit.org/show_bug.cgi?id=143977 Reviewed by Chris Dumez. Added the UI to set the result of an analysis task to 'progression', 'regression', 'unchanged', and 'inconclusive' as well as a boolean indicating whether creating the analysis task was the right thing to do or not. The latter will be a useful metric once we start automatically creating analysis tasks. * init-database.sql: Added two columns to analysis_tasks table. * public/api/analysis-tasks.php: Include the added columns in the JSON. * public/include/db.php: (Database::to_database_boolean): Added. * public/include/json-header.php: (require_match_one_of_values): Added. * public/privileged-api/update-analysis-task.php: Added. Updates 'result' and 'needed' values of an analysis task. (main): * public/v2/analysis.js: (App.AnalysisTask.result): Added. (App.AnalysisTask.needed): Added. We don't use DS.attr('boolean') here since that would coerce null into false and we want to differentiate null from false in order to differentiate the null-ness of the value. (App.AnalysisTask.saveStatus): Added. (App.AnalysisTask.statusLabel): Use 'result' as the label if it's set and all build requests have been processed. * public/v2/app.css: * public/v2/app.js: (App.AnalysisTaskController.analysisResultOptions): Added. (App.AnalysisTaskController.shouldNotHaveBeenCreated): Added. (App.AnalysisTaskController.needsFeedback): Added. Show the checkbox to indicate the analysis task should not have been created if 'no change' is selected. (App.AnalysisTaskController._updateChosenAnalysisResult): Added. (App.AnalysisTaskController.actions.saveStatus): Added. * public/v2/index.html: Extracted a partial template for updating the bug numbers. Also added the UI to update 'result' and 'needed' values of the analysis task. Canonical link: https://commits.webkit.org/161959@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@183041 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-04-21 02:48:22 +00:00
CREATE TYPE analysis_task_result_type as ENUM ('progression', 'regression', 'unchanged', 'inconclusive');
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
CREATE TABLE analysis_tasks (
task_id serial PRIMARY KEY,
task_name varchar(256) NOT NULL,
task_author varchar(256),
Perf dashboard should automatically detect regressions https://bugs.webkit.org/show_bug.cgi?id=141443 Reviewed by Anders Carlsson. Added a node.js script detect-changes.js to detect potential regressions and progressions on the graphs tracked on v2 dashboards. * init-database.sql: Added analysis_strategies table and task_segmentation and task_test_range columns to analysis_tasks to keep the segmentation and test range selection strategies used to create an analysis task. * public/api/analysis-tasks.php: (format_task): Include task_segmentation and analysis_tasks in the results. * public/include/json-header.php: (remote_user_name): Returns null when the privileged API is authenticated as a slave instead of a CSRF prevention token. (should_authenticate_as_slave): Added. (ensure_privileged_api_data_and_token_or_slave): Added. Authenticate as a slave if slaveName and slavePassword are specified. Since detect-changes.js and other slaves are not susceptible to a CSRF attack, we don't need to check a CSRF token. * public/privileged-api/create-analysis-task.php: (main): Use ensure_privileged_api_data_and_token_or_slave to let detect-changes.js create new analysis task. Also add or find segmentation and test range selection strategies if specified. * public/privileged-api/create-test-group.php: (main): Use ensure_privileged_api_data_and_token_or_slave. * public/privileged-api/generate-csrf-token.php: * public/v2/app.js: (App.Pane._computeMovingAverageAndOutliers): _executeStrategy has been moved to Statistics. * public/v2/data.js: Export Measurement, RunsData, TimeSeries. Used in detect-changes.js. (Array.prototype.find): Added a polyfill to be used in node.js. (RunsData.fetchRuns): (RunsData.pathForFetchingRuns): Extracted from fetchRuns. Used in detect-changes.js. (RunsData.createRunsDataInResponse): Extracted from App.Manifest._formatFetchedData to use it in detect-changes.js. (RunsData.unitFromMetricName): Ditto. (RunsData.isSmallerBetter): Ditto. (RunsData.prototype._timeSeriesByTimeInternal): Added secondaryTime to sort points when commit times are identical. (TimeSeries): When commit times are identical, order points based on build time. This is needed for when we trigger two builds at two different OS versions with the same WebKit revision since OS versions don't change the commit times. (TimeSeries.prototype.findPointByIndex): Added. (TimeSeries.prototype.rawValues): Added. * public/v2/js/statistics.js: (Statistics.TestRangeSelectionStrategies.[0]): Use the 99% two-sided probability as claimed in the description of this strategy instead of the default probability. Also fixed a bug that debugging code was referring to non-existent variables. (Statistics.executeStrategy): Moved from App.Pane (app.js). * public/v2/manifest.js: (App.Manifest._formatFetchedData): Various code has been extracted into RunsData in data.js to be used in detect-changes.js. * tools/detect-changes.js: Added. The script fetches the manifest JSON, analyzes each graph in the v2 dashboards, and creates an analysis task for the latest regression or progression detected. It also schedules an A/B testing if possible and notifies another server; e.g. to send an email. (main): Loads the settings JSON specified in the argument. (fetchManifestAndAnalyzeData): The main loop that periodically wakes up to do the analysis. (mapInOrder): Executes callback sequentially (i.e. blocking) on each item in the array. (configurationsForTesting): Finds every (platform, metric) pair to analyze in the v2 dashbaords, and computes various values for when statistically significant changes are detected later. (analyzeConfiguration): Finds potential regressions and progression in the last X days where X is the specified maximum number of days using the specified strategies. Sort the resultant ranges in chronological order and create a new analysis task for the very last change we detected. We'll eventually create an analysis task for all detected changes since we're repeating the analysis in fetchManifestAndAnalyzeData after some time. (computeRangesForTesting): Fetch measured values and compute ranges to test using the specified segmentation and test range selection strategies. Once ranges are found, find overlapping analysis tasks as they need to be filtered out in analyzeConfiguration to avoid creating multiple analysis tasks for the same range (e.g. humans may create one before the script gets to do it). (createAnalysisTaskAndNotify): Create a new analysis task for the specified range, trigger an A/B testing if available, and notify another server with a HTML message as specified. (findStrategyByLabel): (changeTypeForRange): A change is a regression if values are getting larger in a smaller-is-better test or values are getting smaller in a larger-is-better test and vice versa. (summarizeRange): Create a human readable string that summarizes the change detected. e.g. "Potential 3.2% regression detected between 2015-04-20 12:00 and 17:00". (formatTimeRange): (getJSON): (postJSON): (postNotification): Recursively replaces $title and $massage in the specified JSON template. (instantiateNotificationTemplate): (fetchJSON): Canonical link: https://commits.webkit.org/162105@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@183232 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-04-24 01:16:37 +00:00
task_segmentation integer REFERENCES analysis_strategies,
task_test_range integer REFERENCES analysis_strategies,
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
task_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
task_platform integer REFERENCES platforms,
task_metric integer REFERENCES test_metrics,
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
task_start_run integer REFERENCES test_runs,
task_start_run_time timestamp,
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
task_end_run integer REFERENCES test_runs,
task_end_run_time timestamp,
Perf dashboard should have UI to set status on analysis tasks https://bugs.webkit.org/show_bug.cgi?id=143977 Reviewed by Chris Dumez. Added the UI to set the result of an analysis task to 'progression', 'regression', 'unchanged', and 'inconclusive' as well as a boolean indicating whether creating the analysis task was the right thing to do or not. The latter will be a useful metric once we start automatically creating analysis tasks. * init-database.sql: Added two columns to analysis_tasks table. * public/api/analysis-tasks.php: Include the added columns in the JSON. * public/include/db.php: (Database::to_database_boolean): Added. * public/include/json-header.php: (require_match_one_of_values): Added. * public/privileged-api/update-analysis-task.php: Added. Updates 'result' and 'needed' values of an analysis task. (main): * public/v2/analysis.js: (App.AnalysisTask.result): Added. (App.AnalysisTask.needed): Added. We don't use DS.attr('boolean') here since that would coerce null into false and we want to differentiate null from false in order to differentiate the null-ness of the value. (App.AnalysisTask.saveStatus): Added. (App.AnalysisTask.statusLabel): Use 'result' as the label if it's set and all build requests have been processed. * public/v2/app.css: * public/v2/app.js: (App.AnalysisTaskController.analysisResultOptions): Added. (App.AnalysisTaskController.shouldNotHaveBeenCreated): Added. (App.AnalysisTaskController.needsFeedback): Added. Show the checkbox to indicate the analysis task should not have been created if 'no change' is selected. (App.AnalysisTaskController._updateChosenAnalysisResult): Added. (App.AnalysisTaskController.actions.saveStatus): Added. * public/v2/index.html: Extracted a partial template for updating the bug numbers. Also added the UI to update 'result' and 'needed' values of the analysis task. Canonical link: https://commits.webkit.org/161959@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@183041 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-04-21 02:48:22 +00:00
task_result analysis_task_result_type,
task_needed boolean,
CONSTRAINT analysis_task_should_be_unique_for_range UNIQUE(task_start_run, task_end_run),
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
CONSTRAINT analysis_task_must_be_associated_with_run_or_be_custom
CHECK ((task_start_run IS NULL AND task_start_run_time IS NULL
AND task_end_run IS NULL AND task_end_run_time IS NULL
AND task_platform IS NULL AND task_metric IS NULL)
OR (task_start_run IS NOT NULL AND task_start_run_time IS NOT NULL
AND task_end_run IS NOT NULL AND task_end_run_time IS NOT NULL
AND task_platform IS NOT NULL AND task_metric IS NOT NULL)));
New perf dashboard should provide UI to create a new analysis task https://bugs.webkit.org/show_bug.cgi?id=138910 Reviewed by Benjamin Poulain. This patch reverts some parts of r175006 and re-introduces bugs associated with analysis tasks. I'll add UI to show and edit bug numbers associated with an analysis task in a follow up patch. With this patch, we can create a new analysis task by selection a range of points and opening "analysis pane" (renamed from "bugs pane"). Each analysis task created is represented by a yellow bar in the chart hyperlinked to the analysis task. * init-database.sql: Redefined the bugs to be associated with an analysis task instead of a test run. * public/api/analysis-tasks.php: Added the support for querying analysis tasks for a specific metric on a specific platform. Also retrieve and return all bugs associated with analysis tasks. (main): (fetch_and_push_bugs_to_tasks): Added. Fetches all bugs associated with an array of analysis tasks and adds the associated bugs to each task in the array. (format_task): * public/api/runs.php: Reverted changes made in r175006. (fetch_runs_for_config): (format_run): * public/api/test-groups.php: (fetch_test_groups_for_task): Use the newly added Database::select_rows. * public/include/db.php: (Database::select_first_or_last_row): (Database::select_rows): Extracted from select_first_or_last_row. * public/v2/analysis.js: (App.AnalysisTask): Added "bugs" property. (App.Bug): Added now that bugs are regular data store objects. * public/v2/app.js: (App.Pane._fetch): Calls this.fetchAnalyticRanges to fetch analysis tasks as well as test runs. (App.Pane.fetchAnalyticRanges): Added. Fetches analysis tasks for the current metric on the current platform that are associated with a specific range of runs. (App.PaneController.actions.toggleBugsPane): Updated per showingBugsPane to showingAnalysisPane rename. (App.PaneController.actions.associateBug): Deleted. (App.PaneController.actions.createAnalysisTask): Replaced the pre-condition checks with assertions as this action should never be triggered when the pre-condition is not met. Also re-fetch analysis tasks once we've created one. (App.PaneController.toggleSearchPane): Updated per showingBugsPane to showingAnalysisPane rename. (App.PaneController._detailsChanged): Ditto. Removed selectedSinglePoint since it's no longer used. (App.PaneController._showDetails): Call _updateCanAnalyze to update the status of "Analyze" button. (App.PaneController._updateBugs): Deleted. (App.PaneController._updateMarkedPoints): Deleted. (App.PaneController._updateCanAnalyze): Added. Disables the button to create an analysis task when the name is missing or when at most one point is selected. (App.InteractiveChartComponent._constructGraphIfPossible): Update the locations of range rects. (App.InteractiveChartComponent._relayoutDataAndAxes): Ditto. (App.InteractiveChartComponent._mousePointInGraph): Don't return a point unless the mouse cursor is on our svg element to avoid locking the current item when a bar shown for an analysis task is clicked. (App.InteractiveChartComponent._rangesChanged): Added. Creates an array of objects representing clickable bars for analysis tasks. (App.InteractiveChartComponent._updateRangeBarRects): Computes the inline style used by each clickable bar for analysis tasks to place them at the right location. (App.InteractiveChartComponent.actions.openRange): Added. Forwards the action to the parent controller. * public/v2/chart-pane.css: (.chart .extent): Use the same color as the vertical indicator in the highlight behind the selection. (.chart .rangeBar): Added. * public/v2/data.js: (TimeSeries.prototype.nextPoint): Added. Used by _rangesChanged. * public/v2/index.html: Renamed "bugs pane" to "analysis pane" and removed the UI to associate bugs. This ability will be reinstated in a follow up patch. Also added a container div and spans for analysis task bars in the interactive chart component. Canonical link: https://commits.webkit.org/156808@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@176422 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-20 23:25:07 +00:00
Analysis task page should allow specifying commits that caused or fixed a regression or a progression https://bugs.webkit.org/show_bug.cgi?id=155529 Reviewed by Chris Dumez. Added the capability to associate revisions that caused or fixed a progression or a regression for which an analysis task was created. Added task_commits that stores this relationship and added the backend support to retrieve this table in /api/analysis-tasks and an privileged API to update this table at /privileged-api/associate-commit. Also extracted a new component, MutableListView, out of AnalysisTaskPage to render and manipulate a list of mutable items, and used it to render the list of associated bugs and commits. The view takes a list of kinds (e.g. repositories or bug trackers), and accepts a pair of a kind and arbitrary text as a new item value. * init-database.sql: Added task_commits table. * public/api/analysis-tasks.php: (main): (fetch_associated_data_for_tasks): Renamed from fetch_and_push_bugs_to_tasks now that it also fetches the list of commits associated with each analysis task by calling CommitLogFetcher::fetch_for_tasks. Also fixe the bug that we were not taking (format_task): No longer sets 'category' since the computation of category now depends on the list of commits associated with this analysis task which aren't available until fetch_associated_data_for_tasks. (determine_category): Added. Categorize any analysis tasks with "fixes" commits as "closed" and "causes" commits as "identified". * public/include/commit-log-fetcher.php: (CommitLogFetcher::__construct): Remove the unused instance variable. (CommitLogFetcher::fetch_for_tasks): Added. Fetches all commits associated with a list of analysis tasks. Assumes the caller (fetch_associated_data_for_tasks) had setup "fixes" and "causes" fields on each task. * public/privileged-api/associate-commit.php: Added. Updates task_commits table to associate or disassociate a commit with an analysis task. When the specified analysis task and the specified commit are already associated, we simply update the table instead of adding a duplicating entry or error. For dissociation, the front-end specifies the commit ID. (main): Added. * public/v3/index.html: * public/v3/components/mutable-list-view.js: Added. Used by the list associated bugs and commits. (MutableListView): Added. (MutableListView.prototype.setList): Added. (MutableListView.prototype.setKindList): Added. (MutableListView.prototype.setAddCallback): Added. This callback is invoked when the user tries to add a new item to the list. (MutableListView.prototype.render): Added. (MutableListView.prototype._submitted): Added. (MutableListView.cssTemplate): (MutableListView.htmlTemplate): (MutableListItem): Added. RemovalLink could be a hyperlink or a callback and gets involved when the user tries to delete this item. (MutableListItem.prototype.content): * public/v3/models/analysis-task.js: (AnalysisTask): Added the support of the list of commits that fixed and caused changes. (AnalysisTask.prototype.updateSingleton): Ditto. (AnalysisTask.prototype.causes): Added. (AnalysisTask.prototype.fixes): Added. (AnalysisTask.prototype.associateCommit): Added. Use the API added at /privileged-api/associate-commit to associate a new commit with this analysis task. Each commit has either caused or fixed the change. (AnalysisTask.prototype.dissociateCommit): Added. Use the same API to disassociate each commit. (AnalysisTask._constructAnalysisTasksFromRawData): Find all commits associated with each analysis task. Because commit log objects use a fake ID fdue to /api/measurement-set not providing commit IDs, we must use CommitLog.findByRemoteId to find each commit instead of usual CommitLog.findById. (AnalysisTask._constructAnalysisTasksFromRawData.resolveCommits): Added. * public/v3/models/build-request.js: (BuildRequest.prototype.hasFinished): Renamed from hasCompleted since it was confusing for this._status being "completed" wasn't a necessary condition for this function to return true. * public/v3/models/commit-log.js: (CommitLog): Added the static map for actual commit ID instead of a fake ID created in ensureSingleton. (CommitLog.prototype.remoteId): Added. Returns the real commit ID. (CommitLog.findByRemoteId): Added. Finds an CommitLog object using the real ID. * public/v3/models/test-group.js: (TestGroup.prototype.hasFinished): Renamed from hasCompleted to match the rename in BuildRequest. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage): Added lists for the commits that fixed and caused the change using MutableListView. Also adopted MutableListView for the list of associated bugs. (AnalysisTaskPage.prototype.render): Added the code to populate the newly added lists. (AnalysisTaskPage.prototype._makeCommitListItem): Added. (AnalysisTaskPage.prototype._associateBug): Now this is a callback from MutableListView. (AnalysisTaskPage.prototype._associateCommit): Added. (AnalysisTaskPage.prototype._dissociateCommit): Added. (AnalysisTaskPage.htmlTemplate): (AnalysisTaskPage.cssTemplate): * public/v3/remote.js: (getJSON): Spit out the entire responseText when JSON failed to parse to make debugging easier. Canonical link: https://commits.webkit.org/173656@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@198265 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-03-16 07:02:28 +00:00
CREATE TABLE task_commits (
taskcommit_task integer NOT NULL REFERENCES analysis_tasks ON DELETE CASCADE,
taskcommit_commit integer NOT NULL REFERENCES commits ON DELETE CASCADE,
taskcommit_is_fix boolean NOT NULL,
Analysis task page should allow specifying commits that caused or fixed a regression or a progression https://bugs.webkit.org/show_bug.cgi?id=155529 Reviewed by Chris Dumez. Added the capability to associate revisions that caused or fixed a progression or a regression for which an analysis task was created. Added task_commits that stores this relationship and added the backend support to retrieve this table in /api/analysis-tasks and an privileged API to update this table at /privileged-api/associate-commit. Also extracted a new component, MutableListView, out of AnalysisTaskPage to render and manipulate a list of mutable items, and used it to render the list of associated bugs and commits. The view takes a list of kinds (e.g. repositories or bug trackers), and accepts a pair of a kind and arbitrary text as a new item value. * init-database.sql: Added task_commits table. * public/api/analysis-tasks.php: (main): (fetch_associated_data_for_tasks): Renamed from fetch_and_push_bugs_to_tasks now that it also fetches the list of commits associated with each analysis task by calling CommitLogFetcher::fetch_for_tasks. Also fixe the bug that we were not taking (format_task): No longer sets 'category' since the computation of category now depends on the list of commits associated with this analysis task which aren't available until fetch_associated_data_for_tasks. (determine_category): Added. Categorize any analysis tasks with "fixes" commits as "closed" and "causes" commits as "identified". * public/include/commit-log-fetcher.php: (CommitLogFetcher::__construct): Remove the unused instance variable. (CommitLogFetcher::fetch_for_tasks): Added. Fetches all commits associated with a list of analysis tasks. Assumes the caller (fetch_associated_data_for_tasks) had setup "fixes" and "causes" fields on each task. * public/privileged-api/associate-commit.php: Added. Updates task_commits table to associate or disassociate a commit with an analysis task. When the specified analysis task and the specified commit are already associated, we simply update the table instead of adding a duplicating entry or error. For dissociation, the front-end specifies the commit ID. (main): Added. * public/v3/index.html: * public/v3/components/mutable-list-view.js: Added. Used by the list associated bugs and commits. (MutableListView): Added. (MutableListView.prototype.setList): Added. (MutableListView.prototype.setKindList): Added. (MutableListView.prototype.setAddCallback): Added. This callback is invoked when the user tries to add a new item to the list. (MutableListView.prototype.render): Added. (MutableListView.prototype._submitted): Added. (MutableListView.cssTemplate): (MutableListView.htmlTemplate): (MutableListItem): Added. RemovalLink could be a hyperlink or a callback and gets involved when the user tries to delete this item. (MutableListItem.prototype.content): * public/v3/models/analysis-task.js: (AnalysisTask): Added the support of the list of commits that fixed and caused changes. (AnalysisTask.prototype.updateSingleton): Ditto. (AnalysisTask.prototype.causes): Added. (AnalysisTask.prototype.fixes): Added. (AnalysisTask.prototype.associateCommit): Added. Use the API added at /privileged-api/associate-commit to associate a new commit with this analysis task. Each commit has either caused or fixed the change. (AnalysisTask.prototype.dissociateCommit): Added. Use the same API to disassociate each commit. (AnalysisTask._constructAnalysisTasksFromRawData): Find all commits associated with each analysis task. Because commit log objects use a fake ID fdue to /api/measurement-set not providing commit IDs, we must use CommitLog.findByRemoteId to find each commit instead of usual CommitLog.findById. (AnalysisTask._constructAnalysisTasksFromRawData.resolveCommits): Added. * public/v3/models/build-request.js: (BuildRequest.prototype.hasFinished): Renamed from hasCompleted since it was confusing for this._status being "completed" wasn't a necessary condition for this function to return true. * public/v3/models/commit-log.js: (CommitLog): Added the static map for actual commit ID instead of a fake ID created in ensureSingleton. (CommitLog.prototype.remoteId): Added. Returns the real commit ID. (CommitLog.findByRemoteId): Added. Finds an CommitLog object using the real ID. * public/v3/models/test-group.js: (TestGroup.prototype.hasFinished): Renamed from hasCompleted to match the rename in BuildRequest. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage): Added lists for the commits that fixed and caused the change using MutableListView. Also adopted MutableListView for the list of associated bugs. (AnalysisTaskPage.prototype.render): Added the code to populate the newly added lists. (AnalysisTaskPage.prototype._makeCommitListItem): Added. (AnalysisTaskPage.prototype._associateBug): Now this is a callback from MutableListView. (AnalysisTaskPage.prototype._associateCommit): Added. (AnalysisTaskPage.prototype._dissociateCommit): Added. (AnalysisTaskPage.htmlTemplate): (AnalysisTaskPage.cssTemplate): * public/v3/remote.js: (getJSON): Spit out the entire responseText when JSON failed to parse to make debugging easier. Canonical link: https://commits.webkit.org/173656@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@198265 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-03-16 07:02:28 +00:00
CONSTRAINT task_commit_must_be_unique UNIQUE(taskcommit_task, taskcommit_commit));
New perf dashboard should provide UI to create a new analysis task https://bugs.webkit.org/show_bug.cgi?id=138910 Reviewed by Benjamin Poulain. This patch reverts some parts of r175006 and re-introduces bugs associated with analysis tasks. I'll add UI to show and edit bug numbers associated with an analysis task in a follow up patch. With this patch, we can create a new analysis task by selection a range of points and opening "analysis pane" (renamed from "bugs pane"). Each analysis task created is represented by a yellow bar in the chart hyperlinked to the analysis task. * init-database.sql: Redefined the bugs to be associated with an analysis task instead of a test run. * public/api/analysis-tasks.php: Added the support for querying analysis tasks for a specific metric on a specific platform. Also retrieve and return all bugs associated with analysis tasks. (main): (fetch_and_push_bugs_to_tasks): Added. Fetches all bugs associated with an array of analysis tasks and adds the associated bugs to each task in the array. (format_task): * public/api/runs.php: Reverted changes made in r175006. (fetch_runs_for_config): (format_run): * public/api/test-groups.php: (fetch_test_groups_for_task): Use the newly added Database::select_rows. * public/include/db.php: (Database::select_first_or_last_row): (Database::select_rows): Extracted from select_first_or_last_row. * public/v2/analysis.js: (App.AnalysisTask): Added "bugs" property. (App.Bug): Added now that bugs are regular data store objects. * public/v2/app.js: (App.Pane._fetch): Calls this.fetchAnalyticRanges to fetch analysis tasks as well as test runs. (App.Pane.fetchAnalyticRanges): Added. Fetches analysis tasks for the current metric on the current platform that are associated with a specific range of runs. (App.PaneController.actions.toggleBugsPane): Updated per showingBugsPane to showingAnalysisPane rename. (App.PaneController.actions.associateBug): Deleted. (App.PaneController.actions.createAnalysisTask): Replaced the pre-condition checks with assertions as this action should never be triggered when the pre-condition is not met. Also re-fetch analysis tasks once we've created one. (App.PaneController.toggleSearchPane): Updated per showingBugsPane to showingAnalysisPane rename. (App.PaneController._detailsChanged): Ditto. Removed selectedSinglePoint since it's no longer used. (App.PaneController._showDetails): Call _updateCanAnalyze to update the status of "Analyze" button. (App.PaneController._updateBugs): Deleted. (App.PaneController._updateMarkedPoints): Deleted. (App.PaneController._updateCanAnalyze): Added. Disables the button to create an analysis task when the name is missing or when at most one point is selected. (App.InteractiveChartComponent._constructGraphIfPossible): Update the locations of range rects. (App.InteractiveChartComponent._relayoutDataAndAxes): Ditto. (App.InteractiveChartComponent._mousePointInGraph): Don't return a point unless the mouse cursor is on our svg element to avoid locking the current item when a bar shown for an analysis task is clicked. (App.InteractiveChartComponent._rangesChanged): Added. Creates an array of objects representing clickable bars for analysis tasks. (App.InteractiveChartComponent._updateRangeBarRects): Computes the inline style used by each clickable bar for analysis tasks to place them at the right location. (App.InteractiveChartComponent.actions.openRange): Added. Forwards the action to the parent controller. * public/v2/chart-pane.css: (.chart .extent): Use the same color as the vertical indicator in the highlight behind the selection. (.chart .rangeBar): Added. * public/v2/data.js: (TimeSeries.prototype.nextPoint): Added. Used by _rangesChanged. * public/v2/index.html: Renamed "bugs pane" to "analysis pane" and removed the UI to associate bugs. This ability will be reinstated in a follow up patch. Also added a container div and spans for analysis task bars in the interactive chart component. Canonical link: https://commits.webkit.org/156808@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@176422 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-20 23:25:07 +00:00
CREATE TABLE bugs (
bug_id serial PRIMARY KEY,
bug_task integer REFERENCES analysis_tasks NOT NULL,
bug_tracker integer REFERENCES bug_trackers NOT NULL,
bug_number integer NOT NULL);
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
CREATE TABLE build_triggerables (
triggerable_id serial PRIMARY KEY,
Introduce the notion of repository groups to triggerables https://bugs.webkit.org/show_bug.cgi?id=170228 Reviewed by Chris Dumez. On some triggerable, it's desirable to specify multiple sets of repositories that are accepted. For example, if a repository X transitioned from Subversion to Git, and if a triggerable accepted X and some other repository Y, then it's desirable to two sets: (X-Subversion, Y) and (X-Git, Y) since neither (X-Subversion, X-Git) nor (X-Subversion, X-Git, Y) makes sense as a set. This patch introduces triggerable_repository_groups table to represent a set of repositories accepted by a triggerable. It has many to one relationship to build_triggerables and triggerable_repositories in turn now has many to one relationship to triggerable_repository_groups instead of build_triggerables. Also make it possible to disable a triggerable e.g. a set of tests and platforms are no longer supported. We don't want to delete the triggerable completely from the database since it would result in the associated A/B testing results being purged, which is not desirale. To migrate an existing database, run the following transaction: ```sql BEGIN; ALTER TABLE build_triggerables ADD COLUMN triggerable_disabled boolean NOT NULL DEFAULT FALSE; CREATE TABLE triggerable_repository_groups ( repositorygroup_id serial PRIMARY KEY, repositorygroup_triggerable integer REFERENCES build_triggerables NOT NULL, repositorygroup_name varchar(256) NOT NULL, repositorygroup_description varchar(256), repositorygroup_accepts_roots boolean NOT NULL DEFAULT FALSE, CONSTRAINT repository_group_name_must_be_unique_for_triggerable UNIQUE(repositorygroup_triggerable, repositorygroup_name)); INSERT INTO triggerable_repository_groups (repositorygroup_triggerable, repositorygroup_name) SELECT triggerable_id, 'default' FROM build_triggerables; ALTER TABLE triggerable_repositories ADD COLUMN trigrepo_group integer REFERENCES triggerable_repository_groups; UPDATE triggerable_repositories SET trigrepo_group = repositorygroup_id FROM triggerable_repository_groups WHERE trigrepo_triggerable = repositorygroup_triggerable; ALTER TABLE triggerable_repositories ALTER COLUMN trigrepo_group SET NOT NULL; ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_triggerable; ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_sub_roots; END; ``` * init-database.sql: * public/admin/triggerables.php: Use a custom column to make forms to add and configure repository groups. (insert_triggerable_repositories): Added. (generate_repository_list): Added. (generate_repository_form): Added. (generate_repository_checkboxes): Now generates checkboxes for a repository group instead of a triggerable. * public/include/manifest-generator.php: (fetch_triggerables): Fixed the bug that we were not filtering results with query in /api/triggerable. Rewrote it to include an array of repository groups, which in turn contains an array of repositories along with its name and a description, and a boolean indicating whether it accepts a custom root file or not. The boolean will be used when we're adding the support for perf try bots. We will keep acceptedRepositories since it's still used by detect-changes.js. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Resolve repositoriy, test, and platform IDs to their respective objects. * public/v3/models/triggerable.js: (Triggerable): (Triggerable.prototype.isDisabled): Added. (Triggerable.prototype.repositoryGroups): Added. (Triggerable.prototype.acceptsTest): Added. (TriggerableRepositoryGroup): Added. (TriggerableRepositoryGroup.prototype.description): Added. (TriggerableRepositoryGroup.prototype.acceptsCustomRoots): Added. (TriggerableRepositoryGroup.prototype.repositories): Added. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't use a disabled triggerable. * server-tests/api-manifest-tests.js: Updated a test case to test repository groups. * tools/js/database.js: (tableToPrefixMap): Added triggerable_repository_groups. * tools/js/v3-models.js: Imported TriggerableRepositoryGroup from triggerable.js. Canonical link: https://commits.webkit.org/187454@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214975 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-05 23:12:46 +00:00
triggerable_name varchar(64) NOT NULL UNIQUE,
triggerable_disabled boolean NOT NULL DEFAULT FALSE);
CREATE TABLE triggerable_repository_groups (
repositorygroup_id serial PRIMARY KEY,
repositorygroup_triggerable integer REFERENCES build_triggerables NOT NULL,
repositorygroup_name varchar(256) NOT NULL,
repositorygroup_description varchar(256),
repositorygroup_accepts_roots boolean NOT NULL DEFAULT FALSE,
repositorygroup_hidden boolean NOT NULL DEFAULT FALSE,
Introduce the notion of repository groups to triggerables https://bugs.webkit.org/show_bug.cgi?id=170228 Reviewed by Chris Dumez. On some triggerable, it's desirable to specify multiple sets of repositories that are accepted. For example, if a repository X transitioned from Subversion to Git, and if a triggerable accepted X and some other repository Y, then it's desirable to two sets: (X-Subversion, Y) and (X-Git, Y) since neither (X-Subversion, X-Git) nor (X-Subversion, X-Git, Y) makes sense as a set. This patch introduces triggerable_repository_groups table to represent a set of repositories accepted by a triggerable. It has many to one relationship to build_triggerables and triggerable_repositories in turn now has many to one relationship to triggerable_repository_groups instead of build_triggerables. Also make it possible to disable a triggerable e.g. a set of tests and platforms are no longer supported. We don't want to delete the triggerable completely from the database since it would result in the associated A/B testing results being purged, which is not desirale. To migrate an existing database, run the following transaction: ```sql BEGIN; ALTER TABLE build_triggerables ADD COLUMN triggerable_disabled boolean NOT NULL DEFAULT FALSE; CREATE TABLE triggerable_repository_groups ( repositorygroup_id serial PRIMARY KEY, repositorygroup_triggerable integer REFERENCES build_triggerables NOT NULL, repositorygroup_name varchar(256) NOT NULL, repositorygroup_description varchar(256), repositorygroup_accepts_roots boolean NOT NULL DEFAULT FALSE, CONSTRAINT repository_group_name_must_be_unique_for_triggerable UNIQUE(repositorygroup_triggerable, repositorygroup_name)); INSERT INTO triggerable_repository_groups (repositorygroup_triggerable, repositorygroup_name) SELECT triggerable_id, 'default' FROM build_triggerables; ALTER TABLE triggerable_repositories ADD COLUMN trigrepo_group integer REFERENCES triggerable_repository_groups; UPDATE triggerable_repositories SET trigrepo_group = repositorygroup_id FROM triggerable_repository_groups WHERE trigrepo_triggerable = repositorygroup_triggerable; ALTER TABLE triggerable_repositories ALTER COLUMN trigrepo_group SET NOT NULL; ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_triggerable; ALTER TABLE triggerable_repositories DROP COLUMN trigrepo_sub_roots; END; ``` * init-database.sql: * public/admin/triggerables.php: Use a custom column to make forms to add and configure repository groups. (insert_triggerable_repositories): Added. (generate_repository_list): Added. (generate_repository_form): Added. (generate_repository_checkboxes): Now generates checkboxes for a repository group instead of a triggerable. * public/include/manifest-generator.php: (fetch_triggerables): Fixed the bug that we were not filtering results with query in /api/triggerable. Rewrote it to include an array of repository groups, which in turn contains an array of repositories along with its name and a description, and a boolean indicating whether it accepts a custom root file or not. The boolean will be used when we're adding the support for perf try bots. We will keep acceptedRepositories since it's still used by detect-changes.js. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Resolve repositoriy, test, and platform IDs to their respective objects. * public/v3/models/triggerable.js: (Triggerable): (Triggerable.prototype.isDisabled): Added. (Triggerable.prototype.repositoryGroups): Added. (Triggerable.prototype.acceptsTest): Added. (TriggerableRepositoryGroup): Added. (TriggerableRepositoryGroup.prototype.description): Added. (TriggerableRepositoryGroup.prototype.acceptsCustomRoots): Added. (TriggerableRepositoryGroup.prototype.repositories): Added. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't use a disabled triggerable. * server-tests/api-manifest-tests.js: Updated a test case to test repository groups. * tools/js/database.js: (tableToPrefixMap): Added triggerable_repository_groups. * tools/js/v3-models.js: Imported TriggerableRepositoryGroup from triggerable.js. Canonical link: https://commits.webkit.org/187454@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214975 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-05 23:12:46 +00:00
CONSTRAINT repository_group_name_must_be_unique_for_triggerable UNIQUE(repositorygroup_triggerable, repositorygroup_name));
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
CREATE TABLE triggerable_repositories (
trigrepo_repository integer REFERENCES repositories NOT NULL,
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
trigrepo_group integer REFERENCES triggerable_repository_groups NOT NULL,
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
trigrepo_accepts_patch boolean NOT NULL DEFAULT FALSE,
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
CONSTRAINT repository_must_be_unique_for_repository_group UNIQUE(trigrepo_repository, trigrepo_group));
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
CREATE TABLE triggerable_configurations (
trigconfig_test integer REFERENCES tests NOT NULL,
trigconfig_platform integer REFERENCES platforms NOT NULL,
trigconfig_triggerable integer REFERENCES build_triggerables NOT NULL,
CONSTRAINT triggerable_must_be_unique_for_test_and_platform UNIQUE(trigconfig_test, trigconfig_platform));
Add the file uploading capability to the perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=169737 Reviewed by Chris Dumez. Added /privileged-api/upload-file to upload a file, and /api/uploaded-file/ to download the file and retrieve its meta data based on its SHA256. We treat two files with the identical SHA256 as identical since anyone who can upload a file using this mechanism can execute arbitrary code in our bots anyway. This is important for avoiding uploading a large darwinup roots multiple times to the server, saving both user's time/bandwidth and server's disk space. * config.json: Added uploadDirectory, uploadFileLimitInMB, and uploadUserQuotaInMB as options. * init-database.sql: Added uploaded_files table. * public/api/uploaded-file.php: Added. (main): /api/uploaded-file/N would download uploaded_file with id=N. /api/uploaded-file/?sha256=X would return the meta data for uploaded_file with sha256=X. (stream_file_content): Streams the file content in 64KB chunks. We support Range & If-Range HTTP request headers so that browsers can pause and resume downloading of a large root file. (parse_range_header): Parses Range HTTP request header. * public/include/json-header.php: (remote_user_name): Use the default argument of NULL. * public/include/manifest-generator.php: (ManifestGenerator::generate): Include the maximum upload size in the manifest file to let the frontend code preemptively check the file size before attempting to submit a file. * public/include/uploaded-file-helpers.php: Added. (format_uploaded_file): (uploaded_file_path_for_row): * public/privileged-api/upload-file-form.html: Added. For debugging purposes. (fetchCSRFfToken): (upload): * public/privileged-api/upload-file.php: Added. (main): (query_total_file_size): (create_uploaded_file_from_form_data): * public/shared/common-remote.js: (CommonRemoteAPI.prototype.postFormData): Added. (CommonRemoteAPI.prototype.postFormDataWithStatus): Added. (CommonRemoteAPI.prototype.sendHttpRequestWithFormData): Added. (CommonRemoteAPI.prototype._asJSON): Throw an exception instead of calling a non-existent reject. * public/v3/models/uploaded-file.js: Added. (UploadedFile): Added. (UploadedFile.uploadFile): Added. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Added. Finds the file with the same SHA256 in the server to avoid uploading a large custom root multiple times. (UploadedFile._computeSHA256Hash): Added. * public/v3/privileged-api.js: (PrivilegedAPI.prototype.sendRequest): Added the options dictionary as a third argument. For now, only support useFormData boolean. * public/v3/remote.js: (BrowserRemoteAPI.prototype.sendHttpRequestWithFormData): Added. * server-tests/api-manifest.js: Updated per the inclusion of fileUploadSizeLimit in the manifest. * server-tests/api-uploaded-file.js: Added. * server-tests/privileged-api-upload-file-tests.js: Added. * server-tests/resources/temporary-file.js: Added. (TemporaryFile): Added. A helper class for creating a temporary file to upload. (TemporaryFile.makeTemporaryFileOfSizeInMB): (TemporaryFile.makeTemporaryFile): (TemporaryFile.inject): * server-tests/resources/test-server.conf: Set upload_max_filesize and post_max_size for testing. * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): Use uploadFileLimitInMB and uploadUserQuotaInMB of 2MB and 5MB. (TestServer.prototype._ensureDataDirectory): Create a directory to store uploaded files inside the data directory. In a production server, we can place it outside ServerRoot / DocumentRoot. (TestServer.prototype.cleanDataDirectory): Delete the aforementioned directory as needed. * tools/js/database.js: (tableToPrefixMap): Added uploaded_files. * tools/js/remote.js: (NodeRemoteAPI.prototype.sendHttpRequest): Added a dictionary to specify request headers and a callback to process the response as arguments. Fixed the bug that any 2xx code other than 200 was resulting in a rejected promise. Also include the response headers in the result for tests. Finally, when content is a function, call that instead of writing the content since FormData requires a custom logic. (NodeRemoteAPI.prototype.sendHttpRequestWithFormData): Added. * tools/js/v3-models.js: Include uploaded-file.js. * tools/run-tests.py: (main): Add form-data as a new dependency. Canonical link: https://commits.webkit.org/186730@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214065 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-16 20:53:35 +00:00
CREATE TABLE uploaded_files (
file_id serial PRIMARY KEY,
file_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
file_deleted_at timestamp,
file_mime varchar(64),
file_filename varchar(1024) NOT NULL,
file_extension varchar(16),
file_author varchar(256),
file_size bigint NOT NULL,
file_sha256 char(64) NOT NULL);
CREATE INDEX file_author_index ON uploaded_files(file_author);
CREATE UNIQUE INDEX file_sha256_index ON uploaded_files(file_sha256) WHERE file_deleted_at is NULL;
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
CREATE TYPE analysis_test_group_repetition_type as ENUM ('alternating', 'sequential');
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
CREATE TABLE analysis_test_groups (
testgroup_id serial PRIMARY KEY,
testgroup_task integer REFERENCES analysis_tasks NOT NULL,
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
testgroup_name varchar(256),
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
testgroup_author varchar(256),
testgroup_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
Perf dashboard should let user cancel pending A/B testing and hide failed ones https://bugs.webkit.org/show_bug.cgi?id=154433 Reviewed by Chris Dumez. Added a button to hide a test group in the details view (the bottom table) in the analysis task page, and "Show hidden tests" link to show the hidden test groups on demand. When a test group is hidden, all pending requests in the group will also be canceled since a common scenario of using this feature is that the user had triggered an useless A/B testing; e.g. all builds will fail, wrong, etc... We can revisit and add the capability to just cancel the pending requests and leaving the group visible later if necessary. Run `ALTER TYPE build_request_status_type ADD VALUE 'canceled';` to add the new type. * init-database.sql: Added testgroup_hidden column to analysis_test_groups table and added 'canceled' as a value to build_request_status_type table. * public/api/test-groups.php: (format_test_group): Added 'hidden' field in the JSON result. * public/privileged-api/update-test-group.php: (main): Added the support for updating testgroup_hidden column. When this column is set to true, also cancel all pending build requests (by setting its request_status to 'canceled' which will be ignore by the syncing script). * public/v3/components/test-group-results-table.js: (TestGroupResultsTable.prototype.setTestGroup): Reset _renderedTestGroup here so that the next call to render() will update the table; e.g. when build requests' status change from 'Pending' to 'Canceled'. * public/v3/models/build-request.js: (BuildRequest.prototype.hasCompleted): A build request is considered complete/finished if it's canceled. (BuildRequest.prototype.hasPending): Added. (BuildRequest.prototype.statusLabel): Handle 'canceled' status. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): Added to update 'hidden' field. (TestGroup.prototype.isHidden): Added. (TestGroup.prototype.hasPending): Added. (TestGroup.prototype.hasPending): Added. (TestGroup.prototype.updateHiddenFlag): Added. Uses the privileged API to update testgroup_hidden column. The JSON API also updates the status of the 'pending' build requests in the group to 'canceled'. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage): Added _showHiddenTestGroups and _filteredTestGroups as instance variables. (AnalysisTaskPage.prototype._didFetchTestGroups): (AnalysisTaskPage.prototype._showAllTestGroups): Added. (AnalysisTaskPage.prototype._didUpdateTestGroupHiddenState): Extracted from _didFetchTestGroups. (AnalysisTaskPage.prototype._renderTestGroupList): Use the filtered list of test groups to show the list of test groups. When all test groups are shown, we would first show the hidden ones after the regular ones. (AnalysisTaskPage.prototype._createTestGroupListItem): Extracted from _renderTestGroupList. (AnalysisTaskPage.prototype._renderTestGroupDetails): Update the text inside the button to hide the test group. Also show a warning text that the pending requests will be canceled if there are any. (AnalysisTaskPage.prototype._hideCurrentTestGroup): Added. (AnalysisTaskPage.cssTemplate): Updated the style. Canonical link: https://commits.webkit.org/172548@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@196794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-02-19 04:18:27 +00:00
testgroup_hidden boolean NOT NULL DEFAULT FALSE,
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
testgroup_needs_notification boolean NOT NULL DEFAULT FALSE,
testgroup_notification_sent_at timestamp DEFAULT NULL,
Add retry for test groups with failed build requests. https://bugs.webkit.org/show_bug.cgi?id=190188 Reviewed by Ryosuke Niwa. Added retry logic in run-analysis script. Current retry logic will only be triggered when there is at least one successful build request for each commit set in a test group. * init-database.sql: Added 'testgroup_initial_repetition_count' and 'testgroup_may_need_more_requests'. SQL to update existing database: ''' BEGIN; ALTER TABLE analysis_test_groups ADD COLUMN testgroup_initial_repetition_count integer DEFAULT NULL, ADD COLUMN testgroup_may_need_more_requests boolean DEFAULT FALSE; UPDATE analysis_test_groups SET testgroup_initial_repetition_count = ( SELECT DISTINCT(COUNT(*)) FROM build_requests WHERE request_group = testgroup_id AND request_order >= 0 GROUP BY request_commit_set ); ALTER TABLE analysis_test_groups ALTER COLUMN testgroup_initial_repetition_count DROP DEFAULT, ALTER COLUMN testgroup_may_need_more_requests SET NOT NULL; END; ''' 'testgroup_initial_repetition_count' represents the number of successful build request for each commit set when test group is created. 'testgroup_may_need_more_requests' will be set when any build request in test group is set to 'failed'. * public/api/build-requests.php: Added the logic to set 'testgroup_may_need_more_requests'. * public/api/test-groups.php: Updated 'ready-for-notification' to 'ready-for-further-processing' so that it returns finished test groups those either have 'needs_notification' or 'may_need_more_requests' set. * public/include/commit-sets-helpers.php: Set 'initial_repetition_count' to repetition count. * public/privileged-api/update-test-group.php: Added APIs to add build request for a test group and update 'may_need_more_requests' flag. * public/v3/models/test-group.js: (TestGroup): Added '_mayNeedMoreRequests' and '_initialRepetitionCount' field. Refactored code that interacts with '/api/update-test-group'. (TestGroup.prototype.updateSingleton): (TestGroup.prototype.mayNeedMoreRequests): (TestGroup.prototype.initialRepetitionCount): (TestGroup.prototype.async._updateBuildRequest): (TestGroup.prototype.updateName): (TestGroup.prototype.updateHiddenFlag): (TestGroup.prototype.async.didSendNotification): (TestGroup.prototype.async.addMoreBuildRequests): (TestGroup.prototype.async.clearMayNeedMoreBuildRequests): Added API to clear 'may_need_more_requests' flag. (TestGroup.fetchAllReadyForFurtherProcessing): Refactored 'TestGroup.fetchAllWithNotificationReady' to return test groups either have 'needs_notification' or 'may_need_more_requests' set. (TestGroup.fetchAllThatMayNeedMoreRequests): Fetches test groups those may need more build requests. * server-tests/api-test-groups.js: Added unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Added unit tests for 'add-build-requests' API. * server-tests/privileged-api-update-test-group-tests.js: Added unit tests. * server-tests/resources/mock-data.js: (MockData.addMockData): * server-tests/resources/test-server.js: (TestServer.prototype._determinePgsqlDirectory): Fixed a bug that 'childProcess.execFileSync' may return a buffer. * tools/run-analysis.js: Added logic to add extra build request before sennding notification. * tools/js/retry-failed-build-requests.js: (async.createAdditionalBuildRequestsForTestGroupsWithFailedRequests): Module that add extra build requests. * unit-tests/retry-failed-build-requests-tests.js: Added. * unit-tests/test-groups-tests.js: Added unit tests. Canonical link: https://commits.webkit.org/205257@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236861 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-05 00:17:49 +00:00
testgroup_initial_repetition_count integer NOT NULL,
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
testgroup_repetition_type analysis_test_group_repetition_type NOT NULL DEFAULT 'alternating',
Add retry for test groups with failed build requests. https://bugs.webkit.org/show_bug.cgi?id=190188 Reviewed by Ryosuke Niwa. Added retry logic in run-analysis script. Current retry logic will only be triggered when there is at least one successful build request for each commit set in a test group. * init-database.sql: Added 'testgroup_initial_repetition_count' and 'testgroup_may_need_more_requests'. SQL to update existing database: ''' BEGIN; ALTER TABLE analysis_test_groups ADD COLUMN testgroup_initial_repetition_count integer DEFAULT NULL, ADD COLUMN testgroup_may_need_more_requests boolean DEFAULT FALSE; UPDATE analysis_test_groups SET testgroup_initial_repetition_count = ( SELECT DISTINCT(COUNT(*)) FROM build_requests WHERE request_group = testgroup_id AND request_order >= 0 GROUP BY request_commit_set ); ALTER TABLE analysis_test_groups ALTER COLUMN testgroup_initial_repetition_count DROP DEFAULT, ALTER COLUMN testgroup_may_need_more_requests SET NOT NULL; END; ''' 'testgroup_initial_repetition_count' represents the number of successful build request for each commit set when test group is created. 'testgroup_may_need_more_requests' will be set when any build request in test group is set to 'failed'. * public/api/build-requests.php: Added the logic to set 'testgroup_may_need_more_requests'. * public/api/test-groups.php: Updated 'ready-for-notification' to 'ready-for-further-processing' so that it returns finished test groups those either have 'needs_notification' or 'may_need_more_requests' set. * public/include/commit-sets-helpers.php: Set 'initial_repetition_count' to repetition count. * public/privileged-api/update-test-group.php: Added APIs to add build request for a test group and update 'may_need_more_requests' flag. * public/v3/models/test-group.js: (TestGroup): Added '_mayNeedMoreRequests' and '_initialRepetitionCount' field. Refactored code that interacts with '/api/update-test-group'. (TestGroup.prototype.updateSingleton): (TestGroup.prototype.mayNeedMoreRequests): (TestGroup.prototype.initialRepetitionCount): (TestGroup.prototype.async._updateBuildRequest): (TestGroup.prototype.updateName): (TestGroup.prototype.updateHiddenFlag): (TestGroup.prototype.async.didSendNotification): (TestGroup.prototype.async.addMoreBuildRequests): (TestGroup.prototype.async.clearMayNeedMoreBuildRequests): Added API to clear 'may_need_more_requests' flag. (TestGroup.fetchAllReadyForFurtherProcessing): Refactored 'TestGroup.fetchAllWithNotificationReady' to return test groups either have 'needs_notification' or 'may_need_more_requests' set. (TestGroup.fetchAllThatMayNeedMoreRequests): Fetches test groups those may need more build requests. * server-tests/api-test-groups.js: Added unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Added unit tests for 'add-build-requests' API. * server-tests/privileged-api-update-test-group-tests.js: Added unit tests. * server-tests/resources/mock-data.js: (MockData.addMockData): * server-tests/resources/test-server.js: (TestServer.prototype._determinePgsqlDirectory): Fixed a bug that 'childProcess.execFileSync' may return a buffer. * tools/run-analysis.js: Added logic to add extra build request before sennding notification. * tools/js/retry-failed-build-requests.js: (async.createAdditionalBuildRequestsForTestGroupsWithFailedRequests): Module that add extra build requests. * unit-tests/retry-failed-build-requests-tests.js: Added. * unit-tests/test-groups-tests.js: Added unit tests. Canonical link: https://commits.webkit.org/205257@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236861 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-05 00:17:49 +00:00
testgroup_may_need_more_requests boolean DEFAULT FALSE,
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
CONSTRAINT testgroup_name_must_be_unique_for_each_task UNIQUE(testgroup_task, testgroup_name));
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
CREATE INDEX testgroup_task_index ON analysis_test_groups(testgroup_task);
Rename RootSet to CommitSet https://bugs.webkit.org/show_bug.cgi?id=169580 Rubber-stamped by Chris Dumez. Renamed root_sets to commit_sets and roots to commit_set_relationships in the database schema, and renamed related classes in public/v3/ and tools accordingly. RootSet, MeasurementRootSet, and CustomRootSet are respectively renamed to CommitSet, MeasurementCommitSet, and CustomCommitSet. In order to migrate the database, run: ``` BEGIN; ALTER TABLE root_sets RENAME TO commit_sets; ALTER TABLE commit_sets RENAME COLUMN rootset_id TO commitset_id; ALTER TABLE roots RENAME TO commit_set_relationships; ALTER TABLE commit_set_relationships RENAME COLUMN root_set TO commitset_set; ALTER TABLE commit_set_relationships RENAME COLUMN root_commit TO commitset_commit; ALTER TABLE build_requests RENAME COLUMN request_root_set TO request_commit_set; END; ``` * browser-tests/index.html: * init-database.sql: * public/api/build-requests.php: (main): * public/api/test-groups.php: (main): (format_test_group): * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::commit_sets): Renamed from root_sets. (BuildRequestsFetcher::commits): Renamed from roots. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Renamed from fetch_roots_for_set_if_needed. * public/privileged-api/create-test-group.php: (main): (ensure_commit_sets): Renamed from commit_sets_from_root_sets. * public/v3/components/analysis-results-viewer.js: (AnalysisResultsViewer.prototype.buildRowGroups): (AnalysisResultsViewer.prototype._collectCommitSetsInTestGroups): Renamed from _collectRootSetsInTestGroups. (AnalysisResultsViewer.prototype._buildRowsForPointsAndTestGroups): (AnalysisResultsViewer.prototype._buildRowsForPointsAndTestGroups): (AnalysisResultsViewer.CommitSetInTestGroup): Renamed from RootSetInTestGroup. (AnalysisResultsViewer.CommitSetInTestGroup.prototype.constructor): (AnalysisResultsViewer.CommitSetInTestGroup.prototype.commitSet): Renamed from rootSet. (AnalysisResultsViewer.CommitSetInTestGroup.prototype.succeedingCommitSet): Renamed from succeedingRootSet. (AnalysisResultsViewer.TestGroupStackingBlock.prototype.constructor): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.addRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.isComplete): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.startRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.endRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype._computeTestGroupStatus): * public/v3/components/chart-revision-range.js: (ChartRevisionRange.prototype._revisionForPoint): (ChartRevisionRange.prototype._computeRevisionList): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.constructor): (CustomizableTestGroupForm.prototype.setCommitSetMap): Renamed from setRootSetMap. (CustomizableTestGroupForm.prototype._submitted): (CustomizableTestGroupForm.prototype._computeCommitSetMap): Renamed from _computeRootSetMap. (CustomizableTestGroupForm.prototype.render): Renamed from render. (CustomizableTestGroupForm.prototype._constructRevisionRadioButtons): * public/v3/components/results-table.js: (ResultsTable.prototype.render): (ResultsTable.prototype._createRevisionListCells): (ResultsTable.prototype._computeRepositoryList): (ResultsTableRow.prototype.constructor): (ResultsTableRow.prototype.commitSet): Renamed from rootSet. * public/v3/components/test-group-results-table.js: (TestGroupResultsTable.prototype.buildRowGroups): * public/v3/index.html: * public/v3/models/build-request.js: (BuildRequest.prototype.constructor): (BuildRequest.prototype.updateSingleton): (BuildRequest.prototype.commitSet): Renamed from rootSet. (BuildRequest.constructBuildRequestsFromData): * public/v3/models/commit-set.js: Renamed from public/v3/models/root-set.js. (CommitSet): Renamed from RootSet. (CommitSet.containsMultipleCommitsForRepository): (MeasurementCommitSet): Renamed from MeasurementRootSet. (MeasurementCommitSet.prototype.namedStaticMap): (MeasurementCommitSet.prototype.ensureNamedStaticMap): (MeasurementCommitSet.namedStaticMap): (MeasurementCommitSet.ensureNamedStaticMap): (MeasurementCommitSet.ensureSingleton): (CustomCommitSet): Renamed from CustomRootSet. * public/v3/models/measurement-adaptor.js: (MeasurementAdaptor.prototype.applyTo): * public/v3/models/test-group.js: (TestGroup.prototype.constructor): (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.repetitionCount): (TestGroup.prototype.requestedCommitSets): Renamed from requestedRootSets. (TestGroup.prototype.requestsForCommitSet): Renamed from requestsForRootSet. (TestGroup.prototype.labelForCommitSet): Renamed from labelForRootSet. (TestGroup.prototype.didSetResult): (TestGroup.prototype.compareTestResults): (TestGroup.prototype._valuesForCommitSet): Renamed from _valuesForRootSet. (TestGroup.prototype.createAndRefetchTestGroups): * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype.render): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype._createNewTestGroupFromChart): (AnalysisTaskPage.prototype._createNewTestGroupFromViewer): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): * server-tests/api-build-requests-tests.js: * server-tests/resources/mock-data.js: (MockData.resetV3Models): (MockData.addMockData): (MockData.addAnotherMockTestGroup): * tools/detect-changes.js: (createAnalysisTaskAndNotify): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype._propertiesForBuildRequest): (BuildbotSyncer.prototype._revisionSetFromCommitSetWithExclusionList): * tools/js/database.js: (tableToPrefixMap): * tools/js/v3-models.js: * tools/sync-buildbot.js: (syncLoop): * tools/sync-with-buildbot.py: Deleted. No longer used. * unit-tests/analysis-task-tests.js: * unit-tests/build-request-tests.js: (sampleBuildRequestData): * unit-tests/buildbot-syncer-tests.js: (sampleCommitSetData): * unit-tests/measurement-adaptor-tests.js: * unit-tests/measurement-set-tests.js: * unit-tests/resources/mock-v3-models.js: (MockModels.inject): * unit-tests/test-groups-tests.js: (sampleTestGroup): Canonical link: https://commits.webkit.org/186635@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213952 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-14 23:06:40 +00:00
CREATE TABLE commit_sets (
commitset_id serial PRIMARY KEY);
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
CREATE TABLE commit_set_items (
Rename RootSet to CommitSet https://bugs.webkit.org/show_bug.cgi?id=169580 Rubber-stamped by Chris Dumez. Renamed root_sets to commit_sets and roots to commit_set_relationships in the database schema, and renamed related classes in public/v3/ and tools accordingly. RootSet, MeasurementRootSet, and CustomRootSet are respectively renamed to CommitSet, MeasurementCommitSet, and CustomCommitSet. In order to migrate the database, run: ``` BEGIN; ALTER TABLE root_sets RENAME TO commit_sets; ALTER TABLE commit_sets RENAME COLUMN rootset_id TO commitset_id; ALTER TABLE roots RENAME TO commit_set_relationships; ALTER TABLE commit_set_relationships RENAME COLUMN root_set TO commitset_set; ALTER TABLE commit_set_relationships RENAME COLUMN root_commit TO commitset_commit; ALTER TABLE build_requests RENAME COLUMN request_root_set TO request_commit_set; END; ``` * browser-tests/index.html: * init-database.sql: * public/api/build-requests.php: (main): * public/api/test-groups.php: (main): (format_test_group): * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::commit_sets): Renamed from root_sets. (BuildRequestsFetcher::commits): Renamed from roots. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Renamed from fetch_roots_for_set_if_needed. * public/privileged-api/create-test-group.php: (main): (ensure_commit_sets): Renamed from commit_sets_from_root_sets. * public/v3/components/analysis-results-viewer.js: (AnalysisResultsViewer.prototype.buildRowGroups): (AnalysisResultsViewer.prototype._collectCommitSetsInTestGroups): Renamed from _collectRootSetsInTestGroups. (AnalysisResultsViewer.prototype._buildRowsForPointsAndTestGroups): (AnalysisResultsViewer.prototype._buildRowsForPointsAndTestGroups): (AnalysisResultsViewer.CommitSetInTestGroup): Renamed from RootSetInTestGroup. (AnalysisResultsViewer.CommitSetInTestGroup.prototype.constructor): (AnalysisResultsViewer.CommitSetInTestGroup.prototype.commitSet): Renamed from rootSet. (AnalysisResultsViewer.CommitSetInTestGroup.prototype.succeedingCommitSet): Renamed from succeedingRootSet. (AnalysisResultsViewer.TestGroupStackingBlock.prototype.constructor): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.addRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.isComplete): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.startRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.endRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype._computeTestGroupStatus): * public/v3/components/chart-revision-range.js: (ChartRevisionRange.prototype._revisionForPoint): (ChartRevisionRange.prototype._computeRevisionList): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.constructor): (CustomizableTestGroupForm.prototype.setCommitSetMap): Renamed from setRootSetMap. (CustomizableTestGroupForm.prototype._submitted): (CustomizableTestGroupForm.prototype._computeCommitSetMap): Renamed from _computeRootSetMap. (CustomizableTestGroupForm.prototype.render): Renamed from render. (CustomizableTestGroupForm.prototype._constructRevisionRadioButtons): * public/v3/components/results-table.js: (ResultsTable.prototype.render): (ResultsTable.prototype._createRevisionListCells): (ResultsTable.prototype._computeRepositoryList): (ResultsTableRow.prototype.constructor): (ResultsTableRow.prototype.commitSet): Renamed from rootSet. * public/v3/components/test-group-results-table.js: (TestGroupResultsTable.prototype.buildRowGroups): * public/v3/index.html: * public/v3/models/build-request.js: (BuildRequest.prototype.constructor): (BuildRequest.prototype.updateSingleton): (BuildRequest.prototype.commitSet): Renamed from rootSet. (BuildRequest.constructBuildRequestsFromData): * public/v3/models/commit-set.js: Renamed from public/v3/models/root-set.js. (CommitSet): Renamed from RootSet. (CommitSet.containsMultipleCommitsForRepository): (MeasurementCommitSet): Renamed from MeasurementRootSet. (MeasurementCommitSet.prototype.namedStaticMap): (MeasurementCommitSet.prototype.ensureNamedStaticMap): (MeasurementCommitSet.namedStaticMap): (MeasurementCommitSet.ensureNamedStaticMap): (MeasurementCommitSet.ensureSingleton): (CustomCommitSet): Renamed from CustomRootSet. * public/v3/models/measurement-adaptor.js: (MeasurementAdaptor.prototype.applyTo): * public/v3/models/test-group.js: (TestGroup.prototype.constructor): (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.repetitionCount): (TestGroup.prototype.requestedCommitSets): Renamed from requestedRootSets. (TestGroup.prototype.requestsForCommitSet): Renamed from requestsForRootSet. (TestGroup.prototype.labelForCommitSet): Renamed from labelForRootSet. (TestGroup.prototype.didSetResult): (TestGroup.prototype.compareTestResults): (TestGroup.prototype._valuesForCommitSet): Renamed from _valuesForRootSet. (TestGroup.prototype.createAndRefetchTestGroups): * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype.render): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype._createNewTestGroupFromChart): (AnalysisTaskPage.prototype._createNewTestGroupFromViewer): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): * server-tests/api-build-requests-tests.js: * server-tests/resources/mock-data.js: (MockData.resetV3Models): (MockData.addMockData): (MockData.addAnotherMockTestGroup): * tools/detect-changes.js: (createAnalysisTaskAndNotify): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype._propertiesForBuildRequest): (BuildbotSyncer.prototype._revisionSetFromCommitSetWithExclusionList): * tools/js/database.js: (tableToPrefixMap): * tools/js/v3-models.js: * tools/sync-buildbot.js: (syncLoop): * tools/sync-with-buildbot.py: Deleted. No longer used. * unit-tests/analysis-task-tests.js: * unit-tests/build-request-tests.js: (sampleBuildRequestData): * unit-tests/buildbot-syncer-tests.js: (sampleCommitSetData): * unit-tests/measurement-adaptor-tests.js: * unit-tests/measurement-set-tests.js: * unit-tests/resources/mock-v3-models.js: (MockModels.inject): * unit-tests/test-groups-tests.js: (sampleTestGroup): Canonical link: https://commits.webkit.org/186635@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213952 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-14 23:06:40 +00:00
commitset_set integer REFERENCES commit_sets NOT NULL,
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
commitset_commit integer REFERENCES commits,
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
commitset_commit_owner integer REFERENCES commits DEFAULT NULL,
commitset_patch_file integer REFERENCES uploaded_files,
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
commitset_root_file integer REFERENCES uploaded_files,
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
commitset_requires_build boolean DEFAULT FALSE,
CONSTRAINT commitset_must_have_commit_or_root CHECK (commitset_commit IS NOT NULL OR commitset_root_file IS NOT NULL),
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
CONSTRAINT commitset_with_patch_must_have_commit CHECK (commitset_patch_file IS NULL OR commitset_commit IS NOT NULL),
CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE),
CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE));
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
Perf dashboard should let user cancel pending A/B testing and hide failed ones https://bugs.webkit.org/show_bug.cgi?id=154433 Reviewed by Chris Dumez. Added a button to hide a test group in the details view (the bottom table) in the analysis task page, and "Show hidden tests" link to show the hidden test groups on demand. When a test group is hidden, all pending requests in the group will also be canceled since a common scenario of using this feature is that the user had triggered an useless A/B testing; e.g. all builds will fail, wrong, etc... We can revisit and add the capability to just cancel the pending requests and leaving the group visible later if necessary. Run `ALTER TYPE build_request_status_type ADD VALUE 'canceled';` to add the new type. * init-database.sql: Added testgroup_hidden column to analysis_test_groups table and added 'canceled' as a value to build_request_status_type table. * public/api/test-groups.php: (format_test_group): Added 'hidden' field in the JSON result. * public/privileged-api/update-test-group.php: (main): Added the support for updating testgroup_hidden column. When this column is set to true, also cancel all pending build requests (by setting its request_status to 'canceled' which will be ignore by the syncing script). * public/v3/components/test-group-results-table.js: (TestGroupResultsTable.prototype.setTestGroup): Reset _renderedTestGroup here so that the next call to render() will update the table; e.g. when build requests' status change from 'Pending' to 'Canceled'. * public/v3/models/build-request.js: (BuildRequest.prototype.hasCompleted): A build request is considered complete/finished if it's canceled. (BuildRequest.prototype.hasPending): Added. (BuildRequest.prototype.statusLabel): Handle 'canceled' status. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): Added to update 'hidden' field. (TestGroup.prototype.isHidden): Added. (TestGroup.prototype.hasPending): Added. (TestGroup.prototype.hasPending): Added. (TestGroup.prototype.updateHiddenFlag): Added. Uses the privileged API to update testgroup_hidden column. The JSON API also updates the status of the 'pending' build requests in the group to 'canceled'. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage): Added _showHiddenTestGroups and _filteredTestGroups as instance variables. (AnalysisTaskPage.prototype._didFetchTestGroups): (AnalysisTaskPage.prototype._showAllTestGroups): Added. (AnalysisTaskPage.prototype._didUpdateTestGroupHiddenState): Extracted from _didFetchTestGroups. (AnalysisTaskPage.prototype._renderTestGroupList): Use the filtered list of test groups to show the list of test groups. When all test groups are shown, we would first show the hidden ones after the regular ones. (AnalysisTaskPage.prototype._createTestGroupListItem): Extracted from _renderTestGroupList. (AnalysisTaskPage.prototype._renderTestGroupDetails): Update the text inside the button to hide the test group. Also show a warning text that the pending requests will be canceled if there are any. (AnalysisTaskPage.prototype._hideCurrentTestGroup): Added. (AnalysisTaskPage.cssTemplate): Updated the style. Canonical link: https://commits.webkit.org/172548@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@196794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-02-19 04:18:27 +00:00
CREATE TYPE build_request_status_type as ENUM ('pending', 'scheduled', 'running', 'failed', 'completed', 'canceled');
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
CREATE TABLE build_requests (
request_id serial PRIMARY KEY,
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
request_triggerable integer REFERENCES build_triggerables NOT NULL,
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
request_repository_group integer REFERENCES triggerable_repository_groups,
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
request_platform integer REFERENCES platforms NOT NULL,
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
request_test integer REFERENCES tests,
request_group integer REFERENCES analysis_test_groups NOT NULL,
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
request_order integer NOT NULL,
Rename RootSet to CommitSet https://bugs.webkit.org/show_bug.cgi?id=169580 Rubber-stamped by Chris Dumez. Renamed root_sets to commit_sets and roots to commit_set_relationships in the database schema, and renamed related classes in public/v3/ and tools accordingly. RootSet, MeasurementRootSet, and CustomRootSet are respectively renamed to CommitSet, MeasurementCommitSet, and CustomCommitSet. In order to migrate the database, run: ``` BEGIN; ALTER TABLE root_sets RENAME TO commit_sets; ALTER TABLE commit_sets RENAME COLUMN rootset_id TO commitset_id; ALTER TABLE roots RENAME TO commit_set_relationships; ALTER TABLE commit_set_relationships RENAME COLUMN root_set TO commitset_set; ALTER TABLE commit_set_relationships RENAME COLUMN root_commit TO commitset_commit; ALTER TABLE build_requests RENAME COLUMN request_root_set TO request_commit_set; END; ``` * browser-tests/index.html: * init-database.sql: * public/api/build-requests.php: (main): * public/api/test-groups.php: (main): (format_test_group): * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::commit_sets): Renamed from root_sets. (BuildRequestsFetcher::commits): Renamed from roots. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Renamed from fetch_roots_for_set_if_needed. * public/privileged-api/create-test-group.php: (main): (ensure_commit_sets): Renamed from commit_sets_from_root_sets. * public/v3/components/analysis-results-viewer.js: (AnalysisResultsViewer.prototype.buildRowGroups): (AnalysisResultsViewer.prototype._collectCommitSetsInTestGroups): Renamed from _collectRootSetsInTestGroups. (AnalysisResultsViewer.prototype._buildRowsForPointsAndTestGroups): (AnalysisResultsViewer.prototype._buildRowsForPointsAndTestGroups): (AnalysisResultsViewer.CommitSetInTestGroup): Renamed from RootSetInTestGroup. (AnalysisResultsViewer.CommitSetInTestGroup.prototype.constructor): (AnalysisResultsViewer.CommitSetInTestGroup.prototype.commitSet): Renamed from rootSet. (AnalysisResultsViewer.CommitSetInTestGroup.prototype.succeedingCommitSet): Renamed from succeedingRootSet. (AnalysisResultsViewer.TestGroupStackingBlock.prototype.constructor): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.addRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.isComplete): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.startRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype.endRowIndex): (AnalysisResultsViewer.TestGroupStackingBlock.prototype._computeTestGroupStatus): * public/v3/components/chart-revision-range.js: (ChartRevisionRange.prototype._revisionForPoint): (ChartRevisionRange.prototype._computeRevisionList): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.constructor): (CustomizableTestGroupForm.prototype.setCommitSetMap): Renamed from setRootSetMap. (CustomizableTestGroupForm.prototype._submitted): (CustomizableTestGroupForm.prototype._computeCommitSetMap): Renamed from _computeRootSetMap. (CustomizableTestGroupForm.prototype.render): Renamed from render. (CustomizableTestGroupForm.prototype._constructRevisionRadioButtons): * public/v3/components/results-table.js: (ResultsTable.prototype.render): (ResultsTable.prototype._createRevisionListCells): (ResultsTable.prototype._computeRepositoryList): (ResultsTableRow.prototype.constructor): (ResultsTableRow.prototype.commitSet): Renamed from rootSet. * public/v3/components/test-group-results-table.js: (TestGroupResultsTable.prototype.buildRowGroups): * public/v3/index.html: * public/v3/models/build-request.js: (BuildRequest.prototype.constructor): (BuildRequest.prototype.updateSingleton): (BuildRequest.prototype.commitSet): Renamed from rootSet. (BuildRequest.constructBuildRequestsFromData): * public/v3/models/commit-set.js: Renamed from public/v3/models/root-set.js. (CommitSet): Renamed from RootSet. (CommitSet.containsMultipleCommitsForRepository): (MeasurementCommitSet): Renamed from MeasurementRootSet. (MeasurementCommitSet.prototype.namedStaticMap): (MeasurementCommitSet.prototype.ensureNamedStaticMap): (MeasurementCommitSet.namedStaticMap): (MeasurementCommitSet.ensureNamedStaticMap): (MeasurementCommitSet.ensureSingleton): (CustomCommitSet): Renamed from CustomRootSet. * public/v3/models/measurement-adaptor.js: (MeasurementAdaptor.prototype.applyTo): * public/v3/models/test-group.js: (TestGroup.prototype.constructor): (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.repetitionCount): (TestGroup.prototype.requestedCommitSets): Renamed from requestedRootSets. (TestGroup.prototype.requestsForCommitSet): Renamed from requestsForRootSet. (TestGroup.prototype.labelForCommitSet): Renamed from labelForRootSet. (TestGroup.prototype.didSetResult): (TestGroup.prototype.compareTestResults): (TestGroup.prototype._valuesForCommitSet): Renamed from _valuesForRootSet. (TestGroup.prototype.createAndRefetchTestGroups): * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype.render): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype._createNewTestGroupFromChart): (AnalysisTaskPage.prototype._createNewTestGroupFromViewer): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): * server-tests/api-build-requests-tests.js: * server-tests/resources/mock-data.js: (MockData.resetV3Models): (MockData.addMockData): (MockData.addAnotherMockTestGroup): * tools/detect-changes.js: (createAnalysisTaskAndNotify): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype._propertiesForBuildRequest): (BuildbotSyncer.prototype._revisionSetFromCommitSetWithExclusionList): * tools/js/database.js: (tableToPrefixMap): * tools/js/v3-models.js: * tools/sync-buildbot.js: (syncLoop): * tools/sync-with-buildbot.py: Deleted. No longer used. * unit-tests/analysis-task-tests.js: * unit-tests/build-request-tests.js: (sampleBuildRequestData): * unit-tests/buildbot-syncer-tests.js: (sampleCommitSetData): * unit-tests/measurement-adaptor-tests.js: * unit-tests/measurement-set-tests.js: * unit-tests/resources/mock-v3-models.js: (MockModels.inject): * unit-tests/test-groups-tests.js: (sampleTestGroup): Canonical link: https://commits.webkit.org/186635@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213952 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-14 23:06:40 +00:00
request_commit_set integer REFERENCES commit_sets NOT NULL,
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
request_status build_request_status_type NOT NULL DEFAULT 'pending',
request_url varchar(1024),
Introduce the concept of analysis task to perf dashboard https://bugs.webkit.org/show_bug.cgi?id=138517 Reviewed by Andreas Kling. Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on a single platform and used to bisect regressions in the range. Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task. * init-database.sql: Added a bunch of tables to store information about analysis tasks. analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two test runs. Analysis tasks not associated with test runs are used for try new patches. analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results. root_sets - A root set represents a set of roots (or packages) installed in each A/B testing. build_requests - A build request represents a single pending build for A/B testing. * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array. (main): (format_task): * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns the specified analysis task group as well as build requests associated with them. (main): (fetch_test_groups_for_task): (fetch_build_requests_for_task): (fetch_build_requests_for_group): (format_test_group): (format_build_request): * public/include/json-header.php: (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php. (compute_token): * public/privileged-api/associate-bug.php: (main): Fixed a typo. * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range. (main): (ensure_row_by_id): (ensure_config_from_runs): * public/privileged-api/generate-csrf-token.php: Use remote_user_name. * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects. (App.AnalysisTask): (App.AnalysisTask.create): (App.TestGroup): (App.TestGroupAdapter): (App.AnalysisTaskSerializer): (App.TestGroupSerializer): (App.BuildRequest): * public/v2/app.css: Added style rules for the analysis page. * public/v2/app.js: (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest. (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not exactly one selected point as we can still create an analysis task. (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more grammatical and also alert'ed the error message when there is one. (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab. Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it once XHR request has completed. (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename. (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point is selected in the details pane. (App.AnalysisRoute): Added. (App.AnalysisTaskRoute): Added. (App.AnalysisTaskViewModel): Added. (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric. (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show. (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects. (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when the user selects an option for all roots in A or B configuration. (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project. * public/v2/chart-pane.css: Added style rules for details view in the analysis task page. * public/v2/data.js: (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous point so that we can use it in App.AnalysisTaskViewModel.roots. (TimeSeries.prototype.findPointByMeasurementId): Added. (TimeSeries.prototype.seriesBetweenPoints): Added. * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even when multiple points are selected. Finally, added a new template for the analysis task page. * public/v2/manifest.js: (App.Metric.fullName): Added to share code between the charts page and the analysis task page. (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in App.AnalysisTaskViewModel._taskUpdated. Canonical link: https://commits.webkit.org/156348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2014-11-07 23:47:09 +00:00
request_build integer REFERENCES builds,
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
request_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
CONSTRAINT build_request_order_must_be_unique_in_group UNIQUE(request_group, request_order),
CONSTRAINT build_request_order_must_be_positive_for_testing
CHECK ((request_test IS NOT NULL AND request_order >= 0) OR (request_test IS NULL AND request_order < 0)));
CREATE INDEX build_request_triggerable ON build_requests(request_triggerable);
Perf dashboard should have the ability to post A/B testing builds https://bugs.webkit.org/show_bug.cgi?id=140317 Rubber-stamped by Simon Fraser. This patch adds the support for triggering A/B testing from the perf dashboard. We add a few new tables to the database. "build_triggerables", which represents a set of builders that accept A/B testing. "triggerable_repositories" associates each "triggerable" with a fixed set of repositories for which an arbitrary revision can be specified for A/B testing. "triggerable_configurations" specifies a triggerable available on a given test on a given platform. "roots" table which specifies the revision used in a given root set in each repository. * init-database.sql: Added "build_triggerables", "triggerable_repositories", "triggerable_configurations", and "roots" tables. Added references to "build_triggerables", "platforms", and "tests" tables as well as columns to store status, status url, and creation time to build_requests table. Also made each test group's name unique in a given analysis task as it would be confusing to have multiple test groups of the same name. * public/admin/tests.php: Added the UI and the code to associate a test with a triggerable. * public/admin/triggerables.php: Added. Manages the list of triggerables as well as repositories for which a specific revision can be set in an A/B testing on a given triggerable. * public/api/build-requests.php: Added. Returns the list of open build requests on a specified triggerable. Also updates the status' and the status urls of specified build requests when buildRequestUpdates is provided in the raw POST data. (main): * public/api/runs.php: (fetch_runs_for_config): Don't include results associated with a build request, meaning they are results of an A/B testing. * public/api/test-groups.php: (main): Use the newly added BuildRequestsFetcher. Also merged fetch_test_groups_for_task back. * public/api/triggerables.php: Added. (main): Returns a list of triggerables or a triggerable associated with a given analysis task. * public/include/admin-header.php: * public/include/build-requests-fetcher.php: Added. Extracted from public/api/test-groups.php. (BuildRequestsFetcher): This class abstracts the process of fetching a list of builds requests and root sets used in those requests.D (BuildRequestsFetcher::__construct): (BuildRequestsFetcher::fetch_for_task): (BuildRequestsFetcher::fetch_for_group): (BuildRequestsFetcher::fetch_incomplete_requests_for_triggerable): (BuildRequestsFetcher::has_results): (BuildRequestsFetcher::results): (BuildRequestsFetcher::results_with_resolved_ids): (BuildRequestsFetcher::results_internal): (BuildRequestsFetcher::root_sets): (BuildRequestsFetcher::fetch_roots_for_set): * public/include/db.php: (Database::prefixed_column_names): Don't return "$prefix_" when there are no columns. (Database::insert_row): Support taking an empty array for values. This is useful in "root_sets" table since it only has the primary key, id, column. (Database::select_or_insert_row): (Database::update_or_insert_row): (Database::update_row): Added. (Database::_select_update_or_insert_row): Takes an extra argument specifying whether a new row should be inserted when no row matches the specified criteria. This is used while updating build_requests' status and url in public/api/build-requests.php since we shouldn't be inserting new build requests in that API. (Database::select_rows): Also use "1 == 1" in the select query when the query criteria is empty. This is used in public/api/triggerables.php when no analysis task is specified. * public/include/json-header.php: (find_triggerable_for_task): Added. Finds a triggerable available on a given test. We return the triggerable associated with the closest ancestor of the test. Since issuing a new query for each ancestor test is expensive, we retrieve triggerable for all ancestor tests at once and manually find the closest ancestor with a triggerable. * public/include/report-processor.php: (ReportProcessor::process): (ReportProcessor::resolve_build_id): Associate a build request with the newly created build if jobId or buildRequest is specified. * public/include/test-name-resolver.php: (TestNameResolver::map_metrics_to_tests): Store the entire metric row instead of its name so that test_exists_on_platform can use it. The last diff in public/admin/tests.php adopts this change. (TestNameResolver::test_exists_on_platform): Added. Returns true iff the test has ever run on a given platform. * public/include/test-path-resolver.php: Added. (TestPathResolver): This class abstracts the ancestor chains of a test. It retrieves the entire "tests" table to do this since there could be arbitrary number of ancestors for a given test. This class is a lot more lightweight than TestNameResolver, which retrieves a whole bunch of tables in order to compute full test metric names. (TestPathResolver::__construct): (TestPathResolver::ancestors_for_test): Returns the ordered list of ancestors from the closest to the highest (a test without a parent). (TestPathResolver::path_for_test): Returns a test "path", the ordered list of test names from the highest ancestor to the test itself. (TestPathResolver::ensure_id_to_test_map): Fetches "tests" table to construct id_to_test_map. * public/privileged-api/create-test-group.php: Added. An API to create A/B testing groups. (main): (commit_sets_from_root_sets): Given a dictionary of repository names to a pair of revisions for sets A and B respectively, returns a pair of arrays, each of which contains the corresponding set of "commits" for sets A and B respectively. e.g. {"WebKit": [1, 2], "Safari": [3, 4]} will result in [[WebKit commit at r1, Safari commit at r3], [WebKit commit at r2, Safari commit at r4]]. * public/v2/analysis.js: (App.AnalysisTask.testGroups): Takes arguments so that set('testGroups') will invalidate the cache. (App.AnalysisTask.triggerable): Added. Retrieves the triggerable associated with the task lazily. (App.TestGroup.rootSets): Added. Returns the list of root set ids used in this A/B testing group. (App.TestGroup.create): Added. Creates a new A/B testing group. (App.Triggerable): Added. (App.TriggerableAdapter): Added. (App.TriggerableAdapter.buildURL): Added. (App.BuildRequest.testGroup): Renamed from group. (App.BuildRequest.orderLabel): Added. One-based index to be used in labels. (App.BuildRequest.config): Added. Returns either 'A' or 'B' depending on the configuration used in this build request. (App.BuildRequest.status): Added. (App.BuildRequest.statusLabel): Added. Returns a human friendly label for the current status. (App.BuildRequest): Removed buildNumber, buildBuilder, as well as buildTime as they're unused. * public/v2/app.js: (App.AnalysisTaskController.testGroups): Added. (App.AnalysisTaskController.possibleRepetitionCounts): Added. (App.AnalysisTaskController.updateRoots): Renamed from roots. This is also no longer a property but an observer that updates "roots" property. Filter out the repositories that are not accepted by the associated triggerable as they will be ignored. (App.AnalysisTaskController.actions.createTestGroup): Added. * public/v2/index.html: Updated the UI, and added a form element to trigger createTestGroup action. * tools/sync-with-buildbot.py: Added. This scripts posts new builds on buildbot and reports back the status of those builds to the perf dashboard. A similar script can be written to support other continuous builds systems. (main): Fetches the list of pending builds as well as currently running or completed builds from a buildbot, and report new statuses of builds requests to the perf dashboard. It will then schedule a single new build on each builder with no pending builds, and marks the set of open build requests that have been scheduled to run on the buildbot but not found in the first step as stale. (load_config): Loads a JSON that contains the configurations for each builder. e.g. [ { "platform": "mac-mavericks", "test": ["Parser", "html5-full-render.html"], "builder": "Trunk Syrah Production Perf AB Tests", "arguments": { "forcescheduler": "force-mac-mavericks-release-perf", "webkit_revision": "$WebKit", "jobid": "$buildRequest" } } ] (find_request_updates): Return a list of build request status updates to make based on the pending builds as well as in-progress and completed builds on each builder on the buildbot. When a build is completed, we use the special status "failedIfNotCompleted" which results in "failed" status only if the build request had not been completed. This is necessary because a failed build will not report its failed-ness back to the perf dashboard in some cases; e.g. lost slave or svn up failure. (update_and_fetch_build_requests): Submit the build request status updates and retrieve the list of open requests the perf dashboard has. (find_stale_request_updates): Compute the list of build requests that have been scheduled on the buildbot but not found in find_request_updates. These build requests are lost. e.g. a master reboot or human canceling a build may trigger such a state. (schedule_request): Schedules a build with the arguments specified in the configuration JSON after replacing repository names with their revisions and buildRequest with the build request id. (config_for_request): Finds a builder for the test and the platform of a build request. (fetch_json): Fetches a JSON from the specified URL, optionally with BasicAuth. (property_value_from_build): Returns the value of a specific property in a buildbot build. (request_id_from_build): Returns the build request id of a given buildbot build if there is one. Canonical link: https://commits.webkit.org/158312@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@178234 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-01-10 05:26:49 +00:00
CREATE INDEX build_request_build ON build_requests(request_build);