haikuwebkit/Websites/perf.webkit.org/public/privileged-api/add-build-requests.php

146 lines
6.3 KiB
PHP
Raw Permalink Normal View History

Add retry for test groups with failed build requests. https://bugs.webkit.org/show_bug.cgi?id=190188 Reviewed by Ryosuke Niwa. Added retry logic in run-analysis script. Current retry logic will only be triggered when there is at least one successful build request for each commit set in a test group. * init-database.sql: Added 'testgroup_initial_repetition_count' and 'testgroup_may_need_more_requests'. SQL to update existing database: ''' BEGIN; ALTER TABLE analysis_test_groups ADD COLUMN testgroup_initial_repetition_count integer DEFAULT NULL, ADD COLUMN testgroup_may_need_more_requests boolean DEFAULT FALSE; UPDATE analysis_test_groups SET testgroup_initial_repetition_count = ( SELECT DISTINCT(COUNT(*)) FROM build_requests WHERE request_group = testgroup_id AND request_order >= 0 GROUP BY request_commit_set ); ALTER TABLE analysis_test_groups ALTER COLUMN testgroup_initial_repetition_count DROP DEFAULT, ALTER COLUMN testgroup_may_need_more_requests SET NOT NULL; END; ''' 'testgroup_initial_repetition_count' represents the number of successful build request for each commit set when test group is created. 'testgroup_may_need_more_requests' will be set when any build request in test group is set to 'failed'. * public/api/build-requests.php: Added the logic to set 'testgroup_may_need_more_requests'. * public/api/test-groups.php: Updated 'ready-for-notification' to 'ready-for-further-processing' so that it returns finished test groups those either have 'needs_notification' or 'may_need_more_requests' set. * public/include/commit-sets-helpers.php: Set 'initial_repetition_count' to repetition count. * public/privileged-api/update-test-group.php: Added APIs to add build request for a test group and update 'may_need_more_requests' flag. * public/v3/models/test-group.js: (TestGroup): Added '_mayNeedMoreRequests' and '_initialRepetitionCount' field. Refactored code that interacts with '/api/update-test-group'. (TestGroup.prototype.updateSingleton): (TestGroup.prototype.mayNeedMoreRequests): (TestGroup.prototype.initialRepetitionCount): (TestGroup.prototype.async._updateBuildRequest): (TestGroup.prototype.updateName): (TestGroup.prototype.updateHiddenFlag): (TestGroup.prototype.async.didSendNotification): (TestGroup.prototype.async.addMoreBuildRequests): (TestGroup.prototype.async.clearMayNeedMoreBuildRequests): Added API to clear 'may_need_more_requests' flag. (TestGroup.fetchAllReadyForFurtherProcessing): Refactored 'TestGroup.fetchAllWithNotificationReady' to return test groups either have 'needs_notification' or 'may_need_more_requests' set. (TestGroup.fetchAllThatMayNeedMoreRequests): Fetches test groups those may need more build requests. * server-tests/api-test-groups.js: Added unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Added unit tests for 'add-build-requests' API. * server-tests/privileged-api-update-test-group-tests.js: Added unit tests. * server-tests/resources/mock-data.js: (MockData.addMockData): * server-tests/resources/test-server.js: (TestServer.prototype._determinePgsqlDirectory): Fixed a bug that 'childProcess.execFileSync' may return a buffer. * tools/run-analysis.js: Added logic to add extra build request before sennding notification. * tools/js/retry-failed-build-requests.js: (async.createAdditionalBuildRequestsForTestGroupsWithFailedRequests): Module that add extra build requests. * unit-tests/retry-failed-build-requests-tests.js: Added. * unit-tests/test-groups-tests.js: Added unit tests. Canonical link: https://commits.webkit.org/205257@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236861 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-05 00:17:49 +00:00
<?php
require_once('../include/json-header.php');
require_once('../include/commit-sets-helpers.php');
function main() {
$db = connect();
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
$data = ensure_privileged_api_data_and_token_or_worker($db);
Add retry for test groups with failed build requests. https://bugs.webkit.org/show_bug.cgi?id=190188 Reviewed by Ryosuke Niwa. Added retry logic in run-analysis script. Current retry logic will only be triggered when there is at least one successful build request for each commit set in a test group. * init-database.sql: Added 'testgroup_initial_repetition_count' and 'testgroup_may_need_more_requests'. SQL to update existing database: ''' BEGIN; ALTER TABLE analysis_test_groups ADD COLUMN testgroup_initial_repetition_count integer DEFAULT NULL, ADD COLUMN testgroup_may_need_more_requests boolean DEFAULT FALSE; UPDATE analysis_test_groups SET testgroup_initial_repetition_count = ( SELECT DISTINCT(COUNT(*)) FROM build_requests WHERE request_group = testgroup_id AND request_order >= 0 GROUP BY request_commit_set ); ALTER TABLE analysis_test_groups ALTER COLUMN testgroup_initial_repetition_count DROP DEFAULT, ALTER COLUMN testgroup_may_need_more_requests SET NOT NULL; END; ''' 'testgroup_initial_repetition_count' represents the number of successful build request for each commit set when test group is created. 'testgroup_may_need_more_requests' will be set when any build request in test group is set to 'failed'. * public/api/build-requests.php: Added the logic to set 'testgroup_may_need_more_requests'. * public/api/test-groups.php: Updated 'ready-for-notification' to 'ready-for-further-processing' so that it returns finished test groups those either have 'needs_notification' or 'may_need_more_requests' set. * public/include/commit-sets-helpers.php: Set 'initial_repetition_count' to repetition count. * public/privileged-api/update-test-group.php: Added APIs to add build request for a test group and update 'may_need_more_requests' flag. * public/v3/models/test-group.js: (TestGroup): Added '_mayNeedMoreRequests' and '_initialRepetitionCount' field. Refactored code that interacts with '/api/update-test-group'. (TestGroup.prototype.updateSingleton): (TestGroup.prototype.mayNeedMoreRequests): (TestGroup.prototype.initialRepetitionCount): (TestGroup.prototype.async._updateBuildRequest): (TestGroup.prototype.updateName): (TestGroup.prototype.updateHiddenFlag): (TestGroup.prototype.async.didSendNotification): (TestGroup.prototype.async.addMoreBuildRequests): (TestGroup.prototype.async.clearMayNeedMoreBuildRequests): Added API to clear 'may_need_more_requests' flag. (TestGroup.fetchAllReadyForFurtherProcessing): Refactored 'TestGroup.fetchAllWithNotificationReady' to return test groups either have 'needs_notification' or 'may_need_more_requests' set. (TestGroup.fetchAllThatMayNeedMoreRequests): Fetches test groups those may need more build requests. * server-tests/api-test-groups.js: Added unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Added unit tests for 'add-build-requests' API. * server-tests/privileged-api-update-test-group-tests.js: Added unit tests. * server-tests/resources/mock-data.js: (MockData.addMockData): * server-tests/resources/test-server.js: (TestServer.prototype._determinePgsqlDirectory): Fixed a bug that 'childProcess.execFileSync' may return a buffer. * tools/run-analysis.js: Added logic to add extra build request before sennding notification. * tools/js/retry-failed-build-requests.js: (async.createAdditionalBuildRequestsForTestGroupsWithFailedRequests): Module that add extra build requests. * unit-tests/retry-failed-build-requests-tests.js: Added. * unit-tests/test-groups-tests.js: Added unit tests. Canonical link: https://commits.webkit.org/205257@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236861 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-05 00:17:49 +00:00
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
$arguments = validate_arguments($data, array(
'addCount' => 'int',
'group' => 'int',
'commitSet' => 'int?',
));
Add retry for test groups with failed build requests. https://bugs.webkit.org/show_bug.cgi?id=190188 Reviewed by Ryosuke Niwa. Added retry logic in run-analysis script. Current retry logic will only be triggered when there is at least one successful build request for each commit set in a test group. * init-database.sql: Added 'testgroup_initial_repetition_count' and 'testgroup_may_need_more_requests'. SQL to update existing database: ''' BEGIN; ALTER TABLE analysis_test_groups ADD COLUMN testgroup_initial_repetition_count integer DEFAULT NULL, ADD COLUMN testgroup_may_need_more_requests boolean DEFAULT FALSE; UPDATE analysis_test_groups SET testgroup_initial_repetition_count = ( SELECT DISTINCT(COUNT(*)) FROM build_requests WHERE request_group = testgroup_id AND request_order >= 0 GROUP BY request_commit_set ); ALTER TABLE analysis_test_groups ALTER COLUMN testgroup_initial_repetition_count DROP DEFAULT, ALTER COLUMN testgroup_may_need_more_requests SET NOT NULL; END; ''' 'testgroup_initial_repetition_count' represents the number of successful build request for each commit set when test group is created. 'testgroup_may_need_more_requests' will be set when any build request in test group is set to 'failed'. * public/api/build-requests.php: Added the logic to set 'testgroup_may_need_more_requests'. * public/api/test-groups.php: Updated 'ready-for-notification' to 'ready-for-further-processing' so that it returns finished test groups those either have 'needs_notification' or 'may_need_more_requests' set. * public/include/commit-sets-helpers.php: Set 'initial_repetition_count' to repetition count. * public/privileged-api/update-test-group.php: Added APIs to add build request for a test group and update 'may_need_more_requests' flag. * public/v3/models/test-group.js: (TestGroup): Added '_mayNeedMoreRequests' and '_initialRepetitionCount' field. Refactored code that interacts with '/api/update-test-group'. (TestGroup.prototype.updateSingleton): (TestGroup.prototype.mayNeedMoreRequests): (TestGroup.prototype.initialRepetitionCount): (TestGroup.prototype.async._updateBuildRequest): (TestGroup.prototype.updateName): (TestGroup.prototype.updateHiddenFlag): (TestGroup.prototype.async.didSendNotification): (TestGroup.prototype.async.addMoreBuildRequests): (TestGroup.prototype.async.clearMayNeedMoreBuildRequests): Added API to clear 'may_need_more_requests' flag. (TestGroup.fetchAllReadyForFurtherProcessing): Refactored 'TestGroup.fetchAllWithNotificationReady' to return test groups either have 'needs_notification' or 'may_need_more_requests' set. (TestGroup.fetchAllThatMayNeedMoreRequests): Fetches test groups those may need more build requests. * server-tests/api-test-groups.js: Added unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Added unit tests for 'add-build-requests' API. * server-tests/privileged-api-update-test-group-tests.js: Added unit tests. * server-tests/resources/mock-data.js: (MockData.addMockData): * server-tests/resources/test-server.js: (TestServer.prototype._determinePgsqlDirectory): Fixed a bug that 'childProcess.execFileSync' may return a buffer. * tools/run-analysis.js: Added logic to add extra build request before sennding notification. * tools/js/retry-failed-build-requests.js: (async.createAdditionalBuildRequestsForTestGroupsWithFailedRequests): Module that add extra build requests. * unit-tests/retry-failed-build-requests-tests.js: Added. * unit-tests/test-groups-tests.js: Added unit tests. Canonical link: https://commits.webkit.org/205257@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236861 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-05 00:17:49 +00:00
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
$additional_build_request_count = $arguments['addCount'];
$test_group_id = $arguments['group'];
$commit_set_id = array_get($arguments, 'commitSet');
Add retry for test groups with failed build requests. https://bugs.webkit.org/show_bug.cgi?id=190188 Reviewed by Ryosuke Niwa. Added retry logic in run-analysis script. Current retry logic will only be triggered when there is at least one successful build request for each commit set in a test group. * init-database.sql: Added 'testgroup_initial_repetition_count' and 'testgroup_may_need_more_requests'. SQL to update existing database: ''' BEGIN; ALTER TABLE analysis_test_groups ADD COLUMN testgroup_initial_repetition_count integer DEFAULT NULL, ADD COLUMN testgroup_may_need_more_requests boolean DEFAULT FALSE; UPDATE analysis_test_groups SET testgroup_initial_repetition_count = ( SELECT DISTINCT(COUNT(*)) FROM build_requests WHERE request_group = testgroup_id AND request_order >= 0 GROUP BY request_commit_set ); ALTER TABLE analysis_test_groups ALTER COLUMN testgroup_initial_repetition_count DROP DEFAULT, ALTER COLUMN testgroup_may_need_more_requests SET NOT NULL; END; ''' 'testgroup_initial_repetition_count' represents the number of successful build request for each commit set when test group is created. 'testgroup_may_need_more_requests' will be set when any build request in test group is set to 'failed'. * public/api/build-requests.php: Added the logic to set 'testgroup_may_need_more_requests'. * public/api/test-groups.php: Updated 'ready-for-notification' to 'ready-for-further-processing' so that it returns finished test groups those either have 'needs_notification' or 'may_need_more_requests' set. * public/include/commit-sets-helpers.php: Set 'initial_repetition_count' to repetition count. * public/privileged-api/update-test-group.php: Added APIs to add build request for a test group and update 'may_need_more_requests' flag. * public/v3/models/test-group.js: (TestGroup): Added '_mayNeedMoreRequests' and '_initialRepetitionCount' field. Refactored code that interacts with '/api/update-test-group'. (TestGroup.prototype.updateSingleton): (TestGroup.prototype.mayNeedMoreRequests): (TestGroup.prototype.initialRepetitionCount): (TestGroup.prototype.async._updateBuildRequest): (TestGroup.prototype.updateName): (TestGroup.prototype.updateHiddenFlag): (TestGroup.prototype.async.didSendNotification): (TestGroup.prototype.async.addMoreBuildRequests): (TestGroup.prototype.async.clearMayNeedMoreBuildRequests): Added API to clear 'may_need_more_requests' flag. (TestGroup.fetchAllReadyForFurtherProcessing): Refactored 'TestGroup.fetchAllWithNotificationReady' to return test groups either have 'needs_notification' or 'may_need_more_requests' set. (TestGroup.fetchAllThatMayNeedMoreRequests): Fetches test groups those may need more build requests. * server-tests/api-test-groups.js: Added unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Added unit tests for 'add-build-requests' API. * server-tests/privileged-api-update-test-group-tests.js: Added unit tests. * server-tests/resources/mock-data.js: (MockData.addMockData): * server-tests/resources/test-server.js: (TestServer.prototype._determinePgsqlDirectory): Fixed a bug that 'childProcess.execFileSync' may return a buffer. * tools/run-analysis.js: Added logic to add extra build request before sennding notification. * tools/js/retry-failed-build-requests.js: (async.createAdditionalBuildRequestsForTestGroupsWithFailedRequests): Module that add extra build requests. * unit-tests/retry-failed-build-requests-tests.js: Added. * unit-tests/test-groups-tests.js: Added unit tests. Canonical link: https://commits.webkit.org/205257@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236861 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-05 00:17:49 +00:00
$test_group = $db->select_first_row('analysis_test_groups', 'testgroup', array('id' => $test_group_id));
if (!$test_group)
exit_with_error('InvalidTestGroup');
if (Database::is_true($test_group['testgroup_hidden']))
exit_with_error('CannotAddToHiddenTestGroup');
$existing_build_requests = $db->select_rows('build_requests', 'request', array('group' => $test_group_id), 'order');
$current_order = $existing_build_requests[count($existing_build_requests) - 1]['request_order'];
if ($current_order < 0)
exit_with_error('NoTestingBuildRequests');
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
$repetition_type = $test_group['testgroup_repetition_type'];
assert(in_array($repetition_type, ['alternating', 'sequential']));
$is_alternating_type = $repetition_type == 'alternating';
if ($is_alternating_type && $commit_set_id)
exit_with_error('CommitSetNotSupportedForAlternatingRepetitionType');
$existing_test_type_build_requests = array_filter($existing_build_requests, function($build_request) {
return $build_request['request_order'] >= 0;
});
if ($is_alternating_type)
add_alternating_build_requests($db, $existing_test_type_build_requests, $additional_build_request_count, $current_order);
else {
if ($commit_set_id)
add_sequential_build_requests_for_commit_set($db, $existing_test_type_build_requests, $additional_build_request_count, $commit_set_id);
else
add_sequential_build_requests_for_all_commit_sets($db, $existing_build_requests, $additional_build_request_count, $test_group_id);
}
exit_with_success();
}
function add_alternating_build_requests($db, $existing_build_requests, $additional_build_request_count, $order)
{
Add retry for test groups with failed build requests. https://bugs.webkit.org/show_bug.cgi?id=190188 Reviewed by Ryosuke Niwa. Added retry logic in run-analysis script. Current retry logic will only be triggered when there is at least one successful build request for each commit set in a test group. * init-database.sql: Added 'testgroup_initial_repetition_count' and 'testgroup_may_need_more_requests'. SQL to update existing database: ''' BEGIN; ALTER TABLE analysis_test_groups ADD COLUMN testgroup_initial_repetition_count integer DEFAULT NULL, ADD COLUMN testgroup_may_need_more_requests boolean DEFAULT FALSE; UPDATE analysis_test_groups SET testgroup_initial_repetition_count = ( SELECT DISTINCT(COUNT(*)) FROM build_requests WHERE request_group = testgroup_id AND request_order >= 0 GROUP BY request_commit_set ); ALTER TABLE analysis_test_groups ALTER COLUMN testgroup_initial_repetition_count DROP DEFAULT, ALTER COLUMN testgroup_may_need_more_requests SET NOT NULL; END; ''' 'testgroup_initial_repetition_count' represents the number of successful build request for each commit set when test group is created. 'testgroup_may_need_more_requests' will be set when any build request in test group is set to 'failed'. * public/api/build-requests.php: Added the logic to set 'testgroup_may_need_more_requests'. * public/api/test-groups.php: Updated 'ready-for-notification' to 'ready-for-further-processing' so that it returns finished test groups those either have 'needs_notification' or 'may_need_more_requests' set. * public/include/commit-sets-helpers.php: Set 'initial_repetition_count' to repetition count. * public/privileged-api/update-test-group.php: Added APIs to add build request for a test group and update 'may_need_more_requests' flag. * public/v3/models/test-group.js: (TestGroup): Added '_mayNeedMoreRequests' and '_initialRepetitionCount' field. Refactored code that interacts with '/api/update-test-group'. (TestGroup.prototype.updateSingleton): (TestGroup.prototype.mayNeedMoreRequests): (TestGroup.prototype.initialRepetitionCount): (TestGroup.prototype.async._updateBuildRequest): (TestGroup.prototype.updateName): (TestGroup.prototype.updateHiddenFlag): (TestGroup.prototype.async.didSendNotification): (TestGroup.prototype.async.addMoreBuildRequests): (TestGroup.prototype.async.clearMayNeedMoreBuildRequests): Added API to clear 'may_need_more_requests' flag. (TestGroup.fetchAllReadyForFurtherProcessing): Refactored 'TestGroup.fetchAllWithNotificationReady' to return test groups either have 'needs_notification' or 'may_need_more_requests' set. (TestGroup.fetchAllThatMayNeedMoreRequests): Fetches test groups those may need more build requests. * server-tests/api-test-groups.js: Added unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Added unit tests for 'add-build-requests' API. * server-tests/privileged-api-update-test-group-tests.js: Added unit tests. * server-tests/resources/mock-data.js: (MockData.addMockData): * server-tests/resources/test-server.js: (TestServer.prototype._determinePgsqlDirectory): Fixed a bug that 'childProcess.execFileSync' may return a buffer. * tools/run-analysis.js: Added logic to add extra build request before sennding notification. * tools/js/retry-failed-build-requests.js: (async.createAdditionalBuildRequestsForTestGroupsWithFailedRequests): Module that add extra build requests. * unit-tests/retry-failed-build-requests-tests.js: Added. * unit-tests/test-groups-tests.js: Added unit tests. Canonical link: https://commits.webkit.org/205257@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236861 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-05 00:17:49 +00:00
$commit_sets = array();
$build_request_by_commit_set = array();
foreach ($existing_build_requests as $build_request) {
$requested_commit_set = $build_request['request_commit_set'];
if (array_key_exists($requested_commit_set, $build_request_by_commit_set))
continue;
$build_request_by_commit_set[$requested_commit_set] = $build_request;
array_push($commit_sets, $requested_commit_set);
}
$db->begin_transaction();
for ($i = 0; $i < $additional_build_request_count; $i++) {
foreach ($commit_sets as $commit_set) {
$build_request = $build_request_by_commit_set[$commit_set];
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
duplicate_build_request_with_new_order($db, $build_request, ++$order);
Add retry for test groups with failed build requests. https://bugs.webkit.org/show_bug.cgi?id=190188 Reviewed by Ryosuke Niwa. Added retry logic in run-analysis script. Current retry logic will only be triggered when there is at least one successful build request for each commit set in a test group. * init-database.sql: Added 'testgroup_initial_repetition_count' and 'testgroup_may_need_more_requests'. SQL to update existing database: ''' BEGIN; ALTER TABLE analysis_test_groups ADD COLUMN testgroup_initial_repetition_count integer DEFAULT NULL, ADD COLUMN testgroup_may_need_more_requests boolean DEFAULT FALSE; UPDATE analysis_test_groups SET testgroup_initial_repetition_count = ( SELECT DISTINCT(COUNT(*)) FROM build_requests WHERE request_group = testgroup_id AND request_order >= 0 GROUP BY request_commit_set ); ALTER TABLE analysis_test_groups ALTER COLUMN testgroup_initial_repetition_count DROP DEFAULT, ALTER COLUMN testgroup_may_need_more_requests SET NOT NULL; END; ''' 'testgroup_initial_repetition_count' represents the number of successful build request for each commit set when test group is created. 'testgroup_may_need_more_requests' will be set when any build request in test group is set to 'failed'. * public/api/build-requests.php: Added the logic to set 'testgroup_may_need_more_requests'. * public/api/test-groups.php: Updated 'ready-for-notification' to 'ready-for-further-processing' so that it returns finished test groups those either have 'needs_notification' or 'may_need_more_requests' set. * public/include/commit-sets-helpers.php: Set 'initial_repetition_count' to repetition count. * public/privileged-api/update-test-group.php: Added APIs to add build request for a test group and update 'may_need_more_requests' flag. * public/v3/models/test-group.js: (TestGroup): Added '_mayNeedMoreRequests' and '_initialRepetitionCount' field. Refactored code that interacts with '/api/update-test-group'. (TestGroup.prototype.updateSingleton): (TestGroup.prototype.mayNeedMoreRequests): (TestGroup.prototype.initialRepetitionCount): (TestGroup.prototype.async._updateBuildRequest): (TestGroup.prototype.updateName): (TestGroup.prototype.updateHiddenFlag): (TestGroup.prototype.async.didSendNotification): (TestGroup.prototype.async.addMoreBuildRequests): (TestGroup.prototype.async.clearMayNeedMoreBuildRequests): Added API to clear 'may_need_more_requests' flag. (TestGroup.fetchAllReadyForFurtherProcessing): Refactored 'TestGroup.fetchAllWithNotificationReady' to return test groups either have 'needs_notification' or 'may_need_more_requests' set. (TestGroup.fetchAllThatMayNeedMoreRequests): Fetches test groups those may need more build requests. * server-tests/api-test-groups.js: Added unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Added unit tests for 'add-build-requests' API. * server-tests/privileged-api-update-test-group-tests.js: Added unit tests. * server-tests/resources/mock-data.js: (MockData.addMockData): * server-tests/resources/test-server.js: (TestServer.prototype._determinePgsqlDirectory): Fixed a bug that 'childProcess.execFileSync' may return a buffer. * tools/run-analysis.js: Added logic to add extra build request before sennding notification. * tools/js/retry-failed-build-requests.js: (async.createAdditionalBuildRequestsForTestGroupsWithFailedRequests): Module that add extra build requests. * unit-tests/retry-failed-build-requests-tests.js: Added. * unit-tests/test-groups-tests.js: Added unit tests. Canonical link: https://commits.webkit.org/205257@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236861 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-05 00:17:49 +00:00
}
}
$db->commit_transaction();
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
}
Add retry for test groups with failed build requests. https://bugs.webkit.org/show_bug.cgi?id=190188 Reviewed by Ryosuke Niwa. Added retry logic in run-analysis script. Current retry logic will only be triggered when there is at least one successful build request for each commit set in a test group. * init-database.sql: Added 'testgroup_initial_repetition_count' and 'testgroup_may_need_more_requests'. SQL to update existing database: ''' BEGIN; ALTER TABLE analysis_test_groups ADD COLUMN testgroup_initial_repetition_count integer DEFAULT NULL, ADD COLUMN testgroup_may_need_more_requests boolean DEFAULT FALSE; UPDATE analysis_test_groups SET testgroup_initial_repetition_count = ( SELECT DISTINCT(COUNT(*)) FROM build_requests WHERE request_group = testgroup_id AND request_order >= 0 GROUP BY request_commit_set ); ALTER TABLE analysis_test_groups ALTER COLUMN testgroup_initial_repetition_count DROP DEFAULT, ALTER COLUMN testgroup_may_need_more_requests SET NOT NULL; END; ''' 'testgroup_initial_repetition_count' represents the number of successful build request for each commit set when test group is created. 'testgroup_may_need_more_requests' will be set when any build request in test group is set to 'failed'. * public/api/build-requests.php: Added the logic to set 'testgroup_may_need_more_requests'. * public/api/test-groups.php: Updated 'ready-for-notification' to 'ready-for-further-processing' so that it returns finished test groups those either have 'needs_notification' or 'may_need_more_requests' set. * public/include/commit-sets-helpers.php: Set 'initial_repetition_count' to repetition count. * public/privileged-api/update-test-group.php: Added APIs to add build request for a test group and update 'may_need_more_requests' flag. * public/v3/models/test-group.js: (TestGroup): Added '_mayNeedMoreRequests' and '_initialRepetitionCount' field. Refactored code that interacts with '/api/update-test-group'. (TestGroup.prototype.updateSingleton): (TestGroup.prototype.mayNeedMoreRequests): (TestGroup.prototype.initialRepetitionCount): (TestGroup.prototype.async._updateBuildRequest): (TestGroup.prototype.updateName): (TestGroup.prototype.updateHiddenFlag): (TestGroup.prototype.async.didSendNotification): (TestGroup.prototype.async.addMoreBuildRequests): (TestGroup.prototype.async.clearMayNeedMoreBuildRequests): Added API to clear 'may_need_more_requests' flag. (TestGroup.fetchAllReadyForFurtherProcessing): Refactored 'TestGroup.fetchAllWithNotificationReady' to return test groups either have 'needs_notification' or 'may_need_more_requests' set. (TestGroup.fetchAllThatMayNeedMoreRequests): Fetches test groups those may need more build requests. * server-tests/api-test-groups.js: Added unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Added unit tests for 'add-build-requests' API. * server-tests/privileged-api-update-test-group-tests.js: Added unit tests. * server-tests/resources/mock-data.js: (MockData.addMockData): * server-tests/resources/test-server.js: (TestServer.prototype._determinePgsqlDirectory): Fixed a bug that 'childProcess.execFileSync' may return a buffer. * tools/run-analysis.js: Added logic to add extra build request before sennding notification. * tools/js/retry-failed-build-requests.js: (async.createAdditionalBuildRequestsForTestGroupsWithFailedRequests): Module that add extra build requests. * unit-tests/retry-failed-build-requests-tests.js: Added. * unit-tests/test-groups-tests.js: Added unit tests. Canonical link: https://commits.webkit.org/205257@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236861 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-05 00:17:49 +00:00
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
function add_sequential_build_requests_for_all_commit_sets($db, $existing_build_requests, $additional_build_request_count, $test_group_id)
{
$order_shift_by_commit_set = array();
$last_build_request_by_commit_set = array();
foreach ($existing_build_requests as $current_build_request) {
$commit_set_for_current_request = $current_build_request['request_commit_set'];
if (!array_key_exists($commit_set_for_current_request, $last_build_request_by_commit_set))
$order_shift_by_commit_set[$commit_set_for_current_request] = count($last_build_request_by_commit_set) * $additional_build_request_count;
$last_build_request_by_commit_set[$commit_set_for_current_request] = $current_build_request;
}
$db->begin_transaction();
foreach (array_reverse($order_shift_by_commit_set, true) as $commit_set => $shift) {
if (!$shift)
continue;
$db->query('UPDATE build_requests SET request_order = request_order + $1
WHERE request_commit_set = $2 AND request_group = $3',
array($shift, $commit_set, $test_group_id));
}
foreach (array_keys($last_build_request_by_commit_set) as $commit_set) {
$current_build_request = $last_build_request_by_commit_set[$commit_set];
$last_existing_order = $current_build_request['request_order'];
$order_shift = $order_shift_by_commit_set[$commit_set];
for ($i = 1, $starting_order = $last_existing_order + $order_shift; $i <= $additional_build_request_count; $i++)
duplicate_build_request_with_new_order($db, $current_build_request, $starting_order + $i);
}
$db->commit_transaction();
}
function add_sequential_build_requests_for_commit_set($db, $existing_build_requests, $additional_build_request_count, $target_commit_set_id)
{
$found_target = false;
foreach ($existing_build_requests as $current_build_request) {
if ($current_build_request['request_commit_set'] == $target_commit_set_id) {
$found_target = true;
break;
}
}
if (!$found_target)
exit_with_error('NoCommitSetInTestGroup');
$db->begin_transaction();
foreach (array_reverse($existing_build_requests, true) as $current_build_request) {
if ($current_build_request['request_commit_set'] == $target_commit_set_id) {
for ($i = 0; $i < $additional_build_request_count; $i++)
duplicate_build_request_with_new_order($db, $current_build_request, $current_build_request['request_order'] + $i + 1);
break;
} else {
$db->query('UPDATE build_requests SET request_order = request_order + $1 WHERE request_id = $2',
array($additional_build_request_count, $current_build_request['request_id']));
}
}
$db->commit_transaction();
}
function duplicate_build_request_with_new_order($db, $build_request, $order_overwrite)
{
$db->insert_row('build_requests', 'request', array(
'triggerable' => $build_request['request_triggerable'],
'repository_group' => $build_request['request_repository_group'],
'platform' => $build_request['request_platform'],
'test' => $build_request['request_test'],
'group' => $build_request['request_group'],
'order' => $order_overwrite,
'commit_set' => $build_request['request_commit_set']));
Add retry for test groups with failed build requests. https://bugs.webkit.org/show_bug.cgi?id=190188 Reviewed by Ryosuke Niwa. Added retry logic in run-analysis script. Current retry logic will only be triggered when there is at least one successful build request for each commit set in a test group. * init-database.sql: Added 'testgroup_initial_repetition_count' and 'testgroup_may_need_more_requests'. SQL to update existing database: ''' BEGIN; ALTER TABLE analysis_test_groups ADD COLUMN testgroup_initial_repetition_count integer DEFAULT NULL, ADD COLUMN testgroup_may_need_more_requests boolean DEFAULT FALSE; UPDATE analysis_test_groups SET testgroup_initial_repetition_count = ( SELECT DISTINCT(COUNT(*)) FROM build_requests WHERE request_group = testgroup_id AND request_order >= 0 GROUP BY request_commit_set ); ALTER TABLE analysis_test_groups ALTER COLUMN testgroup_initial_repetition_count DROP DEFAULT, ALTER COLUMN testgroup_may_need_more_requests SET NOT NULL; END; ''' 'testgroup_initial_repetition_count' represents the number of successful build request for each commit set when test group is created. 'testgroup_may_need_more_requests' will be set when any build request in test group is set to 'failed'. * public/api/build-requests.php: Added the logic to set 'testgroup_may_need_more_requests'. * public/api/test-groups.php: Updated 'ready-for-notification' to 'ready-for-further-processing' so that it returns finished test groups those either have 'needs_notification' or 'may_need_more_requests' set. * public/include/commit-sets-helpers.php: Set 'initial_repetition_count' to repetition count. * public/privileged-api/update-test-group.php: Added APIs to add build request for a test group and update 'may_need_more_requests' flag. * public/v3/models/test-group.js: (TestGroup): Added '_mayNeedMoreRequests' and '_initialRepetitionCount' field. Refactored code that interacts with '/api/update-test-group'. (TestGroup.prototype.updateSingleton): (TestGroup.prototype.mayNeedMoreRequests): (TestGroup.prototype.initialRepetitionCount): (TestGroup.prototype.async._updateBuildRequest): (TestGroup.prototype.updateName): (TestGroup.prototype.updateHiddenFlag): (TestGroup.prototype.async.didSendNotification): (TestGroup.prototype.async.addMoreBuildRequests): (TestGroup.prototype.async.clearMayNeedMoreBuildRequests): Added API to clear 'may_need_more_requests' flag. (TestGroup.fetchAllReadyForFurtherProcessing): Refactored 'TestGroup.fetchAllWithNotificationReady' to return test groups either have 'needs_notification' or 'may_need_more_requests' set. (TestGroup.fetchAllThatMayNeedMoreRequests): Fetches test groups those may need more build requests. * server-tests/api-test-groups.js: Added unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Added unit tests for 'add-build-requests' API. * server-tests/privileged-api-update-test-group-tests.js: Added unit tests. * server-tests/resources/mock-data.js: (MockData.addMockData): * server-tests/resources/test-server.js: (TestServer.prototype._determinePgsqlDirectory): Fixed a bug that 'childProcess.execFileSync' may return a buffer. * tools/run-analysis.js: Added logic to add extra build request before sennding notification. * tools/js/retry-failed-build-requests.js: (async.createAdditionalBuildRequestsForTestGroupsWithFailedRequests): Module that add extra build requests. * unit-tests/retry-failed-build-requests-tests.js: Added. * unit-tests/test-groups-tests.js: Added unit tests. Canonical link: https://commits.webkit.org/205257@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@236861 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-05 00:17:49 +00:00
}
main();
?>