haikuwebkit/Websites/perf.webkit.org/server-tests/privileged-api-create-test-...

1492 lines
85 KiB
JavaScript
Raw Permalink Normal View History

'use strict';
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
const assert = require('assert');
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
const MockData = require('./resources/mock-data.js');
const TestServer = require('./resources/test-server.js');
const TemporaryFile = require('./resources/temporary-file.js').TemporaryFile;
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
const addWorkerForReport = require('./resources/common-operations.js').addWorkerForReport;
const prepareServerTest = require('./resources/common-operations.js').prepareServerTest;
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
const assertThrows = require('./resources/common-operations.js').assertThrows;
function createAnalysisTask(name, webkitRevisions = ["191622", "191623"])
{
const reportWithRevision = [{
Refactor "builds" table "build_number" row to "build_tag" to fit more generic use cases. https://bugs.webkit.org/show_bug.cgi?id=203270 Reviewed by Ryosuke Niwa. Rename "build_number" to "build_tag" to fit more generic use cases. Add support to be backward compatible to accept reports contain either "buildNumber" or "buildTag". * browser-tests/index.html: * init-database.sql: * migrate-database.sql: Updated migration sql for this change. * public/admin/test-configurations.php: * public/api/measurement-set.php: * public/api/runs.php: * public/api/upload-root.php: Added support for both "buildNumber" and "buildTag". * public/include/report-processor.php: Added support for both "buildNumber" and "buildTag". * public/v3/models/builder.js: (Builder.prototype.urlForBuild): (Builder): (Build): (Build.prototype.buildTag): (Build.prototype.label): (Build.prototype.url): (Build.prototype.buildNumber): Deleted. * public/v3/models/measurement-adaptor.js: (MeasurementAdaptor): (MeasurementAdaptor.prototype.applyTo.return.build): * public/v3/pages/chart-pane-status-view.js: * public/v3/pages/test-freshness-page.js: (TestFreshnessPage.prototype._renderTooltip): * server-tests/admin-platforms-tests.js: (reportsForDifferentPlatforms): * server-tests/admin-reprocess-report-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-manifest-tests.js: * server-tests/api-measurement-set-tests.js: * server-tests/api-report-tests.js: Added unit tests for testing "buildNumber" and "buildTag". (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptySlaveReport): (makeReport): (reportWithSameSubtestName): * server-tests/api-upload-root-tests.js: Added unit tests for testing "buildNumber" and "buildTag". * server-tests/privileged-api-add-build-requests-tests.js: * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async.createAnalysisTask): * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.sampleBuildData): (MockData.finishedBuildData): * server-tests/tools-sync-buildbot-integration-tests.js: * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildTag): (BuildbotBuildEntry.prototype.url): (BuildbotSyncer.prototype.urlForBuildTag): (BuildbotBuildEntry.prototype.buildNumber): Deleted. (BuildbotSyncer.prototype.urlForBuildNumber): Deleted. * unit-tests/analysis-task-tests.js: (measurementCluster): * unit-tests/buildbot-syncer-tests.js: * unit-tests/measurement-adaptor-tests.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/measurement-set-tests.js: * unit-tests/resources/mock-v3-models.js: Canonical link: https://commits.webkit.org/216795@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@251564 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-10-24 22:00:37 +00:00
"buildTag": "124",
"buildTime": "2015-10-27T15:34:51",
"revisions": {
"WebKit": {
"revision": webkitRevisions[0],
"timestamp": '2015-10-27T11:36:56.878473Z',
},
"macOS": {
"revision": "15A284",
}
},
"builderName": "someBuilder",
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
"workerName": "someWorker",
"workerPassword": "somePassword",
"platform": "some platform",
"tests": {
"some test": {
"metrics": {
"Time": ["Arithmetic"],
},
"tests": {
"test1": {
"metrics": {"Time": { "current": [11] }},
}
}
},
}}];
const anotherReportWithRevision = [{
Refactor "builds" table "build_number" row to "build_tag" to fit more generic use cases. https://bugs.webkit.org/show_bug.cgi?id=203270 Reviewed by Ryosuke Niwa. Rename "build_number" to "build_tag" to fit more generic use cases. Add support to be backward compatible to accept reports contain either "buildNumber" or "buildTag". * browser-tests/index.html: * init-database.sql: * migrate-database.sql: Updated migration sql for this change. * public/admin/test-configurations.php: * public/api/measurement-set.php: * public/api/runs.php: * public/api/upload-root.php: Added support for both "buildNumber" and "buildTag". * public/include/report-processor.php: Added support for both "buildNumber" and "buildTag". * public/v3/models/builder.js: (Builder.prototype.urlForBuild): (Builder): (Build): (Build.prototype.buildTag): (Build.prototype.label): (Build.prototype.url): (Build.prototype.buildNumber): Deleted. * public/v3/models/measurement-adaptor.js: (MeasurementAdaptor): (MeasurementAdaptor.prototype.applyTo.return.build): * public/v3/pages/chart-pane-status-view.js: * public/v3/pages/test-freshness-page.js: (TestFreshnessPage.prototype._renderTooltip): * server-tests/admin-platforms-tests.js: (reportsForDifferentPlatforms): * server-tests/admin-reprocess-report-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-manifest-tests.js: * server-tests/api-measurement-set-tests.js: * server-tests/api-report-tests.js: Added unit tests for testing "buildNumber" and "buildTag". (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptySlaveReport): (makeReport): (reportWithSameSubtestName): * server-tests/api-upload-root-tests.js: Added unit tests for testing "buildNumber" and "buildTag". * server-tests/privileged-api-add-build-requests-tests.js: * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async.createAnalysisTask): * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.sampleBuildData): (MockData.finishedBuildData): * server-tests/tools-sync-buildbot-integration-tests.js: * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildTag): (BuildbotBuildEntry.prototype.url): (BuildbotSyncer.prototype.urlForBuildTag): (BuildbotBuildEntry.prototype.buildNumber): Deleted. (BuildbotSyncer.prototype.urlForBuildNumber): Deleted. * unit-tests/analysis-task-tests.js: (measurementCluster): * unit-tests/buildbot-syncer-tests.js: * unit-tests/measurement-adaptor-tests.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/measurement-set-tests.js: * unit-tests/resources/mock-v3-models.js: Canonical link: https://commits.webkit.org/216795@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@251564 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2019-10-24 22:00:37 +00:00
"buildTag": "125",
"buildTime": "2015-10-27T17:27:41",
"revisions": {
"WebKit": {
"revision": webkitRevisions[1],
"timestamp": '2015-10-27T16:38:10.768995Z',
},
"macOS": {
"revision": "15A284",
}
},
"builderName": "someBuilder",
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
"workerName": "someWorker",
"workerPassword": "somePassword",
"platform": "some platform",
"tests": {
"some test": {
"metrics": {
"Time": ["Arithmetic"],
},
"tests": {
"test1": {
"metrics": {"Time": { "current": [12] }},
}
}
},
}}];
const db = TestServer.database();
const remote = TestServer.remoteAPI();
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
return addWorkerForReport(reportWithRevision[0]).then(() => {
return remote.postJSON('/api/report/', reportWithRevision);
}).then(() => {
return remote.postJSON('/api/report/', anotherReportWithRevision);
}).then((result) => {
return Manifest.fetch();
}).then(() => {
const test = Test.findByPath(['some test', 'test1']);
const platform = Platform.findByName('some platform');
return db.selectFirstRow('test_configurations', {metric: test.metrics()[0].id(), platform: platform.id()});
}).then((configRow) => {
return db.selectRows('test_runs', {config: configRow['id']});
}).then((testRuns) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testRuns.length, 2);
return PrivilegedAPI.sendRequest('create-analysis-task', {
name: name,
startRun: testRuns[0]['id'],
endRun: testRuns[1]['id'],
});
}).then((content) => content['taskId']);
}
function addTriggerableAndCreateTask(name, webkitRevisions)
{
const report = {
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
'workerName': 'anotherWorker',
'workerPassword': 'anotherPassword',
'triggerable': 'build-webkit',
'configurations': [
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
{test: MockData.someTestId(), platform: MockData.somePlatformId()},
{test: MockData.someTestId(), platform: MockData.otherPlatformId()},
],
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
'repositoryGroups': [
{name: 'os-only', acceptsRoot: true, repositories: [
{repository: MockData.macosRepositoryId(), acceptsPatch: false},
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
]},
{name: 'webkit-only', acceptsRoot: true, repositories: [
{repository: MockData.webkitRepositoryId(), acceptsPatch: true},
]},
{name: 'system-and-webkit', acceptsRoot: true, repositories: [
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
{repository: MockData.macosRepositoryId(), acceptsPatch: false},
{repository: MockData.webkitRepositoryId(), acceptsPatch: true}
]},
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
{name: 'system-webkit-sjc', acceptsRoot: true, repositories: [
{repository: MockData.macosRepositoryId(), acceptsPatch: false},
{repository: MockData.jscRepositoryId(), acceptsPatch: false},
{repository: MockData.webkitRepositoryId(), acceptsPatch: true}
]},
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
]
};
return MockData.addMockData(TestServer.database()).then(() => {
[perf dashboard] Remove non-inclusive words from perf dashboard. https://bugs.webkit.org/show_bug.cgi?id=223505 Reviewed by Ryosuke Niwa. Removed most of non-inclusive terminology and replaced it with 'worker'. Make impacted APIs backward compatible during transition. The non-inclusive terminology will be removed after transition. * ReadMe.md: Removed non-inclusive words. * init-database.sql: Rename tables with non-inclusive words. * migrate-database.sql: Added migration SQL for existing database. * public/admin/build-workers.php: Renamed from Websites/perf.webkit.org/public/admin/build-slaves.php. * public/api/build-requests.php: * public/api/measurement-set.php: * public/api/report-commits.php: * public/api/runs.php: * public/api/update-triggerable.php: * public/api/upload-root.php: * public/include/admin-header.php: * public/include/json-header.php: * public/include/report-processor.php: * public/privileged-api/add-build-requests.php: * public/privileged-api/create-analysis-task.php: * public/privileged-api/create-test-group.php: * public/privileged-api/update-test-group.php: * public/v2/js/ember.js: * server-tests/api-build-requests-tests.js: * server-tests/api-commits-tests.js: * server-tests/api-report-commits-tests.js: * server-tests/api-report-tests.js: (emptyReport): (reportWitMismatchingCommitTime): (reportWithOneSecondCommitTimeDifference): (emptyWorkerReport): (emptySlaveReport): Deleted. * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (updateWithOSXRepositoryGroup): (updateWithMacWebKitRepositoryGroups): * server-tests/api-upload-root-tests.js: (makeReport): (addWorkerAndCreateRootFile): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): (async addTriggerableAndCreateTask): * server-tests/resources/common-operations.js: * server-tests/resources/mock-data.js: (MockData.addMockConfiguration): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/resources/test-server.js: (TestServer.prototype.testConfig): * server-tests/tools-buildbot-triggerable-tests.js: * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (createTriggerable): * tools/detect-changes.js: (loadServerConfig): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.buildTag): (BuildbotSyncer): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): (BuildbotSyncer._loadConfig): (BuildbotSyncer._validateAndMergeConfig): (BuildbotBuildEntry.prototype.slaveName): Deleted. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.updateTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._scheduleRequestIfWorkerIsAvailable): (BuildbotTriggerable.prototype._scheduleRequestWithLog): (BuildbotTriggerable._testGroupMapForBuildRequests): (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Deleted. * tools/js/database.js: * tools/js/os-build-fetcher.js: (prototype.async _reportCommits): * tools/js/privileged-api.js: (NodePrivilegedAPI.prototype.sendRequest): (NodePrivilegedAPI.configure): (NodePrivilegedAPI): * tools/pull-os-versions.py: (OSBuildFetcher.fetch_and_report_new_builds): * tools/run-analysis.js: (async analysisLoop): * tools/sync-buildbot.js: (syncLoop.const.makeTriggerable): (syncLoop): * tools/sync-commits.py: (Repository.fetch_commits_and_submit): * tools/sync-os-versions.js: (syncLoop): * tools/util.py: (submit_commits): * unit-tests/analysis-task-tests.js: * unit-tests/buildbot-syncer-tests.js: (sampleiOSConfig): (sampleBuildData): (async const): * unit-tests/checkconfig.js: * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/privileged-api-tests.js: * unit-tests/retry-failed-build-requests-tests.js: * unit-tests/test-groups-tests.js: Canonical link: https://commits.webkit.org/235601@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274800 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-22 21:23:38 +00:00
return addWorkerForReport(report);
}).then(() => {
return TestServer.remoteAPI().postJSON('/api/update-triggerable/', report);
}).then(() => {
return createAnalysisTask(name, webkitRevisions);
});
}
describe('/privileged-api/create-test-group', function () {
prepareServerTest(this);
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
TemporaryFile.inject();
it('should return "InvalidName" on an empty request', () => {
return PrivilegedAPI.sendRequest('create-test-group', {}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidName');
});
});
it('should return "InvalidTask" when task is not specified', () => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', commitSets: [[1]]}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidTask');
});
});
it('should return "InvalidTask" when task is not a valid integer', () => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 'foo', commitSets: [[1]]}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidTask');
});
});
it('should return "InvalidCommitSets" when commit sets are not specified', () => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, repetitionCount: 1}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidCommitSets');
});
});
it('should return "InvalidCommitSets" when commit sets is empty', () => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, repetitionCount: 1, commitSets: {}}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidCommitSets');
});
});
it('should return "InvalidTask" when there is no matching task', () => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, repetitionCount: 1, commitSets: {'WebKit': []}}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidTask');
});
});
it('should return "InvalidRepetitionCount" when repetitionCount is not a valid integer', () => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, repetitionCount: 'foo', commitSets: {'WebKit': []}}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidRepetitionCount');
});
});
it('should return "InvalidRepetitionCount" when repetitionCount is a negative integer', () => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, repetitionCount: -5, commitSets: {'WebKit': []}}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidRepetitionCount');
});
});
it('should return "InvalidTask" when there is no matching task', () => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, commitSets: {'WebKit': []}}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidTask');
});
});
it('should return "TriggerableNotFoundForTask" when there is no matching triggerable', () => {
return createAnalysisTask('some task').then((taskId) => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'WebKit': []}}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'TriggerableNotFoundForTask');
});
});
});
it('should return "InvalidCommitSets" when each repository specifies zero revisions', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'WebKit': []}}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidCommitSets');
});
});
});
it('should return "InvalidRevisionSets" when a revision set is empty', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
const webkit = Repository.all().find((repository) => repository.name() == 'WebKit');
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, revisionSets: [{[webkit.id()]: {revision: '191622'}}, {}]}).then((content) => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidRevisionSets');
});
});
});
it('should return "InvalidRevisionSets" when the number of revision sets is less than two', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
const webkit = Repository.all().find((repository) => repository.name() == 'WebKit');
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, revisionSets: [{[webkit.id()]: {revision: '191622'}}]}).then((content) => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidRevisionSets');
});
});
});
it('should return "RepositoryNotFound" when commit sets contains an invalid repository', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'Foo': []}}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'RepositoryNotFound');
});
});
});
it('should return "RevisionNotFound" when commit sets contains an invalid revision', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'WebKit': ['1']}}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'RevisionNotFound');
});
});
});
it('should return "RevisionNotFound" when revision sets contains an invalid revision', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
const webkit = Repository.all().find((repository) => repository.name() == 'WebKit');
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '1a'}}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, revisionSets}).then((content) => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'RevisionNotFound');
});
});
});
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
it('should return "AmbiguousRevision" when there are multiple commits that match the specified revision string', () => {
return addTriggerableAndCreateTask('some task', ['2ceda45d3cd63cde58d0dbf5767714e03d902e43', '2c71a8ddc1f661663ccfd1a29c633ba57e879533']).then((taskId) => {
const webkit = Repository.all().find((repository) => repository.name() == 'WebKit');
const revisionSets = [{[webkit.id()]: {revision: '2ceda'}}, {[webkit.id()]: {revision: '2c'}}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, revisionSets}).then((content) => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'AmbiguousRevision');
});
});
});
it('should return "RevisionNotFound" when the end of a Git hash is specified', () => {
return addTriggerableAndCreateTask('some task', ['2ceda45d3cd63cde58d0dbf5767714e03d902e43', '5471a8ddc1f661663ccfd1a29c633ba57e879533']).then((taskId) => {
const webkit = Repository.all().find((repository) => repository.name() == 'WebKit');
const revisionSets = [{[webkit.id()]: {revision: '2ceda45d3cd63cde58d0dbf5767714e03d902e43'}}, {[webkit.id()]: {revision: '57e879533'}}];
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, revisionSets}).then((content) => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'RevisionNotFound');
});
});
});
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
it('should return "InvalidUploadedFile" when revision sets contains an invalid file ID', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
const webkit = Repository.all().find((repository) => repository.name() == 'WebKit');
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
const revisionSets = [{[webkit.id()]: {revision: '191622'}, 'customRoots': ['1']}, {[webkit.id()]: {revision: '1'}}];
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, revisionSets}).then((content) => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidUploadedFile');
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
});
});
});
it('should return "InvalidRepository" when a revision set uses a repository name instead of a repository id', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
const revisionSets = [{'WebKit': {revision: '191622'}}, {}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, revisionSets}).then((content) => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidRepository');
});
});
});
it('should return "InvalidCommitSets" when commit sets contains an inconsistent number of revisions', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'WebKit': ['191622', '191623'], 'macOS': ['15A284']}}).then((content) => {
assert(false, 'should never be reached');
In-browser and node.js implementations of RemoteAPI should share some code https://bugs.webkit.org/show_bug.cgi?id=169695 Rubber-stamped by Antti Koivisto. Extracted CommonRemoteAPI out of RemoteAPI implementations for node.js and browser. * public/shared/common-remote.js: Added. (CommonRemoteAPI): Added. (CommonRemoteAPI.prototype.postJSON): Extracted from RemoteAPI. (CommonRemoteAPI.prototype.postJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.getJSON): Ditto. (CommonRemoteAPI.prototype.getJSONWithStatus): Ditto. (CommonRemoteAPI.prototype.sendHttpRequest): Added. Needs to implemented by a subclass. (CommonRemoteAPI.prototype._asJSON): Added. (CommonRemoteAPI.prototype._checkStatus): Added. * public/v3/index.html: Include common-remote.js. * public/v3/privileged-api.js: (PrivilegedAPI): Use class now that we don't include data.js. (PrivilegedAPI.sendRequest): Modernized the code. (PrivilegedAPI.requestCSRFToken): Ditto. * public/v3/remote.js: (BrowserRemoteAPI): Renamed from RemoteAPI. window.RemoteAPI is now an instance of this class. (BrowserRemoteAPI.prototype.sendHttpRequest): Moved from RemoteAPI.sendHttpRequest. (BrowserRemoteAPI.prototype.sendHttpRequest): * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests since NodeJSRemoteAPI now throws the JSON status as an error to be consistent with BrowserRemoteAPI. * server-tests/privileged-api-create-test-group-tests.js: Ditto. * server-tests/privileged-api-upate-run-status.js: Ditto. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.syncOnce): Just use postJSONWithStatus instead of manually checking the status. * tools/js/remote.js: (NodeRemoteAPI): Renamed from RemoteAPI. Still exported as RemoteAPI. (NodeRemoteAPI.prototype.constructor): (NodeRemoteAPI.prototype.sendHttpRequest): Modernized the code. Canonical link: https://commits.webkit.org/186683@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@214008 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-03-15 21:59:12 +00:00
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidCommitSets');
});
});
});
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
it('should return "DuplicateTestGroupName" when there is already a test group of the same name', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
const commitSets = {'WebKit': ['191622', '191623']};
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets}).then((content) => {
assert(content['testGroupId']);
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets});
}).then(() => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'DuplicateTestGroupName');
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
});
});
});
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
it('should return "InvalidOwnerRevision" when commit ownership is not valid', () => {
let taskId;
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
const ownedJSC = Repository.all().filter((repository) => repository.name() == 'JavaScriptCore' && repository.ownerId())[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, [ownedJSC.id()]: {revision: 'owned-jsc-6161', ownerRevision: '191621'}},
{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, [ownedJSC.id()]: {revision: 'owned-jsc-9191', ownerRevision: '191622'}}];
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2, revisionSets});
}).then(() => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidOwnerRevision');
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
});
});
it('should return "InvalidCommitOwnership" when commit ownership is not valid', () => {
let taskId;
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
const ownedJSC = Repository.all().filter((repository) => repository.name() == 'JavaScriptCore' && repository.ownerId())[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, [ownedJSC.id()]: {revision: 'owned-jsc-6161', ownerRevision: '191622'}},
{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, [ownedJSC.id()]: {revision: 'owned-jsc-9191', ownerRevision: '191622'}}];
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2, revisionSets});
}).then(() => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'InvalidCommitOwnership');
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
});
});
it('should allow a commit set in which owner of a commit is not in the same set', () => {
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
let taskId;
let groupId;
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
const ownedJSC = Repository.all().filter((repository) => repository.name() == 'JavaScriptCore' && repository.ownerId())[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, [ownedJSC.id()]: {revision: 'owned-jsc-6161', ownerRevision: '191622'}},
{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, [ownedJSC.id()]: {revision: 'owned-jsc-9191', ownerRevision: '192736'}}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 1, revisionSets});
}).then((content) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(content['status'], 'OK');
groupId = content['testGroupId'];
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), groupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 1);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 4);
assert(requests[0].isBuild());
assert(!requests[0].isTest());
assert(requests[1].isBuild());
assert(!requests[1].isTest());
assert(!requests[2].isBuild());
assert(requests[2].isTest());
assert(!requests[3].isBuild());
assert(requests[3].isTest());
const macos = Repository.findById(MockData.macosRepositoryId());
const webkit = Repository.findById(MockData.webkitRepositoryId());
const ownedJSC = Repository.findById(MockData.ownedJSCRepositoryId());
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [ownedJSC, webkit, macos]);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [ownedJSC, webkit, macos]);
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set0.revisionForRepository(ownedJSC), 'owned-jsc-6161');
assert.strictEqual(set1.revisionForRepository(macos), '15A284');
assert.strictEqual(set1.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(ownedJSC), 'owned-jsc-9191');
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
});
});
it('should allow a commit set in which repository of owner commit is not specified at all', () => {
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
let taskId;
let groupId;
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
const jsc = Repository.all().filter((repository) => repository.name() == 'JavaScriptCore' && repository.ownerId())[0];
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const revisionSets = [{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, [jsc.id()]: {revision: 'owned-jsc-6161', ownerRevision: '191622'}},
{[macos.id()]: {revision: '15A284'}, [jsc.id()]: {revision: 'owned-jsc-9191', ownerRevision: '192736'}}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 1, revisionSets});
}).then((content) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(content['status'], 'OK');
groupId = content['testGroupId'];
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), groupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 1);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 4);
assert(requests[0].isBuild());
assert(!requests[0].isTest());
assert(requests[1].isBuild());
assert(!requests[1].isTest());
assert(!requests[2].isBuild());
assert(requests[2].isTest());
assert(!requests[3].isBuild());
assert(requests[3].isTest());
const macos = Repository.findById(MockData.macosRepositoryId());
const webkit = Repository.findById(MockData.webkitRepositoryId());
const ownedJSC = Repository.findById(MockData.ownedJSCRepositoryId());
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [ownedJSC, webkit, macos]);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [ownedJSC, macos]);
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set0.revisionForRepository(ownedJSC), 'owned-jsc-6161');
assert.strictEqual(set1.revisionForRepository(macos), '15A284');
assert.strictEqual(set1.revisionForRepository(ownedJSC), 'owned-jsc-9191');
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
});
});
it('should create a test group from commitSets with the repetition count of one when repetitionCount is omitted', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
let insertedGroupId;
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'macOS': ['15A284', '15A284'], 'WebKit': ['191622', '191623']}}).then((content) => {
insertedGroupId = content['testGroupId'];
Add API to upload a patched build for a custom A/B testing https://bugs.webkit.org/show_bug.cgi?id=171956 Reviewed by Chris Dumez. Added /api/upload-root to upload a root file, the build product of a patch associated with a commit set. Extracted more functions out of privileged-api/upload-file.php to uploaded-file-helpers.php to share code with /api/upload-root.php. * public/api/upload-root.php: Added. (main): (compute_commit_set_items_to_update): Find the list of commit set items to associate this root with. A root can be associated with multiple repositories and there fore commit set items; e.g. if a software is built from multiple repositories and there is a patch associated with one of them, the built product must be associated with all those repositories. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Include the root file is there is one. * public/include/json-header.php: (validate_arguments): Added the support for validating json string. (verify_slave): Return the slave ID found. * public/include/uploaded-file-helpers.php: (validate_uploaded_file): Extracted from /privileged-api/upload-file to be shared with /api/upload-root. (query_total_file_size): Ditto. (create_uploaded_file_from_form_data): Ditto. (upload_file_in_transaction): Ditto. Takes a lambda to do the extra work inside the transaction. * public/privileged-api/upload-file.php: (main): * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Resolve the rootFIle of each commit set item. * public/v3/models/commit-set.js: (CommitSet): Added _repositoryToRootMap and _allRootFiles as instance variables. (CommitSet.prototype.updateSingleton): Added. Previously, each commit set's states never changed after its creation. After this patch, each item can be newly associated with a root so we must update its _repositoryToRootMap and _allRootFiles. For simplicity, we update all states. (CommitSet.prototype._updateFromObject): Extracted from the constructor. (CommitSet.prototype.allRootFiles): Added. Includes custom roots and roots created for patches. (CommitSet.prototype.rootForRepository): Added. (CommitSet.prototype.equals): Fixed the bug that we were comparing _repositoryToPatchMap to _repositoryToCommitMap, and added a check for _repositoryToRootMap. * public/v3/models/test-group.js: (TestGroup.prototype.task): Added. (TestGroup.createWithTask): (TestGroup.createWithCustomConfiguration): (TestGroup.createAndRefetchTestGroups): (TestGroup._fetchTestGroupsForTask): Deleted. Now fetchForTask takes a boolean argument: ignoreCache. (TestGroup.findAllByTask): Added. (TestGroup.fetchForTask): Renamed from fetchByTask. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._fetchRelatedInfoForTaskId): * server-tests/api-build-requests-tests.js: * server-tests/api-upload-root-tests.js: Added. Added tests for /api/upload-root. (makeReport): Added. (addSlaveAndCreateRootFile): Added. (createTestGroupWihPatch): Added. * server-tests/privileged-api-create-test-group-tests.js: * server-tests/resources/mock-data.js: (MockData.sharedRepositoryId): Added. (MockData.addMockData): Added "Shared" repository along with commits. Canonical link: https://commits.webkit.org/188885@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@216647 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-05-11 02:32:00 +00:00
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 1);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 2);
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const macos = Repository.findById(MockData.macosRepositoryId());
const webkit = Repository.findById(MockData.webkitRepositoryId());
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [webkit, macos]);
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(macos), '15A284');
assert.strictEqual(set1.revisionForRepository(webkit), '191623');
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const repositoryGroup = requests[0].repositoryGroup();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(repositoryGroup.name(), 'system-and-webkit');
assert.strictEqual(requests[1].repositoryGroup(), repositoryGroup);
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
assert(repositoryGroup.accepts(set0));
assert(repositoryGroup.accepts(set1));
});
});
});
it('should create a test group from revisionSets with the repetition count of one when repetitionCount is omitted', () => {
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
let webkit;
return addTriggerableAndCreateTask('some task').then((taskId) => {
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const webkit = Repository.findById(MockData.webkitRepositoryId());
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
const params = {name: 'test', task: taskId, revisionSets};
let insertedGroupId;
return PrivilegedAPI.sendRequest('create-test-group', params).then((content) => {
insertedGroupId = content['testGroupId'];
Add API to upload a patched build for a custom A/B testing https://bugs.webkit.org/show_bug.cgi?id=171956 Reviewed by Chris Dumez. Added /api/upload-root to upload a root file, the build product of a patch associated with a commit set. Extracted more functions out of privileged-api/upload-file.php to uploaded-file-helpers.php to share code with /api/upload-root.php. * public/api/upload-root.php: Added. (main): (compute_commit_set_items_to_update): Find the list of commit set items to associate this root with. A root can be associated with multiple repositories and there fore commit set items; e.g. if a software is built from multiple repositories and there is a patch associated with one of them, the built product must be associated with all those repositories. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Include the root file is there is one. * public/include/json-header.php: (validate_arguments): Added the support for validating json string. (verify_slave): Return the slave ID found. * public/include/uploaded-file-helpers.php: (validate_uploaded_file): Extracted from /privileged-api/upload-file to be shared with /api/upload-root. (query_total_file_size): Ditto. (create_uploaded_file_from_form_data): Ditto. (upload_file_in_transaction): Ditto. Takes a lambda to do the extra work inside the transaction. * public/privileged-api/upload-file.php: (main): * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Resolve the rootFIle of each commit set item. * public/v3/models/commit-set.js: (CommitSet): Added _repositoryToRootMap and _allRootFiles as instance variables. (CommitSet.prototype.updateSingleton): Added. Previously, each commit set's states never changed after its creation. After this patch, each item can be newly associated with a root so we must update its _repositoryToRootMap and _allRootFiles. For simplicity, we update all states. (CommitSet.prototype._updateFromObject): Extracted from the constructor. (CommitSet.prototype.allRootFiles): Added. Includes custom roots and roots created for patches. (CommitSet.prototype.rootForRepository): Added. (CommitSet.prototype.equals): Fixed the bug that we were comparing _repositoryToPatchMap to _repositoryToCommitMap, and added a check for _repositoryToRootMap. * public/v3/models/test-group.js: (TestGroup.prototype.task): Added. (TestGroup.createWithTask): (TestGroup.createWithCustomConfiguration): (TestGroup.createAndRefetchTestGroups): (TestGroup._fetchTestGroupsForTask): Deleted. Now fetchForTask takes a boolean argument: ignoreCache. (TestGroup.findAllByTask): Added. (TestGroup.fetchForTask): Renamed from fetchByTask. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._fetchRelatedInfoForTaskId): * server-tests/api-build-requests-tests.js: * server-tests/api-upload-root-tests.js: Added. Added tests for /api/upload-root. (makeReport): Added. (addSlaveAndCreateRootFile): Added. (createTestGroupWihPatch): Added. * server-tests/privileged-api-create-test-group-tests.js: * server-tests/resources/mock-data.js: (MockData.sharedRepositoryId): Added. (MockData.addMockData): Added "Shared" repository along with commits. Canonical link: https://commits.webkit.org/188885@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@216647 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-05-11 02:32:00 +00:00
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 1);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 2);
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.deepStrictEqual(set0.repositories(), [webkit]);
assert.deepStrictEqual(set1.repositories(), [webkit]);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '191623');
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const repositoryGroup = requests[0].repositoryGroup();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(repositoryGroup.name(), 'webkit-only');
assert.strictEqual(repositoryGroup, requests[1].repositoryGroup());
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
assert(repositoryGroup.accepts(set0));
assert(repositoryGroup.accepts(set1));
});
});
});
it('should create a test group with the repetition count of two with two repositories', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
let insertedGroupId;
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2,
commitSets: {'WebKit': ['191622', '191623'], 'macOS': ['15A284', '15A284']}}).then((content) => {
insertedGroupId = content['testGroupId'];
Add API to upload a patched build for a custom A/B testing https://bugs.webkit.org/show_bug.cgi?id=171956 Reviewed by Chris Dumez. Added /api/upload-root to upload a root file, the build product of a patch associated with a commit set. Extracted more functions out of privileged-api/upload-file.php to uploaded-file-helpers.php to share code with /api/upload-root.php. * public/api/upload-root.php: Added. (main): (compute_commit_set_items_to_update): Find the list of commit set items to associate this root with. A root can be associated with multiple repositories and there fore commit set items; e.g. if a software is built from multiple repositories and there is a patch associated with one of them, the built product must be associated with all those repositories. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Include the root file is there is one. * public/include/json-header.php: (validate_arguments): Added the support for validating json string. (verify_slave): Return the slave ID found. * public/include/uploaded-file-helpers.php: (validate_uploaded_file): Extracted from /privileged-api/upload-file to be shared with /api/upload-root. (query_total_file_size): Ditto. (create_uploaded_file_from_form_data): Ditto. (upload_file_in_transaction): Ditto. Takes a lambda to do the extra work inside the transaction. * public/privileged-api/upload-file.php: (main): * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Resolve the rootFIle of each commit set item. * public/v3/models/commit-set.js: (CommitSet): Added _repositoryToRootMap and _allRootFiles as instance variables. (CommitSet.prototype.updateSingleton): Added. Previously, each commit set's states never changed after its creation. After this patch, each item can be newly associated with a root so we must update its _repositoryToRootMap and _allRootFiles. For simplicity, we update all states. (CommitSet.prototype._updateFromObject): Extracted from the constructor. (CommitSet.prototype.allRootFiles): Added. Includes custom roots and roots created for patches. (CommitSet.prototype.rootForRepository): Added. (CommitSet.prototype.equals): Fixed the bug that we were comparing _repositoryToPatchMap to _repositoryToCommitMap, and added a check for _repositoryToRootMap. * public/v3/models/test-group.js: (TestGroup.prototype.task): Added. (TestGroup.createWithTask): (TestGroup.createWithCustomConfiguration): (TestGroup.createAndRefetchTestGroups): (TestGroup._fetchTestGroupsForTask): Deleted. Now fetchForTask takes a boolean argument: ignoreCache. (TestGroup.findAllByTask): Added. (TestGroup.fetchForTask): Renamed from fetchByTask. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._fetchRelatedInfoForTaskId): * server-tests/api-build-requests-tests.js: * server-tests/api-upload-root-tests.js: Added. Added tests for /api/upload-root. (makeReport): Added. (addSlaveAndCreateRootFile): Added. (createTestGroupWihPatch): Added. * server-tests/privileged-api-create-test-group-tests.js: * server-tests/resources/mock-data.js: (MockData.sharedRepositoryId): Added. (MockData.addMockData): Added "Shared" repository along with commits. Canonical link: https://commits.webkit.org/188885@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@216647 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-05-11 02:32:00 +00:00
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 4);
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests[2].commitSet(), set0);
assert.strictEqual(requests[3].commitSet(), set1);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [webkit, macos]);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set1.revisionForRepository(webkit), '191623');
assert.strictEqual(set1.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.commitForRepository(macos), set1.commitForRepository(macos));
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const repositoryGroup = requests[0].repositoryGroup();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(repositoryGroup.name(), 'system-and-webkit');
assert.strictEqual(requests[1].repositoryGroup(), repositoryGroup);
assert.strictEqual(requests[2].repositoryGroup(), repositoryGroup);
assert.strictEqual(requests[3].repositoryGroup(), repositoryGroup);
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
assert(repositoryGroup.accepts(set0));
assert(repositoryGroup.accepts(set1));
});
});
});
it('should create a test group using Git partial hashes', () => {
let webkit;
let macos;
return addTriggerableAndCreateTask('some task', ['2ceda45d3cd63cde58d0dbf5767714e03d902e43', '5471a8ddc1f661663ccfd1a29c633ba57e879533']).then((taskId) => {
webkit = Repository.findById(MockData.webkitRepositoryId());
macos = Repository.findById(MockData.macosRepositoryId());
const revisionSets = [{[macos.id()]: {revision: '15A284'}, [webkit.id()]: {revision: '2ceda'}},
{[macos.id()]: {revision: '15A284'}, [webkit.id()]: {revision: '5471a'}}];
const params = {name: 'test', task: taskId, repetitionCount: 2, revisionSets};
let insertedGroupId;
return PrivilegedAPI.sendRequest('create-test-group', params).then((content) => {
insertedGroupId = content['testGroupId'];
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 4);
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [webkit, macos]);
assert.strictEqual(set0.revisionForRepository(webkit), '2ceda45d3cd63cde58d0dbf5767714e03d902e43');
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set1.revisionForRepository(webkit), '5471a8ddc1f661663ccfd1a29c633ba57e879533');
assert.strictEqual(set1.revisionForRepository(macos), '15A284');
const repositoryGroup0 = requests[0].repositoryGroup();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(repositoryGroup0.name(), 'system-and-webkit');
assert.strictEqual(repositoryGroup0, requests[2].repositoryGroup());
const repositoryGroup1 = requests[1].repositoryGroup();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(repositoryGroup1, repositoryGroup0);
assert(repositoryGroup0.accepts(set0));
assert(repositoryGroup0.accepts(set1));
});
});
});
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
it('should create a test group using different repository groups if needed', () => {
let webkit;
let macos;
return addTriggerableAndCreateTask('some task').then((taskId) => {
webkit = Repository.findById(MockData.webkitRepositoryId());
macos = Repository.findById(MockData.macosRepositoryId());
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
const revisionSets = [{[macos.id()]: {revision: '15A284'}, [webkit.id()]: {revision: '191622'}},
{[webkit.id()]: {revision: '191623'}}];
const params = {name: 'test', task: taskId, repetitionCount: 2, revisionSets};
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
let insertedGroupId;
return PrivilegedAPI.sendRequest('create-test-group', params).then((content) => {
insertedGroupId = content['testGroupId'];
Add API to upload a patched build for a custom A/B testing https://bugs.webkit.org/show_bug.cgi?id=171956 Reviewed by Chris Dumez. Added /api/upload-root to upload a root file, the build product of a patch associated with a commit set. Extracted more functions out of privileged-api/upload-file.php to uploaded-file-helpers.php to share code with /api/upload-root.php. * public/api/upload-root.php: Added. (main): (compute_commit_set_items_to_update): Find the list of commit set items to associate this root with. A root can be associated with multiple repositories and there fore commit set items; e.g. if a software is built from multiple repositories and there is a patch associated with one of them, the built product must be associated with all those repositories. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Include the root file is there is one. * public/include/json-header.php: (validate_arguments): Added the support for validating json string. (verify_slave): Return the slave ID found. * public/include/uploaded-file-helpers.php: (validate_uploaded_file): Extracted from /privileged-api/upload-file to be shared with /api/upload-root. (query_total_file_size): Ditto. (create_uploaded_file_from_form_data): Ditto. (upload_file_in_transaction): Ditto. Takes a lambda to do the extra work inside the transaction. * public/privileged-api/upload-file.php: (main): * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Resolve the rootFIle of each commit set item. * public/v3/models/commit-set.js: (CommitSet): Added _repositoryToRootMap and _allRootFiles as instance variables. (CommitSet.prototype.updateSingleton): Added. Previously, each commit set's states never changed after its creation. After this patch, each item can be newly associated with a root so we must update its _repositoryToRootMap and _allRootFiles. For simplicity, we update all states. (CommitSet.prototype._updateFromObject): Extracted from the constructor. (CommitSet.prototype.allRootFiles): Added. Includes custom roots and roots created for patches. (CommitSet.prototype.rootForRepository): Added. (CommitSet.prototype.equals): Fixed the bug that we were comparing _repositoryToPatchMap to _repositoryToCommitMap, and added a check for _repositoryToRootMap. * public/v3/models/test-group.js: (TestGroup.prototype.task): Added. (TestGroup.createWithTask): (TestGroup.createWithCustomConfiguration): (TestGroup.createAndRefetchTestGroups): (TestGroup._fetchTestGroupsForTask): Deleted. Now fetchForTask takes a boolean argument: ignoreCache. (TestGroup.findAllByTask): Added. (TestGroup.fetchForTask): Renamed from fetchByTask. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._fetchRelatedInfoForTaskId): * server-tests/api-build-requests-tests.js: * server-tests/api-upload-root-tests.js: Added. Added tests for /api/upload-root. (makeReport): Added. (addSlaveAndCreateRootFile): Added. (createTestGroupWihPatch): Added. * server-tests/privileged-api-create-test-group-tests.js: * server-tests/resources/mock-data.js: (MockData.sharedRepositoryId): Added. (MockData.addMockData): Added "Shared" repository along with commits. Canonical link: https://commits.webkit.org/188885@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@216647 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-05-11 02:32:00 +00:00
return TestGroup.fetchForTask(taskId, true);
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 4);
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
assert.deepStrictEqual(set1.repositories(), [webkit]);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set1.revisionForRepository(webkit), '191623');
assert.strictEqual(set1.revisionForRepository(macos), null);
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
const repositoryGroup0 = requests[0].repositoryGroup();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(repositoryGroup0.name(), 'system-and-webkit');
assert.strictEqual(repositoryGroup0, requests[2].repositoryGroup());
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
assert(repositoryGroup0.accepts(set0));
assert(!repositoryGroup0.accepts(set1));
const repositoryGroup1 = requests[1].repositoryGroup();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(repositoryGroup1.name(), 'webkit-only');
assert.strictEqual(repositoryGroup1, requests[3].repositoryGroup());
Each build request should be associated with a repository group https://bugs.webkit.org/show_bug.cgi?id=170528 Rubber-stamped by Chris Dumez. Make the buildbot syncing script use the concept of repository groups so that each repository group can post a different set of properties to buildbot. In order to do this, we associate each build request with a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via /api/update-triggerable just the same way the set of the supported platform, test pairs are updated. Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name to either a string value or a repository name enclosed in < and >: ```js "repositoryGroups": { "webkit-svn": { "repositories": ["WebKit", "macOS"], "properties": {"os": "<macOS>", "wk": "<WebKit>"} } } ``` With this, removed the support for specifying a repository to use in generic dictionary of properties via a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in build requests. After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group. Run the appropriate database queries to set the repository group on each build request. Because of this change, this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests. Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same test group if the next build request to be scheduled does not specify a repository group. * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for repository and group pair in triggerable_repositories table. * public/api/update-triggerable.php: (main): Validate and insert repository groups. (validate_configurations): Extracted from main. (validate_repository_groups): Added. * public/v3/models/repository.js: (Repository.findTopLevelByName): Added. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response. * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given triggerable for a list of repositories. (RepositoryGroupFinder): Added. (RepositoryGroupFinder::__construct): Added. (RepositoryGroupFinder::find_by_repositories): Added. (RepositoryGroupFinder::populate_map): Added. * public/privileged-api/create-test-group.php: (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set. Use that to set the repository group in each new build request. (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group. (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]} * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.triggerable): Added. (BuildRequest.prototype.repositoryGroup): Added. (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group. * public/v3/models/triggerable.js: (Triggerable.prototype.name): Added. (Triggerable.prototype.acceptedRepositories): Deleted. (TriggerableRepositoryGroup): (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request. * server-tests/api-manifest-tests.js: Added assertions for the repository groups. * server-tests/api-report-tests.js: (.emptyReport): (.reportWithTwoLevelsOfAggregations): * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with a triggerable. (.updateWithOSXRepositoryGroup): (.mapRepositoriesByGroup): * server-tests/privileged-api-create-test-group-tests.js: (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups in existing test cases, and added a test case for creating a test group with two different repository groups. * server-tests/resources/mock-data.js: (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps. (MockData.emptyTriggeragbleId): Added. (MockData.macosRepositoryId): Added. (MockData.webkitRepositoryId): Added. (MockData.gitWebkitRepositoryId): Added. (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks. (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js. (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js. (MockData.addAnotherMockTestGroup): Cleanup. (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test updating the set of repository groups in addition to the set of test, platform pairs. (.refetchManifest): Added. * tools/js/buildbot-syncer.js: (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument, and buildRequestArgument as the third argument. (BuildbotSyncer.prototype.repositoryGroups): Added. (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find. Also added an assertion that the build request is associated with a repository group. (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to specify a revision in favor of explicity listing each property and repository name in a repository group. (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties shared across syncers, the name of properties which specifies the build slave name and build request ID. These values are not stored as top-level properties and superseded by the concept of repository groups. (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups. (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of properties or buildRequestArgument (often inherited from shared). (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of allowed proeprties in each configuration now that they're specified as top-level properties. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups. (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is not included in the list of valid build requests. (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes those that lack a repository group set. (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds the next build request to be scheduled for the test group. (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of a test group information since syncOnce now calls _nextRequestInGroup to find the next build request. * tools/js/v3-models.js: * unit-tests/build-request-tests.js: Fixed the test name. * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests for parsing repository groups. (sampleiOSConfig): Updated the mock configuration per code changes. (sampleiOSConfigWithExpansions): Ditto. (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration. Various test cases have been updated to reflect this. (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock configurations. The git hash was there to test "rootOptions", which this patch removed. (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding" which, again, this patch removed. (sampleInProgressBuild): Ditto. (sampleFinishedBuild): Ditto. * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Added ock repository groups so that existing tests will continue to function. Canonical link: https://commits.webkit.org/187490@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-06 21:56:59 +00:00
assert(!repositoryGroup1.accepts(set0));
assert(repositoryGroup1.accepts(set1));
});
});
});
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
it('should create a test group with a custom root', () => {
return addTriggerableAndCreateTask('some task').then((taskId) => {
let insertedGroupId;
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
let uploadedFile;
return TemporaryFile.makeTemporaryFile('some.dat', 'some content').then((stream) => {
return PrivilegedAPI.sendRequest('upload-file', {newFile: stream}, {useFormData: true});
}).then((response) => {
uploadedFile = response['uploadedFile'];
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
const revisionSets = [{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}},
{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, 'customRoots': [uploadedFile['id']]}];
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2, revisionSets}).then((content) => {
insertedGroupId = content['testGroupId'];
Add API to upload a patched build for a custom A/B testing https://bugs.webkit.org/show_bug.cgi?id=171956 Reviewed by Chris Dumez. Added /api/upload-root to upload a root file, the build product of a patch associated with a commit set. Extracted more functions out of privileged-api/upload-file.php to uploaded-file-helpers.php to share code with /api/upload-root.php. * public/api/upload-root.php: Added. (main): (compute_commit_set_items_to_update): Find the list of commit set items to associate this root with. A root can be associated with multiple repositories and there fore commit set items; e.g. if a software is built from multiple repositories and there is a patch associated with one of them, the built product must be associated with all those repositories. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Include the root file is there is one. * public/include/json-header.php: (validate_arguments): Added the support for validating json string. (verify_slave): Return the slave ID found. * public/include/uploaded-file-helpers.php: (validate_uploaded_file): Extracted from /privileged-api/upload-file to be shared with /api/upload-root. (query_total_file_size): Ditto. (create_uploaded_file_from_form_data): Ditto. (upload_file_in_transaction): Ditto. Takes a lambda to do the extra work inside the transaction. * public/privileged-api/upload-file.php: (main): * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Resolve the rootFIle of each commit set item. * public/v3/models/commit-set.js: (CommitSet): Added _repositoryToRootMap and _allRootFiles as instance variables. (CommitSet.prototype.updateSingleton): Added. Previously, each commit set's states never changed after its creation. After this patch, each item can be newly associated with a root so we must update its _repositoryToRootMap and _allRootFiles. For simplicity, we update all states. (CommitSet.prototype._updateFromObject): Extracted from the constructor. (CommitSet.prototype.allRootFiles): Added. Includes custom roots and roots created for patches. (CommitSet.prototype.rootForRepository): Added. (CommitSet.prototype.equals): Fixed the bug that we were comparing _repositoryToPatchMap to _repositoryToCommitMap, and added a check for _repositoryToRootMap. * public/v3/models/test-group.js: (TestGroup.prototype.task): Added. (TestGroup.createWithTask): (TestGroup.createWithCustomConfiguration): (TestGroup.createAndRefetchTestGroups): (TestGroup._fetchTestGroupsForTask): Deleted. Now fetchForTask takes a boolean argument: ignoreCache. (TestGroup.findAllByTask): Added. (TestGroup.fetchForTask): Renamed from fetchByTask. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._fetchRelatedInfoForTaskId): * server-tests/api-build-requests-tests.js: * server-tests/api-upload-root-tests.js: Added. Added tests for /api/upload-root. (makeReport): Added. (addSlaveAndCreateRootFile): Added. (createTestGroupWihPatch): Added. * server-tests/privileged-api-create-test-group-tests.js: * server-tests/resources/mock-data.js: (MockData.sharedRepositoryId): Added. (MockData.addMockData): Added "Shared" repository along with commits. Canonical link: https://commits.webkit.org/188885@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@216647 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-05-11 02:32:00 +00:00
return TestGroup.fetchForTask(taskId, true);
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
});
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 4);
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests[2].commitSet(), set0);
assert.strictEqual(requests[3].commitSet(), set1);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [webkit, macos]);
assert.deepStrictEqual(set1.customRoots(), [UploadedFile.ensureSingleton(uploadedFile['id'], uploadedFile)]);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set0.revisionForRepository(webkit), set1.revisionForRepository(webkit));
assert.strictEqual(set0.commitForRepository(webkit), set1.commitForRepository(webkit));
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.commitForRepository(macos), set1.commitForRepository(macos));
assert.strictEqual(set0.revisionForRepository(macos), set1.revisionForRepository(macos));
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
assert(!set0.equals(set1));
});
});
});
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
it('should create a build test group with a patch', () => {
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
let taskId;
let webkit;
let macos;
let insertedGroupId;
let uploadedFile;
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
return TemporaryFile.makeTemporaryFile('some.dat', 'some content');
}).then((stream) => {
return PrivilegedAPI.sendRequest('upload-file', {newFile: stream}, {useFormData: true});
}).then((response) => {
const rawFile = response['uploadedFile'];
uploadedFile = UploadedFile.ensureSingleton(rawFile.id, rawFile);
const revisionSets = [{[webkit.id()]: {revision: '191622', patch: uploadedFile.id()}, [macos.id()]: {revision: '15A284'}},
{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2, revisionSets});
}).then((content) => {
insertedGroupId = content['testGroupId'];
Add API to upload a patched build for a custom A/B testing https://bugs.webkit.org/show_bug.cgi?id=171956 Reviewed by Chris Dumez. Added /api/upload-root to upload a root file, the build product of a patch associated with a commit set. Extracted more functions out of privileged-api/upload-file.php to uploaded-file-helpers.php to share code with /api/upload-root.php. * public/api/upload-root.php: Added. (main): (compute_commit_set_items_to_update): Find the list of commit set items to associate this root with. A root can be associated with multiple repositories and there fore commit set items; e.g. if a software is built from multiple repositories and there is a patch associated with one of them, the built product must be associated with all those repositories. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Include the root file is there is one. * public/include/json-header.php: (validate_arguments): Added the support for validating json string. (verify_slave): Return the slave ID found. * public/include/uploaded-file-helpers.php: (validate_uploaded_file): Extracted from /privileged-api/upload-file to be shared with /api/upload-root. (query_total_file_size): Ditto. (create_uploaded_file_from_form_data): Ditto. (upload_file_in_transaction): Ditto. Takes a lambda to do the extra work inside the transaction. * public/privileged-api/upload-file.php: (main): * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Resolve the rootFIle of each commit set item. * public/v3/models/commit-set.js: (CommitSet): Added _repositoryToRootMap and _allRootFiles as instance variables. (CommitSet.prototype.updateSingleton): Added. Previously, each commit set's states never changed after its creation. After this patch, each item can be newly associated with a root so we must update its _repositoryToRootMap and _allRootFiles. For simplicity, we update all states. (CommitSet.prototype._updateFromObject): Extracted from the constructor. (CommitSet.prototype.allRootFiles): Added. Includes custom roots and roots created for patches. (CommitSet.prototype.rootForRepository): Added. (CommitSet.prototype.equals): Fixed the bug that we were comparing _repositoryToPatchMap to _repositoryToCommitMap, and added a check for _repositoryToRootMap. * public/v3/models/test-group.js: (TestGroup.prototype.task): Added. (TestGroup.createWithTask): (TestGroup.createWithCustomConfiguration): (TestGroup.createAndRefetchTestGroups): (TestGroup._fetchTestGroupsForTask): Deleted. Now fetchForTask takes a boolean argument: ignoreCache. (TestGroup.findAllByTask): Added. (TestGroup.fetchForTask): Renamed from fetchByTask. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._fetchRelatedInfoForTaskId): * server-tests/api-build-requests-tests.js: * server-tests/api-upload-root-tests.js: Added. Added tests for /api/upload-root. (makeReport): Added. (addSlaveAndCreateRootFile): Added. (createTestGroupWihPatch): Added. * server-tests/privileged-api-create-test-group-tests.js: * server-tests/resources/mock-data.js: (MockData.sharedRepositoryId): Added. (MockData.addMockData): Added "Shared" repository along with commits. Canonical link: https://commits.webkit.org/188885@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@216647 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-05-11 02:32:00 +00:00
return TestGroup.fetchForTask(taskId, true);
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.test(), Test.findById(MockData.someTestId()));
assert.strictEqual(group.platform(), Platform.findById(MockData.somePlatformId()));
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 6);
assert.strictEqual(requests[0].isBuild(), true);
assert.strictEqual(requests[1].isBuild(), true);
assert.strictEqual(requests[2].isBuild(), false);
assert.strictEqual(requests[3].isBuild(), false);
assert.strictEqual(requests[4].isBuild(), false);
assert.strictEqual(requests[5].isBuild(), false);
assert.strictEqual(requests[0].isTest(), false);
assert.strictEqual(requests[1].isTest(), false);
assert.strictEqual(requests[2].isTest(), true);
assert.strictEqual(requests[3].isTest(), true);
assert.strictEqual(requests[4].isTest(), true);
assert.strictEqual(requests[5].isTest(), true);
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests[2].commitSet(), set0);
assert.strictEqual(requests[3].commitSet(), set1);
assert.strictEqual(requests[4].commitSet(), set0);
assert.strictEqual(requests[5].commitSet(), set1);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [webkit, macos]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set0.revisionForRepository(webkit), set1.revisionForRepository(webkit));
assert.strictEqual(set0.commitForRepository(webkit), set1.commitForRepository(webkit));
assert.strictEqual(set0.patchForRepository(webkit), uploadedFile);
assert.strictEqual(set1.patchForRepository(webkit), null);
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.revisionForRepository(macos), set1.revisionForRepository(macos));
assert.strictEqual(set0.commitForRepository(macos), set1.commitForRepository(macos));
assert.strictEqual(set0.patchForRepository(macos), null);
assert.strictEqual(set1.patchForRepository(macos), null);
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
assert(!set0.equals(set1));
});
});
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
it('should create a build test group with a owned commits even when one of group does not contain an owned commit', () => {
let taskId;
let webkit;
let jsc;
let macos;
let insertedGroupId;
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
jsc = Repository.all().filter((repository) => repository.name() == 'JavaScriptCore')[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}},
{[webkit.id()]: {revision: '192736'}, [macos.id()]: {revision: '15A284'}, [jsc.id()]: {revision: 'owned-jsc-9191', ownerRevision: '192736'}}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2, revisionSets});
}).then((content) => {
insertedGroupId = content['testGroupId'];
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.test(), Test.findById(MockData.someTestId()));
assert.strictEqual(group.platform(), Platform.findById(MockData.somePlatformId()));
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 6);
assert.strictEqual(requests[0].isBuild(), true);
assert.strictEqual(requests[1].isBuild(), true);
assert.strictEqual(requests[2].isBuild(), false);
assert.strictEqual(requests[3].isBuild(), false);
assert.strictEqual(requests[4].isBuild(), false);
assert.strictEqual(requests[5].isBuild(), false);
assert.strictEqual(requests[0].isTest(), false);
assert.strictEqual(requests[1].isTest(), false);
assert.strictEqual(requests[2].isTest(), true);
assert.strictEqual(requests[3].isTest(), true);
assert.strictEqual(requests[4].isTest(), true);
assert.strictEqual(requests[5].isTest(), true);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests[2].commitSet(), set0);
assert.strictEqual(requests[3].commitSet(), set1);
assert.strictEqual(requests[4].commitSet(), set0);
assert.strictEqual(requests[5].commitSet(), set1);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [jsc, webkit, macos]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '192736');
assert.strictEqual(set0.patchForRepository(webkit), null);
assert.strictEqual(set1.patchForRepository(webkit), null);
assert.strictEqual(set0.requiresBuildForRepository(webkit), false);
assert.strictEqual(set1.requiresBuildForRepository(webkit), false);
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.revisionForRepository(macos), set1.revisionForRepository(macos));
assert.strictEqual(set0.commitForRepository(macos), set1.commitForRepository(macos));
assert.strictEqual(set0.patchForRepository(macos), null);
assert.strictEqual(set1.patchForRepository(macos), null);
assert.strictEqual(set0.requiresBuildForRepository(macos), false);
assert.strictEqual(set1.requiresBuildForRepository(macos), false);
assert.strictEqual(set0.revisionForRepository(jsc), null);
assert.strictEqual(set1.revisionForRepository(jsc), 'owned-jsc-9191');
assert.ok(!set0.patchForRepository(jsc));
assert.strictEqual(set1.patchForRepository(jsc), null);
assert.strictEqual(set0.ownerRevisionForRepository(jsc), null);
assert.strictEqual(set1.ownerRevisionForRepository(jsc), '192736');
assert.strictEqual(set1.requiresBuildForRepository(jsc), true);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
assert(!set0.equals(set1));
});
});
it('should create a test group with a owned commits even when no patch is specified', () => {
let taskId;
let webkit;
let jsc;
let macos;
let insertedGroupId;
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
jsc = Repository.all().filter((repository) => repository.name() == 'JavaScriptCore')[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, [jsc.id()]: {revision: 'owned-jsc-6161', ownerRevision: '191622'}},
{[webkit.id()]: {revision: '192736'}, [macos.id()]: {revision: '15A284'}, [jsc.id()]: {revision: 'owned-jsc-9191', ownerRevision: '192736'}}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2, revisionSets});
}).then((content) => {
insertedGroupId = content['testGroupId'];
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.test(), Test.findById(MockData.someTestId()));
assert.strictEqual(group.platform(), Platform.findById(MockData.somePlatformId()));
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 6);
assert.strictEqual(requests[0].isBuild(), true);
assert.strictEqual(requests[1].isBuild(), true);
assert.strictEqual(requests[2].isBuild(), false);
assert.strictEqual(requests[3].isBuild(), false);
assert.strictEqual(requests[4].isBuild(), false);
assert.strictEqual(requests[5].isBuild(), false);
assert.strictEqual(requests[0].isTest(), false);
assert.strictEqual(requests[1].isTest(), false);
assert.strictEqual(requests[2].isTest(), true);
assert.strictEqual(requests[3].isTest(), true);
assert.strictEqual(requests[4].isTest(), true);
assert.strictEqual(requests[5].isTest(), true);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests[2].commitSet(), set0);
assert.strictEqual(requests[3].commitSet(), set1);
assert.strictEqual(requests[4].commitSet(), set0);
assert.strictEqual(requests[5].commitSet(), set1);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [jsc, webkit, macos]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [jsc, webkit, macos]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '192736');
assert.strictEqual(set0.patchForRepository(webkit), null);
assert.strictEqual(set1.patchForRepository(webkit), null);
assert.strictEqual(set0.requiresBuildForRepository(webkit), false);
assert.strictEqual(set1.requiresBuildForRepository(webkit), false);
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.revisionForRepository(macos), set1.revisionForRepository(macos));
assert.strictEqual(set0.commitForRepository(macos), set1.commitForRepository(macos));
assert.strictEqual(set0.patchForRepository(macos), null);
assert.strictEqual(set1.patchForRepository(macos), null);
assert.strictEqual(set0.requiresBuildForRepository(macos), false);
assert.strictEqual(set1.requiresBuildForRepository(macos), false);
assert.strictEqual(set0.revisionForRepository(jsc), 'owned-jsc-6161');
assert.strictEqual(set1.revisionForRepository(jsc), 'owned-jsc-9191');
assert.strictEqual(set0.patchForRepository(jsc), null);
assert.strictEqual(set1.patchForRepository(jsc), null);
assert.strictEqual(set0.ownerRevisionForRepository(jsc), '191622');
assert.strictEqual(set1.ownerRevisionForRepository(jsc), '192736');
assert.strictEqual(set0.requiresBuildForRepository(jsc), true);
assert.strictEqual(set1.requiresBuildForRepository(jsc), true);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
assert(!set0.equals(set1));
});
});
it('should create a test group with a owned commits and a patch', () => {
let taskId;
let webkit;
let macos;
let jsc;
let insertedGroupId;
let uploadedFile;
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
jsc = Repository.all().filter((repository) => repository.name() == 'JavaScriptCore')[0];
return TemporaryFile.makeTemporaryFile('some.dat', 'some content');
}).then((stream) => {
return PrivilegedAPI.sendRequest('upload-file', {newFile: stream}, {useFormData: true});
}).then((response) => {
const rawFile = response['uploadedFile'];
uploadedFile = UploadedFile.ensureSingleton(rawFile.id, rawFile);
const revisionSets = [{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, [jsc.id()]: {revision: 'owned-jsc-6161', ownerRevision: '191622'}},
{[webkit.id()]: {revision: '192736', patch: uploadedFile.id()}, [macos.id()]: {revision: '15A284'}, [jsc.id()]: {revision: 'owned-jsc-9191', ownerRevision: '192736'}}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2, revisionSets});
}).then((content) => {
insertedGroupId = content['testGroupId'];
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.test(), Test.findById(MockData.someTestId()));
assert.strictEqual(group.platform(), Platform.findById(MockData.somePlatformId()));
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 6);
assert.strictEqual(requests[0].isBuild(), true);
assert.strictEqual(requests[1].isBuild(), true);
assert.strictEqual(requests[2].isBuild(), false);
assert.strictEqual(requests[3].isBuild(), false);
assert.strictEqual(requests[4].isBuild(), false);
assert.strictEqual(requests[5].isBuild(), false);
assert.strictEqual(requests[0].isTest(), false);
assert.strictEqual(requests[1].isTest(), false);
assert.strictEqual(requests[2].isTest(), true);
assert.strictEqual(requests[3].isTest(), true);
assert.strictEqual(requests[4].isTest(), true);
assert.strictEqual(requests[5].isTest(), true);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests[2].commitSet(), set0);
assert.strictEqual(requests[3].commitSet(), set1);
assert.strictEqual(requests[4].commitSet(), set0);
assert.strictEqual(requests[5].commitSet(), set1);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [jsc, webkit, macos]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [jsc, webkit, macos]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '192736');
assert.strictEqual(set0.patchForRepository(webkit), null);
assert.strictEqual(set1.patchForRepository(webkit), uploadedFile);
assert.strictEqual(set0.ownerRevisionForRepository(webkit), null);
assert.strictEqual(set1.ownerRevisionForRepository(webkit), null);
assert.strictEqual(set0.requiresBuildForRepository(webkit), true);
assert.strictEqual(set1.requiresBuildForRepository(webkit), true);
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.revisionForRepository(macos), set1.revisionForRepository(macos));
assert.strictEqual(set0.commitForRepository(macos), set1.commitForRepository(macos));
assert.strictEqual(set0.patchForRepository(macos), null);
assert.strictEqual(set1.patchForRepository(macos), null);
assert.strictEqual(set0.ownerRevisionForRepository(macos), null);
assert.strictEqual(set1.ownerRevisionForRepository(macos), null);
assert.strictEqual(set0.requiresBuildForRepository(macos), false);
assert.strictEqual(set1.requiresBuildForRepository(macos), false);
assert.strictEqual(set0.revisionForRepository(jsc), 'owned-jsc-6161');
assert.strictEqual(set1.revisionForRepository(jsc), 'owned-jsc-9191');
assert.strictEqual(set0.patchForRepository(jsc), null);
assert.strictEqual(set1.patchForRepository(jsc), null);
assert.strictEqual(set0.ownerRevisionForRepository(jsc), '191622');
assert.strictEqual(set1.ownerRevisionForRepository(jsc), '192736');
assert.strictEqual(set0.requiresBuildForRepository(jsc), true);
assert.strictEqual(set1.requiresBuildForRepository(jsc), true);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
assert(!set0.equals(set1));
});
});
it('should still work even if components with same name but one is owned, one is not', () => {
let taskId;
let webkit;
let macos;
let jsc;
let ownedJSC;
let insertedGroupId;
let uploadedFile;
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
return MockData.addAnotherTriggerable(TestServer.database());
}).then(() => {
webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
jsc = Repository.all().filter((repository) => repository.name() == 'JavaScriptCore' && !repository.ownerId())[0];
ownedJSC = Repository.all().filter((repository) => repository.name() == 'JavaScriptCore' && repository.ownerId())[0];
return TemporaryFile.makeTemporaryFile('some.dat', 'some content');
}).then((stream) => {
return PrivilegedAPI.sendRequest('upload-file', {newFile: stream}, {useFormData: true});
}).then((response) => {
const rawFile = response['uploadedFile'];
uploadedFile = UploadedFile.ensureSingleton(rawFile.id, rawFile);
const revisionSets = [{[jsc.id()]: {revision: 'jsc-6161'}, [webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284'}, [ownedJSC.id()]: {revision: 'owned-jsc-6161', ownerRevision: '191622'}},
{[jsc.id()]: {revision: 'jsc-9191'}, [webkit.id()]: {revision: '192736', patch: uploadedFile.id()}, [macos.id()]: {revision: '15A284'}, [ownedJSC.id()]: {revision: 'owned-jsc-9191', ownerRevision: '192736'}}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2, revisionSets});
}).then((content) => {
insertedGroupId = content['testGroupId'];
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 2);
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.test(), Test.findById(MockData.someTestId()));
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.platform(), Platform.findById(MockData.somePlatformId()));
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 6);
assert.strictEqual(requests[0].isBuild(), true);
assert.strictEqual(requests[1].isBuild(), true);
assert.strictEqual(requests[2].isBuild(), false);
assert.strictEqual(requests[3].isBuild(), false);
assert.strictEqual(requests[4].isBuild(), false);
assert.strictEqual(requests[5].isBuild(), false);
assert.strictEqual(requests[0].isTest(), false);
assert.strictEqual(requests[1].isTest(), false);
assert.strictEqual(requests[2].isTest(), true);
assert.strictEqual(requests[3].isTest(), true);
assert.strictEqual(requests[4].isTest(), true);
assert.strictEqual(requests[5].isTest(), true);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests[2].commitSet(), set0);
assert.strictEqual(requests[3].commitSet(), set1);
assert.strictEqual(requests[4].commitSet(), set0);
assert.strictEqual(requests[5].commitSet(), set1);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [ownedJSC, jsc, webkit, macos]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [ownedJSC, jsc, webkit, macos]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '192736');
assert.strictEqual(set0.patchForRepository(webkit), null);
assert.strictEqual(set1.patchForRepository(webkit), uploadedFile);
assert.strictEqual(set0.ownerRevisionForRepository(webkit), null);
assert.strictEqual(set1.ownerRevisionForRepository(webkit), null);
assert.strictEqual(set0.requiresBuildForRepository(webkit), true);
assert.strictEqual(set1.requiresBuildForRepository(webkit), true);
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.revisionForRepository(macos), set1.revisionForRepository(macos));
assert.strictEqual(set0.commitForRepository(macos), set1.commitForRepository(macos));
assert.strictEqual(set0.patchForRepository(macos), null);
assert.strictEqual(set1.patchForRepository(macos), null);
assert.strictEqual(set0.ownerRevisionForRepository(macos), null);
assert.strictEqual(set1.ownerRevisionForRepository(macos), null);
assert.strictEqual(set0.requiresBuildForRepository(macos), false);
assert.strictEqual(set1.requiresBuildForRepository(macos), false);
assert.strictEqual(set0.revisionForRepository(ownedJSC), 'owned-jsc-6161');
assert.strictEqual(set1.revisionForRepository(ownedJSC), 'owned-jsc-9191');
assert.strictEqual(set0.patchForRepository(ownedJSC), null);
assert.strictEqual(set1.patchForRepository(ownedJSC), null);
assert.strictEqual(set0.ownerRevisionForRepository(ownedJSC), '191622');
assert.strictEqual(set1.ownerRevisionForRepository(ownedJSC), '192736');
assert.strictEqual(set0.requiresBuildForRepository(ownedJSC), true);
assert.strictEqual(set1.requiresBuildForRepository(ownedJSC), true);
assert.strictEqual(set0.revisionForRepository(jsc), 'jsc-6161');
assert.strictEqual(set1.revisionForRepository(jsc), 'jsc-9191');
assert.strictEqual(set0.patchForRepository(jsc), null);
assert.strictEqual(set1.patchForRepository(jsc), null);
assert.strictEqual(set0.ownerRevisionForRepository(jsc), null);
assert.strictEqual(set1.ownerRevisionForRepository(jsc), null);
assert.strictEqual(set0.requiresBuildForRepository(jsc), false);
assert.strictEqual(set1.requiresBuildForRepository(jsc), false);
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
assert(!set0.equals(set1));
});
});
it('should not create a build request to build a patch when the commit set does not have a patch', () => {
let taskId;
let webkit;
let macos;
let insertedGroupId;
let uploadedFile;
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
return TemporaryFile.makeTemporaryFile('some.dat', 'some content');
}).then((stream) => {
return PrivilegedAPI.sendRequest('upload-file', {newFile: stream}, {useFormData: true});
}).then((response) => {
const rawFile = response['uploadedFile'];
uploadedFile = UploadedFile.ensureSingleton(rawFile.id, rawFile);
const revisionSets = [{[macos.id()]: {revision: '15A284'}},
{[webkit.id()]: {revision: '191622', patch: uploadedFile.id()}, [macos.id()]: {revision: '15A284'}}];
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2, revisionSets});
}).then((content) => {
insertedGroupId = content['testGroupId'];
return TestGroup.fetchForTask(taskId, true);
}).then((testGroups) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.test(), Test.findById(MockData.someTestId()));
assert.strictEqual(group.platform(), Platform.findById(MockData.somePlatformId()));
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 5);
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests[0].isBuild(), true);
assert.strictEqual(requests[1].isBuild(), false);
assert.strictEqual(requests[2].isBuild(), false);
assert.strictEqual(requests[3].isBuild(), false);
assert.strictEqual(requests[4].isBuild(), false);
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests[0].isTest(), false);
assert.strictEqual(requests[1].isTest(), true);
assert.strictEqual(requests[2].isTest(), true);
assert.strictEqual(requests[3].isTest(), true);
assert.strictEqual(requests[4].isTest(), true);
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests[2].commitSet(), set0);
assert.strictEqual(requests[3].commitSet(), set1);
assert.strictEqual(requests[4].commitSet(), set0);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [macos]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), null);
assert.strictEqual(set0.patchForRepository(webkit), uploadedFile);
assert.strictEqual(set0.revisionForRepository(macos), '15A284');
assert.strictEqual(set0.revisionForRepository(macos), set1.revisionForRepository(macos));
assert.strictEqual(set0.commitForRepository(macos), set1.commitForRepository(macos));
assert.strictEqual(set0.patchForRepository(macos), null);
assert.strictEqual(set1.patchForRepository(macos), null);
assert(!set0.equals(set1));
});
});
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
it('should return "PatchNotAccepted" when a patch is specified for a repository that does not accept a patch', () => {
let taskId;
let webkit;
let macos;
let uploadedFile;
return addTriggerableAndCreateTask('some task').then((id) => taskId = id).then(() => {
webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
return TemporaryFile.makeTemporaryFile('some.dat', 'some content');
}).then((stream) => {
return PrivilegedAPI.sendRequest('upload-file', {newFile: stream}, {useFormData: true});
}).then((response) => {
const rawFile = response['uploadedFile'];
uploadedFile = UploadedFile.ensureSingleton(rawFile.id, rawFile);
const revisionSets = [{[webkit.id()]: {revision: '191622'}, [macos.id()]: {revision: '15A284', patch: uploadedFile.id()}},
Performance Dashboard backend should support A/B testing for owned components. https://bugs.webkit.org/show_bug.cgi?id=175978 Reviewed by Ryosuke Niwa. Add backend change for Performance Dashboard to support A/B testing for owned components. Added 'commitset_commit_owner' and 'commitset_requires_build' columns to 'commit_set_items' table. 'commitset_commit_owner' referrs to determine a commit with owner. 'commitset_requires_build' indicates whether a root build is required. This will be set true whenever commit_set_item specifies a patch file, or commit_set_item is commit with owner commit, or any other commit from same repository and in same build-request group requires build. SQL for updating existing database: 'BEGIN; ALTER TABLE commit_set_items ADD COLUMN commitset_commit_owner integer REFERENCES commits DEFAULT NULL, ADD COLUMN commitset_requires_build boolean DEFAULT FALSE; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_patch_file IS NOT NULL; UPDATE commit_set_items SET commitset_requires_build = TRUE WHERE commitset_set IN (SELECT requests1.request_commit_set FROM build_requests as requests1 JOIN build_requests as requests2 ON requests1.request_group = requests2.request_group JOIN commit_set_items as item ON item.commitset_set = requests2.request_commit_set WHERE item.commitset_patch_file IS NOT NULL); ALTER TABLE commit_set_items ADD CONSTRAINT commitset_item_with_patch_must_requires_build CHECK (commitset_patch_file IS NULL OR commitset_requires_build = TRUE), ADD CONSTRAINT commitset_item_with_owned_commit_must_requires_build CHECK (commitset_commit_owner IS NULL OR commitset_requires_build = TRUE); END;' * init-database.sql: Updated 'commit_set_items' table. * public/admin/triggerables.php: Only top level repository should show on triggerables page. * public/include/build-requests-fetcher.php: Added 'commitOwner' and 'requireBuild' to 'revision_items'. Added 'commitOwner' field to a commit. * public/include/db.php: Should be able to insert boolean value to database without explicted convert to 't' or 'f'. * public/privileged-api/create-test-group.php: Added logic to process 'commitOwner' and 'requireBuild' in 'commit_set_items'. Removed a 'FIXME' that has been addressed before this commit. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Set 'commitOwner' field for a commit set item. * public/v3/models/commit-set.js: (CommitSet): Added maps for repository to commit owner and whether a repository requires builds. (CommitSet.prototype.updateSingleton): (CommitSet.prototype._updateFromObject): (CommitSet.prototype.ownerRevisionForRepository): Returns owner revision for a given repository in current commit set. (CommitSet.prototype.requiresBuildForRepository): Returns whether a repository need to build. (CommitSet.prototype.equals): Equality check should include 2 new maps. (CustomCommitSet): CustomCommitSet should be able to store commit with an owner commit. (CustomCommitSet.prototype.setRevisionForRepository): Added each revision list entry should have 'ownerRevision'(null by default). (CustomCommitSet.prototype.equals): Equality check should also check the equality of 'ownerRevision'. (CustomCommitSet.prototype.ownerRevisionForRepository): Returns a owner revision for a given repository. * public/v3/models/repository.js: (Repository.prototype.findOwnedRepositoryByName): Return an repository owned by current repository with a given name. * public/v3/models/test-group.js: Added 'ownerRevision' field in each entry of revisionSet. * server-tests/api-build-requests-tests.js: Added tests. * server-tests/privileged-api-create-test-group-tests.js: Added tests. * server-tests/privileged-api-upload-file-tests.js: Fix unit tests by setting'requires_build' field to be true when updating commit_set_item which has a patch.. * server-tests/resources/mock-data.js: Added mock build requests with commit sets contain owned commits. (MockData.jscRepositoryId): Returns id for JavaScriptsCore repository. (MockData.addMockConfiguration): Added mock JavaScriptCore and owned JavaScriptCore repositories and commits associated with them. (MockData.ownedJSCRepositoryId): Added a JavaScriptCore repository with WebKit as owner. (MockData.addMockConfiguration): Added mock data for test cases those require a commit with a owner commit. (MockData.addTestGroupWithOwnedCommits): Added mock data for analysis tasks, the build requires of which contains owned commits. (MockData.set addAnotherTriggerable): Added another triggerable which has mac, webkit and javascript core repositories as triggerable repository group. (MockData.set addAnotherMockTestGroup): Added another mock test group. * tools/js/v3-models.js: Import CustomCommitSet. * unit-tests/resources/mock-v3-models.js: Added an owned webkit repository. * unit-tests/commit-set-tests.js: Added unit tests CustomCommitSet. Canonical link: https://commits.webkit.org/193541@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222219 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-09-19 20:14:45 +00:00
{[webkit.id()]: {revision: '192736'}, [macos.id()]: {revision: '15A284'}}];
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2, revisionSets});
}).then(() => {
assert(false, 'should never be reached');
}, (error) => {
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(error, 'PatchNotAccepted');
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
});
});
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
it('should create a test group with an analysis task with needs-notification flag set', async () => {
await addTriggerableAndCreateTask('some task');
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
let result = await PrivilegedAPI.sendRequest('create-test-group',
{name: 'test', taskName: 'other task', platform: MockData.somePlatformId(), test: MockData.someTestId(), needsNotification: true, revisionSets});
const insertedGroupId = result['testGroupId'];
const [analysisTask, testGroups] = await Promise.all([AnalysisTask.fetchById(result['taskId']), TestGroup.fetchForTask(result['taskId'], true)]);
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(analysisTask.name(), 'other task');
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 1);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(group.needsNotification());
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.deepStrictEqual(set0.repositories(), [webkit]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(set1.repositories(), [webkit]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '191623');
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
});
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
it('should be able to create a test group with needs-notification flag unset', async () => {
await addTriggerableAndCreateTask('some task');
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
let result = await PrivilegedAPI.sendRequest('create-test-group',
{name: 'test', taskName: 'other task', platform: MockData.somePlatformId(), test: MockData.someTestId(), needsNotification: false, revisionSets});
const insertedGroupId = result['testGroupId'];
const [analysisTask, testGroups] = await Promise.all([AnalysisTask.fetchById(result['taskId']), TestGroup.fetchForTask(result['taskId'], true)]);
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(analysisTask.name(), 'other task');
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 1);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
const group = testGroups[0];
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(group.id(), insertedGroupId);
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(group.initialRepetitionCount(), 1);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
assert.ok(!group.needsNotification());
const requests = group.buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 2);
Added sending notification feature when test group finishes. https://bugs.webkit.org/show_bug.cgi?id=184340 Reviewed by Ryosuke Niwa. Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group has a pending notification. Added 'testgroup_notification_sent_at' to record the last notification sent time. SQL queries to update existing database are: 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;' 'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;' Updated 'run-analysis' script to be able to send notification when test group finishes. Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group. * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test. * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage. * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table. * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its 'testgroup_needs_notification' is true will be returned by this API. * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids. * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification' while create an analysis_test_groups. * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'. * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'. Extended this API to allow authentication both from CSRF token and slave. * public/v3/components/custom-configuration-test-group-form.js: (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/components/test-group-form.js: (TestGroupForm): Added '_notifyOnCompletion' instance variable. (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox. (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents 'testgroup_needs_notification' from API perspective. (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list. * public/v3/models/analysis-results.js: Export 'AnalysisResults'. (AnalysisResults.fetch): Update API path to use absolute url. (AnalysisResults): * public/v3/models/analysis-task.js: (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as 'needsNotification' to send to server. (AnalysisTask): * public/v3/models/test-group.js: (TestGroup): Added '_needsNotification' field. (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field. (TestGroup.prototype.needsNotification): Returns '_needsNotification' field. (TestGroup.prototype.author): Returns author information. (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true. (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched. (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as 'needsNotification' to send to server. (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'. * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype._retryCurrentTestGroup): (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const): (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype._createCustomTestGroup): * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox. (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async._analyzeRange): * public/v3/pages/create-analysis-task-page.js: Adapted API change. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'. * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change. Added new tests. * server-tests/privileged-api-create-test-group-tests.js: Added new tests. * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API. * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true. * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change. (async.createTestGroupWihPatch): (createTestGroupWihOwnedCommit): * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups. (AnalysisResultsNotifier): (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups): (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification. (AnalysisResultsNotifier.prototype._constructMessageByRules): (AnalysisResultsNotifier._matchesRule): (AnalysisResultsNotifier._applyUpdate): (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group. (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task. (AnalysisResultsNotifier._instantiateNotificationTemplate): * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result. (TestGroupResultPage): (TestGroupResultPage.prototype.async.setTestGroup): (TestGroupResultPage._urlForAnalysisTask): (TestGroupResultPage.prototype._URLForAnalysisTask): (TestGroupResultPage.prototype.constructTables): (TestGroupResultPage.prototype._constructTableForMetric): (TestGroupResultPage.): (TestGroupResultPage.prototype.get pageContent): (TestGroupResultPage.prototype.get styleTemplate): (BarGraph): (BarGraph.prototype.setWidth): (BarGraph.prototype._constructBarGraph): (BarGraph.prototype.get pageContent): (BarGraph.prototype.get styleTemplate): * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change. (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/v3-models.js: * tools/run-analysis.js: Added the logic that sends notification for completed test groups. (main): (async.analysisLoop): * unit-tests/analysis-task-tests.js: * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'. * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change. * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'. * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits. Canonical link: https://commits.webkit.org/201783@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-06-08 03:44:28 +00:00
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.deepStrictEqual(set0.repositories(), [webkit]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(set1.repositories(), [webkit]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '191623');
Add the UI for scheduling a A/B testing with a custom root https://bugs.webkit.org/show_bug.cgi?id=170622 Reviewed by Anders Carlsson. This patch adds the support for creating a new analysis task with a custom darwinup roots. A follow up patch would update the syncing script to schedule such an A/B testing job to a buildbot instance. * ReadMe.md: Updated instructions for backing up and restoring the database so that it's easier to replace the file path for the backup. * init-database.sql: Make task_platform and task_metric optional in each analysis task. Also added a column to store the root file in commit_set_relationships. * public/api/build-requests.php: (main): Include the uploaded files. * public/api/commits.php: (main): Added the support for querying the latest commits for a given platform. This is used in a new page to create a custom analysis task to autofill the latest revisions for a given platform. * public/api/test-groups.php: (main): Include the uploaded files. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::__construct): Added a list of uploaded_files and a map from its id. (BuildRequestsFetcher::uploaded_files): Added. (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Added the support for including custom roots' id in each commit set, and inserting its meta data in the list of uplaoded files. * public/include/commit-log-fetcher.php: (CommitLogFetcher::fetch_latest_for_platform): Added. Finds the latest commit for a given platform. Ideally, we should be finding the latest commit for a given platform, but this is very slow so instead find the commit of the latest build for a given platform. * public/privileged-api/create-test-group.php: (main): Added the support for creating an analysis task along with a group. (commit_sets_from_revision_sets): Added the support for custom roots. Verify the specified uploaded file exists and include it in commit_set_relationships. Because commits and upload files are stored in a different column in commit_set_relationships, this function now stores the information for each row of commit_set_relationships except the commit set ID, which is unknown until the set is created, instead of a commit ID. (ensure_commit_sets): Made the each entry in a commit set a row instead of a commit ID as done. As this format is only by v2 UI and detect-changes.js, we don't add the support for specifying custom roots here. * public/privileged-api/upload-file.php: (main): Fixed a typo. Also added one more error check. * public/v3/components/custom-analysis-task-configurator.js: Added. The UI for selecting a test, a platform, and a set of revisions, as well as custom roots for a custom A/B testing job. The first set of revision with custom roots is referred as "baseline", and the second configuration is referred as "comparison" in this class. (CustomAnalysisTaskConfigurator): (CustomAnalysisTaskConfigurator.prototype.tests): Added. (CustomAnalysisTaskConfigurator.prototype.platform): Added. (CustomAnalysisTaskConfigurator.prototype.commitSets): Added. Returns a pair of baseline and comparsion if both have been configured by the user. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): Added. (CustomAnalysisTaskConfigurator.prototype._configureComparison): Added. Called when the user is to configu the "comparison" configuration. (CustomAnalysisTaskConfigurator.prototype.render): Added. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Added. Renders the list of top-level tests that can be scheduled by a triggerable. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Added. Renders the list of platforms that can be schedule with the currently selected list of tests by a triggerable. Note that the current UI only lets the user select a single test but the intent is to allow multiple tests to be selected in the near future. (CustomAnalysisTaskConfigurator.prototype._buildCheckboxList): Added. Creates a list of radio boxes to select an item with a callback for each. It automatically sets "selected" class on the selected item. It's used to render both the list of tests and platforms. (CustomAnalysisTaskConfigurator.prototype._updateTriggerable): Added. Finds the triggerable for a given list of tests and platforms. Returns an error when some tests belong to another triggearalbe. (CustomAnalysisTaskConfigurator.prototype._updateRepositoryGroups): Added. Finds a repository group to use when the current triggerable has changed. We try to use the repository group of the same name if there is any, and defaults to the first repository group if there is none. This allows the set of repositories to be specified to more or less persist across different triggerables. For example, if iOS platforms and Mac platforms use two distinct triggerables , and both triggerables have two repository groups: one that only specify the OS and the other that specifies both teh OS and WebKit revision, then this code allows the choice the user had made to specify either just the OS or the OS and WebKit will be preserved when the user switches from an iOS platform to a Mac platform. (CustomAnalysisTaskConfigurator.prototype._updateCommitSetMap): Added. Create a commit set map, the format that TestGroup.createWithTask accepts given "baseline" and "comparison" commit sets. Pretend "comparison" is not set if two sets are identical since it makes no sense to schedule an A/B testing job when A and B are identical. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Added. Creates a commit set using the revisions and the csutom roots the user had specified. (CustomAnalysisTaskConfigurator.prototype._renderRepositoryPanes): Added. Renders the pane to specify revisions and custom roots for "baseline" and "comparison". (CustomAnalysisTaskConfigurator.prototype._renderBaselineRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._renderComparisonRevisionTable): Added. (CustomAnalysisTaskConfigurator.prototype._optionalRepositoryList): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Added. Creates a table for specifying revisions and custom roots along with a list of repository groups to pick. The set of repositories and custom roots are shown at the all if all repository groups require them. Otherwise, they are grouped at the bottom as optional. (CustomAnalysisTaskConfigurator.prototype._buildRepositoryGroupList): Added. (CustomAnalysisTaskConfigurator.prototype._selectRepositoryGroup): Added. (CustomAnalysisTaskConfigurator.prototype._buildRevisionInput): Added. Creates an input element to specify a revision for a given repository. Autofills it with the latest commit for the currently selected platform if the user had not modified the field by the time the revisions are fetched. (CustomAnalysisTaskConfigurator.htmlTemplate): Added. (CustomAnalysisTaskConfigurator.cssTemplate): Added. * public/v3/components/instant-file-uploader.js: Added. A form to upload a custom darwinup root in "baseline" or "comparison" configurations of CustomAnalysisTaskConfigurator. It's "instant" because it auto-detects when a file to be uploaded had already been uploaded elsewhere by checking its SHA-256 hash. (InstantFileUploader): (InstantFileUploader.prototype.hasFileToUpload): Added. (InstantFileUploader.prototype.uploadedFiles): Added. (InstantFileUploader.prototype.addUploadedFile): Added. It's called on the uploader for "comparison" configuration when the uploader for "baseline" configuration dipsatches "uploadedFile" action to automatically mirror the newly uploaded custom root to "comparision" configuration. (InstantFileUploader.prototype.didConstructShadowTree): Added. (InstantFileUploader.prototype.render): Added. (InstantFileUploader.prototype._renderUploadedFiles): Added. Renders the list of the uploaded files. (InstantFileUploader.prototype._renderPreuploadFiles): Added. Renders the list of the files to be uploaded with a progress bar. (InstantFileUploader.prototype._updateUploadStatus): Added. Updates the progress bar for uploading the file. (InstantFileUploader.prototype._formatUploadError): Added. (InstantFileUploader.prototype._didFileInputChange): Added. Called when the user picks a file to uploaded on the input element. Fetch the meta data for the uploaded file with the same SHA-256 hash if there is any, and start uploading the file if there isn't one. (InstantFileUploader.prototype._removeUploadedFile): Added. (InstantFileUploader.prototype._didUploadFile): Added. Move a file from the list of files to be uploaded to the list of uploaded files. (InstantFileUploader.htmlTemplate): Added. (InstantFileUploader.cssTemplate): Added. * public/v3/index.html: * public/v3/models/analysis-task.js: (AnalysisTask): Made platform and metric optional as it is now. (AnalysisTask.findByPlatformAndMetric): Skip analysis tasks without a platform or a metric. (AnalysisTask.prototype.isCustom): Added. Returns true for a custom analysis task. (AnalysisTask.fetchRelatedTasks): Skip custom analysis tasks. (AnalysisTask._constructAnalysisTasksFromRawData): Construct analysis tasks even if they were missing a metric or a platform instead of silently skipping them. * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Construct uploaded file objects returned by /api/build-requests. * public/v3/models/commit-log.js: (CommitLog.fetchLatestCommitForPlatform): Added. * public/v3/models/commit-set.js: (CommitSet): Added this._customRoots. (CommitSet.prototype.customRoots): Returns this._customRoots. (CommitSet.prototype.equals): Returns false when the set of custom roots are not equal. (CommitSet.areCustomRootsEqual): Added. (CustomCommitSet): (CustomCommitSet.prototype.equals): Added. (CustomCommitSet.prototype.customRoots): Added. (CustomCommitSet.prototype.addCustomRoot): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Store fileUploadSizeLimit in the manifest as UploadedFile.fileUploadSizeLimit. This allows a file size check in the client size instead of uploading it to the server and receiving an error. * public/v3/models/metric.js: (Metric.formatTime): Moved from ChartPaneStatusView to be also used by InstantFileUploader._renderUploadedFiles. * public/v3/models/test-group.js: (TestGroup.prototype.createWithTask): Added. (TestGroup.prototype.createAndRefetchTestGroups): (TestGroup.prototype._revisionSetsFromCommitSets): Added. Extracted from createAndRefetchTestGroups. (TestGroup.prototype._fetchTestGroupsForTask): Added. Extracted from createAndRefetchTestGroups. * public/v3/models/triggerable.js: (Triggerable.triggerablePlatformsForTests): Added. (Triggerable.sortByNamePreferringSmallerRepositories): Added. * public/v3/models/uploaded-file.js: (UploadedFile.prototype.createdAt): Added. (UploadedFile.prototype.filename): Added. (UploadedFile.prototype.author): Added. (UploadedFile.prototype.size): Added. (UploadedFile.uploadFile): Added a client-side check for the file size using UploadedFile.fileUploadSizeLimit. (UploadedFile.fetchUnloadedFileWithIdenticalHash): Ditto. Also fixed a bug that 404 was resulting in a rejected promise instead of a resolved promise with null. * public/v3/pages/analysis-category-page.js: (AnalysisCategoryPage.prototype._reconstructTaskList): Modernized the code. Added the support for platform and metric being null for some analysis tasks. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._didFetchTask): Don't fetch the measurement set or create a chart for custom tasks. (AnalysisTaskPage.prototype.render): Don't display the charts or the stacking table for custom tasks. (AnalysisTaskPage.prototype._renderTaskNameAndStatus): Don't try to show the full test name for custom tasks since it's not associated with exactly one pair. * public/v3/pages/chart-pane-status-view.js: (ChartPaneStatusView.prototype._renderBuildRevisionTable): (ChartPaneStatusView.prototype._formatTime): Moved to Metric.formatTime. * public/v3/pages/chart-pane.js: (ChartPane.prototype._analyzeRange): Set inProgress to true to hide CustomAnalysisTaskConfigurator in CreateAnalysisTaskPage when creating a non-custom analysis task for a specific range. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage): This page now shows CustomAnalysisTaskConfigurator by default, and lets a user create a custom analysis task by picking a test, a platform, and a set of revisions and custom darwinup roots. (CreateAnalysisTaskPage.prototype.updateFromSerializedState): Show a message when inProgress is set. This is the old behavior of this page. (CreateAnalysisTaskPage.prototype.didConstructShadowTree): Added. (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): Added. (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): Added. Hides CustomAnalysisTaskConfigurator and the select element to specify the numebr of iterations when a message is set. (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * public/v3/pages/page-router.js: (PageRouter.prototype.route): Always enqueue the page to re-render when the route has changed. * server-tests/api-build-requests-tests.js: Updated test cases now that the response contains a list of uploaded files associated with build requests. * server-tests/privileged-api-create-test-group-tests.js: Added test cases for creating a custom analysis task and a test group with custom roots. * server-tests/resources/mock-data.js: (MockData.addMockData): Updated the mock data to satisfy new constraint on analysis-tasks table. * tools/js/remote.js: Include global.FormData from form-data.js. * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated the mock response. * unit-tests/buildbot-syncer-tests.js: (createSampleBuildRequest): Ditto. * unit-tests/test-groups-tests.js: (sampleTestGroup): Ditto. Canonical link: https://commits.webkit.org/187631@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215205 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-10 21:45:11 +00:00
});
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
it('should create a custom test group for an existing custom analysis task', () => {
let firstResult;
let secondResult;
let webkit;
let test = MockData.someTestId();
return addTriggerableAndCreateTask('some task').then(() => {
webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
return PrivilegedAPI.sendRequest('create-test-group',
{name: 'test1', taskName: 'other task', platform: MockData.somePlatformId(), test, revisionSets});
}).then((result) => {
firstResult = result;
Add the support for scheduling a A/B testing with a patch. https://bugs.webkit.org/show_bug.cgi?id=171209 Reviewed by Chris Dumez. Added the support for creating a custom test group with a patch applied. First, each repository in a repository group has a boolean indicating whether a given repository can have a patch applied or not. When any configuration in a test group contains a patch, we create build requests without a test specified in order to "build" those patches. These build requests have negative order numbers to differentiate them from regular build requests. We can't simply build ones with patches since there could be differences in SDK, build options, etc... when patches are applied. The JSON format for commit sets returned by /api/build-requests have been changed from using an array of commit IDs to an array of dictionaries indicate commit and acceptsPatch boolean. /api/update-triggerable now uses a dictionary with two keys: repository and acceptsPatch to specify a set of repositories associated with a repository group, and /privileged-api-create-test-group uses a dictionary with two keys: revision and patch instead of a revision string to specify commit sets. Furthermore, the syncing script's configuration have been updated to use a dictionary of repository names to an options dictionary instead of an array of repositories names. For now, the only supported option is acceptsPatch but will be extended when we add the support for rolling back system components. e.g. {"WebKit": {acceptsPatch: true}, "macOS": {}} instead of ["WebKit", "macOS"] On the UI side, InstantFileUploader has been changed to accept only one file by default, and added a new method allowMultipleFiles() to allow multiple files to be selected for custom roots. Also replaced the input element with type=file by a button with a custom label to show labels such as "Apply a patch" or "Add a new root" instead of the generic label like "choose a file". * init-database.sql: Added trigrepo_accepts_patch to triggerable_repositories to indicate whether a given repository can have a patch applied or not. Made request_test optional in build_requests for when a build request is created to build patches. Such a build request have a negative request_order. Updated the related constraints accordingly. * public/admin/triggerables.php: Added the support for updating whether a given repository can have a patch applied in each repository group. Only show the repositories in the repository group for this purpose since there is no way to accept a patch on a repository without it being a part of the group. (generate_repository_form): Now takes the markup for checkboxes instead of generating one itself. (generate_repository_checkboxes): Now takes an array of repositories to generate checkboxes. The checkbox is shown when the repository ID exists as a key in this array, and is checked when its value is true. The new capability to skip repositories not in the array is used to hide repositories not associated with the group in the list of checkboxes to indicate a repository accepts a patch. * public/api/update-triggerable.php: (main): Now updates the description and acceptsRoots states of each repository group, and sets acceptsPatch boolean for each repository in the group if set in the update. (validate_repository_groups): Use a reference to $repository_groups in order to set repository_id_list, which contains an array of repository IDs to find the existing repository group that matches the set via RepositoryGroupFinder's find_by_repositories. Also added a various validations for acceptsRoots, a dictionary specifying repository and acceptsPatch. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Instead of returning an array of commit IDs as "commits", it now returns an array of dictionaries with "commit" and "patch" keys specifying the commit ID and the patch file's ID respectively as "revisionItems". (BuildRequestsFetcher::add_uploaded_file): Added. Extracted from fetch_commits_for_set_if_needed. Used to add either a patch file or a custom root file in the list of uploaded files in the result. * public/include/manifest-generator.php: (fetch_triggerables): Each element in repository group's "repositories" field is now an array of dictionaries with "repository" and "acceptsPatch" as keys. * public/include/repository-group-finder.php: (RepositoryGroupFinder::__construct): Added a map for boolean indicating whether a given repository group allows a patch on a repository. Used in /privileged-api/create-test-group. (RepositoryGroupFinder::accepts_patch): Added. (RepositoryGroupFinder::populate_map): Build up the map for acceptsPatch boolean per repository per group. * public/privileged-api/create-test-group.php: (main): Fixed a bug that we were not explicitly checking for a duplicate test group name (with a test). Create build requests to "build" patches if there is any patch file specified. (commit_sets_from_revision_sets): Updated to take a dictionary with "revision" and "patch" as keys to specify a revision and a patch if any instead of just a revision string for each repository. Also validate that each repository is allowed to have a patch once the repository group has been found for the set of repositories. (ensure_commit_sets): * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator): Added _patchUploaders as an instance variable, which is a dictionary of configuration names to a map of InstantFileUploader's used to upload a patch. Also renamed _fileUploaders to _customRootUploaders for clarity. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree.createRootUploader): Added. (CustomAnalysisTaskConfigurator.prototype.didConstructShadowTree): (CustomAnalysisTaskConfigurator.prototype._ensurePatchUploader): Added. Creates an instant file uploader for patches. We only allow a single patch per repository. (CustomAnalysisTaskConfigurator.prototype._computeCommitSet): Include a patch in the commit set as needed. (CustomAnalysisTaskConfigurator.prototype._buildRevisionTable): Show the patch file uploader for repositories which can have patches in the current repository group. (CustomAnalysisTaskConfigurator.cssTemplate): Show borders between every rows instead of just between tbody's now that each row can have a patch file uploader. * public/v3/components/instant-file-uploader.js: (InstantFileUploader): Added _fileInput and _allowMultipleFiles as instance variables. We now show a button in the UI instead of an input with type=file. _fileInput is a hidden input with type=file used inside a click event of the button to let the user pick a file. (InstantFileUploader.prototype.allowMultipleFiles): Added. Allows this instance to accept multiple files. (InstantFileUploader.prototype.didConstructShadowTree): Synthetically click on the hidden input element when the newly added button element is clicked to open the browser's file picker. (InstantFileUploader.prototype.render): Hide the button to add a file if this instance can only select one file and there is already some file being uploaded in this instance. (InstantFileUploader.htmlTemplate): Replaced the input element with type=file with a button. Its label comes from the default slot content. * public/v3/models/build-request.js: (BuildRequest): Made the test optional. (BuildRequest.prototype.isBuild): Returns true if this is a build request for building a patch. (BuildRequest.prototype.isTest): Returns true if this is a build request for running tests. (BuildRequest.constructBuildRequestsFromData): Create each commit log here instead of relying on CommitSet's constructor to construct its commit logs. Also updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/commit-set.js: (CommitSet): Made _repositoryToCommitMap a real Map object. Also added _repositoryToPatchMap. Also got rid of the code to instantiate commit logs since that's now done in BuildRequest.constructBuildRequestsFromData. (CommitSet.prototype.commitForRepository): (CommitSet.prototype.revisionForRepository): (CommitSet.prototype.patchForRepository): Added. (CommitSet.prototype.latestCommitTime): Modernized the code. (CommitSet.prototype.equals): Modernized the code. Also added the check for patches. (MeasurementCommitSet): Updated per the change to make _repositoryToCommitMap a real Map. (CustomCommitSet.prototype.setRevisionForRepository): (CustomCommitSet.prototype.equals): Added the check for patches. (CustomCommitSet.prototype.revisionForRepository): (CustomCommitSet.prototype.patchForRepository): Added. * public/v3/models/manifest.js: (Manifest._didFetchManifest): Updated per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * public/v3/models/repository.js: (Repository.prototype.ownerId): Renamed from owner for clarity. * public/v3/models/test-group.js: (TestGroup): Modernized the code by using LazilyEvaluatedFunction. Removed _requestsAreInOrder since it's not necessary anymore with LazilyEvaluatedFunction. (TestGroup.prototype.addBuildRequest): (TestGroup.prototype.test): Use the last build request's test since the first few requests could be requests to build patches. (TestGroup.prototype.platform): Ditto. (TestGroup.prototype._lastRequest): Added. (TestGroup.prototype._orderedBuildRequests): Added. (TestGroup.prototype.repetitionCount): Only count the build requests for testing (skipping any requests to build patches). (TestGroup.prototype.requestedCommitSets): Simply call _computeRequestedCommitSetsLazily. (TestGroup.prototype._computeRequestedCommitSets): Extracted from requestedCommitSets. (TestGroup.prototype.requestsForCommitSet): (TestGroup.prototype.labelForCommitSet): Rewritten. Just compute the label here instead of relying on _commitSetToLabel since requestedSets is always of the length two at the moment. (TestGroup._revisionSetsFromCommitSets): Specify both the revision and the patch in the revision set. * public/v3/models/triggerable.js: (TriggerableRepositoryGroup): Added _patchAcceptingSet as an instance variable. Use sortByNamePreferringOnesWithURL to sort repositories instead of simple sortByName. (TriggerableRepositoryGroup.prototype.accepts): Added checks for the custom roots and patches. (TriggerableRepositoryGroup.prototype.acceptsPatchForRepository): Added. * server-tests/api-build-requests-tests.js: Updated the test cases per the replacement of an array of commit IDs by an array of dictionaries with commit and patch properties. * server-tests/api-manifest-tests.js: Updated the test case per the name of Repository's owner to ownerId. * server-tests/api-update-triggerable.js: Updated the test case per the name of Repository's owner to ownerId, and added a test case for updating whether a given repository group allows custom roots as well as patches on repositories via /api/update-triggerable. (.updateWithOSXRepositoryGroup): Updated the sample syncing script configuration per the format change. (.refetchManifest): Added. * server-tests/privileged-api-create-test-group-tests.js: Updated per the syncing script configuration format change. Also added a test for creating a test group with a duplicate name, which is expected to fail with DuplicateTestGroupName, and creating a test group with a patch both when it's allowed and when it's not allowed in the matching repository group. (.addTriggerableAndCreateTask): Updated per the format change. * server-tests/resources/mock-data.js: (MockData.addEmptyTriggerable): Added a metric and its configuration to make it appear in the manifest file. The new test case in api-update-triggerable.js requires this. (MockData.mockTestSyncConfigWithSingleBuilder): Updated per the syncing script configuration format change. (MockData.mockTestSyncConfigWithTwoBuilders): Ditto. * server-tests/tools-buildbot-triggerable-tests.js: Removed the useless assertions about test configurations, and added assertions about custom roots and patches in the test case for updateTriggerables. * tools/js/buildbot-syncer.js: (BuildbotSyncer._parseRepositoryGroup): Made each assertion explicitly refer to the specific repository group to make it more user friendly. Now each repository group uses a dictionary of repository names to its options in the syncing script configurations. When parsed, we insert it as an array of dictionaries with repository ID and acceptsPatch boolean specified separately since this is the format /api/update-triggerable expects. * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype.updateTriggerable): * unit-tests/build-request-tests.js: (sampleBuildRequestData): Updated per the commit sets format change in /api/build-requests. * unit-tests/buildbot-syncer-tests.js: Updated the existing tests per various format changes and added a couple of new test cases for the syncing script's configuration validation. (sampleiOSConfig): (smallConfiguration): (createSampleBuildRequest): * unit-tests/resources/mock-v3-models.js: (MockModels.inject): Updated per the repository group format change. * unit-tests/test-groups-tests.js: (sampleTestGroup): Updated per the commit sets format change in /api/build-requests. Canonical link: https://commits.webkit.org/188378@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-30 18:11:47 +00:00
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '192736'}}];
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
return PrivilegedAPI.sendRequest('create-test-group',
{name: 'test2', task: result['taskId'], platform: MockData.otherPlatformId(), test, revisionSets, repetitionCount: 2});
}).then((result) => {
secondResult = result;
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
// pg funtions will return all data as string, so we need make sure that we compare as int
assert.strictEqual(parseInt(firstResult['taskId']), parseInt(secondResult['taskId']));
Add API to upload a patched build for a custom A/B testing https://bugs.webkit.org/show_bug.cgi?id=171956 Reviewed by Chris Dumez. Added /api/upload-root to upload a root file, the build product of a patch associated with a commit set. Extracted more functions out of privileged-api/upload-file.php to uploaded-file-helpers.php to share code with /api/upload-root.php. * public/api/upload-root.php: Added. (main): (compute_commit_set_items_to_update): Find the list of commit set items to associate this root with. A root can be associated with multiple repositories and there fore commit set items; e.g. if a software is built from multiple repositories and there is a patch associated with one of them, the built product must be associated with all those repositories. * public/include/build-requests-fetcher.php: (BuildRequestsFetcher::fetch_commits_for_set_if_needed): Include the root file is there is one. * public/include/json-header.php: (validate_arguments): Added the support for validating json string. (verify_slave): Return the slave ID found. * public/include/uploaded-file-helpers.php: (validate_uploaded_file): Extracted from /privileged-api/upload-file to be shared with /api/upload-root. (query_total_file_size): Ditto. (create_uploaded_file_from_form_data): Ditto. (upload_file_in_transaction): Ditto. Takes a lambda to do the extra work inside the transaction. * public/privileged-api/upload-file.php: (main): * public/v3/models/build-request.js: (BuildRequest.constructBuildRequestsFromData): Resolve the rootFIle of each commit set item. * public/v3/models/commit-set.js: (CommitSet): Added _repositoryToRootMap and _allRootFiles as instance variables. (CommitSet.prototype.updateSingleton): Added. Previously, each commit set's states never changed after its creation. After this patch, each item can be newly associated with a root so we must update its _repositoryToRootMap and _allRootFiles. For simplicity, we update all states. (CommitSet.prototype._updateFromObject): Extracted from the constructor. (CommitSet.prototype.allRootFiles): Added. Includes custom roots and roots created for patches. (CommitSet.prototype.rootForRepository): Added. (CommitSet.prototype.equals): Fixed the bug that we were comparing _repositoryToPatchMap to _repositoryToCommitMap, and added a check for _repositoryToRootMap. * public/v3/models/test-group.js: (TestGroup.prototype.task): Added. (TestGroup.createWithTask): (TestGroup.createWithCustomConfiguration): (TestGroup.createAndRefetchTestGroups): (TestGroup._fetchTestGroupsForTask): Deleted. Now fetchForTask takes a boolean argument: ignoreCache. (TestGroup.findAllByTask): Added. (TestGroup.fetchForTask): Renamed from fetchByTask. * public/v3/pages/analysis-task-page.js: (AnalysisTaskPage.prototype._fetchRelatedInfoForTaskId): * server-tests/api-build-requests-tests.js: * server-tests/api-upload-root-tests.js: Added. Added tests for /api/upload-root. (makeReport): Added. (addSlaveAndCreateRootFile): Added. (createTestGroupWihPatch): Added. * server-tests/privileged-api-create-test-group-tests.js: * server-tests/resources/mock-data.js: (MockData.sharedRepositoryId): Added. (MockData.addMockData): Added "Shared" repository along with commits. Canonical link: https://commits.webkit.org/188885@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@216647 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-05-11 02:32:00 +00:00
return Promise.all([AnalysisTask.fetchById(result['taskId']), TestGroup.fetchForTask(result['taskId'], true)]);
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
}).then((result) => {
const [analysisTask, testGroups] = result;
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(analysisTask.name(), 'other task');
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups.length, 2);
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
TestGroup.sortByName(testGroups);
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(testGroups[0].name(), 'test1');
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(testGroups[0].initialRepetitionCount(), 1);
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
let requests = testGroups[0].buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 2);
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
let set0 = requests[0].commitSet();
let set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.deepStrictEqual(set0.repositories(), [webkit]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(set1.repositories(), [webkit]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '191623');
assert.strictEqual(testGroups[1].name(), 'test2');
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
assert.strictEqual(testGroups[1].initialRepetitionCount(), 2);
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
requests = testGroups[1].buildRequests();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.strictEqual(requests.length, 4);
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
set0 = requests[0].commitSet();
set1 = requests[1].commitSet();
Make server test run with new node version https://bugs.webkit.org/show_bug.cgi?id=222463 Change assert.equal to assert.strictEqual since node deprecated this API Add a new function console.assert to make sure it will throw an error Reviewed by Ryosuke Niwa. * public/v3/models/repository.js: (Repository.sortByNamePreferringOnesWithURL): (Repository): * server-tests/admin-platforms-tests.js: * server-tests/admin-reprocess-report-tests.js: * server-tests/api-build-requests-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-commits-tests.js: (assertCommitIsSameAsOneSubmitted): (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-manifest-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/api-measurement-set-tests.js: * server-tests/api-report-commits-tests.js: (emptyReport.then): (async initialReportCommits): (async setUpTestsWithExpectedStatus): (async testWithExpectedFailure): * server-tests/api-report-tests.js: (emptyReport.then): * server-tests/api-test-groups.js: * server-tests/api-update-triggerable-tests.js: (update.then): (then.response.db.selectAll.string_appeared_here.then): (then.db.selectAll.string_appeared_here.then): (then.Manifest.fetch.then): * server-tests/api-upload-root-tests.js: * server-tests/api-uploaded-file-tests.js: (return.TestServer.remoteAPI.getJSON.string_appeared_here.then): * server-tests/privileged-api-add-build-requests-tests.js: (async createAnalysisTask): (const.commitSet.of.updatedGroups.0.requestedCommitSets): * server-tests/privileged-api-create-analysis-task-tests.js: * server-tests/privileged-api-create-test-group-tests.js: (createAnalysisTask): (return.createAnalysisTask.string_appeared_here.then): (return.addTriggerableAndCreateTask.string_appeared_here.then): (string_appeared_here.then): * server-tests/privileged-api-upate-run-status.js: * server-tests/privileged-api-update-test-group-tests.js: (async createAnalysisTask): * server-tests/privileged-api-upload-file-tests.js: (TestServer.testConfig.uploadFileLimitInMB.1.then): * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/temporary-file.js: (TemporaryFile.makeTemporaryFileOfSizeInMB): * server-tests/tools-buildbot-triggerable-tests.js: (assertRequestAndResolve): (then.refetchManifest.then): (getBuilderNameToIDMapPromise.then): * server-tests/tools-os-build-fetcher-tests.js: * server-tests/tools-sync-buildbot-integration-tests.js: (assertAndResolveRequest): * tools/js/assert-override.js: Added. (makeConsoleAssertThrow): * tools/js/buildbot-syncer.js: (BuildbotBuildEntry.prototype.initialize): (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): (BuildbotBuildEntry): (BuildbotSyncer.prototype.scheduleRequest): (BuildbotSyncer._loadConfig): (BuildbotSyncer._parseRepositoryGroup): (BuildbotSyncer._validateAndMergeConfig): (BuildbotSyncer): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable.prototype._validateRequests): * tools/js/markup-component.js: (const.MarkupDocument.new.MarkupDocument.prototype._idForClone): (const.MarkupDocument.new.MarkupDocument.prototype.markup): * tools/js/os-build-fetcher.js: (prototype._addOwnedCommitsForBuild): * tools/js/remote.js: (NodeRemoteAPI.prototype.configure): * tools/js/v3-models.js: * unit-tests/commit-set-tests.js: Canonical link: https://commits.webkit.org/234972@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274036 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-03-06 14:31:56 +00:00
assert.deepStrictEqual(requests[2].commitSet(), set0);
assert.deepStrictEqual(requests[3].commitSet(), set1);
assert.deepStrictEqual(set0.repositories(), [webkit]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(set1.repositories(), [webkit]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '192736');
Add the support for creating a custom test group in the analysis task page Make it possible to create more custom test groups in the analysis task page https://bugs.webkit.org/show_bug.cgi?id=171138 Rubber-stamped by Chris Dumez. Extracted CustomConfigurationTestGroupForm out of CreateAnalysisTaskPage and added it to AnalysisTaskPage inside AnalysisTaskConfiguratorPane. This allows configuration of a new test group within a custom analysis task. * public/privileged-api/create-test-group.php: (main): Fixed the bug that the triggerable wasn't resolved when creating a test group in a custom analysis task. * public/v3/components/custom-analysis-task-configurator.js: (CustomAnalysisTaskConfigurator.prototype.selectTests): Added. Used by CustomConfigurationTestGroupForm's setConfigurations. (CustomAnalysisTaskConfigurator.prototype.selectPlatform): Ditto. (CustomAnalysisTaskConfigurator.prototype.setCommitSets): Ditto. (CustomAnalysisTaskConfigurator.prototype._setUploadedFilesIfEmpty): Added. (CustomAnalysisTaskConfigurator.prototype._revisionMapFromCommitSet): Added. (CustomAnalysisTaskConfigurator.prototype.render): Update the currently selected platforms and tests now that they can be set externally via selectTests and selectPlatform. (CustomAnalysisTaskConfigurator.prototype._renderTriggerableTests): Return the result of _renderRadioButtonList so that the caller can update the currently selected tests without having to reconstruct the list. (CustomAnalysisTaskConfigurator.prototype._renderTriggerablePlatforms): Ditto. (CustomAnalysisTaskConfigurator.prototype._renderRadioButtonList): Renamed from _buildCheckboxList. Now returns a function which updates the currently selected items. We still pretend that multiple items can be selected to make it future-proof. * public/v3/components/custom-configuration-test-group-form.js: Added. (CustomConfigurationTestGroupForm): Added. Inherits from TestGroupForm. Extracted from CreateAnalysisTaskPage. (CustomConfigurationTestGroupForm.prototype.setHasTask): Added. (CustomConfigurationTestGroupForm.prototype.hasCommitSets): Added. (CustomConfigurationTestGroupForm.prototype.setConfigurations): Added. Used by AnalysisTaskConfiguratorPane to set the default configuration to what the latest test group used. (CustomConfigurationTestGroupForm.prototype.startTesting): Added. Dispatches "startTesting" action with platform, test, taskName in addition to what CustomizedTestGroupForm emits. (CustomConfigurationTestGroupForm.prototype.didConstructShadowTree): Added. (CustomConfigurationTestGroupForm.prototype.render): Added. (CustomConfigurationTestGroupForm.prototype._updateTestGroupName): Added. (CustomConfigurationTestGroupForm.cssTemplate): Added. (CustomConfigurationTestGroupForm.htmlTemplate): Added. * public/v3/components/test-group-form.js: (TestGroupForm.cssTemplate): Make the form display: block. * public/v3/index.html: * public/v3/models/test-group.js: (TestGroup.prototype.test): Added. (TestGroup.prototype.platform): Added. (TestGroup.createWithCustomConfiguration): Added. Creates a custom test group with an existing analysis task. * public/v3/models/uploaded-file.js: (UploadedFile): Fixed a bug that _deletedAt was set to a Date object even when object.deletedAt is null. * public/v3/pages/analysis-task-page.js: (AnalysisTaskConfiguratorPane): Added. (AnalysisTaskConfiguratorPane.prototype.didConstructShadowTree): Added. Dispatch createCustomTestGroup action in turn when receiving startTesting from CustomConfigurationTestGroupForm. (AnalysisTaskConfiguratorPane.prototype.setTestGroups): Added. (AnalysisTaskConfiguratorPane.prototype.render): Added. (AnalysisTaskConfiguratorPane.htmlTemplate): Added. We override this instead of formContent to display the "Start" button at the end instead of at the beginnning. (AnalysisTaskConfiguratorPane.cssTemplate): Added. (AnalysisTaskPage.prototype.didConstructShadowTree): Listen to createCustomTestGroup. (AnalysisTaskPage.prototype.render): Hide AnalysisTaskConfiguratorPane when the analysis task is not custom. (AnalysisTaskPage.prototype._showTestGroup): Let AnalysisTaskConfiguratorPane know of the current test group so that it can update the default configuration if the user hasn't modified yet. (AnalysisTaskPage.prototype._createCustomTestGroup): Added. * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.didConstructShadowTree): (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup): (CreateAnalysisTaskPage.prototype.render): (CreateAnalysisTaskPage.prototype._renderMessage): (CreateAnalysisTaskPage.htmlTemplate): (CreateAnalysisTaskPage.cssTemplate): * server-tests/privileged-api-create-test-group-tests.js: Added a test case for creating a custom test group for an existing analysis task. * server-tests/resources/mock-data.js: (MockData.otherPlatformId): Added. (MockData.addMockData): Added a test configuration for otherPlatformId. Canonical link: https://commits.webkit.org/188040@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215636 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-04-21 21:08:36 +00:00
});
});
[perf dashboard] Add sequential mode for perf dashboard A/B testing. https://bugs.webkit.org/show_bug.cgi?id=223886 Reviewed by Ryosuke Niwa. Add 'testgroup_repetition_type' to 'analysis_test_groups' table to store repetition type which defaults to 'alternating'. Add support to schedule 'sequential' A/B testing which schedules all iterations for the first configuration (A) before the iterations for the second configuration (B). Before this patch, all A/B testing alternated between two different configurations for each iteration. Add retry logic for 'sequential' A/B testing. Update syncing script to not proceed with next configuration the current configuration successfully completed more iterations than the initially requested or retries exceeded the list. Fix a potential race in syncing script that 'Promise.all' may cause test group not scheduled in order. * browser-tests/test-group-form-tests.js: Updated and added unit tests. * init-database.sql: Added 'testgroup_repetition_type' field to 'analysis_test_groups' table. * migrate-database.sql: Updated migration script to conditionally add 'testgroup_repetition_type' field. * public/api/test-groups.php: Added 'repetitionType' field in api return value. * public/include/build-requests-fetcher.php: Fetching build requests under a triggerable should include test group with all test groups finished but still need retry. This will allow syncing script to resolve the build request by ID and block other request to be scheduled on the builder. * public/include/commit-sets-helpers.php: Added code to create build requests based on test mode. * public/privileged-api/add-build-requests.php: Added support to only add build requests to one commit set. * public/privileged-api/create-analysis-task.php: Added 'repetitionType' support. * public/privileged-api/create-test-group.php: Added 'repetitionType' support. * public/privileged-api/update-test-group.php: * public/shared/common-component-base.js: * public/v3/components/custom-configuration-test-group-form.js: Added 'repetitionType' support. (CustomConfigurationTestGroupForm.prototype.startTesting): * public/v3/components/customizable-test-group-form.js: (CustomizableTestGroupForm.prototype.startTesting): * public/v3/components/test-group-form.js: (TestGroupForm.prototype.startTesting): * public/v3/models/analysis-task.js: (AnalysisTask.async create): (AnalysisTask): * public/v3/models/build-request.js: (BuildRequest): (BuildRequest.prototype.updateSingleton): Allow build request order to be updated. * public/v3/models/test-group.js: (TestGroup): (TestGroup.prototype.updateSingleton): (TestGroup.prototype.repetitionType): (TestGroup.prototype.repetitionCountForCommitSet): Refactored `repetitionCount` since repetition may be different between configurations. (TestGroup.prototype.hasRetries): A helper function to tell if a test group has retry runs. (TestGroup.prototype.additionalRepetitionNeededToReachInitialRepetitionCount): (TestGroup.prototype.successfulTestCount): (TestGroup.prototype.isFirstTestRequest): (TestGroup.prototype.precedingBuildRequest): (TestGroup.prototype.retryCountForCommitSet): (TestGroup.prototype.retryCountsAreSameForAllCommitSets): (TestGroup.prototype.async _createAlternatingRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.async _createSequentialRetriesForTestGroup): Moved from 'retry-failed-build-requests' module. (TestGroup.prototype.async cancelPendingRequests): (TestGroup.prototype.async didSendNotification): (TestGroup.prototype.async addMoreBuildRequests): (TestGroup.async createWithTask): (TestGroup.async createWithCustomConfiguration): (TestGroup.async createAndRefetchTestGroups): (TestGroup.prototype.async scheduleMoreRequestsOrClearFlag): Moved from 'retry-failed-build-requests' module. (TestGroup.async fetchById): Added support to fetch test group by ID. * public/v3/pages/analysis-task-page.js: (AnalysisTaskChartPane.prototype.didConstructShadowTree): (AnalysisTaskResultsPane.prototype.didConstructShadowTree): (AnalysisTaskConfiguratorPane.prototype.setTestGroups): (AnalysisTaskConfiguratorPane): (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree): (AnalysisTaskTestGroupPane.prototype._renderCurrentTestGroup): (AnalysisTaskTestGroupPane.prototype._retrySummary): (AnalysisTaskPage.prototype.didConstructShadowTree): (AnalysisTaskPage.prototype.async _retryCurrentTestGroup): (AnalysisTaskPage.prototype.async _bisectCurrentTestGroup): (AnalysisTaskPage.set const): (AnalysisTaskPage.prototype.async _createTestGroupAfterVerifyingCommitSetList): (AnalysisTaskPage.prototype.async _createCustomTestGroup): * public/v3/pages/chart-pane.js: (ChartPane.prototype.didConstructShadowTree): (ChartPane.prototype.async _analyzeRange): * public/v3/pages/create-analysis-task-page.js: (CreateAnalysisTaskPage.prototype.async _createAnalysisTaskWithGroup): * server-tests/api-build-requests-tests.js: Added a unit test. * server-tests/api-upload-root-tests.js: Updated unit tests. * server-tests/privileged-api-add-build-requests-tests.js: Updated and added unit tests. * server-tests/privileged-api-create-analysis-task-tests.js: Added unit tests. * server-tests/privileged-api-create-test-group-tests.js: Updated and added unit tests. * server-tests/privileged-api-update-test-group-tests.js: Updated and added unit tests. * server-tests/resources/common-operations.js: (async assertThrows): * server-tests/resources/mock-data.js: (MockData.addMockData): (MockData.set addMockBuildRequestsWithRoots): (MockData.set addAnotherMockTestGroup): (MockData.set mockTestSyncConfigWithSingleBuilder): (MockData.sampleBuildData): * server-tests/tools-buildbot-triggerable-tests.js: Updated and added unit tests. * server-tests/tools-sync-buildbot-integration-tests.js: Updated unit tests. (async createTestGroupWithPatch): (createTestGroupWihOwnedCommit): * tools/js/buildbot-syncer.js: (BuildbotSyncer.prototype.scheduleRequestInGroupIfAvailable): (BuildbotSyncer.prototype.pullBuildbot): * tools/js/buildbot-triggerable.js: (BuildbotTriggerable): (BuildbotTriggerable.prototype.async syncOnce): (BuildbotTriggerable.prototype.async _scheduleRequest): (BuildbotTriggerable.prototype._shouldDeferSequentialTestingRequestWithNewCommitSet): * tools/js/measurement-set-analyzer.js: (MeasurementSetAnalyzer.prototype.async _analyzeMeasurementSet): (MeasurementSetAnalyzer): * tools/js/retry-failed-build-requests.js: Removed. * tools/run-analysis.js: Use updated API to retry. (async analysisLoop): (async processTestGroupMayNeedMoreRequests): * tools/sync-buildbot.js: Added command line option '--max-retry-factor'. * unit-tests/analysis-task-tests.js: Updated unit tests. * unit-tests/buildbot-syncer-tests.js: Updated unit tests. (createSampleBuildRequest): * unit-tests/measurement-set-analyzer-tests.js: * unit-tests/retry-failed-build-requests-tests.js: Moved unit tests to test-group-tests.js. * unit-tests/test-groups-tests.js: Added unit tests from 'retry-failed-build-requests-tests.js'. Canonical link: https://commits.webkit.org/238151@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278072 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2021-05-26 01:23:22 +00:00
it('should create a sequential test group with an analysis task', async () => {
await addTriggerableAndCreateTask('some task');
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
let result = await PrivilegedAPI.sendRequest('create-test-group',
{name: 'test', taskName: 'other task', platform: MockData.somePlatformId(), test: MockData.someTestId(),
needsNotification: true, revisionSets, repetitionType: 'sequential', repetitionCount: 2});
const insertedGroupId = result['testGroupId'];
const [analysisTask, testGroups] = await Promise.all([AnalysisTask.fetchById(result['taskId']), TestGroup.fetchForTask(result['taskId'], true)]);
assert.strictEqual(analysisTask.name(), 'other task');
assert.strictEqual(testGroups.length, 1);
const group = testGroups[0];
assert.strictEqual(group.id(), insertedGroupId);
assert.strictEqual(group.initialRepetitionCount(), 2);
assert.strictEqual(group.repetitionType(), 'sequential')
assert.ok(group.needsNotification());
const requests = group.buildRequests();
assert.strictEqual(requests.length, 4);
assert.strictEqual(requests[0].order(), 0);
assert.strictEqual(requests[1].order(), 1);
assert.strictEqual(requests[2].order(), 2);
assert.strictEqual(requests[3].order(), 3);
assert.strictEqual(group.requestedCommitSets().length, 2)
assert(requests[0].commitSet().equals(requests[1].commitSet()));
assert(requests[2].commitSet().equals(requests[3].commitSet()));
const set0 = requests[0].commitSet();
const set1 = requests[2].commitSet();
assert.deepStrictEqual(set0.repositories(), [webkit]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(set1.repositories(), [webkit]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '191623');
});
it('should create an alternating test group with an analysis task', async () => {
await addTriggerableAndCreateTask('some task');
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
let result = await PrivilegedAPI.sendRequest('create-test-group',
{name: 'test', taskName: 'other task', platform: MockData.somePlatformId(), test: MockData.someTestId(),
needsNotification: true, revisionSets, repetitionType: 'alternating', repetitionCount: 2});
const insertedGroupId = result['testGroupId'];
const [analysisTask, testGroups] = await Promise.all([AnalysisTask.fetchById(result['taskId']), TestGroup.fetchForTask(result['taskId'], true)]);
assert.strictEqual(analysisTask.name(), 'other task');
assert.strictEqual(testGroups.length, 1);
const group = testGroups[0];
assert.strictEqual(group.id(), insertedGroupId);
assert.strictEqual(group.initialRepetitionCount(), 2);
assert.strictEqual(group.repetitionType(), 'alternating')
assert.ok(group.needsNotification());
const requests = group.buildRequests();
assert.strictEqual(requests.length, 4);
assert.strictEqual(requests[0].order(), 0);
assert.strictEqual(requests[1].order(), 1);
assert.strictEqual(requests[2].order(), 2);
assert.strictEqual(requests[3].order(), 3);
assert.strictEqual(group.requestedCommitSets().length, 2)
assert(requests[0].commitSet().equals(requests[2].commitSet()));
assert(requests[1].commitSet().equals(requests[3].commitSet()));
const set0 = requests[0].commitSet();
const set1 = requests[1].commitSet();
assert.deepStrictEqual(set0.repositories(), [webkit]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(set1.repositories(), [webkit]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '191623');
});
it('should create a sequential test group in an existing analysis task', async () => {
const taskId = await addTriggerableAndCreateTask('some task');
const ignoreCache = true;
let [analysisTask, testGroups] = await Promise.all([AnalysisTask.fetchById(taskId), TestGroup.fetchForTask(taskId, ignoreCache)]);
assert.strictEqual(analysisTask.name(), 'some task');
assert.strictEqual(testGroups.length, 0);
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
let result = await PrivilegedAPI.sendRequest('create-test-group',
{name: 'test', task: taskId, platform: MockData.somePlatformId(), test: MockData.someTestId(),
needsNotification: true, revisionSets, repetitionType: 'sequential', repetitionCount: 2});
const insertedGroupId = result['testGroupId'];
[analysisTask, testGroups] = await Promise.all([AnalysisTask.fetchById(result['taskId']), TestGroup.fetchForTask(result['taskId'], ignoreCache)]);
assert.strictEqual(analysisTask.name(), 'some task');
assert.strictEqual(testGroups.length, 1);
const group = testGroups[0];
assert.strictEqual(group.id(), insertedGroupId);
assert.strictEqual(group.initialRepetitionCount(), 2);
assert.strictEqual(group.repetitionType(), 'sequential')
assert.ok(group.needsNotification());
const requests = group.buildRequests();
assert.strictEqual(requests.length, 4);
assert.strictEqual(requests[0].order(), 0);
assert.strictEqual(requests[1].order(), 1);
assert.strictEqual(requests[2].order(), 2);
assert.strictEqual(requests[3].order(), 3);
assert.strictEqual(group.requestedCommitSets().length, 2)
assert(requests[0].commitSet().equals(requests[1].commitSet()));
assert(requests[2].commitSet().equals(requests[3].commitSet()));
const set0 = requests[0].commitSet();
const set1 = requests[2].commitSet();
assert.deepStrictEqual(set0.repositories(), [webkit]);
assert.deepStrictEqual(set0.customRoots(), []);
assert.deepStrictEqual(set1.repositories(), [webkit]);
assert.deepStrictEqual(set1.customRoots(), []);
assert.strictEqual(set0.revisionForRepository(webkit), '191622');
assert.strictEqual(set1.revisionForRepository(webkit), '191623');
});
it('should reject with "InvalidRepetitionType" if repetition type is not "alternating" or "sequential"', async () => {
await addTriggerableAndCreateTask('some task');
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
await assertThrows('InvalidRepetitionType', () => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', taskName: 'other task',
platform: MockData.somePlatformId(), test: MockData.someTestId(),
needsNotification: true, repetitionType: 'invalid-mode', repetitionCount: 2, revisionSets})
});
});
it('should reject with "InvalidRepetitionType" when creating a test group with a bad repetition type for an existing analysis task', async () => {
const taskId = await addTriggerableAndCreateTask('some task');
const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
await assertThrows('InvalidRepetitionType', () => {
return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId,
platform: MockData.somePlatformId(), test: MockData.someTestId(),
needsNotification: true, repetitionType: 'invalid-mode', repetitionCount: 2, revisionSets})
});
});
});