haikuwebkit/Tools/Scripts/merge-result-jsons

63 lines
2.5 KiB
Plaintext
Raw Permalink Normal View History

#!/usr/bin/env python -u
compare-results should be able to parse multiple JSONs at once by merging them https://bugs.webkit.org/show_bug.cgi?id=213685 Reviewed by Saam Barati. This patch improves the compare-results script to enable it to merge JSON output files. This is handy when running scripts by hand where you'll get many a and b JSONs. To do the merging this patch moves the logic of merge-result-jsons into a shared library. compare-results can take multiple files sequentally or by passing the '-a'/'-b' flags multiple times. e.g. `/compare-results -a OG-1.json -a OG-2.json -a OG-3.json -a OG-4.json -b MC-1.json MC-2.json MC-3.json MC-4.json` will behave as you'd expect; combining all the OG JSONs and the MC JSONs before computing the result. Lastly, the benchmark_results script now can handle duplicates of an aggregator in the aggregator list. This is useful because otherwise the logic of merging JSONs is significantly more complicated. * Scripts/compare-results: (getOptions): (main): * Scripts/merge-result-jsons: (readJSONFile): (deepAppend): Deleted. (mergeJSONs): Deleted. * Scripts/webkitpy/benchmark_runner/benchmark_json_merge.py: Copied from Tools/Scripts/merge-result-jsons. (deepAppend): (mergeJSONs): * Scripts/webkitpy/benchmark_runner/benchmark_results.py: (BenchmarkResults._aggregate_results_for_test): (BenchmarkResults._lint_subtest_results): * Scripts/webkitpy/benchmark_runner/benchmark_results_unittest.py: Canonical link: https://commits.webkit.org/226494@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@263629 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-06-27 22:18:41 +00:00
# Copyright (C) 2019-2020 Apple Inc. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# 3. Neither the name of Apple Inc. ("Apple") nor the names of
# its contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
import argparse
import json
compare-results should be able to parse multiple JSONs at once by merging them https://bugs.webkit.org/show_bug.cgi?id=213685 Reviewed by Saam Barati. This patch improves the compare-results script to enable it to merge JSON output files. This is handy when running scripts by hand where you'll get many a and b JSONs. To do the merging this patch moves the logic of merge-result-jsons into a shared library. compare-results can take multiple files sequentally or by passing the '-a'/'-b' flags multiple times. e.g. `/compare-results -a OG-1.json -a OG-2.json -a OG-3.json -a OG-4.json -b MC-1.json MC-2.json MC-3.json MC-4.json` will behave as you'd expect; combining all the OG JSONs and the MC JSONs before computing the result. Lastly, the benchmark_results script now can handle duplicates of an aggregator in the aggregator list. This is useful because otherwise the logic of merging JSONs is significantly more complicated. * Scripts/compare-results: (getOptions): (main): * Scripts/merge-result-jsons: (readJSONFile): (deepAppend): Deleted. (mergeJSONs): Deleted. * Scripts/webkitpy/benchmark_runner/benchmark_json_merge.py: Copied from Tools/Scripts/merge-result-jsons. (deepAppend): (mergeJSONs): * Scripts/webkitpy/benchmark_runner/benchmark_results.py: (BenchmarkResults._aggregate_results_for_test): (BenchmarkResults._lint_subtest_results): * Scripts/webkitpy/benchmark_runner/benchmark_results_unittest.py: Canonical link: https://commits.webkit.org/226494@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@263629 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-06-27 22:18:41 +00:00
from webkitpy.benchmark_runner.benchmark_json_merge import mergeJSONs
def readJSONFile(path):
with open(path, 'r') as contents:
return json.load(contents)
def main():
parser = argparse.ArgumentParser(description="Merge the resulting json files from multiple invocations of the run_benchmark script.")
parser.add_argument("-o",
type=str,
required=False,
help="File to put the merged json into prints to standard out if nothing is passed")
parser.add_argument("jsons",
type=str,
nargs='+',
help="The json files to be merged.")
# parse_args will error on our list of incomming JSON files...
args = parser.parse_args()
result = mergeJSONs(list(map(readJSONFile, args.jsons)))
if args.o:
with open(args.o, 'w') as f:
json.dump(result, f)
else:
print(json.dumps(result))
if __name__ == "__main__":
main()