summaryrefslogtreecommitdiffstats
path: root/tools/auto_bisect
Commit message (Collapse)AuthorAgeFilesLines
* Make failed telemetry test on perf try job red.simonhatch2015-03-132-2/+21
| | | | | | | | | | | | | The bisect RunPerformanceTestAndParseResults function allows flaky runs (since either it's bisecting return codes, or it's only concerned about gathering the metrics). When we run perf tests though, we probably don't want to allow flakes to occur. This patch adds an "allow_flakes" parameter to optionally return a failure code if any run fails. Without patch: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/1109 With patch: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/1117 BUG=467058 Review URL: https://codereview.chromium.org/1004793002 Cr-Commit-Position: refs/heads/master@{#320544}
* Make error message for out-of-order revisions more detailed.qyearsley2015-03-131-2/+4
| | | | | | | | BUG=466737 Review URL: https://codereview.chromium.org/1006613003 Cr-Commit-Position: refs/heads/master@{#320412}
* Make android-chrome bisect work with archives.prasadv2015-03-115-51/+120
| | | | | | | | | BUG=465868 NOTRY=true Review URL: https://codereview.chromium.org/994183002 Cr-Commit-Position: refs/heads/master@{#320142}
* Add android-chrome-perf as builder_type choiceprasadv2015-03-021-1/+2
| | | | | | | | | BUG=463265 NOTRY=true Review URL: https://codereview.chromium.org/973653002 Cr-Commit-Position: refs/heads/master@{#318799}
* Make perf try jobs for win x64 build x64 binaries.prasadv2015-02-285-5/+46
| | | | | | | | | | Perf try jobs running on Win x64 bisect bots should build and run tests on x64 binaries. BUG=462639 Review URL: https://codereview.chromium.org/961193002 Cr-Commit-Position: refs/heads/master@{#318570}
* Tweak logging statements in fetch_build.py.qyearsley2015-02-201-3/+3
| | | | | | | | BUG= Review URL: https://codereview.chromium.org/927943005 Cr-Commit-Position: refs/heads/master@{#317206}
* Fix a couple minor typos in auto_bisect.qyearsley2015-02-174-5/+5
| | | | | | | | | | I just discovered a source code spell checker tool called `codespell`. BUG= Review URL: https://codereview.chromium.org/925703002 Cr-Commit-Position: refs/heads/master@{#316657}
* Add "" as a valid value for --builder_type in bisect_perf_regression.py.qyearsley2015-02-171-2/+3
| | | | | | | | BUG= Review URL: https://codereview.chromium.org/916543002 Cr-Commit-Position: refs/heads/master@{#316630}
* Update the builder name of linux full bisect builder.qyearsley2015-02-122-3/+4
| | | | | | | | | | | This should be submitted after https://codereview.chromium.org/897313003/ is submitted. BUG= Review URL: https://codereview.chromium.org/919463002 Cr-Commit-Position: refs/heads/master@{#316011}
* Fix formatting and printing of help of bisect_perf_regression.py.qyearsley2015-02-102-7/+11
| | | | | | | | BUG=457024 Review URL: https://codereview.chromium.org/916503002 Cr-Commit-Position: refs/heads/master@{#315638}
* Make changes to read target_arch param from config.prasadv2015-02-103-44/+113
| | | | | | | | | BUG=455845 NOTRY=true Review URL: https://codereview.chromium.org/907963003 Cr-Commit-Position: refs/heads/master@{#315622}
* Handling test failures on re-test of good/bad revisions for confidence.robertocn2015-02-051-1/+7
| | | | | | | | BUG=455467 Review URL: https://codereview.chromium.org/872033003 Cr-Commit-Position: refs/heads/master@{#314891}
* Obtain confidence score based off last known good and first known bad ↵robertocn2015-02-033-12/+111
| | | | | | | | | | revision results. BUG=448817 Review URL: https://codereview.chromium.org/850013004 Cr-Commit-Position: refs/heads/master@{#314257}
* Run "git stash" and "git stash apply" before and after a dry test run.qyearsley2015-01-291-1/+18
| | | | | | | | | | | | | | Reason: We don't want to destroy local uncommited changes when running tests. One solution was to mock bisect_utils.GitRun -- doing that caused the course of the run to change, making it so that there were no results (since git log is used to get a list of revisions, even for the dry run). So if we want to solve this problem by mocking git, we should replace it with a fake RunGit that returns different canned values depending on what the input is. The advantage of stash/apply is that its simpler, but it's also a little bit slower than the above way. BUG= Review URL: https://codereview.chromium.org/847393004 Cr-Commit-Position: refs/heads/master@{#313650}
* Fixing typo in the docstring for bisect_perf_regression.pyrobertocn2015-01-281-1/+1
| | | | | | | | NOTRY=True Review URL: https://codereview.chromium.org/873313010 Cr-Commit-Position: refs/heads/master@{#313607}
* Use builder_type when requesting/fetching builds, and add support for full ↵qyearsley2015-01-285-234/+171
| | | | | | | | | | | | | | | | | | | | | | | | linux builds. Changes in this CL: - In request_build.py, add a URL for linux try bot. - Add sample config for functional bisect. - Make perf-related config parameters optional. - Use builder type when starting build try jobs, and when fetching builds. A few other changes: - Update bisect_perf_regression.py file docstring. - Rename _BuilderTryjob to _StartBuilderTryJob and refactor. - Remove unused code in request_build.py. Proposed follow-up: - Move functions related to builder to be methods of one class, so that the builder-specific info/logic (e.g. builder bot and archive location) is in one place. BUG= Review URL: https://codereview.chromium.org/806943007 Cr-Commit-Position: refs/heads/master@{#313415}
* Add builder_type to fetch builds for android-chrome.prasadv2015-01-261-4/+57
| | | | | | | | | BUG= NOTRY=True Review URL: https://codereview.chromium.org/866573003 Cr-Commit-Position: refs/heads/master@{#313127}
* Avoid finding mean for an empty metrics.prasadv2015-01-232-2/+4
| | | | | | | | | BUG=451476 NOTRY=True Review URL: https://codereview.chromium.org/861033004 Cr-Commit-Position: refs/heads/master@{#312897}
* Averaging nested result lists instead of flattening them.robertocn2015-01-141-14/+2
| | | | | | | | | | Also, removing special case where certain metrics where summed instead of averaged since they are now handled in the dashboard by averaging, no longer making them special cases. BUG=447588 Review URL: https://codereview.chromium.org/847273002 Cr-Commit-Position: refs/heads/master@{#311377}
* Replace builder_host and builder_port with builder_type.qyearsley2015-01-129-86/+75
| | | | | | | | BUG=402669 Review URL: https://codereview.chromium.org/847603002 Cr-Commit-Position: refs/heads/master@{#311065}
* Fix an error while syncing revision set as Noneprasadv2015-01-091-1/+2
| | | | | | | | | BUG= NOTRY=True Review URL: https://codereview.chromium.org/836783003 Cr-Commit-Position: refs/heads/master@{#310888}
* Temporarily disable 'relative-import' pylint error.prasadv2015-01-092-3/+6
| | | | | | | | | BUG= NOTRY=true Review URL: https://codereview.chromium.org/848433002 Cr-Commit-Position: refs/heads/master@{#310865}
* Sync android-chrome revision along with chromium on android-chrome platform.prasadv2015-01-094-6/+73
| | | | | | | | | | | | | | android-chrome revision is overwritten with TOT revision when we sync only chromium revision. This causes inconsistency between android-chrome and chromium revisions that we want to bisect. Therefore we pass android-chrome revision along with chromium revision to make sure we are bisect chromium on proper android-chrome revision. BUG=447348 NOTRY=true Review URL: https://codereview.chromium.org/800143003 Cr-Commit-Position: refs/heads/master@{#310681}
* [bisect] - Verify suspected culprit by comparing ToT with and without patch.simonhatch2015-01-074-36/+271
| | | | | | | | | | | | | | | | | | | | | | | | | This CL adds a new step to compare ToT with and without the culprit CL. If the confidence in the original bisect is extremely high, we sync to ToT, attempts to revert the culprit CL, and try running the performance test again. If the culprit CL is in the root depot (ie. chromium), we issue a request to build remotely. If the culprit CL is in another depot, we build locally since we can't post a patch to multiple depots. We could in theory also change the confidence of the original bisect, but for now we only add information about the retest results and some possible warnings to the output. Example runs: Revert wasn't clean: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_perf_bisect/builds/1000 http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/1334 Revert succeeded: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/1531 http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/1369 BUG=383716 NOTRY=true Review URL: https://codereview.chromium.org/806853006 Cr-Commit-Position: refs/heads/master@{#310385}
* Avoid skipping DEPS change revisions for chromium repo.prasadv2014-12-181-3/+5
| | | | | | | | | BUG=442809 NOTRY=true Review URL: https://codereview.chromium.org/818503002 Cr-Commit-Position: refs/heads/master@{#309082}
* Make changes to support bisect for android-chrome.prasadv2014-12-124-28/+29
| | | | | | | | | BUG= NOTRY=true Review URL: https://codereview.chromium.org/792183003 Cr-Commit-Position: refs/heads/master@{#308002}
* Lower "confidence score" required in order to not abort, and refactor test.qyearsley2014-12-052-70/+68
| | | | | | | | BUG= Review URL: https://codereview.chromium.org/764733005 Cr-Commit-Position: refs/heads/master@{#307083}
* Fixing incorrect initial regression confidence calculation.robertocn2014-12-032-5/+55
| | | | | | | | BUG=438410 Review URL: https://codereview.chromium.org/773843004 Cr-Commit-Position: refs/heads/master@{#306499}
* [bisect] - Fix parsing cloud links from telemetry.simonhatch2014-12-011-1/+2
| | | | | | | | | | | | | | The url output from telemetry was recently changed to make internal buckets more accessible (to whoever has the right permissions). There's also a flag being added that will allow you to specify the upload bucket, so we need to take that into account as well. Cloud url change: https://codereview.chromium.org/745753002/ BUG=433456 TEST=Submit a perf try job with tracing enabled and --upload-results. http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/1047 Review URL: https://codereview.chromium.org/735743003 Cr-Commit-Position: refs/heads/master@{#306246}
* Modify bisect_perf_regression.py to use functions in fetch_build.pyqyearsley2014-11-227-289/+158
| | | | | | | | BUG=402669 Review URL: https://codereview.chromium.org/696133003 Cr-Commit-Position: refs/heads/master@{#305333}
* Add unit tests for functions in math_utils.pyqyearsley2014-11-202-36/+96
| | | | | | | | | | | | | | | | | | | | Specifically, this adds tests for: TruncatedMean Variance StandardDeviation StandardError which were not covered before. I also added a couple notes about things we may want to change in the behavior of the math_utils functions. Still not covered is PooledStandardError, unit test coverage can be added for that function in separate CL. BUG= Review URL: https://codereview.chromium.org/736573002 Cr-Commit-Position: refs/heads/master@{#305052}
* Removing misleading comment.robertocn2014-11-141-2/+0
| | | | | | | | | BUG= NOTRY=TRUE Review URL: https://codereview.chromium.org/725193002 Cr-Commit-Position: refs/heads/master@{#304242}
* Fix formattting error while printing other bisect regressions.prasadv2014-11-141-1/+2
| | | | | | | | | BUG= NOTRY=true Review URL: https://codereview.chromium.org/729533002 Cr-Commit-Position: refs/heads/master@{#304230}
* Parse profiler file links from perf try bot jobs. This will allow us to kick ↵simonhatch2014-11-141-1/+1
| | | | | | | | | | | off try jobs that pass back traces. Telemetry changes: https://codereview.chromium.org/721763003/ BUG= Review URL: https://codereview.chromium.org/721113004 Cr-Commit-Position: refs/heads/master@{#304211}
* Enable bisecting skia repo.prasadv2014-11-133-103/+19
| | | | | | | | | | | | | Before crrev.com/270272, skia repo used to maintain separate mirror of skia/src, skia/include and skia/gyp and used to checkout subdirectories from svn repo. Due to crrev.com/270272, which changed Skia DEPS to pull skia source from googlesource (git). BUG=406599 NOTRY=true Review URL: https://codereview.chromium.org/721033002 Cr-Commit-Position: refs/heads/master@{#304101}
* Make correction to the browser links of repositories.prasadv2014-11-132-52/+26
| | | | | | | | | | | | Since all the repos are migrated to Git, links to browser revisions should points to https://chromium.googlesource.com instead of svn repo. BUG= NOTRY=true Review URL: https://codereview.chromium.org/718213003 Cr-Commit-Position: refs/heads/master@{#304094}
* Special-casing -1 as an issue_id for bisects not linked to a bug.robertocn2014-11-121-5/+9
| | | | | | Review URL: https://codereview.chromium.org/717333002 Cr-Commit-Position: refs/heads/master@{#303920}
* Fix bisect to process angle repoprasadv2014-11-122-1/+6
| | | | | | | | | BUG=430558 NOTRY=true Review URL: https://codereview.chromium.org/694623003 Cr-Commit-Position: refs/heads/master@{#303755}
* Minor reformatting of "failed to reproduce, early abort" message.qyearsley2014-11-111-9/+10
| | | | | | | | BUG= Review URL: https://codereview.chromium.org/684993008 Cr-Commit-Position: refs/heads/master@{#303698}
* Aborting bisect early when the bug specified in the bisect config is closed.robertocn2014-11-037-6/+540
| | | | | | | | | | | | | | | Also, renaming valid_params and valid_parameters when validating bisect options to required_params and required_parameters respectively to better reflect what they actually are. Fixing some runtime errors introduced by the refactoring to use ArgsParser instead of OptionsParser. BUG=424688 Review URL: https://codereview.chromium.org/697713003 Cr-Commit-Position: refs/heads/master@{#302468}
* Change configs/try.py to get the list of config files to use from args.qyearsley2014-11-036-71/+63
| | | | | | | | | | | | | | For example, after uploading this CL, I just ran: ./try.py mac.bisect.blink_perf.cfg linux.perf_test.tab_switching.cfg. I expect it should only start two try jobs. If you run ./try.py (with no arguments), it prints usage help and a "too few arguments" message. BUG=415890 Review URL: https://codereview.chromium.org/687733003 Cr-Commit-Position: refs/heads/master@{#302414}
* Refactoring auto-bisect botsergiyb2014-10-316-596/+156
| | | | | | | | | | | | | | | | | | | | | | | - BisectPerformanceMetrics class: - Added src_cwd to constructor to allow easier mocking. Using os.getcwd() creates non-obvious expectations from the caller. - Exposed printer field to allow it being reused outside. - Added a comment explaining arguments to the constructor. - BisectOptions class: - Changed from deprecated optparse to argparse - Split _CreateCommandLineParser - Moved logic from ParseCommandLine to argparse where possible - Removed code related to ChromeOS BUG=425652 R=qyearsley@chromium.org CC=ojan@chromium.org Review URL: https://codereview.chromium.org/691553002 Cr-Commit-Position: refs/heads/master@{#302256}
* Update style according to advice from pylint.qyearsley2014-10-306-39/+42
| | | | | | | | BUG= Review URL: https://codereview.chromium.org/681103002 Cr-Commit-Position: refs/heads/master@{#302139}
* Add --goma_threads option to bisection toolswolenetz2014-10-292-2/+7
| | | | | | | | | | | | | | | | | Lets user of run-bisect-manual-test.py specify a non-default value for number of goma threads, and hooks this into Builder used by bisect_perf_regression.py. Example usage: python tools/run-bisect-manual-test.py --browser=release -b 100 -g 50 -w ../../tmp --goma_threads 50 R=qyearsley@chromium.org TEST=script passes through specified value that defaults to 64 Bug=none Review URL: https://codereview.chromium.org/679353002 Cr-Commit-Position: refs/heads/master@{#301896}
* Making a lack of regression confidence not return an error, but a warning.robertocn2014-10-283-15/+32
| | | | | | | | BUG=427535 Review URL: https://codereview.chromium.org/676403004 Cr-Commit-Position: refs/heads/master@{#301734}
* Update OWNERS file for src/tools/auto-bisect.qyearsley2014-10-281-0/+2
| | | | | | | | BUG= Review URL: https://codereview.chromium.org/681143002 Cr-Commit-Position: refs/heads/master@{#301651}
* Make bisect tool check for new V8 roll description, "based on <git-hash>"prasadv2014-10-271-3/+10
| | | | | | | | | BUG=419297 NOTRY=true Review URL: https://codereview.chromium.org/676403005 Cr-Commit-Position: refs/heads/master@{#301444}
* Add bisect-config-trying script and a few example configs.qyearsley2014-10-278-0/+268
| | | | | | | | | | This is my initial attempt at the script suggested in http://crbug.com/415890. BUG=415890 Review URL: https://codereview.chromium.org/652543005 Cr-Commit-Position: refs/heads/master@{#301409}
* Enabling logging for bisect script.robertocn2014-10-272-23/+51
| | | | | | | | Setting default configuration to print any INFO and above logging without formatting to standard out. Review URL: https://codereview.chromium.org/660393002 Cr-Commit-Position: refs/heads/master@{#301389}
* Fix bisect failures for V8 rollsprasadv2014-10-241-87/+142
| | | | | | | | | BUG=425171 NOTRY=true Review URL: https://codereview.chromium.org/676793002 Cr-Commit-Position: refs/heads/master@{#301202}
generated by cgit v1.1 at 2025-03-16 01:34:57 +0000