summaryrefslogtreecommitdiffstats
path: root/tools/isolate_driver.py
Commit message (Collapse)AuthorAgeFilesLines
* lib32/ doesn't exist on OSX x64 build. Update browser_tests accordingly.maruel2014-09-261-1/+1
| | | | | | | | | | | Add nexe to isolate_driver.py. R=vadimsh@chromium.org BUG=415188 Review URL: https://codereview.chromium.org/574753004 Cr-Commit-Position: refs/heads/master@{#296991}
* Make isolate_driver.py only add .dll/.so/.dylib for component builds' ↵jam2014-09-221-0/+6
| | | | | | | | | | | | generated files. Right now it's adding a lot of other files that aren't needed to run the tests, which slows down the uploading and downloading steps when using component builds. Since this preprocessing step is only meant to add extra binaries that are the result of the component build, just whitelist .dll/.so/.dylib. BUG=414808 Review URL: https://codereview.chromium.org/592143002 Cr-Commit-Position: refs/heads/master@{#296037}
* Fix a bug in isolate_driver.py where it would map directories.maruel2014-09-171-5/+7
| | | | | | | | | | | | | | This would cause <(PRODUCT_DIR)/remoting/remoting.webapp to be mapped, while it should have been <(PRODUCT_DIR)/remoting/remoting.webapp/. In any case, it's not useful for isolate_driver.py to map directories, so skip these entries completely. R=vadimsh@chromium.org BUG= Review URL: https://codereview.chromium.org/581713002 Cr-Commit-Position: refs/heads/master@{#295313}
* Add new isolation mode "prepare".vadimsh2014-09-091-2/+23
| | | | | | | | | | | | | | | | In that mode isolate_driver.py will collect all arguments needed for isolate.py invocation into *.isolated.gen.json file, but won't actually run the isolation itself. It's a feature of isolate_driver.py. Isolate client itself knows nothing about this mode. BUG=389227 R=maruel@chromium.org Review URL: https://codereview.chromium.org/555553002 Cr-Commit-Position: refs/heads/master@{#293885}
* Fixed bug in dependency recursion in isolate_driver.py.kbr@chromium.org2014-05-311-17/+15
| | | | | | | | | | | | | | | | | | | When run in a component build, isolate_driver recursively descends the dependency graph to discover implicit shared library dependencies. A bug in the recursion caused dependencies at certain depths in the graph to be missed. Also changed the recursion to use a set to record newly added dependencies to avoid adding them to the isolate multiple times. Expanded the ignored set of suffixes slightly. BUG=378021 TEST=GLES2 conformance tests run via isolate in component build on Mac OS R=maruel@chromium.org,vadimsh@chromium.org,csharp@chromium.org Review URL: https://codereview.chromium.org/304993006 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@274023 0039d316-1c4b-4281-b951-d872f2087c98
* Revert of Enable the ninja parsing code all the time. ↵falken@chromium.org2014-05-091-6/+158
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (https://codereview.chromium.org/270333002/) Reason for revert: Speculative revert. Maybe the cause of various bot failures involving "A .isolate file is required" Logs: http://build.chromium.org/p/chromium.linux/builders/Linux%20Tests/builds/10820 http://build.chromium.org/p/chromium.win/builders/Vista%20Tests%20%283%29/builds/39788 Snippet from Linux Tests log: /usr/bin/python /mnt/data/b/build/slave/Linux_Tests/build/src/tools/swarming_client/isolate.py run --isolated /mnt/data/b/build/slave/Linux_Tests/build/src/out/Release/interactive_ui_tests.isolated -v -- --no-cr --test-launcher-bot-mode --gtest_output=xml:/mnt/data/b/build/slave/Linux_Tests/build/gtest-results/interactive_ui_tests/interactive_ui_tests.xml --test-launcher-summary-output=/tmp/tmpldQYtj [------ Swarming Error ------] A .isolate file is required. Traceback (most recent call last): File "/mnt/data/b/build/slave/Linux_Tests/build/src/tools/swarming_client/isolate.py", line 1516, in main return dispatcher.execute(OptionParserIsolate(version=__version__), argv) File "/mnt/data/b/build/slave/Linux_Tests/build/src/tools/swarming_client/third_party/depot_tools/subcommand.py", line 242, in execute return command(parser, args[1:]) File "/mnt/data/b/build/slave/Linux_Tests/build/src/tools/swarming_client/isolate.py", line 1251, in CMDrun options, os.getcwd(), None, options.skip_refresh) File "/mnt/data/b/build/slave/Linux_Tests/build/src/tools/swarming_client/isolate.py", line 863, in load_complete_state raise ExecutionError('A .isolate file is required.') ExecutionError: A .isolate file is required. Original issue's description: > isolate_driver: Enable ninja parsing code all the time. > > Split the ninja parsing code in its own file so that isolate_driver is focused > on .isolate management. > > - This removes the need to specify the majority of binary runtime dependencies. > Sadly, this means a few unrelated binaries are now pulled in, for example d8, > libvpx_obj_int_extract and tls_edit. That's a maintenance/performance trade > off. > - Removed some dependencies for unit_tests.isolate to ensure this works. > > R=vadimsh@chromium.org > BUG=368543 > > Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=269140 TBR=vadimsh@chromium.org,maruel@chromium.org NOTREECHECKS=true NOTRY=true BUG=368543 Review URL: https://codereview.chromium.org/272113002 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@269163 0039d316-1c4b-4281-b951-d872f2087c98
* isolate_driver: Enable ninja parsing code all the time.maruel@chromium.org2014-05-091-158/+6
| | | | | | | | | | | | | | | | | | Split the ninja parsing code in its own file so that isolate_driver is focused on .isolate management. - This removes the need to specify the majority of binary runtime dependencies. Sadly, this means a few unrelated binaries are now pulled in, for example d8, libvpx_obj_int_extract and tls_edit. That's a maintenance/performance trade off. - Removed some dependencies for unit_tests.isolate to ensure this works. R=vadimsh@chromium.org BUG=368543 Review URL: https://codereview.chromium.org/270333002 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@269140 0039d316-1c4b-4281-b951-d872f2087c98
* Assume the foo_test_run depends on everything needed to run the test.maruel@chromium.org2014-05-061-2/+7
| | | | | | | | | | | | It previously assumed foo_test was the target to process. This doesn't work in cases where there is no foo_test target defined. R=kbr@chromium.org BUG=368543 Review URL: https://codereview.chromium.org/266923004 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@268409 0039d316-1c4b-4281-b951-d872f2087c98
* Make isolate_driver.py process build.ninja and extract dependencies. (bis)maruel@chromium.org2014-04-111-60/+166
| | | | | | | | | | | | | | | | | | | | | | | | | This uses a few assumption: - This basically breaks non-ninja build for component builds. This never worked anyway. - This assumes the file format of .ninja files. This will likely be quite obvious when this breaks. - It makes some assumptions about the build steps, for example '.so.TOC' -> '.so'. On the other hand, it creates a deterministic dependency tree, which is awesome. Technically it would work as well for non-component builds but I don't want to go this far yet. But in the end, that's the goal that nobody has to enter the binary dependencies in the .isolate files. Was reverted in r263072, second try. Added a check to ensure the files added exist and have the +x bit set. TBR=vadimsh@chromium.org BUG=360223,333473 Review URL: https://codereview.chromium.org/234433002 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@263240 0039d316-1c4b-4281-b951-d872f2087c98
* Revert 263058 "Make isolate_driver.py process build.ninja and ex..."viettrungluu@chromium.org2014-04-101-162/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Seems to have busted Mac builds, e.g., http://build.chromium.org/p/chromium.mac/buildstatus?builder=Mac%20Builder%20%28dbg%29&number=65094: FAILED: cd ../../net; export BUILT_PRODUCTS_DIR=/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/out/Debug; export CONFIGURATION=Debug; export PRODUCT_NAME=net_unittests_run; export SDKROOT=/Developer/SDKs/MacOSX10.6.sdk; export SRCROOT=/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/out/Debug/../../net; export SOURCE_ROOT="${SRCROOT}"; export TARGET_BUILD_DIR=/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/out/Debug; export TEMP_DIR="${TMPDIR}";python ../tools/isolate_driver.py check --isolated "../out/Debug/net_unittests.isolated" --isolate "net_unittests.isolate" --path-variable PRODUCT_DIR "../out/Debug " --extra-variable "version_full=36.0.1935.0" --config-variable "OS=mac" --config-variable "chromeos=0" --config-variable "component=shared_library" --config-variable "internal_gles2_conform_tests=0" --config-variable "icu_use_data_file_flag=1" --config-variable "libpeer_target_type=static_library" --config-variable "use_openssl=0" --extra-variable mac_product_name Chromium --ignore_broken_items [------ Swarming Error ------] File /Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/out/Debug/net_resources_target_net_resources_d558be7de16a8d003a6e7f7fae73e582 doesn't exist Traceback (most recent call last): File "/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/tools/swarming_client/isolate.py", line 1516, in main return dispatcher.execute(OptionParserIsolate(version=__version__), argv) File "/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/tools/swarming_client/third_party/depot_tools/subcommand.py", line 242, in execute return command(parser, args[1:]) File "/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/tools/swarming_client/isolate.py", line 1082, in CMDcheck options, os.getcwd(), options.subdir, False) File "/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/tools/swarming_client/isolate.py", line 888, in load_complete_state options.extra_variables, options.ignore_broken_items) File "/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/tools/swarming_client/isolate.py", line 762, in load_isolate ignore_broken_items) File "/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/tools/swarming_client/isolate.py", line 63, in expand_directories_and_symlinks indir, relfile, blacklist, follow_symlinks)) File "/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/tools/swarming_client/isolateserver.py", line 1450, in expand_directory_and_symlink relfile, symlinks = expand_symlinks(indir, relfile) File "/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/tools/swarming_client/isolateserver.py", line 1358, in expand_symlinks todo = file_path.fix_native_path_case(done, todo) File "/Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/tools/swarming_client/utils/file_path.py", line 488, in fix_native_path_case os.path.join(native_case_path, raw_part)) OSError: File /Volumes/data/b/build/slave/Mac_Builder__dbg_/build/src/out/Debug/net_resources_target_net_resources_d558be7de16a8d003a6e7f7fae73e582 doesn't exist [----------------------------] > Make isolate_driver.py process build.ninja and extract dependencies. > > This uses a few assumption: > - This basically breaks non-ninja build for component builds. This never worked > anyway. > - This assumes the file format of .ninja files. This will likely be quite > obvious when this breaks. > - It makes some assumptions about the build steps, for example '.so.TOC' -> > '.so'. > > On the other hand, it creates a deterministic dependency tree, which is awesome. > Technically it would work as well for non-component builds but I don't want to > go this far yet. But in the end, that's the goal that nobody has to enter the > binary dependencies in the .isolate files. > > R=csharp@chromium.org > BUG=360223,333473 > > Review URL: https://codereview.chromium.org/228463003 TBR=maruel@chromium.org Review URL: https://codereview.chromium.org/233743002 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@263072 0039d316-1c4b-4281-b951-d872f2087c98
* Make isolate_driver.py process build.ninja and extract dependencies.maruel@chromium.org2014-04-101-60/+162
| | | | | | | | | | | | | | | | | | | | | | This uses a few assumption: - This basically breaks non-ninja build for component builds. This never worked anyway. - This assumes the file format of .ninja files. This will likely be quite obvious when this breaks. - It makes some assumptions about the build steps, for example '.so.TOC' -> '.so'. On the other hand, it creates a deterministic dependency tree, which is awesome. Technically it would work as well for non-component builds but I don't want to go this far yet. But in the end, that's the goal that nobody has to enter the binary dependencies in the .isolate files. R=csharp@chromium.org BUG=360223,333473 Review URL: https://codereview.chromium.org/228463003 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@263058 0039d316-1c4b-4281-b951-d872f2087c98
* Disable the hack in isolate_driver.py, it is not sufficient.maruel@chromium.org2014-04-071-1/+3
| | | | | | | | | | | | | Will have to implement a proper ninja parser to enable support for component build. NOTRY=true TBR=csharp@chromium.org BUG=360223,333473 Review URL: https://codereview.chromium.org/227683007 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@262187 0039d316-1c4b-4281-b951-d872f2087c98
* Implement work around to detect libraries being linked.maruel@chromium.org2014-04-071-10/+45
| | | | | | | | | | | | | This is very temporary to unblock using isolated testing with component builds. It basically scans the dlls twice and removes the ones that changed size. The proper solution is to use primitive ninja parsing. R=csharp@chromium.org BUG=360223,333473 Review URL: https://codereview.chromium.org/227093002 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@262155 0039d316-1c4b-4281-b951-d872f2087c98
* Disable component builds hack in isolate_driver.pymaruel@chromium.org2014-04-041-1/+2
| | | | | | | | | | | It causes file sharing access issues. TBR=scottmg@chromium.org BUG=360223,333473 Review URL: https://codereview.chromium.org/226743002 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@261888 0039d316-1c4b-4281-b951-d872f2087c98
* Add a wrapper script which modifies the .isolate on the fly.maruel@chromium.org2014-04-041-0/+132
This tool will ultimately process the .ninja files, in the meantime it archives all the DLLs. It's cheezy but this will unblock the component build short term. This is very crude but this will get folks going. This means once it's done, we can remove all the .isolate representing the dynamic libraries. Remove swarming_client/ as a dependency in the .isolate files, it's not needed anymore. R=vadimsh@chromium.org BUG=333473 Review URL: https://codereview.chromium.org/196283016 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@261652 0039d316-1c4b-4281-b951-d872f2087c98
generated by cgit v1.1 at 2025-03-26 19:39:15 +0000