summaryrefslogtreecommitdiffstats
path: root/courgette
Commit message (Collapse)AuthorAgeFilesLines
* Move base::FreeDeleter into its own header.dcheng2016-03-261-0/+1
| | | | | | | | | | Final blocker to wholly deleting base/memory/scoped_ptr.h. BUG=554298 Review URL: https://codereview.chromium.org/1837483003 Cr-Commit-Position: refs/heads/master@{#383417}
* [Courgette] Add and use AddressTranslator::PointerToTargetRVA(); Update ↵huangs2016-03-238-56/+103
| | | | | | | | | | | | | | | | | | | comments. Addresses in Courgette (abs32 and rel32) are represented in these forms: (1) Location RVA. (2) Location FileOffset. (3) Pointer in image. (4) Target VA. (5) Target RVA. We already have (1) -> (2), (2) -> (1), (2) -> (3), (1) -> (3) for existing usage. Now we add (3) -> (5) and refactor accordingly (with helpers to do (4) -> (5) for PE files). PointerToTargetRVA() will be used again we apply LabelManager to save 25% peak RAM. Review URL: https://codereview.chromium.org/1807293003 Cr-Commit-Position: refs/heads/master@{#382920}
* [Courgette] Clean up Disassembler; fix ELF Memory leaks.huangs2016-03-1419-880/+888
| | | | | | | | | | | | | | | | | | | | | Cleaning up code surrounding Disassembler: - Extract AddressTranslator interface to be used across subclasses. - Use FileOffset = size_t by context. - Detailed comments & TODOs in DisassemblerElf32ARM. - Fix DisassemblerElf32ARM memory leaks. - Lots of superficial stylistic changes. Except for AddressTranslator routines and unit tests, shying away from control flow and logic changes. BUG=579206 Committed: https://crrev.com/58b822d441f5c982e879e536fa3c1cbac8fd339a Cr-Commit-Position: refs/heads/master@{#380881} Review URL: https://codereview.chromium.org/1676683002 Cr-Commit-Position: refs/heads/master@{#380987}
* Revert of [Courgette] Clean up Disassembler; fix ELF Memory leaks. (patchset ↵scottmg2016-03-1219-882/+880
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | #15 id:270001 of https://codereview.chromium.org/1676683002/ ) Reason for revert: Regressed linux sizes (iostream maybe?) https://build.chromium.org/p/chromium/builders/Linux/builds/72899/steps/sizes/logs/stdio Original issue's description: > [Courgette] Clean up Disassembler; fix ELF Memory leaks. > > Cleaning up code surrounding Disassembler: > - Extract AddressTranslator interface to be used across subclasses. > - Use FileOffset = size_t by context. > - Detailed comments & TODOs in DisassemblerElf32ARM. > - Fix DisassemblerElf32ARM memory leaks. > - Lots of superficial stylistic changes. > > Except for AddressTranslator routines and unit tests, shying away > from control flow and logic changes. > > BUG=579206 > > Committed: https://crrev.com/58b822d441f5c982e879e536fa3c1cbac8fd339a > Cr-Commit-Position: refs/heads/master@{#380881} TBR=grt@chromium.org,wfh@chromium.org,chrisha@chromium.org,andrewhayden@chromium.org,huangs@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG=579206 Review URL: https://codereview.chromium.org/1792603006 Cr-Commit-Position: refs/heads/master@{#380885}
* [Courgette] Clean up Disassembler; fix ELF Memory leaks.huangs2016-03-1219-880/+882
| | | | | | | | | | | | | | | | | | Cleaning up code surrounding Disassembler: - Extract AddressTranslator interface to be used across subclasses. - Use FileOffset = size_t by context. - Detailed comments & TODOs in DisassemblerElf32ARM. - Fix DisassemblerElf32ARM memory leaks. - Lots of superficial stylistic changes. Except for AddressTranslator routines and unit tests, shying away from control flow and logic changes. BUG=579206 Review URL: https://codereview.chromium.org/1676683002 Cr-Commit-Position: refs/heads/master@{#380881}
* [Courgette] Add huangs@ to OWNER.huangs2016-02-291-1/+2
| | | | | | | | Also sorting alphabetically per review. Review URL: https://codereview.chromium.org/1746783002 Cr-Commit-Position: refs/heads/master@{#378241}
* gn: Make courgette_unittests pass on swarmingthakis2016-02-111-0/+4
| | | | | | | | | | | | | | | | | | courgette_unittests.isolate lists this, but gn-based swarming doesn't use the checked-in isolate files. Verified that this makes things pass by running python tools\mb\mb.py run //out/gn courgette_unittests locally. Without this patch, this command fails locally. BUG=498033 TBR=wfh Review URL: https://codereview.chromium.org/1689623005 Cr-Commit-Position: refs/heads/master@{#374919}
* [Courgette] Fix AssemblyProgram parsing for ELF-ARM.huangs2016-02-043-40/+20
| | | | | | | | | | | | | | | | | | | This fixes two problems: (A) In DisassemblerElf32, RVAToFileOffset() used Program Segment Header, but FileOffsetToRVA() used Section Header. For consistency, both should use the same one. We choose the latter (for now). (B) Even if a section has sh_type=SHT_PROGBITS, it can still have sh_addr=0. Extracting Rel32 address from these sections would add overlapping RVA chaos, and so should be avoided. Also using elf-arm7 in a unit test. The test fails before the fix and passes after. BUG= 579206 Review URL: https://codereview.chromium.org/1658463002 Cr-Commit-Position: refs/heads/master@{#373639}
* [Courgette] Add ELF-ARM test binary.Will Harris2016-02-031-0/+0
| | | | | | | | | | | | | | | | | | | | | | | Adding ELF ARM (v7) test binary that fails Courgette-dis. The binary has a non-code .bad section that if parsed, yields an invalid RVA that fails check later. The commands to create the file are: echo -e 'int main(){return 0;}' | arm-linux-gnueabi-g++ -xc++ -march=armv7-a -mtune=generic-armv7-a - echo '00 20 00 0A 00 00 00 00' | xxd -r -p - bad_section arm-linux-gnueabihf-objcopy a.out elf-armv7 --add-section .bad=bad_section rm a.out bad_section The problematic code is 00000000: 00 20 00 0A => beq loc_00008008 The target RVA (00008008) is valid, but the site RVA (00000000) is bad. BUG=579206 R=huangs@chromium.org Review URL: https://codereview.chromium.org/1661723004 . Cr-Commit-Position: refs/heads/master@{#373372}
* Revert of [Courgette] Add ELF-ARM test binary. (patchset #5 id:80001 of ↵vabr2016-02-031-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://codereview.chromium.org/1662633002/ ) Reason for revert: Broke checkperms, see http://crbug.com/579206#c9. Original issue's description: > [Courgette] Add ELF-ARM test binary. > > Adding ELF ARM (v7) test binary that fails Courgette-dis. The binary > has a non-code .bad section that if parsed, yields an invalid RVA that > fails check latter. The commands to create the file are: > > echo -e 'int main(){return 0;}' | arm-linux-gnueabi-g++ -xc++ -march=armv7-a -mtune=generic-armv7-a - > echo '00 20 00 0A 00 00 00 00' | xxd -r -p - bad_section > arm-linux-gnueabihf-objcopy a.out elf-armv7 --add-section .bad=bad_section > rm a.out bad_section > > The problematic code is > 00000000: 00 20 00 0A => beq loc_00008008 > > The target RVA (00008008) is valid, but the site RVA (00000000) is bad. > > BUG=579206 > NOPRESUBMIT=true > > Committed: https://crrev.com/5dd21d43c6da1cd24f80131811158fada2fc8610 > Cr-Commit-Position: refs/heads/master@{#373162} TBR=wfh@chromium.org,huangs@google.com,huangs@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG=579206 Review URL: https://codereview.chromium.org/1663033002 Cr-Commit-Position: refs/heads/master@{#373210}
* [Courgette] Add ELF-ARM test binary.huangs2016-02-031-0/+0
| | | | | | | | | | | | | | | | | | | | | | | Adding ELF ARM (v7) test binary that fails Courgette-dis. The binary has a non-code .bad section that if parsed, yields an invalid RVA that fails check latter. The commands to create the file are: echo -e 'int main(){return 0;}' | arm-linux-gnueabi-g++ -xc++ -march=armv7-a -mtune=generic-armv7-a - echo '00 20 00 0A 00 00 00 00' | xxd -r -p - bad_section arm-linux-gnueabihf-objcopy a.out elf-armv7 --add-section .bad=bad_section rm a.out bad_section The problematic code is 00000000: 00 20 00 0A => beq loc_00008008 The target RVA (00008008) is valid, but the site RVA (00000000) is bad. BUG=579206 NOPRESUBMIT=true Review URL: https://codereview.chromium.org/1662633002 Cr-Commit-Position: refs/heads/master@{#373162}
* [Courgette] Refactor: Manage AssemblyProgram and EncodedProgram with scoped_ptr.huangs2016-01-2918-300/+321
| | | | | | | | | | | | | | | | | | | | | Previously naked pointers AssemblyProgram and EncodedProgram are used over the place, and are deallocated using Delete{AssemblyProgram, EncodedProgram}(). In this CL we use scoped_ptr to manage the life cycles of these objects. - Removed DeleteAssemblyProgram() and DeleteEncodedProgram() and replaced calls with e.g., program.reset(nullptr); if the manual deallocation is a peak memory optimization. - Moved Encode() and ReadEncodedProgram() to the .h files matching the .cc files. - Extracted DetectExecutableType() and ParseDetectedExecutable() from disassembly.* to new files program_detector*c, since Disassembly is really an implementation that caller's don't care about. Committed: https://crrev.com/0a9cbf1781a114b35a4e0f4a834f2d24ade2e917 Cr-Commit-Position: refs/heads/master@{#372212} Review URL: https://codereview.chromium.org/1629703002 Cr-Commit-Position: refs/heads/master@{#372436}
* Revert of [Courgette] Refactor: Manage AssemblyProgram and EncodedProgram ↵dcheng2016-01-2918-321/+300
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | with scoped_ptr. (patchset #5 id:80001 of https://codereview.chromium.org/1629703002/ ) Reason for revert: Breaks ninja build on Linux GN. Original issue's description: > [Courgette] Refactor: Manage AssemblyProgram and EncodedProgram with scoped_ptr. > > Previously naked pointers AssemblyProgram and EncodedProgram are used over the > place, and are deallocated using Delete{AssemblyProgram, EncodedProgram}(). > In this CL we use scoped_ptr to manage the life cycles of these objects. > > - Removed DeleteAssemblyProgram() and DeleteEncodedProgram() and replaced calls > with e.g., program.reset(nullptr); if the manual deallocation is a peak > memory optimization. > - Moved Encode() and ReadEncodedProgram() to the .h files matching the .cc files. > - Extracted DetectExecutableType() and ParseDetectedExecutable() from > disassembly.* to new files program_detector*c, since Disassembly is really an > implementation that caller's don't care about. > > Committed: https://crrev.com/0a9cbf1781a114b35a4e0f4a834f2d24ade2e917 > Cr-Commit-Position: refs/heads/master@{#372212} TBR=wfh@chromium.org,grt@chromium.org,huangs@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true Review URL: https://codereview.chromium.org/1650013002 Cr-Commit-Position: refs/heads/master@{#372274}
* [Courgette] Refactor: Manage AssemblyProgram and EncodedProgram with scoped_ptr.huangs2016-01-2918-300/+321
| | | | | | | | | | | | | | | | | | Previously naked pointers AssemblyProgram and EncodedProgram are used over the place, and are deallocated using Delete{AssemblyProgram, EncodedProgram}(). In this CL we use scoped_ptr to manage the life cycles of these objects. - Removed DeleteAssemblyProgram() and DeleteEncodedProgram() and replaced calls with e.g., program.reset(nullptr); if the manual deallocation is a peak memory optimization. - Moved Encode() and ReadEncodedProgram() to the .h files matching the .cc files. - Extracted DetectExecutableType() and ParseDetectedExecutable() from disassembly.* to new files program_detector*c, since Disassembly is really an implementation that caller's don't care about. Review URL: https://codereview.chromium.org/1629703002 Cr-Commit-Position: refs/heads/master@{#372212}
* allocator cleanup: remove dependencies on allocator from all targetsprimiano2016-01-282-12/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Overview of the cleanup: ------------------------ - In the context of the discussion in https://goo.gl/K2m649 our illustrious base/ owners suggested that having base being the only target depending on allocator (and having the other targets inherit that recursively) would be a more scalable solution, as opposite to having to remember to add an exec -> allocator to each target. - This base -> allocator dep landed in crrev.com/1616793003. - After that CL, until this point, many targets got two paths that lead to allocator: 1. The indirect one via base (content_shell -> base -> allocator) which is what we want to preserve. 2. The direct one (content_shell -> allocator) which is the inconsistent one we want to drop. This CL gets rid of all the instances of 2. The nice property of this sequencing of CLs is that the effect of this large change on the final .ninja files is minimal. See details below. Effect on the produced ninja files: ----------------------------------- GYP, Linux, static build: https://paste.ee/p/RaJLj Just some small reordering of include paths. Dropping --rdynamic (as expected) in keyboard_unittests GN, Linux, static build: https://paste.ee/p/zYtrQ Various targets lose the dependency on allocator.stamp (expected, the order is enforced by depending on base which depends on allocator.stamp) GN, Android (both component and static are similar): https://paste.ee/p/Iq6DD As above. GN, Linux, component build: https://paste.ee/p/jHUmv This is the most juicy change. As expected all the ODR goes away and random targets stop re-linking tcmalloc .o files. GYP, Win, static: https://paste.ee/p/j0IjL Extremely minimal changes: All.ninja stops depending on libcmt.lib. Sounds fine as base (which depends on that) is definitely depending on libcmt. GN, Win, static: https://paste.ee/p/Q6zTo Minimal change, like Linux GN, dropping dependencies on the .stamp files. BUG=564618 CQ_INCLUDE_TRYBOTS=tryserver.blink:linux_blink_rel Review URL: https://codereview.chromium.org/1647453002 Cr-Commit-Position: refs/heads/master@{#372167}
* [Courgette] Fold TrimLabels() into ParseDetectedExecutable().huangs2016-01-227-57/+18
| | | | | | | | | | | | | | | | | | TrimLabels() was a post-processing step after ParseDetectedExecutable(). Soon we'll do trimming earlier in the flow. However, existing code calls TrimLabels() right after ParseDetectedExecutable(). This CL moves TrimLabels() into ParseDetectedExecutable() to simplify the transition. Existing flows that call ParseDetectedExecutable() are also inconsistent in TrimLabel() usage; {-gen, -apply, -dis} do, and debugging flows {-disadj, -gen1a, -gen1u} don't. This CL makes the debugging flows use TrimLabel(). This side effect is likely positive, since it improves consistently. Meanwhile, the actual impact is small: TrimLabels() is active only for ELF ARM anyway. Review URL: https://codereview.chromium.org/1615513002 Cr-Commit-Position: refs/heads/master@{#370916}
* [Courgette] Simplify EncodedProgram Label addition code; removed "1.01 x" ↵huangs2016-01-1914-142/+217
| | | | | | | | | | | | | | | | | | | | | | | | | | memory fix. This CL simplifies how Labels get flattened to a list of RVAs. In the past EncodedProgram used DefineAbs32Label() / DefineRel32Label(), which let callers add one Label at a time. Complexity arose from: - Function pointer usage to avoid duplicate code for abs32 and rel32. - Need for EncodedProgram to dynamically adjust size of RVA list. This led to inefficient array resizing, which was fixed by the "1.01 x" memory growth. Change: We now pass the collection of abs32 and rel32 Labels to EncodedProgram. This simplifies the interface, and allows EncodedProgram to find the max indexes and preallocated buffers. The trade-off is increased test code complexity, since we'd need to create Label collection. Other changes: - Update namespace{} for EncodedProgram and its tests. - Add more Label constructors (for testing). - Add LabelManager::GetIndexBound(), for LabelVector and RVAToLabel. - Add kUnassignedRVA in image_utils.h, with checks for its absence in images. Review URL: https://codereview.chromium.org/1571913003 Cr-Commit-Position: refs/heads/master@{#370200}
* [Courgette] Make Courgette-dis trim labels.huangs2016-01-181-0/+7
| | | | | | | | | | Courgette-gen and Courgette-apply both call TrimLabels() for the assembly programs generated from input image. The testing mode Courgette-dis should also do so. Review URL: https://codereview.chromium.org/1600953002 Cr-Commit-Position: refs/heads/master@{#370052}
* [Courgette] Fix memory leak in AssemblyProgram::TrimLabels().huangs2016-01-181-16/+13
| | | | | | | | | | | | | | | | | The leaky code is only used by EXE_ELF_32_ARM flow. In AssemblyProgram::TrimLabels(), originally we (A) remove underused Labels from std::map<RVA, Label*> that stores it, (B) convert all instructions that use underused Labels to a non-Label instruction. Problem is (A) does not deallocate Label* (owned by the std::map), and memory leak ensues. Naively deallocating Labels in (A) would lead to use-after free. So we do (B) first, then do (A) while deallocating Label*. Review URL: https://codereview.chromium.org/1570173003 Cr-Commit-Position: refs/heads/master@{#370045}
* Move bit_cast from base/macros.h to its own headertapted2016-01-141-0/+1
| | | | | | | | | | | | | | | | | | There isn't much that uses bit_cast. bit_cast requires memcpy which requires <string.h> On Mac, <string.h> includes AvailabilityInternal.h which is half a megabyte and defines dozens of macros. Providing <string.h> in base/macros.h also tempts IWYU violations. Try to avoid it. Add base/bit_cast.h and include it from the 17 files that need it. Then remove the <string.h> #include from base/macros.h and IWYU. BUG=395001 Review URL: https://codereview.chromium.org/399313006 Cr-Commit-Position: refs/heads/master@{#369348}
* [Courgette] Make LabelManager an interface; move code to LabelManagerImpl.huangs2016-01-083-70/+108
| | | | | | | | | | | | | | | | | | | | LabelManager aims to reduce Courgette peak memory, and we still need to use it in production. To reduce transition risk and pain, the plan is: 1. Make LabelManager an interface, move the implementation to LabelMangerImpl. 2. More cleanup of Label-related code in production. 3. Add LabelManagerLegacy implementation of LabelManager and move active Label-related code there. Update callers to call the LabelManager interfaces. No change in behavior. 4. After extensive testing, switch to using LabelManagerImpl with relatively little code change. 5. Remove LabelManagerLegacy. This CL implements step #1. Review URL: https://codereview.chromium.org/1567133002 Cr-Commit-Position: refs/heads/master@{#368378}
* [Courgette] Port basic index assignment routines to LabelManager.huangs2016-01-063-10/+403
| | | | | | | | | Copying UnassignIndexes(), DefaultAssignIndexes(), and AssignRemainingIndexes() from AssemblyProgram. Also adding unit tests. Review URL: https://codereview.chromium.org/1527813002 Cr-Commit-Position: refs/heads/master@{#367741}
* Switch to standard integer types in courgette/.avi2015-12-2165-664/+797
| | | | | | | | | | BUG=138542 TBR=wfh@chromium.org NOPRESUBMIT=true Review URL: https://codereview.chromium.org/1543643002 Cr-Commit-Position: refs/heads/master@{#366439}
* Drop 'const' declaration from courgette unittest.mkwst2015-12-081-1/+1
| | | | | | | | | | | | | | | | | | --- ../../courgette/label_manager_unittest.cc:41:3: error: 'const' type qualifier on return type has no effect [-Werror,-Wignored-qualifiers] const size_t LabelCount() const { return labels_.size(); }; ^~~~~~ 1 error generated. --- BUG=567505 TBR=huangs@chromium.org NOTRY=true NOTREECHECKS=true Review URL: https://codereview.chromium.org/1501413004 Cr-Commit-Position: refs/heads/master@{#363756}
* [Courgette] Initial Implementation of LabelManagerhuangs2015-12-089-17/+439
| | | | | | | | | | | | This is part of the effort to reduce Courgette's peak memory. Main changes: - Moving Label to image_utils.h, and change Label::count_ from int to int32. - Adding utility class ConsecutiveRangeVisitor, with tests. - Adding LabelManager, with tests. The new code is not yet used in production. Review URL: https://codereview.chromium.org/1491703003 Cr-Commit-Position: refs/heads/master@{#363688}
* Cleanup: Correctly spell success(ful).thestig2015-11-231-1/+1
| | | | | | | | | TBR=brettw@chromium.org NOPRESUBMIT=true Review URL: https://codereview.chromium.org/1469783002 Cr-Commit-Position: refs/heads/master@{#361166}
* [Courgette] Move the "1.01 x" memory fix from NoThrowBuffer::reserve() to ↵huangs2015-11-192-7/+13
| | | | | | | | | | | | | | | | | | | | | caller. This amends crrev.com/565753002, which optimizes incessant calls to NoThrowBuffer::reserve() by reserving 1.01 times the requested memory. Key changes: - Exposing NoThrowBuffer::capacity(), like std::vector. - Fixing the single call site that's responsible for the incessant calls by using the same logic as the old fix. - Removing NoThrowBuffer::reserve()'s 1.01 factor. The change is verified for Courgette-apply and Courgette-gen. Cursory checks show no appreciable memory saving, but a ~2% speed up. BUG=167622 Review URL: https://codereview.chromium.org/1447323004 Cr-Commit-Position: refs/heads/master@{#360651}
* Use base::UncheckedMalloc rather than std::nothrow in courgette.grt2015-11-064-101/+147
| | | | | | | | | | | Allocating via std::nothrow will still crash the process if there's insufficient ram. BUG=530624 Review URL: https://codereview.chromium.org/1429243002 Cr-Commit-Position: refs/heads/master@{#358439}
* Cleanup: Pass std::string as const reference from courgette/ki.stfu2015-09-211-5/+4
| | | | | | | | | | | | Passing std::string by reference can prevent extra copying of object. BUG=367418 TEST= R=wfh@chromium.org Review URL: https://codereview.chromium.org/1353333002 Cr-Commit-Position: refs/heads/master@{#349951}
* [Courgette] Refactor: Adding Rel32FinderWin32X86_Basic and Unittestshuangs2015-09-1815-102/+640
| | | | | | | | | | | | | | | | We're planning to improve heuristic to find Rel32 addresses for Win32 x86. First refactor by by extraacting the feature into its own class. This also lead to the extraction of RVA and various helpers from disassembler.h to image_utils.h. Also adding unittests for Rel32FinderWin32X86_Basic. The test is data-driven by adding a simple parser for test data. BUG= Review URL: https://codereview.chromium.org/1344173003 Cr-Commit-Position: refs/heads/master@{#349727}
* [Courgette] Store delta encoding as signed int32.huangs2015-09-094-55/+152
| | | | | | | | | | | | | | | | Owing to AdjustmentMethod permuting Rel32Address items, difference of successive items may be negative. Previously Courgette stores deltas vectors using uint32, so negative values can take 5 bytes each. This CL changes storage of deltas vectors to signed int32. Experiment (goo.gl/vvVADx) shows total diff file reduction of ~3%, and zipped diff file reduction of ~1.5%. We apply the same change for Abs32Address items for consistency. Review URL: https://codereview.chromium.org/1328703002 Cr-Commit-Position: refs/heads/master@{#348039}
* Fix sbox_validation_tests & Make isolate_driver.py include the executable itselfmaruel2015-09-091-1/+0
| | | | | | | | | | | | | | | | | | | | | | This is a redo of https://codereview.chromium.org/1245873002 c6649f2e014a24a909 with fix for Official build. Further reduce the boilerplate in .isolate files and is necessary to add debug symbols, to be done in a follow up. Slightly accelerate ninja file parsing by blacklisting more extensions. sbox_validation_tests wasn't run on Windows since https://codereview.chromium.org/1212003003/ (~2 months ago). Thankfully this CL uncovered this copy-paste error. TBR=jam@chromium.org BUG=98637 CQ_INCLUDE_TRYBOTS=tryserver.blink:linux_blink_rel Review URL: https://codereview.chromium.org/1320793004 Cr-Commit-Position: refs/heads/master@{#347912}
* Revert of [Courgette] Adding CourgetteConfig singleton and --experimental ↵huangs2015-09-087-93/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | flag. (patchset #5 id:80001 of https://chromiumcodereview.appspot.com/1306193004/ ) Reason for revert: The original CL assumes Courgette had to deal with forward/backward compatibility issues. Turns out that (as seen in http://crbug.com/459505): - Courgette executable is shipped with new releases. - This allows Courgette workflow to proceed without being tied down to legacy. These new (to huangs@) facts render the original CL redundant. Original issue's description: > [Courgette] Adding CourgetteConfig singleton and --experimental flag. > > This prepare for upcoming experiments to reduce patch size. We want to > avoid disrupting existing functionality. > > Committed: https://crrev.com/fc0511d9067759336f42d04e918e8a3ce0b84376 > Cr-Commit-Position: refs/heads/master@{#346746} TBR=wfh@chromium.org NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true Review URL: https://codereview.chromium.org/1323263007 Cr-Commit-Position: refs/heads/master@{#347746}
* Revert of Fix sbox_validation_tests & Make isolate_driver.py include the ↵stgao2015-09-031-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | executable itself (patchset #3 id:90001 of https://codereview.chromium.org/1245873002/ ) Reason for revert: This CL seems to break compile http://build.chromium.org/p/chromium.chrome/builders/Google%20Chrome%20Linux%20x64/builds/3900/. If the revert doesn't fix it, I will re-land this CL. Original issue's description: > Fix sbox_validation_tests & Make isolate_driver.py include the executable itself > > Further reduce the boilerplate in .isolate files and is necessary to add debug > symbols, to be done in a follow up. > > Slightly accelerate ninja file parsing by blacklisting more extensions. > > sbox_validation_tests wasn't run on Windows since > https://codereview.chromium.org/1212003003/ (~2 months ago). Thankfully this CL > uncovered this copy-paste error. > > TBR=jam@chromium.org > BUG=98637 > CQ_INCLUDE_TRYBOTS=tryserver.blink:linux_blink_rel > > Committed: https://crrev.com/c6649f2e014a24a9093140a06fcadcaf2a06513c > Cr-Commit-Position: refs/heads/master@{#347027} TBR=jam@chromium.org,maruel@chromium.org NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG=98637 Review URL: https://codereview.chromium.org/1329783003 Cr-Commit-Position: refs/heads/master@{#347071}
* Fix sbox_validation_tests & Make isolate_driver.py include the executable itselfmaruel2015-09-021-1/+0
| | | | | | | | | | | | | | | | | | | Further reduce the boilerplate in .isolate files and is necessary to add debug symbols, to be done in a follow up. Slightly accelerate ninja file parsing by blacklisting more extensions. sbox_validation_tests wasn't run on Windows since https://codereview.chromium.org/1212003003/ (~2 months ago). Thankfully this CL uncovered this copy-paste error. TBR=jam@chromium.org BUG=98637 CQ_INCLUDE_TRYBOTS=tryserver.blink:linux_blink_rel Review URL: https://codereview.chromium.org/1245873002 Cr-Commit-Position: refs/heads/master@{#347027}
* [Courgette] Adding CourgetteConfig singleton and --experimental flag.huangs2015-09-017-4/+93
| | | | | | | | | This prepare for upcoming experiments to reduce patch size. We want to avoid disrupting existing functionality. Review URL: https://codereview.chromium.org/1306193004 Cr-Commit-Position: refs/heads/master@{#346746}
* Annotate GN executables and shared_libraries with sanitizer deps.brettw2015-08-311-0/+2
| | | | | | | | | | | | | | | | | These are needed to link in asan/lsan/etc. mode. GYP injects these automatically but GN doesn't have automatic dependency injection. All tests and components (which cover the vast majority of such targets in actual use) are templates which add these dependencies already, so most of the time it never comes up. Only a few of these changes are actually necessary to link all tests run on the asan bot (which is also what engineers will typically build, since almost nobody will build all in asan mode). But to reduce confusion from linker errors in asan mode, I annotated all executables and shared libraries I could find with the sanitizer dependency. I moved a config out of a target in yasm, and sorted a fews deps on other targets. TBR=xhwang@chromium.org (widevine) TBR=achuith@chromium.org (tools/perf, tools/telemetry) Review URL: https://codereview.chromium.org/1318343003 Cr-Commit-Position: refs/heads/master@{#346490}
* Hand over courgette ownership.dgarrett2015-08-201-2/+1
| | | | | | | | | | I haven't touched this code in years now. BUG=None Review URL: https://codereview.chromium.org/1274433004 Cr-Commit-Position: refs/heads/master@{#344523}
* [Courgette] Optimize QSufSort to speed up courgette-gen.huangs2015-08-193-78/+94
| | | | | | | | | | | | | | | | | | We apply the following optimizations to QSufSort: - split()'s interface is changed to start/end instead of start/length, so many unnecessary adds are removed. - split()'s 3-way partition is now done in a single pass. - search()'s binary search is no longer recursive. Experiment (goo.gl/XboS1f) shows this leads to 35% speed up in qsufsort(), and 10% speed up in courgette-gen. search() was slightly (0.5%) slower up much, perhaps because tail recursion was compiler-optimized. However, the new code is cleaner and takes two fewer params. Review URL: https://codereview.chromium.org/1288703002 Cr-Commit-Position: refs/heads/master@{#344353}
* [Courgette] Unnest maximize loop in adjustment_method_2.cc to speed up ↵huangs2015-08-181-12/+12
| | | | | | | | | | | | | | | | courgette-gen. In adjustment_method_2.cc, AssignmentProblem::AddPatternToLabelQueue(), a maximize loop seems nested one level too deep. Moving it up one level improves courgette-gen. Experiment (goo.gl/OlPjj6) shows this leads to 10% speed up in AdjustmentMethod, and 5% speed up in courgette-gen. We also verified that final patch is unchanged, although in theory the code may cause patch change owing to change in kUnwieldy early exit condition. Review URL: https://codereview.chromium.org/1286173005 Cr-Commit-Position: refs/heads/master@{#343912}
* [Courgette] Extract qsufsort module and add tests.huangs2015-08-116-163/+338
| | | | | | | | | This is to prepare for a follow-up CL to optimize qsufsort(), which reduces run time by ~40%, and "courgette -gen" run time by ~10%. Review URL: https://codereview.chromium.org/1272453003 Cr-Commit-Position: refs/heads/master@{#342883}
* Courgette: Remove 2 duplicates calls to std::sort.sebmarchand2015-08-114-4/+4
| | | | | | | | | | There's already a call to std::sort at the end of ParseRelocs. It's a small improvement, but it's still better than nothing :) BUG= Review URL: https://codereview.chromium.org/1284733002 Cr-Commit-Position: refs/heads/master@{#342865}
* Courgette: Skip the relocs that live outside of the image.sebmarchand2015-08-072-0/+10
| | | | | | | | BUG=517475 Review URL: https://codereview.chromium.org/1277583004 Cr-Commit-Position: refs/heads/master@{#342241}
* [Courgette] Fix NoThrowBuffer::end() off-by-1; add unittests.huangs2015-07-204-4/+87
| | | | | | | | | | | | | | | | | | | NoThrowBuffer::end() should be an exclusive upperbound, but the old implementation was inclusive. The impact is that std::sort() statements in - EncodedProgram::GeneratePeRelocations() - EncodedProgram::GenerateElfRelocations() will not sort the last element, leading to slight change in results. Also found potential use-after-free in NoThrowBuffer::reserve(), but using DCHECK() to block offending case Added basic unit tests for NoThrowBuffer. TEST=courgette_unittests --gtest_filter=MemoryAllocatorTest.NoThrowBuffer Review URL: https://codereview.chromium.org/1242263003 Cr-Commit-Position: refs/heads/master@{#339529}
* Replace StringPrintf("%d", ...) with IntToString()ricea2015-07-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | IntToString() is faster, more compact and clearer than StringPrintf("%d", ...). This CL reduces the size of the text segment of a Linux "Official" build by 1001 bytes. This CL also adds a perftest to verify that IntToString() is faster than StringPrintf("%d", ...). Notes for reviewers: * courgette/memory_monitor.cc contains LOG(INFO) statements. I bypassed presubmit rather than remove them. * components/browser_watcher/watcher_client_win_unittest.cc passed a HANDLE through an int. I don't know why that worked on Win64. I have tried to make it 64-bit safe. BUG=504372 TEST=compile, selected unit tests TBR=stevenjb@chromium.org NOPRESUBMIT=true Review URL: https://codereview.chromium.org/1213443002 Cr-Commit-Position: refs/heads/master@{#337027}
* Update {virtual,override} to follow C++11 style in courgette.nick2015-04-241-4/+4
| | | | | | | | | | | | The Google style guide states that only one of {virtual,override,final} should be used for each declaration, since override implies virtual and final implies both virtual and override. This patch was manually generated using a regex and a text editor. BUG=417463 Review URL: https://codereview.chromium.org/1052683004 Cr-Commit-Position: refs/heads/master@{#326831}
* Robust ELF header parsing.halyavin2015-03-244-30/+36
| | | | | | | | | | | Check all out-of-bounds errors in ELF header parsing. BUG= none TEST= courgette_unittests Review URL: https://codereview.chromium.org/1031513002 Cr-Commit-Position: refs/heads/master@{#322003}
* Remove misleading comment.halyavin2015-03-241-2/+0
| | | | | | | | | | | | This comment was left behind when corresponding code was removed in https://codereview.chromium.org/6597038 (rev fbd31eb8). TEST= none BUG= none Review URL: https://codereview.chromium.org/1025523002 Cr-Commit-Position: refs/heads/master@{#321954}
* Update out-of-date comments and error messages.halyavin2015-03-202-9/+6
| | | | | | | | | TEST= none BUG= none Review URL: https://codereview.chromium.org/1027883002 Cr-Commit-Position: refs/heads/master@{#321635}
* Add support for RIP relative addresses on x86_64.wfh2015-03-202-5/+24
| | | | | | | | | | | | | | | | | | | | | | | | | Visual studio uses rip-relative addressing (rra) extensively in 64 bit binaries. ParseRel32RelocsFromSection does not find these addresses, which causes courgette to miss a lot of rva pointers, and thus missing a lot of compresssion opportunities. This patch makes the ParseRel32RelocsFromSection find rip relative call/jmp/lea. It also finds mov's that load from memory using rra's. Based on CL by niels.werensteijn.teamspeak@gmail.com in crrev.com/212563003 This change gives a noticeable improvement on 64-bit binaries. Against test binaries (64-bit chrome.dll 40.0.2214.115->43.0.2317.0) the patch sizes were: Uncompressed: before: 10,948,152 after: 9,948,442 (9.1% reduction) Compressed (7z ultra): before: 6,084,670 after: 5,581,502 (8.3% reduction) BUG=459064 TEST=courgette_unittests Review URL: https://codereview.chromium.org/878043002 Cr-Commit-Position: refs/heads/master@{#321524}
generated by cgit v1.1 at 2024-04-28 12:36:58 +0000