2012-05-22 00:51:05 +00:00
|
|
|
/*
|
2018-02-16 21:38:53 +00:00
|
|
|
* Copyright (C) 2012, 2013, 2016 Apple Inc. All rights reserved.
|
2012-05-22 00:51:05 +00:00
|
|
|
*
|
|
|
|
* Redistribution and use in source and binary forms, with or without
|
|
|
|
* modification, are permitted provided that the following conditions
|
|
|
|
* are met:
|
|
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer.
|
|
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
|
|
* documentation and/or other materials provided with the distribution.
|
|
|
|
*
|
|
|
|
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
|
|
|
|
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
|
|
|
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
|
|
|
|
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
|
|
|
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
|
|
|
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
|
|
|
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
|
|
|
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
|
|
|
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
|
|
|
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
|
|
*/
|
|
|
|
|
[DFG][FTL] Implement ES6 Generators in DFG / FTL
https://bugs.webkit.org/show_bug.cgi?id=152723
Reviewed by Filip Pizlo.
JSTests:
* stress/generator-fib-ftl-and-array.js: Added.
(fib):
* stress/generator-fib-ftl-and-object.js: Added.
(fib):
* stress/generator-fib-ftl-and-string.js: Added.
(fib):
* stress/generator-fib-ftl.js: Added.
(fib):
* stress/generator-frame-empty.js: Added.
(shouldThrow):
(shouldThrow.fib):
* stress/generator-reduced-save-point-put-to-scope.js: Added.
(shouldBe):
(gen):
* stress/generator-transfer-register-beyond-mutiple-yields.js: Added.
(shouldBe):
(gen):
Source/JavaScriptCore:
This patch introduces DFG and FTL support for ES6 generators.
ES6 generator is compiled by the BytecodeGenerator. But at the last phase, BytecodeGenerator performs "generatorification" onto the unlinked code.
In BytecodeGenerator phase, we just emit op_yield for each yield point. And we don't emit any generator related switch, save, and resume sequences
here. Those are emitted by the generatorification phase.
So the graph is super simple! Before the generatorification, the graph looks like this.
op_enter -> ...... -> op_yield -> ..... -> op_yield -> ...
Roughly speaking, in the generatorification phase, we turn out which variables should be saved and resumed at each op_yield.
This is done by liveness analysis. After that, we convert op_yield to the sequence of "op_put_to_scope", "op_ret", and "op_get_from_scope".
op_put_to_scope and op_get_from_scope sequences are corresponding to the save and resume sequences. We set up the scope for the generator frame and
perform op_put_to_scope and op_get_from_scope onto it. The live registers are saved and resumed over the generator's next() calls by using this
special generator frame scope. And we also set up the global switch for the generator.
In the generatorification phase,
1. We construct the BytecodeGraph from the unlinked instructions. This constructs the basic blocks, and it is used in the subsequent analysis.
2. We perform the analysis onto the unlinked code. We extract the live variables at each op_yield.
3. We insert the get_from_scope and put_to_scope at each op_yield. Which registers should be saved and resumed is offered by (2).
Then, clip the op_yield themselves. And we also insert the switch_imm. The jump targets of this switch are just after this op_switch_imm and each op_yield point.
One interesting point is the try-range. We split the try-range at the op_yield point in BytecodeGenerator phase.
This drops the hacky thing that is introduced in [1].
If the try-range covers the resume sequences, the exception handler's use-registers are incorrectly transferred to the entry block.
For example,
handler uses r2
try-range
label:(entry block can jump here) ^
r1 = get_from_scope # resume sequence starts | use r2 is transferred to the entry block!
r2 = get_from_scope |
starts usual sequences |
... |
Handler's r2 use should be considered at the `r1 = get_from_scope` point.
Previously, we handle this edge case by treating op_resume specially in the liveness analysis[1].
To drop this workaround, we split the try-range not to cover this resume sequence.
handler uses r2
try-range
label:(entry block can jump here)
r1 = get_from_scope # resume sequence starts
r2 = get_from_scope
starts usual sequences ^ try-range should start from here.
... |
OK. Let's show the detailed example.
1. First, there is the normal bytecode sequence. Here, | represents the offsets, and [] represents the bytecodes.
bytecodes | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |
try-range <----------------------------------->
2. When we emit the op_yield in the bytecode generator, we carefully split the try-range.
bytecodes | [ ] | [ ] | [op_yield] | [ ] | [ ] | [ ] |
try-range <-----------> <----------------->
3. And in the generatorification phase, we insert the switch's jump target and save & resume sequences. And we also drop op_yield.
Insert save seq Insert resume seq
before op_yield. after op_yield's point.
v v
bytecodes | [ ] | [ ] | [op_yield] | [ ] | [ ] | [ ] |
try-range <-----------> ^ <----------------->
^ |
Jump to here. Drop this op_yield.
4. The final layout is the following.
bytecodes | [ ] | [ ][save seq][op_ret] | [resume seq] | [ ] | [ ] | [ ] |
try-range <-----------------------------> <---------------->
^
Jump to here.
The rewriting done by the BytecodeRewriter is executed in a batch manner. Since these modification changes the basic blocks and size of unlinked instructions,
BytecodeRewriter also performs the offset adjustment for UnlinkedCodeBlock. So, this rewriting is performed onto the BytecodeGraph rather than BytecodeBasicBlock.
The reason why we take this design is simple: we don't want to newly create the basic blocks and opcodes for this early phase like DFG. Instead, we perform the
modification and adjustment to the unlinked instructions and UnlinkedCodeBlock in a in-place manner.
Bytecode rewriting functionality is offered by BytecodeRewriter. BytecodeRewriter allows us to insert any bytecodes to any places
in a in-place manner. BytecodeRewriter handles the original bytecode offsets as labels. And you can insert bytecodes before and after
these labels. You can also insert any jumps to any places. When you insert jumps, you need to specify jump target with this labels.
These labels (original bytecode offsets) are automatically converted to the appropriate offsets by BytecodeRewriter.
After that phase, the data flow of the generator-saved-and-resumed-registers are explicitly represented by the get_from_scope and put_to_scope.
And the switch is inserted to represent the actual control flow for the generator. And op_yield is removed. Since we use the existing bytecodes (op_switch_imm, op_put_to_scope
op_ret, and op_get_from_scope), DFG and FTL changes are not necessary. This patch also drops data structures and implementations for the old generator,
op_resume, op_save implementations and GeneratorFrame.
Note that this patch does not leverage the recent multi entrypoints support in B3. After this patch is introduced, we will submit a new patch that leverages the multi
entrypoints for generator's resume and sees the performance gain.
Microbenchmarks related to generators show up to 2.9x improvements.
Baseline Patched
generator-fib 102.0116+-3.2880 ^ 34.9670+-0.2221 ^ definitely 2.9174x faster
generator-sunspider-access-nsieve 5.8596+-0.0371 ^ 4.9051+-0.0720 ^ definitely 1.1946x faster
generator-with-several-types 332.1478+-4.2425 ^ 124.6642+-2.4826 ^ definitely 2.6643x faster
<geometric> 58.2998+-0.7758 ^ 27.7425+-0.2577 ^ definitely 2.1015x faster
In ES6SampleBench's Basic, we can observe 41% improvement (Macbook Pro).
Baseline:
Geometric Mean Result: 133.55 ms +- 4.49 ms
Benchmark First Iteration Worst 2% Steady State
Air 54.03 ms +- 7.51 ms 29.06 ms +- 3.13 ms 2276.59 ms +- 61.17 ms
Basic 30.18 ms +- 1.86 ms 18.85 ms +- 0.45 ms 2851.16 ms +- 41.87 ms
Patched:
Geometric Mean Result: 121.78 ms +- 3.96 ms
Benchmark First Iteration Worst 2% Steady State
Air 52.09 ms +- 6.89 ms 29.59 ms +- 3.16 ms 2239.90 ms +- 54.60 ms
Basic 29.28 ms +- 1.46 ms 16.26 ms +- 0.66 ms 2025.15 ms +- 38.56 ms
[1]: https://bugs.webkit.org/show_bug.cgi?id=159281
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* builtins/GeneratorPrototype.js:
(globalPrivate.generatorResume):
* bytecode/BytecodeBasicBlock.cpp:
(JSC::BytecodeBasicBlock::shrinkToFit):
(JSC::BytecodeBasicBlock::computeImpl):
(JSC::BytecodeBasicBlock::compute):
(JSC::isBranch): Deleted.
(JSC::isUnconditionalBranch): Deleted.
(JSC::isTerminal): Deleted.
(JSC::isThrow): Deleted.
(JSC::linkBlocks): Deleted.
(JSC::computeBytecodeBasicBlocks): Deleted.
* bytecode/BytecodeBasicBlock.h:
(JSC::BytecodeBasicBlock::isEntryBlock):
(JSC::BytecodeBasicBlock::isExitBlock):
(JSC::BytecodeBasicBlock::leaderOffset):
(JSC::BytecodeBasicBlock::totalLength):
(JSC::BytecodeBasicBlock::offsets):
(JSC::BytecodeBasicBlock::successors):
(JSC::BytecodeBasicBlock::index):
(JSC::BytecodeBasicBlock::addSuccessor):
(JSC::BytecodeBasicBlock::BytecodeBasicBlock):
(JSC::BytecodeBasicBlock::addLength):
(JSC::BytecodeBasicBlock::leaderBytecodeOffset): Deleted.
(JSC::BytecodeBasicBlock::totalBytecodeLength): Deleted.
(JSC::BytecodeBasicBlock::bytecodeOffsets): Deleted.
(JSC::BytecodeBasicBlock::addBytecodeLength): Deleted.
* bytecode/BytecodeGeneratorification.cpp: Added.
(JSC::BytecodeGeneratorification::BytecodeGeneratorification):
(JSC::BytecodeGeneratorification::graph):
(JSC::BytecodeGeneratorification::yields):
(JSC::BytecodeGeneratorification::enterPoint):
(JSC::BytecodeGeneratorification::storageForGeneratorLocal):
(JSC::GeneratorLivenessAnalysis::GeneratorLivenessAnalysis):
(JSC::GeneratorLivenessAnalysis::computeDefsForBytecodeOffset):
(JSC::GeneratorLivenessAnalysis::computeUsesForBytecodeOffset):
(JSC::GeneratorLivenessAnalysis::run):
(JSC::BytecodeGeneratorification::run):
(JSC::performGeneratorification):
* bytecode/BytecodeGeneratorification.h: Copied from Source/JavaScriptCore/bytecode/BytecodeLivenessAnalysisInlines.h.
* bytecode/BytecodeGraph.h: Added.
(JSC::BytecodeGraph::codeBlock):
(JSC::BytecodeGraph::instructions):
(JSC::BytecodeGraph::basicBlocksInReverseOrder):
(JSC::BytecodeGraph::blockContainsBytecodeOffset):
(JSC::BytecodeGraph::findBasicBlockForBytecodeOffset):
(JSC::BytecodeGraph::findBasicBlockWithLeaderOffset):
(JSC::BytecodeGraph::size):
(JSC::BytecodeGraph::at):
(JSC::BytecodeGraph::operator[]):
(JSC::BytecodeGraph::begin):
(JSC::BytecodeGraph::end):
(JSC::BytecodeGraph::first):
(JSC::BytecodeGraph::last):
(JSC::BytecodeGraph<Block>::BytecodeGraph):
* bytecode/BytecodeList.json:
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::BytecodeLivenessAnalysis):
(JSC::BytecodeLivenessAnalysis::computeDefsForBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::computeUsesForBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::getLivenessInfoAtBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::computeFullLiveness):
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
(JSC::BytecodeLivenessAnalysis::compute):
(JSC::isValidRegisterForLiveness): Deleted.
(JSC::getLeaderOffsetForBasicBlock): Deleted.
(JSC::findBasicBlockWithLeaderOffset): Deleted.
(JSC::blockContainsBytecodeOffset): Deleted.
(JSC::findBasicBlockForBytecodeOffset): Deleted.
(JSC::stepOverInstruction): Deleted.
(JSC::computeLocalLivenessForBytecodeOffset): Deleted.
(JSC::computeLocalLivenessForBlock): Deleted.
(JSC::BytecodeLivenessAnalysis::runLivenessFixpoint): Deleted.
* bytecode/BytecodeLivenessAnalysis.h:
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::isValidRegisterForLiveness):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::computeLocalLivenessForBytecodeOffset):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::computeLocalLivenessForBlock):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::getLivenessInfoAtBytecodeOffset):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/BytecodeRewriter.cpp: Added.
(JSC::BytecodeRewriter::applyModification):
(JSC::BytecodeRewriter::execute):
(JSC::BytecodeRewriter::adjustJumpTargetsInFragment):
(JSC::BytecodeRewriter::insertImpl):
(JSC::BytecodeRewriter::adjustJumpTarget):
* bytecode/BytecodeRewriter.h: Added.
(JSC::BytecodeRewriter::InsertionPoint::InsertionPoint):
(JSC::BytecodeRewriter::InsertionPoint::operator<):
(JSC::BytecodeRewriter::InsertionPoint::operator==):
(JSC::BytecodeRewriter::Insertion::length):
(JSC::BytecodeRewriter::Fragment::Fragment):
(JSC::BytecodeRewriter::Fragment::appendInstruction):
(JSC::BytecodeRewriter::BytecodeRewriter):
(JSC::BytecodeRewriter::insertFragmentBefore):
(JSC::BytecodeRewriter::insertFragmentAfter):
(JSC::BytecodeRewriter::removeBytecode):
(JSC::BytecodeRewriter::graph):
(JSC::BytecodeRewriter::adjustAbsoluteOffset):
(JSC::BytecodeRewriter::adjustJumpTarget):
(JSC::BytecodeRewriter::calculateDifference):
* bytecode/BytecodeUseDef.h:
(JSC::computeUsesForBytecodeOffset):
(JSC::computeDefsForBytecodeOffset):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::dumpBytecode):
(JSC::CodeBlock::finishCreation):
(JSC::CodeBlock::handlerForIndex):
(JSC::CodeBlock::shrinkToFit):
(JSC::CodeBlock::valueProfileForBytecodeOffset):
(JSC::CodeBlock::livenessAnalysisSlow):
* bytecode/CodeBlock.h:
(JSC::CodeBlock::isConstantRegisterIndex):
(JSC::CodeBlock::livenessAnalysis):
(JSC::CodeBlock::liveCalleeLocalsAtYield): Deleted.
* bytecode/HandlerInfo.h:
(JSC::HandlerInfoBase::handlerForIndex):
* bytecode/Opcode.h:
(JSC::isBranch):
(JSC::isUnconditionalBranch):
(JSC::isTerminal):
(JSC::isThrow):
* bytecode/PreciseJumpTargets.cpp:
(JSC::getJumpTargetsForBytecodeOffset):
(JSC::computePreciseJumpTargetsInternal):
(JSC::computePreciseJumpTargets):
(JSC::recomputePreciseJumpTargets):
(JSC::findJumpTargetsForBytecodeOffset):
* bytecode/PreciseJumpTargets.h:
* bytecode/PreciseJumpTargetsInlines.h: Added.
(JSC::extractStoredJumpTargetsForBytecodeOffset):
* bytecode/UnlinkedCodeBlock.cpp:
(JSC::UnlinkedCodeBlock::handlerForBytecodeOffset):
(JSC::UnlinkedCodeBlock::handlerForIndex):
(JSC::UnlinkedCodeBlock::applyModification):
* bytecode/UnlinkedCodeBlock.h:
(JSC::UnlinkedStringJumpTable::offsetForValue):
(JSC::UnlinkedCodeBlock::numCalleeLocals):
* bytecode/VirtualRegister.h:
* bytecompiler/BytecodeGenerator.cpp:
(JSC::BytecodeGenerator::generate):
(JSC::BytecodeGenerator::BytecodeGenerator):
(JSC::BytecodeGenerator::emitComplexPopScopes):
(JSC::prepareJumpTableForStringSwitch):
(JSC::BytecodeGenerator::emitYieldPoint):
(JSC::BytecodeGenerator::emitSave): Deleted.
(JSC::BytecodeGenerator::emitResume): Deleted.
(JSC::BytecodeGenerator::emitGeneratorStateLabel): Deleted.
(JSC::BytecodeGenerator::beginGenerator): Deleted.
(JSC::BytecodeGenerator::endGenerator): Deleted.
* bytecompiler/BytecodeGenerator.h:
(JSC::BytecodeGenerator::generatorStateRegister):
(JSC::BytecodeGenerator::generatorValueRegister):
(JSC::BytecodeGenerator::generatorResumeModeRegister):
(JSC::BytecodeGenerator::generatorFrameRegister):
* bytecompiler/NodesCodegen.cpp:
(JSC::FunctionNode::emitBytecode):
* dfg/DFGOperations.cpp:
* interpreter/Interpreter.cpp:
(JSC::findExceptionHandler):
(JSC::GetCatchHandlerFunctor::operator()):
(JSC::UnwindFunctor::operator()):
* interpreter/Interpreter.h:
* interpreter/InterpreterInlines.h: Copied from Source/JavaScriptCore/bytecode/PreciseJumpTargets.h.
(JSC::Interpreter::getOpcodeID):
* jit/JIT.cpp:
(JSC::JIT::privateCompileMainPass):
* jit/JIT.h:
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_save): Deleted.
(JSC::JIT::emit_op_resume): Deleted.
* llint/LowLevelInterpreter.asm:
* parser/Parser.cpp:
(JSC::Parser<LexerType>::parseInner):
(JSC::Parser<LexerType>::parseGeneratorFunctionSourceElements):
(JSC::Parser<LexerType>::createGeneratorParameters):
* parser/Parser.h:
* runtime/CommonSlowPaths.cpp:
(JSC::SLOW_PATH_DECL): Deleted.
* runtime/CommonSlowPaths.h:
* runtime/GeneratorFrame.cpp: Removed.
(JSC::GeneratorFrame::GeneratorFrame): Deleted.
(JSC::GeneratorFrame::finishCreation): Deleted.
(JSC::GeneratorFrame::createStructure): Deleted.
(JSC::GeneratorFrame::create): Deleted.
(JSC::GeneratorFrame::save): Deleted.
(JSC::GeneratorFrame::resume): Deleted.
(JSC::GeneratorFrame::visitChildren): Deleted.
* runtime/GeneratorFrame.h: Removed.
(JSC::GeneratorFrame::locals): Deleted.
(JSC::GeneratorFrame::localAt): Deleted.
(JSC::GeneratorFrame::offsetOfLocals): Deleted.
(JSC::GeneratorFrame::allocationSizeForLocals): Deleted.
* runtime/JSGeneratorFunction.h:
* runtime/VM.cpp:
(JSC::VM::VM):
* runtime/VM.h:
Source/WTF:
* wtf/FastBitVector.h:
(WTF::FastBitVector::FastBitVector):
Canonical link: https://commits.webkit.org/179373@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@204994 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-08-25 22:55:10 +00:00
|
|
|
#pragma once
|
2012-05-22 00:51:05 +00:00
|
|
|
|
2013-11-12 20:35:34 +00:00
|
|
|
#include <string.h>
|
Make MarkedBlock state tracking support overlapped allocation and marking state
https://bugs.webkit.org/show_bug.cgi?id=161581
Reviewed by Geoffrey Garen.
JSTests:
Add a microbenchmark for why we want to reclaim empty blocks from other allocators.
* microbenchmarks/switching-size-classes.js: Added.
Source/JavaScriptCore:
Concurrent GCs must allow for mutation and allocation during collection. We already know
how to mutate during collection. We have a write barrier for that. Allocation during
collection is more involved: the collector modifies the the mark bits, as well as other
kinds of MarkedBlock state, in-place during a collection. The allocator uses that same
MarkedBlock state to decide which regions of memory are free. This works if the allocator
never runs while the collector is running, but if we want to allow them to run at the same
time, then we need to have two versions of the state: one version built up by the
collector and another consumed by the allocator. We clear the collector state at the
beginning of collection, and splat the collector state onto the allocator state after
collection.
This could be super expensive, but we can make it cheap with some cleverness. The biggest
observation is just that most of the state is a handful of bits per block: is the block
free-listed? is it completely full? completely empty? in the incremental sweeper's
snapshot? is it retired? is it in eden? There is also state inside blocks, like the mark
bits, but I have a solid plan there and I'll save it for another patch. Once we view the
state of blocks as bits, we can put that state into bitvectors, so that if the collector
needs to transform the state of some blocks, it can do it with a single operation over
bitvectors. I like to think of this as 32-way parallelizing block operations, since
doing one operation on a 32-bit word in one of those bitvectors instantly affects 32
blocks.
This change converts all previous collections of MarkedBlocks, along with the MarkedBlock
state, into 8 bitvectors (live, empty, allocated, canAllocateButNotEmpty, eden, unswept,
markingNotEmpty, and markingRetired). The bitvectors separate allocator state (empty,
allocated, canAllocateButNotEmpty) from marking state (markingNotEmpty, markingRetired).
As a nice side-effect of switching to bitvectors, we get size class rebalancing for free.
It used to be that if a MarkedAllocator had an empty block, we would only allow that
memory to be reused by a different MarkedAllocator if we did an incremental sweep or a
full eager sweep. Now we hunt down all destructorless empty blocks before allocating new
MarkedBlocks. It would be relatively easy to also hunt down destructor empty blocks, but
the theory is that those might be expensive to sweep, so it might still be better to leave
those to the incremental sweeper.
This change is perf-neutral all around. I did some tests with two different kinds of
allocation strategies - something that is somewhat easier to do now that you can look for
blocks that are candidates for allocation by just scanning some bitvectors. I tried two
variants:
- Allocate out of non-empty blocks first, leaving empty blocks for last in case a
different allocator needed them. This is sort of a best-fit strategy. I tried this
first, and it can be expressed as:
m_allocationCursor = m_canAllocateButNotEmpty.findBit(m_allocationCursor, true)
- Allocate out of lower-indexed blocks first, treating empty and canAllocateButNotEmpty
blocks equally. This is sort of a first-fit strategy. This is what I ended up settling
on, and it can be expressed as:
m_allocationCursor = (m_canAllocateButNotEmpty | m_empty).findBit(m_allocationCursor, true)
The best-fit strategy meant 1% regressions in LongSpider and Octane overall, and a 11%
regression on Octane/earley. First-fit means perf-neutrality. Most great allocators skew
towards first-fit because it's empirically better, so this result is not surprising.
Overall, the performance of this patch on my machine is as follows, where "neutral" means
less than 1% and not statistically significant.
run-jsc-benchmarks:
SunSpider: neutral
LongSpider: 0.6% slower
V8Spider: neutral
Octane: neutral
Kraken: neutral
Microbenchmarks: 0.37% slower
AsmBench: neutral
CompressionBench: maybe 1% faster
For browser benchmarks, I report the ratio of means (bigger / smaller) along with a T-test
from Mathematica reported as % chance of not [sic] the null hypothesis. Note that we
normally consider anything less than 95% confidence to be inconclusive.
Browser benchmarks:
PLT3: 0.3% faster with 67% confidence
membuster:
Snap2FinishedLoadingPost: 0.68% more memory with 50% confidence
Snap3EndPost: 2.4% more memory with 61% confidence
JetStream: 0.2% slower with 32% confidence
Speedometer: 0.7% faster with 82% confidence
Additionally, Octane/splay's heap capacity goes down to ~180KB from ~200KB, so about a 10%
progression. This is due to the allocator rebalancing feature.
Finally, this breaks --useImmortalObjects. It was already broken as far as I can tell. I
filed a bug to reimplement it (bug 162296). Unless someone urgently needs this internal
tool, it's probably best to reimplement it after I'm done refactoring MarkedSpace.
* JavaScriptCore.xcodeproj/project.pbxproj:
* debugger/Debugger.cpp:
* heap/CellContainer.h:
* heap/CellContainerInlines.h:
(JSC::CellContainer::vm):
(JSC::CellContainer::heap):
(JSC::CellContainer::isMarkedOrNewlyAllocated):
(JSC::CellContainer::aboutToMark):
(JSC::CellContainer::isMarked): Deleted.
(JSC::CellContainer::flipIfNecessary): Deleted.
* heap/ConservativeRoots.cpp:
* heap/Heap.cpp:
(JSC::Heap::beginMarking):
(JSC::Heap::endMarking):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collectImpl):
(JSC::Heap::snapshotMarkedSpace):
(JSC::Heap::prepareForAllocation):
(JSC::Heap::zombifyDeadObjects):
(JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): Deleted.
(JSC::MarkedBlockSnapshotFunctor::operator()): Deleted.
(JSC::Heap::resetAllocators): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isMarked):
(JSC::Heap::isMarkedConcurrently):
(JSC::Heap::testAndSetMarked):
* heap/HeapStatistics.cpp:
* heap/HeapUtil.h:
(JSC::HeapUtil::findGCObjectPointersForMarking):
(JSC::HeapUtil::isPointerGCObjectJSCell):
* heap/HeapVerifier.cpp:
* heap/IncrementalSweeper.cpp:
(JSC::IncrementalSweeper::IncrementalSweeper):
(JSC::IncrementalSweeper::doSweep):
(JSC::IncrementalSweeper::sweepNextBlock):
(JSC::IncrementalSweeper::startSweeping):
(JSC::IncrementalSweeper::willFinishSweeping):
* heap/IncrementalSweeper.h:
* heap/LargeAllocation.h:
(JSC::LargeAllocation::isMarked):
(JSC::LargeAllocation::isMarkedConcurrently):
(JSC::LargeAllocation::isMarkedOrNewlyAllocated):
(JSC::LargeAllocation::aboutToMark):
(JSC::LargeAllocation::isMarkedDuringWeakVisiting): Deleted.
(JSC::LargeAllocation::flipIfNecessary): Deleted.
(JSC::LargeAllocation::flipIfNecessaryDuringMarking): Deleted.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::isPagedOut):
(JSC::MarkedAllocator::findEmptyBlock):
(JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
(JSC::MarkedAllocator::allocateIn):
(JSC::MarkedAllocator::tryAllocateIn):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
(JSC::MarkedAllocator::tryAllocateBlock):
(JSC::MarkedAllocator::addBlock):
(JSC::MarkedAllocator::removeBlock):
(JSC::MarkedAllocator::stopAllocating):
(JSC::MarkedAllocator::prepareForAllocation):
(JSC::MarkedAllocator::lastChanceToFinalize):
(JSC::MarkedAllocator::resumeAllocating):
(JSC::MarkedAllocator::beginMarkingForFullCollection):
(JSC::MarkedAllocator::endMarking):
(JSC::MarkedAllocator::snapshotForEdenCollection):
(JSC::MarkedAllocator::snapshotForFullCollection):
(JSC::MarkedAllocator::findBlockToSweep):
(JSC::MarkedAllocator::sweep):
(JSC::MarkedAllocator::shrink):
(JSC::MarkedAllocator::assertSnapshotEmpty):
(JSC::MarkedAllocator::dump):
(JSC::MarkedAllocator::dumpBits):
(JSC::MarkedAllocator::retire): Deleted.
(JSC::MarkedAllocator::filterNextBlock): Deleted.
(JSC::MarkedAllocator::setNextBlockToSweep): Deleted.
(JSC::MarkedAllocator::reset): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::forEachBitVector):
(JSC::MarkedAllocator::forEachBitVectorWithName):
(JSC::MarkedAllocator::nextAllocator):
(JSC::MarkedAllocator::setNextAllocator):
(JSC::MarkedAllocator::forEachBlock):
(JSC::MarkedAllocator::resumeAllocating): Deleted.
* heap/MarkedBlock.cpp:
(JSC::MarkedBlock::tryCreate):
(JSC::MarkedBlock::Handle::Handle):
(JSC::MarkedBlock::Handle::~Handle):
(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::Handle::specializedSweep):
(JSC::MarkedBlock::Handle::sweep):
(JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectEmptyMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectHasNewlyAllocated):
(JSC::MarkedBlock::Handle::sweepHelperSelectSweepMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectFlipMode):
(JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated):
(JSC::MarkedBlock::Handle::setIsFreeListed):
(JSC::MarkedBlock::Handle::stopAllocating):
(JSC::MarkedBlock::Handle::lastChanceToFinalize):
(JSC::MarkedBlock::Handle::resumeAllocating):
(JSC::MarkedBlock::aboutToMarkSlow):
(JSC::MarkedBlock::clearMarks):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::didConsumeFreeList):
(JSC::MarkedBlock::markCount):
(JSC::MarkedBlock::Handle::isEmpty):
(JSC::MarkedBlock::noteMarkedSlow):
(JSC::MarkedBlock::Handle::removeFromAllocator):
(JSC::MarkedBlock::Handle::didAddToAllocator):
(JSC::MarkedBlock::Handle::didRemoveFromAllocator):
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessarySlow): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarkingSlow): Deleted.
(JSC::MarkedBlock::Handle::willRemoveBlock): Deleted.
(WTF::printInternal): Deleted.
* heap/MarkedBlock.h:
(JSC::MarkedBlock::Handle::isFreeListed):
(JSC::MarkedBlock::Handle::index):
(JSC::MarkedBlock::aboutToMark):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::isMarkedConcurrently):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::isOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::state): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipForEdenCollection): Deleted.
(JSC::MarkedBlock::isMarkedDuringWeakVisiting): Deleted.
(JSC::MarkedBlock::Handle::isLive): Deleted.
(JSC::MarkedBlock::Handle::isLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachDeadCell): Deleted.
(JSC::MarkedBlock::Handle::needsSweeping): Deleted.
(JSC::MarkedBlock::Handle::isAllocated): Deleted.
(JSC::MarkedBlock::Handle::isMarked): Deleted.
* heap/MarkedBlockInlines.h: Added.
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::forEachLiveCell):
(JSC::MarkedBlock::Handle::forEachDeadCell):
(JSC::MarkedBlock::resetVersion):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::MarkedSpace):
(JSC::MarkedSpace::allocate):
(JSC::MarkedSpace::tryAllocate):
(JSC::MarkedSpace::sweep):
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::shrink):
(JSC::MarkedSpace::clearNewlyAllocated):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::endMarking):
(JSC::MarkedSpace::didAllocateInBlock):
(JSC::MarkedSpace::findEmptyBlock):
(JSC::MarkedSpace::snapshot):
(JSC::MarkedSpace::assertSnapshotEmpty):
(JSC::MarkedSpace::dumpBits):
(JSC::MarkedSpace::zombifySweep): Deleted.
(JSC::MarkedSpace::resetAllocators): Deleted.
(JSC::VerifyMarked::operator()): Deleted.
(JSC::MarkedSpace::flip): Deleted.
* heap/MarkedSpace.h:
(JSC::MarkedSpace::nextVersion):
(JSC::MarkedSpace::firstAllocator):
(JSC::MarkedSpace::allocatorForEmptyAllocation):
(JSC::MarkedSpace::forEachAllocator):
(JSC::MarkedSpace::blocksWithNewObjects): Deleted.
(JSC::MarkedSpace::setIsMarking): Deleted.
(JSC::MarkedSpace::forEachLiveCell): Deleted.
(JSC::MarkedSpace::forEachDeadCell): Deleted.
* heap/MarkedSpaceInlines.h: Added.
(JSC::MarkedSpace::forEachLiveCell):
(JSC::MarkedSpace::forEachDeadCell):
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::setMarkedAndAppendToMarkStack):
(JSC::SlotVisitor::markAuxiliary):
(JSC::SlotVisitor::visitChildren):
* heap/Weak.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue):
(WTF::HashTraits<JSC::Weak<T>>::peek):
* heap/WeakBlock.cpp:
(JSC::WeakBlock::specializedVisit):
(JSC::WeakBlock::reap):
* heap/WeakInlines.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue): Deleted.
(WTF::HashTraits<JSC::Weak<T>>::peek): Deleted.
* jit/JITThunks.h:
* runtime/JSGlobalObject.cpp:
* runtime/PrototypeMap.h:
* runtime/SamplingProfiler.cpp:
* runtime/WeakGCMap.h:
* tools/JSDollarVMPrototype.cpp:
Source/WTF:
The main change here is to bring back FastBitVector.cpp, so that I could outline some
large slow path functions. This also adds some utilities, like atomicSetAndCheck() and
isEmpty(). The GC uses these.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/FastBitVector.cpp: Added.
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorImpl::isEmpty):
(WTF::FastBitVector::atomicSetAndCheck):
(WTF::FastBitVector::operator[]): Deleted.
Tools:
Remove the always-trigger-copy-phase configuration.
* Scripts/run-jsc-stress-tests:
Canonical link: https://commits.webkit.org/180306@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206154 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-20 18:12:18 +00:00
|
|
|
#include <wtf/Atomics.h>
|
2012-05-22 00:51:05 +00:00
|
|
|
#include <wtf/FastMalloc.h>
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
#include <wtf/PrintStream.h>
|
2012-05-22 00:51:05 +00:00
|
|
|
#include <wtf/StdLibExtras.h>
|
|
|
|
|
|
|
|
namespace WTF {
|
|
|
|
|
2013-11-27 23:15:48 +00:00
|
|
|
class PrintStream;
|
|
|
|
|
Experiment: create lots of different malloc zones for easier accounting of memory use
https://bugs.webkit.org/show_bug.cgi?id=186422
Patch by Yusuke Suzuki <ysuzuki@apple.com> and Simon Fraser <simon.fraser@apple.com> on 2020-01-02
Reviewed by Saam Barati.
Source/bmalloc:
* bmalloc/BPlatform.h:
* bmalloc/Environment.cpp:
(bmalloc::Environment::computeIsDebugHeapEnabled):
* bmalloc/IsoHeap.h:
(bmalloc::api::IsoHeap::IsoHeap):
* bmalloc/IsoHeapInlines.h:
(bmalloc::api::IsoHeap<Type>::IsoHeap):
* bmalloc/IsoTLSInlines.h:
(bmalloc::IsoTLS::allocateSlow):
(bmalloc::IsoTLS::deallocateSlow):
Source/JavaScriptCore:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* assembler/AssemblerBuffer.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* assembler/AssemblerBuffer.h:
(JSC::AssemblerData::AssemblerData):
(JSC::AssemblerData::operator=):
(JSC::AssemblerData::~AssemblerData):
(JSC::AssemblerData::grow):
* bytecode/AccessCase.cpp:
* bytecode/AccessCase.h:
* bytecode/BytecodeBasicBlock.cpp:
* bytecode/BytecodeBasicBlock.h:
* bytecode/CodeBlock.cpp:
* bytecode/CodeBlock.h:
* bytecode/InstructionStream.cpp:
* bytecode/InstructionStream.h:
* bytecode/PolymorphicAccess.cpp:
* bytecode/PolymorphicAccess.h:
* bytecode/UnlinkedMetadataTable.cpp:
(JSC::UnlinkedMetadataTable::finalize):
* bytecode/UnlinkedMetadataTable.h:
* bytecode/UnlinkedMetadataTableInlines.h:
(JSC::UnlinkedMetadataTable::UnlinkedMetadataTable):
(JSC::UnlinkedMetadataTable::~UnlinkedMetadataTable):
(JSC::UnlinkedMetadataTable::link):
(JSC::UnlinkedMetadataTable::unlink):
* bytecode/ValueProfile.h:
(JSC::ValueProfileAndVirtualRegisterBuffer::ValueProfileAndVirtualRegisterBuffer):
* bytecode/Watchpoint.cpp:
* bytecode/Watchpoint.h:
* dfg/DFGBasicBlock.cpp:
* dfg/DFGBasicBlock.h:
* dfg/DFGNode.cpp:
* dfg/DFGNode.h:
* dfg/DFGSpeculativeJIT.cpp:
* dfg/DFGSpeculativeJIT.h:
* heap/BlockDirectory.cpp:
* heap/BlockDirectory.h:
* heap/FastMallocAlignedMemoryAllocator.cpp:
(JSC::FastMallocAlignedMemoryAllocator::FastMallocAlignedMemoryAllocator):
(JSC::FastMallocAlignedMemoryAllocator::tryAllocateAlignedMemory):
(JSC::FastMallocAlignedMemoryAllocator::freeAlignedMemory):
(JSC::FastMallocAlignedMemoryAllocator::tryAllocateMemory):
(JSC::FastMallocAlignedMemoryAllocator::freeMemory):
(JSC::FastMallocAlignedMemoryAllocator::tryReallocateMemory):
* heap/FastMallocAlignedMemoryAllocator.h:
* heap/GCSegmentedArray.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp.
* heap/GCSegmentedArray.h:
* heap/GCSegmentedArrayInlines.h:
(JSC::GCArraySegment<T>::create):
(JSC::GCArraySegment<T>::destroy):
* heap/GigacageAlignedMemoryAllocator.cpp:
(JSC::GigacageAlignedMemoryAllocator::GigacageAlignedMemoryAllocator):
(JSC::GigacageAlignedMemoryAllocator::tryAllocateAlignedMemory):
(JSC::GigacageAlignedMemoryAllocator::freeAlignedMemory):
(JSC::GigacageAlignedMemoryAllocator::tryAllocateMemory):
(JSC::GigacageAlignedMemoryAllocator::freeMemory):
(JSC::GigacageAlignedMemoryAllocator::tryReallocateMemory):
* heap/GigacageAlignedMemoryAllocator.h:
* heap/IsoAlignedMemoryAllocator.cpp:
(JSC::IsoAlignedMemoryAllocator::IsoAlignedMemoryAllocator):
(JSC::IsoAlignedMemoryAllocator::~IsoAlignedMemoryAllocator):
(JSC::IsoAlignedMemoryAllocator::tryAllocateAlignedMemory):
(JSC::IsoAlignedMemoryAllocator::freeAlignedMemory):
(JSC::IsoAlignedMemoryAllocator::tryAllocateMemory):
(JSC::IsoAlignedMemoryAllocator::freeMemory):
* heap/IsoAlignedMemoryAllocator.h:
* heap/IsoSubspace.cpp:
(JSC::IsoSubspace::IsoSubspace):
* heap/MarkedBlock.cpp:
* heap/MarkedBlock.h:
* heap/WeakBlock.cpp:
(JSC::WeakBlock::create):
(JSC::WeakBlock::destroy):
* heap/WeakBlock.h:
* jit/JITCode.cpp:
* jit/JITCode.h:
* jit/RegisterAtOffsetList.cpp:
* jit/RegisterAtOffsetList.h:
* parser/Nodes.cpp:
* parser/Nodes.h:
* parser/ParserArena.cpp:
(JSC::ParserArena::deallocateObjects):
(JSC::ParserArena::allocateFreeablePool):
* parser/ParserArena.h:
* parser/SourceProvider.cpp:
* parser/SourceProvider.h:
* parser/SourceProviderCache.cpp:
* parser/SourceProviderCache.h:
* parser/SourceProviderCacheItem.h:
(JSC::SourceProviderCacheItem::create):
* runtime/CachePayload.cpp:
(JSC::CachePayload::makeMallocPayload):
* runtime/CachePayload.h:
* runtime/CachedBytecode.h:
(JSC::CachedBytecode::create):
* runtime/CachedTypes.cpp:
(JSC::Encoder::release):
(JSC::Encoder::Page::Page):
(JSC::CachedVector::encode):
(JSC::CachedVector::decode const):
(JSC::CachedInstructionStream::decode const):
* runtime/PropertyMapHashTable.h:
(JSC::PropertyTable::rehash):
* runtime/PropertyTable.cpp:
(JSC::PropertyTable::PropertyTable):
(JSC::PropertyTable::~PropertyTable):
* runtime/SymbolTable.cpp:
* runtime/SymbolTable.h:
* runtime/VM.cpp:
(JSC::VM::~VM):
* runtime/VM.h:
(JSC::ScratchBuffer::create):
(JSC::VM::exceptionFuzzingBuffer):
* wasm/WasmInstance.cpp:
(JSC::Wasm::Instance::Instance):
* wasm/WasmInstance.h:
* wasm/WasmTable.cpp:
(JSC::Wasm::Table::Table):
(JSC::Wasm::FuncRefTable::FuncRefTable):
* wasm/WasmTable.h:
Source/WebCore:
* Sources.txt:
* WebCore.xcodeproj/project.pbxproj:
* bindings/js/SerializedScriptValue.cpp:
* bindings/js/SerializedScriptValue.h:
* css/CSSFontFace.cpp:
* css/CSSFontFace.h:
* css/CSSSelector.cpp:
* css/CSSSelector.h:
* css/CSSValue.cpp:
* css/CSSValue.h:
* css/StyleProperties.cpp:
(WebCore::ImmutableStyleProperties::create):
* css/StyleProperties.h:
* css/StyleRule.cpp:
* css/StyleRule.h:
* dom/ElementData.cpp:
(WebCore::ShareableElementData::createWithAttributes):
(WebCore::UniqueElementData::makeShareableCopy const):
* dom/ElementData.h:
* dom/NodeRareData.cpp:
* dom/NodeRareData.h:
* dom/QualifiedName.cpp:
* dom/QualifiedName.h:
* html/parser/HTMLDocumentParser.cpp:
* html/parser/HTMLDocumentParser.h:
* loader/DocumentLoader.cpp:
* loader/DocumentLoader.h:
* loader/ResourceLoader.cpp:
* loader/ResourceLoader.h:
* loader/cache/CachedResource.cpp:
* loader/cache/CachedResource.h:
* page/PerformanceEntry.cpp:
* page/PerformanceEntry.h:
* platform/graphics/Font.cpp:
* platform/graphics/Font.h:
* platform/graphics/FontCascadeFonts.cpp:
* platform/graphics/FontCascadeFonts.h:
* platform/graphics/Region.cpp:
* platform/graphics/Region.h:
* platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm:
(WebCore::releaseUint8Vector):
* platform/graphics/cg/ImageBufferCG.cpp:
(WebCore::ImageBuffer::ImageBuffer):
* platform/graphics/nicosia/NicosiaBuffer.cpp:
(Nicosia::Buffer::Buffer):
* platform/network/ResourceHandle.cpp:
* platform/network/ResourceHandleInternal.h:
* platform/network/cf/FormDataStreamCFNet.cpp:
(WebCore::closeCurrentStream):
(WebCore::advanceCurrentStream):
* rendering/RenderLayer.cpp:
* rendering/RenderLayer.h:
* rendering/TableLayout.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp.
* rendering/TableLayout.h:
* rendering/style/RenderStyle.cpp:
* rendering/style/RenderStyle.h:
* rendering/style/SVGRenderStyle.cpp:
* rendering/style/SVGRenderStyle.h:
* rendering/style/SVGRenderStyleDefs.cpp:
* rendering/style/SVGRenderStyleDefs.h:
* rendering/style/StyleBoxData.cpp:
* rendering/style/StyleBoxData.h:
* rendering/style/StyleInheritedData.cpp:
* rendering/style/StyleInheritedData.h:
* rendering/style/StyleRareInheritedData.cpp:
* rendering/style/StyleRareInheritedData.h:
* rendering/style/StyleRareNonInheritedData.cpp:
* rendering/style/StyleRareNonInheritedData.h:
* rendering/style/StyleSurroundData.cpp:
* rendering/style/StyleSurroundData.h:
* rendering/style/StyleTransformData.cpp:
* rendering/style/StyleTransformData.h:
* style/StyleTreeResolver.cpp:
* style/StyleTreeResolver.h:
* svg/animation/SMILTimeContainer.cpp:
* svg/animation/SMILTimeContainer.h:
Source/WebKit:
* Shared/ShareableBitmap.cpp:
(WebKit::ShareableBitmap::create):
(WebKit::ShareableBitmap::~ShareableBitmap):
* UIProcess/mac/LegacySessionStateCoding.cpp:
(WebKit::HistoryEntryDataEncoder::HistoryEntryDataEncoder):
(WebKit::HistoryEntryDataEncoder::finishEncoding):
(WebKit::encodeSessionHistoryEntryData):
(WebKit::encodeLegacySessionState):
Source/WTF:
This patch introduces ENABLE(MALLOC_HEAP_BREAKDOWN). If this is enabled, we allocate malloc_zone per malloc kind.
This offers the way to investigate the usage of memory per kind by using vmmap, like the following.
VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION
MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT
=========== ======= ========= ========= ========= ========= ========= ========= ====== ======
StringImpl_0x116efd000 188.0M 69.3M 30.9M 0K 139456 18.0M 12.9M 42% 34
DefaultMallocZone_0x10f487000 176.0M 53.9M 14.1M 0K 115956 9955K 4497K 32% 22
Vector_0x116eff000 162.0M 56.3M 55.3M 0K 140715 17.3M 37.9M 69% 36
MetadataTable_0x11843b000 152.0M 17.5M 17.5M 0K 14200 2353K 15.2M 87% 26
WebKit Using System Malloc_0x114cbe000 150.0M 31.6M 21.8M 0K 87422 16.7M 5278K 24% 23
InstructionStream_0x118469000 150.0M 5764K 5764K 0K 14470 4688K 1076K 19% 24
AssemblerData_0x117ee6000 150.0M 1928K 1928K 0K 1 16 1928K 100% 24
To achieve this goal without making very large change, we put a template type in various containers.
For example, Vector will take Malloc parameter (the default one is FastMalloc allocator). If ENABLE(MALLOC_HEAP_BREAKDOWN) is enabled, we change this to
specific VectorMalloc allocator, and vmmap can show memory usage of this allocator. This patch also supports malloc_zone per IsoHeap. So we can see memory
allocation per IsoHeap in vmmap.
To use this feature, we need to flip two compile time flags, ENABLE(MALLOC_HEAP_BREAKDOWN) in WTF and BENABLE_MALLOC_HEAP_BREAKDOWN in bmalloc.
And use `vmmap $PID` to dump malloc zones. To allocate objects of a class with a specific malloc-zone, use WTF_MAKE_FAST_ALLOCATED_WITH_HEAP_IDENTIFIER(HeapIdentifier) for the class,
and define allocator by DECLARE_ALLOCATOR_WITH_HEAP_IDENTIFIER(HeapIdentifier) in a header and DEFINE_ALLOCATOR_WITH_HEAP_IDENTIFIER(HeapIdentifier) in a cpp file.
This patch also introduce callstack collector for malloc. Vector, HashMap etc. are used to allocate various things, but the above malloc_zone feature only tells thing like "Vector
takes XXX MB memory". But what we want to know in this case is what Vector is consuming memory. We collect StackShot for each malloc call, and combine these information to tell
which callsite is consuming much memory, which tell us that what Vector is consuming memory.
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp.
* wtf/Bag.h:
(WTF::Private::BagNode::BagNode): Deleted.
* wtf/BitVector.cpp:
(WTF::BitVector::OutOfLineBits::create):
(WTF::BitVector::OutOfLineBits::destroy):
* wtf/CMakeLists.txt:
* wtf/ConcurrentBuffer.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp.
* wtf/ConcurrentBuffer.h:
* wtf/DebugHeap.cpp: Copied from Source/JavaScriptCore/runtime/CachePayload.cpp.
(WTF::DebugHeap::DebugHeap):
(WTF::DebugHeap::malloc):
(WTF::DebugHeap::calloc):
(WTF::DebugHeap::memalign):
(WTF::DebugHeap::realloc):
(WTF::DebugHeap::free):
* wtf/DebugHeap.h: Added.
* wtf/FastBitVector.cpp:
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
* wtf/FastMalloc.cpp:
(WTF::fastMallocDumpMallocStats):
(WTF::AvoidRecordingScope::AvoidRecordingScope):
(WTF::AvoidRecordingScope::~AvoidRecordingScope):
(WTF::MallocCallTracker::MallocSiteData::MallocSiteData):
(WTF::MallocCallTracker::singleton):
(WTF::MallocCallTracker::MallocCallTracker):
(WTF::MallocCallTracker::recordMalloc):
(WTF::MallocCallTracker::recordRealloc):
(WTF::MallocCallTracker::recordFree):
(WTF::MallocCallTracker::dumpStats):
(WTF::fastMalloc):
(WTF::fastRealloc):
(WTF::fastFree):
(WTF::fastAlignedMalloc):
(WTF::tryFastAlignedMalloc):
(WTF::fastAlignedFree):
* wtf/FastMalloc.h:
(WTF::FastMalloc::zeroedMalloc):
(WTF::FastMalloc::tryZeroedMalloc):
* wtf/Forward.h:
* wtf/HashTable.cpp:
* wtf/HashTable.h:
(WTF::KeyTraits>::allocateTable):
(WTF::KeyTraits>::deallocateTable):
(WTF::KeyTraits>::rehash):
* wtf/MallocPtr.h:
(WTF::MallocPtr::MallocPtr):
(WTF::MallocPtr::malloc):
(WTF::MallocPtr::zeroedMalloc):
(WTF::MallocPtr::tryMalloc):
(WTF::MallocPtr::tryZeroedMalloc):
(WTF::adoptMallocPtr):
* wtf/MetaAllocator.cpp:
(WTF::MetaAllocator::allocFreeSpaceNode):
(WTF::MetaAllocator::freeFreeSpaceNode):
* wtf/MetaAllocatorHandle.h:
* wtf/Platform.h:
* wtf/RefCountedArray.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/RefCountedArray.h:
(WTF::RefCountedArray::RefCountedArray):
(WTF::RefCountedArray::~RefCountedArray):
(WTF::RefCountedArray::assign):
* wtf/SegmentedVector.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/SegmentedVector.h:
* wtf/SmallPtrSet.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/SmallPtrSet.h:
(WTF::SmallPtrSet::~SmallPtrSet):
(WTF::SmallPtrSet::grow):
* wtf/UniqueArray.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/UniqueArray.h:
(WTF::UniqueArrayFree::operator() const):
(WTF::UniqueArrayFree<T::operator() const):
* wtf/Vector.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/Vector.h:
(WTF::VectorBufferBase::allocateBuffer):
(WTF::VectorBufferBase::tryAllocateBuffer):
(WTF::VectorBufferBase::reallocateBuffer):
(WTF::VectorBufferBase::deallocateBuffer):
(WTF::VectorBufferBase::releaseBuffer):
(WTF::VectorBuffer::releaseBuffer):
(WTF::Vector::swap):
(WTF::Malloc>::Vector):
(WTF::=):
(WTF::Malloc>::contains const):
(WTF::Malloc>::findMatching const):
(WTF::Malloc>::find const):
(WTF::Malloc>::reverseFind const):
(WTF::Malloc>::appendIfNotContains):
(WTF::Malloc>::fill):
(WTF::Malloc>::appendRange):
(WTF::Malloc>::expandCapacity):
(WTF::Malloc>::tryExpandCapacity):
(WTF::Malloc>::resize):
(WTF::Malloc>::resizeToFit):
(WTF::Malloc>::shrink):
(WTF::Malloc>::grow):
(WTF::Malloc>::asanSetInitialBufferSizeTo):
(WTF::Malloc>::asanSetBufferSizeToFullCapacity):
(WTF::Malloc>::asanBufferSizeWillChangeTo):
(WTF::Malloc>::reserveCapacity):
(WTF::Malloc>::tryReserveCapacity):
(WTF::Malloc>::reserveInitialCapacity):
(WTF::Malloc>::shrinkCapacity):
(WTF::Malloc>::append):
(WTF::Malloc>::tryAppend):
(WTF::Malloc>::constructAndAppend):
(WTF::Malloc>::tryConstructAndAppend):
(WTF::Malloc>::appendSlowCase):
(WTF::Malloc>::constructAndAppendSlowCase):
(WTF::Malloc>::tryConstructAndAppendSlowCase):
(WTF::Malloc>::uncheckedAppend):
(WTF::Malloc>::uncheckedConstructAndAppend):
(WTF::Malloc>::appendVector):
(WTF::Malloc>::insert):
(WTF::Malloc>::insertVector):
(WTF::Malloc>::remove):
(WTF::Malloc>::removeFirst):
(WTF::Malloc>::removeFirstMatching):
(WTF::Malloc>::removeAll):
(WTF::Malloc>::removeAllMatching):
(WTF::Malloc>::reverse):
(WTF::Malloc>::map const):
(WTF::Malloc>::releaseBuffer):
(WTF::Malloc>::checkConsistency):
(WTF::swap):
(WTF::operator==):
(WTF::operator!=):
(WTF::Malloc>::isolatedCopy const):
(WTF::removeRepeatedElements):
(WTF::minCapacity>::Vector): Deleted.
(WTF::minCapacity>::contains const): Deleted.
(WTF::minCapacity>::findMatching const): Deleted.
(WTF::minCapacity>::find const): Deleted.
(WTF::minCapacity>::reverseFind const): Deleted.
(WTF::minCapacity>::appendIfNotContains): Deleted.
(WTF::minCapacity>::fill): Deleted.
(WTF::minCapacity>::appendRange): Deleted.
(WTF::minCapacity>::expandCapacity): Deleted.
(WTF::minCapacity>::tryExpandCapacity): Deleted.
(WTF::minCapacity>::resize): Deleted.
(WTF::minCapacity>::resizeToFit): Deleted.
(WTF::minCapacity>::shrink): Deleted.
(WTF::minCapacity>::grow): Deleted.
(WTF::minCapacity>::asanSetInitialBufferSizeTo): Deleted.
(WTF::minCapacity>::asanSetBufferSizeToFullCapacity): Deleted.
(WTF::minCapacity>::asanBufferSizeWillChangeTo): Deleted.
(WTF::minCapacity>::reserveCapacity): Deleted.
(WTF::minCapacity>::tryReserveCapacity): Deleted.
(WTF::minCapacity>::reserveInitialCapacity): Deleted.
(WTF::minCapacity>::shrinkCapacity): Deleted.
(WTF::minCapacity>::append): Deleted.
(WTF::minCapacity>::tryAppend): Deleted.
(WTF::minCapacity>::constructAndAppend): Deleted.
(WTF::minCapacity>::tryConstructAndAppend): Deleted.
(WTF::minCapacity>::appendSlowCase): Deleted.
(WTF::minCapacity>::constructAndAppendSlowCase): Deleted.
(WTF::minCapacity>::tryConstructAndAppendSlowCase): Deleted.
(WTF::minCapacity>::uncheckedAppend): Deleted.
(WTF::minCapacity>::uncheckedConstructAndAppend): Deleted.
(WTF::minCapacity>::appendVector): Deleted.
(WTF::minCapacity>::insert): Deleted.
(WTF::minCapacity>::insertVector): Deleted.
(WTF::minCapacity>::remove): Deleted.
(WTF::minCapacity>::removeFirst): Deleted.
(WTF::minCapacity>::removeFirstMatching): Deleted.
(WTF::minCapacity>::removeAll): Deleted.
(WTF::minCapacity>::removeAllMatching): Deleted.
(WTF::minCapacity>::reverse): Deleted.
(WTF::minCapacity>::map const): Deleted.
(WTF::minCapacity>::releaseBuffer): Deleted.
(WTF::minCapacity>::checkConsistency): Deleted.
(WTF::minCapacity>::isolatedCopy const): Deleted.
* wtf/text/CString.cpp:
(WTF::CStringBuffer::createUninitialized):
* wtf/text/CString.h:
* wtf/text/StringBuffer.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/text/StringBuffer.h:
(WTF::StringBuffer::StringBuffer):
(WTF::StringBuffer::~StringBuffer):
(WTF::StringBuffer::resize):
(WTF::StringBuffer::release):
* wtf/text/StringImpl.cpp:
(WTF::StringImpl::~StringImpl):
(WTF::StringImpl::destroy):
(WTF::StringImpl::createUninitializedInternalNonEmpty):
(WTF::StringImpl::reallocateInternal):
* wtf/text/StringImpl.h:
(WTF::StringImpl::StringImpl):
(WTF::StringImpl::createSubstringSharingImpl):
(WTF::StringImpl::tryCreateUninitialized):
(WTF::StringImpl::adopt):
* wtf/text/cf/StringImplCF.cpp:
(WTF::StringWrapperCFAllocator::allocate):
(WTF::StringWrapperCFAllocator::reallocate):
(WTF::StringWrapperCFAllocator::deallocate):
Canonical link: https://commits.webkit.org/218863@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@253987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-01-03 02:36:43 +00:00
|
|
|
DECLARE_ALLOCATOR_WITH_HEAP_IDENTIFIER(FastBitVector);
|
|
|
|
|
2019-11-14 09:37:12 +00:00
|
|
|
inline constexpr size_t fastBitVectorArrayLength(size_t numBits) { return (numBits + 31) / 32; }
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
|
|
|
|
class FastBitVectorWordView {
|
2019-08-12 20:57:15 +00:00
|
|
|
WTF_MAKE_FAST_ALLOCATED;
|
2012-05-22 00:51:05 +00:00
|
|
|
public:
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
typedef FastBitVectorWordView ViewType;
|
|
|
|
|
|
|
|
FastBitVectorWordView() { }
|
|
|
|
|
|
|
|
FastBitVectorWordView(const uint32_t* array, size_t numBits)
|
|
|
|
: m_words(array)
|
|
|
|
, m_numBits(numBits)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
size_t numBits() const
|
|
|
|
{
|
|
|
|
return m_numBits;
|
|
|
|
}
|
|
|
|
|
|
|
|
uint32_t word(size_t index) const
|
|
|
|
{
|
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(index < fastBitVectorArrayLength(numBits()));
|
|
|
|
return m_words[index];
|
|
|
|
}
|
|
|
|
|
|
|
|
private:
|
|
|
|
const uint32_t* m_words { nullptr };
|
|
|
|
size_t m_numBits { 0 };
|
|
|
|
};
|
[DFG][FTL] Implement ES6 Generators in DFG / FTL
https://bugs.webkit.org/show_bug.cgi?id=152723
Reviewed by Filip Pizlo.
JSTests:
* stress/generator-fib-ftl-and-array.js: Added.
(fib):
* stress/generator-fib-ftl-and-object.js: Added.
(fib):
* stress/generator-fib-ftl-and-string.js: Added.
(fib):
* stress/generator-fib-ftl.js: Added.
(fib):
* stress/generator-frame-empty.js: Added.
(shouldThrow):
(shouldThrow.fib):
* stress/generator-reduced-save-point-put-to-scope.js: Added.
(shouldBe):
(gen):
* stress/generator-transfer-register-beyond-mutiple-yields.js: Added.
(shouldBe):
(gen):
Source/JavaScriptCore:
This patch introduces DFG and FTL support for ES6 generators.
ES6 generator is compiled by the BytecodeGenerator. But at the last phase, BytecodeGenerator performs "generatorification" onto the unlinked code.
In BytecodeGenerator phase, we just emit op_yield for each yield point. And we don't emit any generator related switch, save, and resume sequences
here. Those are emitted by the generatorification phase.
So the graph is super simple! Before the generatorification, the graph looks like this.
op_enter -> ...... -> op_yield -> ..... -> op_yield -> ...
Roughly speaking, in the generatorification phase, we turn out which variables should be saved and resumed at each op_yield.
This is done by liveness analysis. After that, we convert op_yield to the sequence of "op_put_to_scope", "op_ret", and "op_get_from_scope".
op_put_to_scope and op_get_from_scope sequences are corresponding to the save and resume sequences. We set up the scope for the generator frame and
perform op_put_to_scope and op_get_from_scope onto it. The live registers are saved and resumed over the generator's next() calls by using this
special generator frame scope. And we also set up the global switch for the generator.
In the generatorification phase,
1. We construct the BytecodeGraph from the unlinked instructions. This constructs the basic blocks, and it is used in the subsequent analysis.
2. We perform the analysis onto the unlinked code. We extract the live variables at each op_yield.
3. We insert the get_from_scope and put_to_scope at each op_yield. Which registers should be saved and resumed is offered by (2).
Then, clip the op_yield themselves. And we also insert the switch_imm. The jump targets of this switch are just after this op_switch_imm and each op_yield point.
One interesting point is the try-range. We split the try-range at the op_yield point in BytecodeGenerator phase.
This drops the hacky thing that is introduced in [1].
If the try-range covers the resume sequences, the exception handler's use-registers are incorrectly transferred to the entry block.
For example,
handler uses r2
try-range
label:(entry block can jump here) ^
r1 = get_from_scope # resume sequence starts | use r2 is transferred to the entry block!
r2 = get_from_scope |
starts usual sequences |
... |
Handler's r2 use should be considered at the `r1 = get_from_scope` point.
Previously, we handle this edge case by treating op_resume specially in the liveness analysis[1].
To drop this workaround, we split the try-range not to cover this resume sequence.
handler uses r2
try-range
label:(entry block can jump here)
r1 = get_from_scope # resume sequence starts
r2 = get_from_scope
starts usual sequences ^ try-range should start from here.
... |
OK. Let's show the detailed example.
1. First, there is the normal bytecode sequence. Here, | represents the offsets, and [] represents the bytecodes.
bytecodes | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |
try-range <----------------------------------->
2. When we emit the op_yield in the bytecode generator, we carefully split the try-range.
bytecodes | [ ] | [ ] | [op_yield] | [ ] | [ ] | [ ] |
try-range <-----------> <----------------->
3. And in the generatorification phase, we insert the switch's jump target and save & resume sequences. And we also drop op_yield.
Insert save seq Insert resume seq
before op_yield. after op_yield's point.
v v
bytecodes | [ ] | [ ] | [op_yield] | [ ] | [ ] | [ ] |
try-range <-----------> ^ <----------------->
^ |
Jump to here. Drop this op_yield.
4. The final layout is the following.
bytecodes | [ ] | [ ][save seq][op_ret] | [resume seq] | [ ] | [ ] | [ ] |
try-range <-----------------------------> <---------------->
^
Jump to here.
The rewriting done by the BytecodeRewriter is executed in a batch manner. Since these modification changes the basic blocks and size of unlinked instructions,
BytecodeRewriter also performs the offset adjustment for UnlinkedCodeBlock. So, this rewriting is performed onto the BytecodeGraph rather than BytecodeBasicBlock.
The reason why we take this design is simple: we don't want to newly create the basic blocks and opcodes for this early phase like DFG. Instead, we perform the
modification and adjustment to the unlinked instructions and UnlinkedCodeBlock in a in-place manner.
Bytecode rewriting functionality is offered by BytecodeRewriter. BytecodeRewriter allows us to insert any bytecodes to any places
in a in-place manner. BytecodeRewriter handles the original bytecode offsets as labels. And you can insert bytecodes before and after
these labels. You can also insert any jumps to any places. When you insert jumps, you need to specify jump target with this labels.
These labels (original bytecode offsets) are automatically converted to the appropriate offsets by BytecodeRewriter.
After that phase, the data flow of the generator-saved-and-resumed-registers are explicitly represented by the get_from_scope and put_to_scope.
And the switch is inserted to represent the actual control flow for the generator. And op_yield is removed. Since we use the existing bytecodes (op_switch_imm, op_put_to_scope
op_ret, and op_get_from_scope), DFG and FTL changes are not necessary. This patch also drops data structures and implementations for the old generator,
op_resume, op_save implementations and GeneratorFrame.
Note that this patch does not leverage the recent multi entrypoints support in B3. After this patch is introduced, we will submit a new patch that leverages the multi
entrypoints for generator's resume and sees the performance gain.
Microbenchmarks related to generators show up to 2.9x improvements.
Baseline Patched
generator-fib 102.0116+-3.2880 ^ 34.9670+-0.2221 ^ definitely 2.9174x faster
generator-sunspider-access-nsieve 5.8596+-0.0371 ^ 4.9051+-0.0720 ^ definitely 1.1946x faster
generator-with-several-types 332.1478+-4.2425 ^ 124.6642+-2.4826 ^ definitely 2.6643x faster
<geometric> 58.2998+-0.7758 ^ 27.7425+-0.2577 ^ definitely 2.1015x faster
In ES6SampleBench's Basic, we can observe 41% improvement (Macbook Pro).
Baseline:
Geometric Mean Result: 133.55 ms +- 4.49 ms
Benchmark First Iteration Worst 2% Steady State
Air 54.03 ms +- 7.51 ms 29.06 ms +- 3.13 ms 2276.59 ms +- 61.17 ms
Basic 30.18 ms +- 1.86 ms 18.85 ms +- 0.45 ms 2851.16 ms +- 41.87 ms
Patched:
Geometric Mean Result: 121.78 ms +- 3.96 ms
Benchmark First Iteration Worst 2% Steady State
Air 52.09 ms +- 6.89 ms 29.59 ms +- 3.16 ms 2239.90 ms +- 54.60 ms
Basic 29.28 ms +- 1.46 ms 16.26 ms +- 0.66 ms 2025.15 ms +- 38.56 ms
[1]: https://bugs.webkit.org/show_bug.cgi?id=159281
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* builtins/GeneratorPrototype.js:
(globalPrivate.generatorResume):
* bytecode/BytecodeBasicBlock.cpp:
(JSC::BytecodeBasicBlock::shrinkToFit):
(JSC::BytecodeBasicBlock::computeImpl):
(JSC::BytecodeBasicBlock::compute):
(JSC::isBranch): Deleted.
(JSC::isUnconditionalBranch): Deleted.
(JSC::isTerminal): Deleted.
(JSC::isThrow): Deleted.
(JSC::linkBlocks): Deleted.
(JSC::computeBytecodeBasicBlocks): Deleted.
* bytecode/BytecodeBasicBlock.h:
(JSC::BytecodeBasicBlock::isEntryBlock):
(JSC::BytecodeBasicBlock::isExitBlock):
(JSC::BytecodeBasicBlock::leaderOffset):
(JSC::BytecodeBasicBlock::totalLength):
(JSC::BytecodeBasicBlock::offsets):
(JSC::BytecodeBasicBlock::successors):
(JSC::BytecodeBasicBlock::index):
(JSC::BytecodeBasicBlock::addSuccessor):
(JSC::BytecodeBasicBlock::BytecodeBasicBlock):
(JSC::BytecodeBasicBlock::addLength):
(JSC::BytecodeBasicBlock::leaderBytecodeOffset): Deleted.
(JSC::BytecodeBasicBlock::totalBytecodeLength): Deleted.
(JSC::BytecodeBasicBlock::bytecodeOffsets): Deleted.
(JSC::BytecodeBasicBlock::addBytecodeLength): Deleted.
* bytecode/BytecodeGeneratorification.cpp: Added.
(JSC::BytecodeGeneratorification::BytecodeGeneratorification):
(JSC::BytecodeGeneratorification::graph):
(JSC::BytecodeGeneratorification::yields):
(JSC::BytecodeGeneratorification::enterPoint):
(JSC::BytecodeGeneratorification::storageForGeneratorLocal):
(JSC::GeneratorLivenessAnalysis::GeneratorLivenessAnalysis):
(JSC::GeneratorLivenessAnalysis::computeDefsForBytecodeOffset):
(JSC::GeneratorLivenessAnalysis::computeUsesForBytecodeOffset):
(JSC::GeneratorLivenessAnalysis::run):
(JSC::BytecodeGeneratorification::run):
(JSC::performGeneratorification):
* bytecode/BytecodeGeneratorification.h: Copied from Source/JavaScriptCore/bytecode/BytecodeLivenessAnalysisInlines.h.
* bytecode/BytecodeGraph.h: Added.
(JSC::BytecodeGraph::codeBlock):
(JSC::BytecodeGraph::instructions):
(JSC::BytecodeGraph::basicBlocksInReverseOrder):
(JSC::BytecodeGraph::blockContainsBytecodeOffset):
(JSC::BytecodeGraph::findBasicBlockForBytecodeOffset):
(JSC::BytecodeGraph::findBasicBlockWithLeaderOffset):
(JSC::BytecodeGraph::size):
(JSC::BytecodeGraph::at):
(JSC::BytecodeGraph::operator[]):
(JSC::BytecodeGraph::begin):
(JSC::BytecodeGraph::end):
(JSC::BytecodeGraph::first):
(JSC::BytecodeGraph::last):
(JSC::BytecodeGraph<Block>::BytecodeGraph):
* bytecode/BytecodeList.json:
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::BytecodeLivenessAnalysis):
(JSC::BytecodeLivenessAnalysis::computeDefsForBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::computeUsesForBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::getLivenessInfoAtBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::computeFullLiveness):
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
(JSC::BytecodeLivenessAnalysis::compute):
(JSC::isValidRegisterForLiveness): Deleted.
(JSC::getLeaderOffsetForBasicBlock): Deleted.
(JSC::findBasicBlockWithLeaderOffset): Deleted.
(JSC::blockContainsBytecodeOffset): Deleted.
(JSC::findBasicBlockForBytecodeOffset): Deleted.
(JSC::stepOverInstruction): Deleted.
(JSC::computeLocalLivenessForBytecodeOffset): Deleted.
(JSC::computeLocalLivenessForBlock): Deleted.
(JSC::BytecodeLivenessAnalysis::runLivenessFixpoint): Deleted.
* bytecode/BytecodeLivenessAnalysis.h:
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::isValidRegisterForLiveness):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::computeLocalLivenessForBytecodeOffset):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::computeLocalLivenessForBlock):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::getLivenessInfoAtBytecodeOffset):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/BytecodeRewriter.cpp: Added.
(JSC::BytecodeRewriter::applyModification):
(JSC::BytecodeRewriter::execute):
(JSC::BytecodeRewriter::adjustJumpTargetsInFragment):
(JSC::BytecodeRewriter::insertImpl):
(JSC::BytecodeRewriter::adjustJumpTarget):
* bytecode/BytecodeRewriter.h: Added.
(JSC::BytecodeRewriter::InsertionPoint::InsertionPoint):
(JSC::BytecodeRewriter::InsertionPoint::operator<):
(JSC::BytecodeRewriter::InsertionPoint::operator==):
(JSC::BytecodeRewriter::Insertion::length):
(JSC::BytecodeRewriter::Fragment::Fragment):
(JSC::BytecodeRewriter::Fragment::appendInstruction):
(JSC::BytecodeRewriter::BytecodeRewriter):
(JSC::BytecodeRewriter::insertFragmentBefore):
(JSC::BytecodeRewriter::insertFragmentAfter):
(JSC::BytecodeRewriter::removeBytecode):
(JSC::BytecodeRewriter::graph):
(JSC::BytecodeRewriter::adjustAbsoluteOffset):
(JSC::BytecodeRewriter::adjustJumpTarget):
(JSC::BytecodeRewriter::calculateDifference):
* bytecode/BytecodeUseDef.h:
(JSC::computeUsesForBytecodeOffset):
(JSC::computeDefsForBytecodeOffset):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::dumpBytecode):
(JSC::CodeBlock::finishCreation):
(JSC::CodeBlock::handlerForIndex):
(JSC::CodeBlock::shrinkToFit):
(JSC::CodeBlock::valueProfileForBytecodeOffset):
(JSC::CodeBlock::livenessAnalysisSlow):
* bytecode/CodeBlock.h:
(JSC::CodeBlock::isConstantRegisterIndex):
(JSC::CodeBlock::livenessAnalysis):
(JSC::CodeBlock::liveCalleeLocalsAtYield): Deleted.
* bytecode/HandlerInfo.h:
(JSC::HandlerInfoBase::handlerForIndex):
* bytecode/Opcode.h:
(JSC::isBranch):
(JSC::isUnconditionalBranch):
(JSC::isTerminal):
(JSC::isThrow):
* bytecode/PreciseJumpTargets.cpp:
(JSC::getJumpTargetsForBytecodeOffset):
(JSC::computePreciseJumpTargetsInternal):
(JSC::computePreciseJumpTargets):
(JSC::recomputePreciseJumpTargets):
(JSC::findJumpTargetsForBytecodeOffset):
* bytecode/PreciseJumpTargets.h:
* bytecode/PreciseJumpTargetsInlines.h: Added.
(JSC::extractStoredJumpTargetsForBytecodeOffset):
* bytecode/UnlinkedCodeBlock.cpp:
(JSC::UnlinkedCodeBlock::handlerForBytecodeOffset):
(JSC::UnlinkedCodeBlock::handlerForIndex):
(JSC::UnlinkedCodeBlock::applyModification):
* bytecode/UnlinkedCodeBlock.h:
(JSC::UnlinkedStringJumpTable::offsetForValue):
(JSC::UnlinkedCodeBlock::numCalleeLocals):
* bytecode/VirtualRegister.h:
* bytecompiler/BytecodeGenerator.cpp:
(JSC::BytecodeGenerator::generate):
(JSC::BytecodeGenerator::BytecodeGenerator):
(JSC::BytecodeGenerator::emitComplexPopScopes):
(JSC::prepareJumpTableForStringSwitch):
(JSC::BytecodeGenerator::emitYieldPoint):
(JSC::BytecodeGenerator::emitSave): Deleted.
(JSC::BytecodeGenerator::emitResume): Deleted.
(JSC::BytecodeGenerator::emitGeneratorStateLabel): Deleted.
(JSC::BytecodeGenerator::beginGenerator): Deleted.
(JSC::BytecodeGenerator::endGenerator): Deleted.
* bytecompiler/BytecodeGenerator.h:
(JSC::BytecodeGenerator::generatorStateRegister):
(JSC::BytecodeGenerator::generatorValueRegister):
(JSC::BytecodeGenerator::generatorResumeModeRegister):
(JSC::BytecodeGenerator::generatorFrameRegister):
* bytecompiler/NodesCodegen.cpp:
(JSC::FunctionNode::emitBytecode):
* dfg/DFGOperations.cpp:
* interpreter/Interpreter.cpp:
(JSC::findExceptionHandler):
(JSC::GetCatchHandlerFunctor::operator()):
(JSC::UnwindFunctor::operator()):
* interpreter/Interpreter.h:
* interpreter/InterpreterInlines.h: Copied from Source/JavaScriptCore/bytecode/PreciseJumpTargets.h.
(JSC::Interpreter::getOpcodeID):
* jit/JIT.cpp:
(JSC::JIT::privateCompileMainPass):
* jit/JIT.h:
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_save): Deleted.
(JSC::JIT::emit_op_resume): Deleted.
* llint/LowLevelInterpreter.asm:
* parser/Parser.cpp:
(JSC::Parser<LexerType>::parseInner):
(JSC::Parser<LexerType>::parseGeneratorFunctionSourceElements):
(JSC::Parser<LexerType>::createGeneratorParameters):
* parser/Parser.h:
* runtime/CommonSlowPaths.cpp:
(JSC::SLOW_PATH_DECL): Deleted.
* runtime/CommonSlowPaths.h:
* runtime/GeneratorFrame.cpp: Removed.
(JSC::GeneratorFrame::GeneratorFrame): Deleted.
(JSC::GeneratorFrame::finishCreation): Deleted.
(JSC::GeneratorFrame::createStructure): Deleted.
(JSC::GeneratorFrame::create): Deleted.
(JSC::GeneratorFrame::save): Deleted.
(JSC::GeneratorFrame::resume): Deleted.
(JSC::GeneratorFrame::visitChildren): Deleted.
* runtime/GeneratorFrame.h: Removed.
(JSC::GeneratorFrame::locals): Deleted.
(JSC::GeneratorFrame::localAt): Deleted.
(JSC::GeneratorFrame::offsetOfLocals): Deleted.
(JSC::GeneratorFrame::allocationSizeForLocals): Deleted.
* runtime/JSGeneratorFunction.h:
* runtime/VM.cpp:
(JSC::VM::VM):
* runtime/VM.h:
Source/WTF:
* wtf/FastBitVector.h:
(WTF::FastBitVector::FastBitVector):
Canonical link: https://commits.webkit.org/179373@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@204994 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-08-25 22:55:10 +00:00
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
class FastBitVectorWordOwner {
|
2019-08-12 20:57:15 +00:00
|
|
|
WTF_MAKE_FAST_ALLOCATED;
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
public:
|
|
|
|
typedef FastBitVectorWordView ViewType;
|
|
|
|
|
|
|
|
FastBitVectorWordOwner() = default;
|
|
|
|
|
|
|
|
FastBitVectorWordOwner(FastBitVectorWordOwner&& other)
|
|
|
|
: m_words(std::exchange(other.m_words, nullptr))
|
[DFG][FTL] Implement ES6 Generators in DFG / FTL
https://bugs.webkit.org/show_bug.cgi?id=152723
Reviewed by Filip Pizlo.
JSTests:
* stress/generator-fib-ftl-and-array.js: Added.
(fib):
* stress/generator-fib-ftl-and-object.js: Added.
(fib):
* stress/generator-fib-ftl-and-string.js: Added.
(fib):
* stress/generator-fib-ftl.js: Added.
(fib):
* stress/generator-frame-empty.js: Added.
(shouldThrow):
(shouldThrow.fib):
* stress/generator-reduced-save-point-put-to-scope.js: Added.
(shouldBe):
(gen):
* stress/generator-transfer-register-beyond-mutiple-yields.js: Added.
(shouldBe):
(gen):
Source/JavaScriptCore:
This patch introduces DFG and FTL support for ES6 generators.
ES6 generator is compiled by the BytecodeGenerator. But at the last phase, BytecodeGenerator performs "generatorification" onto the unlinked code.
In BytecodeGenerator phase, we just emit op_yield for each yield point. And we don't emit any generator related switch, save, and resume sequences
here. Those are emitted by the generatorification phase.
So the graph is super simple! Before the generatorification, the graph looks like this.
op_enter -> ...... -> op_yield -> ..... -> op_yield -> ...
Roughly speaking, in the generatorification phase, we turn out which variables should be saved and resumed at each op_yield.
This is done by liveness analysis. After that, we convert op_yield to the sequence of "op_put_to_scope", "op_ret", and "op_get_from_scope".
op_put_to_scope and op_get_from_scope sequences are corresponding to the save and resume sequences. We set up the scope for the generator frame and
perform op_put_to_scope and op_get_from_scope onto it. The live registers are saved and resumed over the generator's next() calls by using this
special generator frame scope. And we also set up the global switch for the generator.
In the generatorification phase,
1. We construct the BytecodeGraph from the unlinked instructions. This constructs the basic blocks, and it is used in the subsequent analysis.
2. We perform the analysis onto the unlinked code. We extract the live variables at each op_yield.
3. We insert the get_from_scope and put_to_scope at each op_yield. Which registers should be saved and resumed is offered by (2).
Then, clip the op_yield themselves. And we also insert the switch_imm. The jump targets of this switch are just after this op_switch_imm and each op_yield point.
One interesting point is the try-range. We split the try-range at the op_yield point in BytecodeGenerator phase.
This drops the hacky thing that is introduced in [1].
If the try-range covers the resume sequences, the exception handler's use-registers are incorrectly transferred to the entry block.
For example,
handler uses r2
try-range
label:(entry block can jump here) ^
r1 = get_from_scope # resume sequence starts | use r2 is transferred to the entry block!
r2 = get_from_scope |
starts usual sequences |
... |
Handler's r2 use should be considered at the `r1 = get_from_scope` point.
Previously, we handle this edge case by treating op_resume specially in the liveness analysis[1].
To drop this workaround, we split the try-range not to cover this resume sequence.
handler uses r2
try-range
label:(entry block can jump here)
r1 = get_from_scope # resume sequence starts
r2 = get_from_scope
starts usual sequences ^ try-range should start from here.
... |
OK. Let's show the detailed example.
1. First, there is the normal bytecode sequence. Here, | represents the offsets, and [] represents the bytecodes.
bytecodes | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |
try-range <----------------------------------->
2. When we emit the op_yield in the bytecode generator, we carefully split the try-range.
bytecodes | [ ] | [ ] | [op_yield] | [ ] | [ ] | [ ] |
try-range <-----------> <----------------->
3. And in the generatorification phase, we insert the switch's jump target and save & resume sequences. And we also drop op_yield.
Insert save seq Insert resume seq
before op_yield. after op_yield's point.
v v
bytecodes | [ ] | [ ] | [op_yield] | [ ] | [ ] | [ ] |
try-range <-----------> ^ <----------------->
^ |
Jump to here. Drop this op_yield.
4. The final layout is the following.
bytecodes | [ ] | [ ][save seq][op_ret] | [resume seq] | [ ] | [ ] | [ ] |
try-range <-----------------------------> <---------------->
^
Jump to here.
The rewriting done by the BytecodeRewriter is executed in a batch manner. Since these modification changes the basic blocks and size of unlinked instructions,
BytecodeRewriter also performs the offset adjustment for UnlinkedCodeBlock. So, this rewriting is performed onto the BytecodeGraph rather than BytecodeBasicBlock.
The reason why we take this design is simple: we don't want to newly create the basic blocks and opcodes for this early phase like DFG. Instead, we perform the
modification and adjustment to the unlinked instructions and UnlinkedCodeBlock in a in-place manner.
Bytecode rewriting functionality is offered by BytecodeRewriter. BytecodeRewriter allows us to insert any bytecodes to any places
in a in-place manner. BytecodeRewriter handles the original bytecode offsets as labels. And you can insert bytecodes before and after
these labels. You can also insert any jumps to any places. When you insert jumps, you need to specify jump target with this labels.
These labels (original bytecode offsets) are automatically converted to the appropriate offsets by BytecodeRewriter.
After that phase, the data flow of the generator-saved-and-resumed-registers are explicitly represented by the get_from_scope and put_to_scope.
And the switch is inserted to represent the actual control flow for the generator. And op_yield is removed. Since we use the existing bytecodes (op_switch_imm, op_put_to_scope
op_ret, and op_get_from_scope), DFG and FTL changes are not necessary. This patch also drops data structures and implementations for the old generator,
op_resume, op_save implementations and GeneratorFrame.
Note that this patch does not leverage the recent multi entrypoints support in B3. After this patch is introduced, we will submit a new patch that leverages the multi
entrypoints for generator's resume and sees the performance gain.
Microbenchmarks related to generators show up to 2.9x improvements.
Baseline Patched
generator-fib 102.0116+-3.2880 ^ 34.9670+-0.2221 ^ definitely 2.9174x faster
generator-sunspider-access-nsieve 5.8596+-0.0371 ^ 4.9051+-0.0720 ^ definitely 1.1946x faster
generator-with-several-types 332.1478+-4.2425 ^ 124.6642+-2.4826 ^ definitely 2.6643x faster
<geometric> 58.2998+-0.7758 ^ 27.7425+-0.2577 ^ definitely 2.1015x faster
In ES6SampleBench's Basic, we can observe 41% improvement (Macbook Pro).
Baseline:
Geometric Mean Result: 133.55 ms +- 4.49 ms
Benchmark First Iteration Worst 2% Steady State
Air 54.03 ms +- 7.51 ms 29.06 ms +- 3.13 ms 2276.59 ms +- 61.17 ms
Basic 30.18 ms +- 1.86 ms 18.85 ms +- 0.45 ms 2851.16 ms +- 41.87 ms
Patched:
Geometric Mean Result: 121.78 ms +- 3.96 ms
Benchmark First Iteration Worst 2% Steady State
Air 52.09 ms +- 6.89 ms 29.59 ms +- 3.16 ms 2239.90 ms +- 54.60 ms
Basic 29.28 ms +- 1.46 ms 16.26 ms +- 0.66 ms 2025.15 ms +- 38.56 ms
[1]: https://bugs.webkit.org/show_bug.cgi?id=159281
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* builtins/GeneratorPrototype.js:
(globalPrivate.generatorResume):
* bytecode/BytecodeBasicBlock.cpp:
(JSC::BytecodeBasicBlock::shrinkToFit):
(JSC::BytecodeBasicBlock::computeImpl):
(JSC::BytecodeBasicBlock::compute):
(JSC::isBranch): Deleted.
(JSC::isUnconditionalBranch): Deleted.
(JSC::isTerminal): Deleted.
(JSC::isThrow): Deleted.
(JSC::linkBlocks): Deleted.
(JSC::computeBytecodeBasicBlocks): Deleted.
* bytecode/BytecodeBasicBlock.h:
(JSC::BytecodeBasicBlock::isEntryBlock):
(JSC::BytecodeBasicBlock::isExitBlock):
(JSC::BytecodeBasicBlock::leaderOffset):
(JSC::BytecodeBasicBlock::totalLength):
(JSC::BytecodeBasicBlock::offsets):
(JSC::BytecodeBasicBlock::successors):
(JSC::BytecodeBasicBlock::index):
(JSC::BytecodeBasicBlock::addSuccessor):
(JSC::BytecodeBasicBlock::BytecodeBasicBlock):
(JSC::BytecodeBasicBlock::addLength):
(JSC::BytecodeBasicBlock::leaderBytecodeOffset): Deleted.
(JSC::BytecodeBasicBlock::totalBytecodeLength): Deleted.
(JSC::BytecodeBasicBlock::bytecodeOffsets): Deleted.
(JSC::BytecodeBasicBlock::addBytecodeLength): Deleted.
* bytecode/BytecodeGeneratorification.cpp: Added.
(JSC::BytecodeGeneratorification::BytecodeGeneratorification):
(JSC::BytecodeGeneratorification::graph):
(JSC::BytecodeGeneratorification::yields):
(JSC::BytecodeGeneratorification::enterPoint):
(JSC::BytecodeGeneratorification::storageForGeneratorLocal):
(JSC::GeneratorLivenessAnalysis::GeneratorLivenessAnalysis):
(JSC::GeneratorLivenessAnalysis::computeDefsForBytecodeOffset):
(JSC::GeneratorLivenessAnalysis::computeUsesForBytecodeOffset):
(JSC::GeneratorLivenessAnalysis::run):
(JSC::BytecodeGeneratorification::run):
(JSC::performGeneratorification):
* bytecode/BytecodeGeneratorification.h: Copied from Source/JavaScriptCore/bytecode/BytecodeLivenessAnalysisInlines.h.
* bytecode/BytecodeGraph.h: Added.
(JSC::BytecodeGraph::codeBlock):
(JSC::BytecodeGraph::instructions):
(JSC::BytecodeGraph::basicBlocksInReverseOrder):
(JSC::BytecodeGraph::blockContainsBytecodeOffset):
(JSC::BytecodeGraph::findBasicBlockForBytecodeOffset):
(JSC::BytecodeGraph::findBasicBlockWithLeaderOffset):
(JSC::BytecodeGraph::size):
(JSC::BytecodeGraph::at):
(JSC::BytecodeGraph::operator[]):
(JSC::BytecodeGraph::begin):
(JSC::BytecodeGraph::end):
(JSC::BytecodeGraph::first):
(JSC::BytecodeGraph::last):
(JSC::BytecodeGraph<Block>::BytecodeGraph):
* bytecode/BytecodeList.json:
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::BytecodeLivenessAnalysis):
(JSC::BytecodeLivenessAnalysis::computeDefsForBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::computeUsesForBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::getLivenessInfoAtBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::computeFullLiveness):
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
(JSC::BytecodeLivenessAnalysis::compute):
(JSC::isValidRegisterForLiveness): Deleted.
(JSC::getLeaderOffsetForBasicBlock): Deleted.
(JSC::findBasicBlockWithLeaderOffset): Deleted.
(JSC::blockContainsBytecodeOffset): Deleted.
(JSC::findBasicBlockForBytecodeOffset): Deleted.
(JSC::stepOverInstruction): Deleted.
(JSC::computeLocalLivenessForBytecodeOffset): Deleted.
(JSC::computeLocalLivenessForBlock): Deleted.
(JSC::BytecodeLivenessAnalysis::runLivenessFixpoint): Deleted.
* bytecode/BytecodeLivenessAnalysis.h:
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::isValidRegisterForLiveness):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::computeLocalLivenessForBytecodeOffset):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::computeLocalLivenessForBlock):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::getLivenessInfoAtBytecodeOffset):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/BytecodeRewriter.cpp: Added.
(JSC::BytecodeRewriter::applyModification):
(JSC::BytecodeRewriter::execute):
(JSC::BytecodeRewriter::adjustJumpTargetsInFragment):
(JSC::BytecodeRewriter::insertImpl):
(JSC::BytecodeRewriter::adjustJumpTarget):
* bytecode/BytecodeRewriter.h: Added.
(JSC::BytecodeRewriter::InsertionPoint::InsertionPoint):
(JSC::BytecodeRewriter::InsertionPoint::operator<):
(JSC::BytecodeRewriter::InsertionPoint::operator==):
(JSC::BytecodeRewriter::Insertion::length):
(JSC::BytecodeRewriter::Fragment::Fragment):
(JSC::BytecodeRewriter::Fragment::appendInstruction):
(JSC::BytecodeRewriter::BytecodeRewriter):
(JSC::BytecodeRewriter::insertFragmentBefore):
(JSC::BytecodeRewriter::insertFragmentAfter):
(JSC::BytecodeRewriter::removeBytecode):
(JSC::BytecodeRewriter::graph):
(JSC::BytecodeRewriter::adjustAbsoluteOffset):
(JSC::BytecodeRewriter::adjustJumpTarget):
(JSC::BytecodeRewriter::calculateDifference):
* bytecode/BytecodeUseDef.h:
(JSC::computeUsesForBytecodeOffset):
(JSC::computeDefsForBytecodeOffset):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::dumpBytecode):
(JSC::CodeBlock::finishCreation):
(JSC::CodeBlock::handlerForIndex):
(JSC::CodeBlock::shrinkToFit):
(JSC::CodeBlock::valueProfileForBytecodeOffset):
(JSC::CodeBlock::livenessAnalysisSlow):
* bytecode/CodeBlock.h:
(JSC::CodeBlock::isConstantRegisterIndex):
(JSC::CodeBlock::livenessAnalysis):
(JSC::CodeBlock::liveCalleeLocalsAtYield): Deleted.
* bytecode/HandlerInfo.h:
(JSC::HandlerInfoBase::handlerForIndex):
* bytecode/Opcode.h:
(JSC::isBranch):
(JSC::isUnconditionalBranch):
(JSC::isTerminal):
(JSC::isThrow):
* bytecode/PreciseJumpTargets.cpp:
(JSC::getJumpTargetsForBytecodeOffset):
(JSC::computePreciseJumpTargetsInternal):
(JSC::computePreciseJumpTargets):
(JSC::recomputePreciseJumpTargets):
(JSC::findJumpTargetsForBytecodeOffset):
* bytecode/PreciseJumpTargets.h:
* bytecode/PreciseJumpTargetsInlines.h: Added.
(JSC::extractStoredJumpTargetsForBytecodeOffset):
* bytecode/UnlinkedCodeBlock.cpp:
(JSC::UnlinkedCodeBlock::handlerForBytecodeOffset):
(JSC::UnlinkedCodeBlock::handlerForIndex):
(JSC::UnlinkedCodeBlock::applyModification):
* bytecode/UnlinkedCodeBlock.h:
(JSC::UnlinkedStringJumpTable::offsetForValue):
(JSC::UnlinkedCodeBlock::numCalleeLocals):
* bytecode/VirtualRegister.h:
* bytecompiler/BytecodeGenerator.cpp:
(JSC::BytecodeGenerator::generate):
(JSC::BytecodeGenerator::BytecodeGenerator):
(JSC::BytecodeGenerator::emitComplexPopScopes):
(JSC::prepareJumpTableForStringSwitch):
(JSC::BytecodeGenerator::emitYieldPoint):
(JSC::BytecodeGenerator::emitSave): Deleted.
(JSC::BytecodeGenerator::emitResume): Deleted.
(JSC::BytecodeGenerator::emitGeneratorStateLabel): Deleted.
(JSC::BytecodeGenerator::beginGenerator): Deleted.
(JSC::BytecodeGenerator::endGenerator): Deleted.
* bytecompiler/BytecodeGenerator.h:
(JSC::BytecodeGenerator::generatorStateRegister):
(JSC::BytecodeGenerator::generatorValueRegister):
(JSC::BytecodeGenerator::generatorResumeModeRegister):
(JSC::BytecodeGenerator::generatorFrameRegister):
* bytecompiler/NodesCodegen.cpp:
(JSC::FunctionNode::emitBytecode):
* dfg/DFGOperations.cpp:
* interpreter/Interpreter.cpp:
(JSC::findExceptionHandler):
(JSC::GetCatchHandlerFunctor::operator()):
(JSC::UnwindFunctor::operator()):
* interpreter/Interpreter.h:
* interpreter/InterpreterInlines.h: Copied from Source/JavaScriptCore/bytecode/PreciseJumpTargets.h.
(JSC::Interpreter::getOpcodeID):
* jit/JIT.cpp:
(JSC::JIT::privateCompileMainPass):
* jit/JIT.h:
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_save): Deleted.
(JSC::JIT::emit_op_resume): Deleted.
* llint/LowLevelInterpreter.asm:
* parser/Parser.cpp:
(JSC::Parser<LexerType>::parseInner):
(JSC::Parser<LexerType>::parseGeneratorFunctionSourceElements):
(JSC::Parser<LexerType>::createGeneratorParameters):
* parser/Parser.h:
* runtime/CommonSlowPaths.cpp:
(JSC::SLOW_PATH_DECL): Deleted.
* runtime/CommonSlowPaths.h:
* runtime/GeneratorFrame.cpp: Removed.
(JSC::GeneratorFrame::GeneratorFrame): Deleted.
(JSC::GeneratorFrame::finishCreation): Deleted.
(JSC::GeneratorFrame::createStructure): Deleted.
(JSC::GeneratorFrame::create): Deleted.
(JSC::GeneratorFrame::save): Deleted.
(JSC::GeneratorFrame::resume): Deleted.
(JSC::GeneratorFrame::visitChildren): Deleted.
* runtime/GeneratorFrame.h: Removed.
(JSC::GeneratorFrame::locals): Deleted.
(JSC::GeneratorFrame::localAt): Deleted.
(JSC::GeneratorFrame::offsetOfLocals): Deleted.
(JSC::GeneratorFrame::allocationSizeForLocals): Deleted.
* runtime/JSGeneratorFunction.h:
* runtime/VM.cpp:
(JSC::VM::VM):
* runtime/VM.h:
Source/WTF:
* wtf/FastBitVector.h:
(WTF::FastBitVector::FastBitVector):
Canonical link: https://commits.webkit.org/179373@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@204994 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-08-25 22:55:10 +00:00
|
|
|
, m_numBits(std::exchange(other.m_numBits, 0))
|
2012-05-22 07:10:51 +00:00
|
|
|
{
|
|
|
|
}
|
[DFG][FTL] Implement ES6 Generators in DFG / FTL
https://bugs.webkit.org/show_bug.cgi?id=152723
Reviewed by Filip Pizlo.
JSTests:
* stress/generator-fib-ftl-and-array.js: Added.
(fib):
* stress/generator-fib-ftl-and-object.js: Added.
(fib):
* stress/generator-fib-ftl-and-string.js: Added.
(fib):
* stress/generator-fib-ftl.js: Added.
(fib):
* stress/generator-frame-empty.js: Added.
(shouldThrow):
(shouldThrow.fib):
* stress/generator-reduced-save-point-put-to-scope.js: Added.
(shouldBe):
(gen):
* stress/generator-transfer-register-beyond-mutiple-yields.js: Added.
(shouldBe):
(gen):
Source/JavaScriptCore:
This patch introduces DFG and FTL support for ES6 generators.
ES6 generator is compiled by the BytecodeGenerator. But at the last phase, BytecodeGenerator performs "generatorification" onto the unlinked code.
In BytecodeGenerator phase, we just emit op_yield for each yield point. And we don't emit any generator related switch, save, and resume sequences
here. Those are emitted by the generatorification phase.
So the graph is super simple! Before the generatorification, the graph looks like this.
op_enter -> ...... -> op_yield -> ..... -> op_yield -> ...
Roughly speaking, in the generatorification phase, we turn out which variables should be saved and resumed at each op_yield.
This is done by liveness analysis. After that, we convert op_yield to the sequence of "op_put_to_scope", "op_ret", and "op_get_from_scope".
op_put_to_scope and op_get_from_scope sequences are corresponding to the save and resume sequences. We set up the scope for the generator frame and
perform op_put_to_scope and op_get_from_scope onto it. The live registers are saved and resumed over the generator's next() calls by using this
special generator frame scope. And we also set up the global switch for the generator.
In the generatorification phase,
1. We construct the BytecodeGraph from the unlinked instructions. This constructs the basic blocks, and it is used in the subsequent analysis.
2. We perform the analysis onto the unlinked code. We extract the live variables at each op_yield.
3. We insert the get_from_scope and put_to_scope at each op_yield. Which registers should be saved and resumed is offered by (2).
Then, clip the op_yield themselves. And we also insert the switch_imm. The jump targets of this switch are just after this op_switch_imm and each op_yield point.
One interesting point is the try-range. We split the try-range at the op_yield point in BytecodeGenerator phase.
This drops the hacky thing that is introduced in [1].
If the try-range covers the resume sequences, the exception handler's use-registers are incorrectly transferred to the entry block.
For example,
handler uses r2
try-range
label:(entry block can jump here) ^
r1 = get_from_scope # resume sequence starts | use r2 is transferred to the entry block!
r2 = get_from_scope |
starts usual sequences |
... |
Handler's r2 use should be considered at the `r1 = get_from_scope` point.
Previously, we handle this edge case by treating op_resume specially in the liveness analysis[1].
To drop this workaround, we split the try-range not to cover this resume sequence.
handler uses r2
try-range
label:(entry block can jump here)
r1 = get_from_scope # resume sequence starts
r2 = get_from_scope
starts usual sequences ^ try-range should start from here.
... |
OK. Let's show the detailed example.
1. First, there is the normal bytecode sequence. Here, | represents the offsets, and [] represents the bytecodes.
bytecodes | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |
try-range <----------------------------------->
2. When we emit the op_yield in the bytecode generator, we carefully split the try-range.
bytecodes | [ ] | [ ] | [op_yield] | [ ] | [ ] | [ ] |
try-range <-----------> <----------------->
3. And in the generatorification phase, we insert the switch's jump target and save & resume sequences. And we also drop op_yield.
Insert save seq Insert resume seq
before op_yield. after op_yield's point.
v v
bytecodes | [ ] | [ ] | [op_yield] | [ ] | [ ] | [ ] |
try-range <-----------> ^ <----------------->
^ |
Jump to here. Drop this op_yield.
4. The final layout is the following.
bytecodes | [ ] | [ ][save seq][op_ret] | [resume seq] | [ ] | [ ] | [ ] |
try-range <-----------------------------> <---------------->
^
Jump to here.
The rewriting done by the BytecodeRewriter is executed in a batch manner. Since these modification changes the basic blocks and size of unlinked instructions,
BytecodeRewriter also performs the offset adjustment for UnlinkedCodeBlock. So, this rewriting is performed onto the BytecodeGraph rather than BytecodeBasicBlock.
The reason why we take this design is simple: we don't want to newly create the basic blocks and opcodes for this early phase like DFG. Instead, we perform the
modification and adjustment to the unlinked instructions and UnlinkedCodeBlock in a in-place manner.
Bytecode rewriting functionality is offered by BytecodeRewriter. BytecodeRewriter allows us to insert any bytecodes to any places
in a in-place manner. BytecodeRewriter handles the original bytecode offsets as labels. And you can insert bytecodes before and after
these labels. You can also insert any jumps to any places. When you insert jumps, you need to specify jump target with this labels.
These labels (original bytecode offsets) are automatically converted to the appropriate offsets by BytecodeRewriter.
After that phase, the data flow of the generator-saved-and-resumed-registers are explicitly represented by the get_from_scope and put_to_scope.
And the switch is inserted to represent the actual control flow for the generator. And op_yield is removed. Since we use the existing bytecodes (op_switch_imm, op_put_to_scope
op_ret, and op_get_from_scope), DFG and FTL changes are not necessary. This patch also drops data structures and implementations for the old generator,
op_resume, op_save implementations and GeneratorFrame.
Note that this patch does not leverage the recent multi entrypoints support in B3. After this patch is introduced, we will submit a new patch that leverages the multi
entrypoints for generator's resume and sees the performance gain.
Microbenchmarks related to generators show up to 2.9x improvements.
Baseline Patched
generator-fib 102.0116+-3.2880 ^ 34.9670+-0.2221 ^ definitely 2.9174x faster
generator-sunspider-access-nsieve 5.8596+-0.0371 ^ 4.9051+-0.0720 ^ definitely 1.1946x faster
generator-with-several-types 332.1478+-4.2425 ^ 124.6642+-2.4826 ^ definitely 2.6643x faster
<geometric> 58.2998+-0.7758 ^ 27.7425+-0.2577 ^ definitely 2.1015x faster
In ES6SampleBench's Basic, we can observe 41% improvement (Macbook Pro).
Baseline:
Geometric Mean Result: 133.55 ms +- 4.49 ms
Benchmark First Iteration Worst 2% Steady State
Air 54.03 ms +- 7.51 ms 29.06 ms +- 3.13 ms 2276.59 ms +- 61.17 ms
Basic 30.18 ms +- 1.86 ms 18.85 ms +- 0.45 ms 2851.16 ms +- 41.87 ms
Patched:
Geometric Mean Result: 121.78 ms +- 3.96 ms
Benchmark First Iteration Worst 2% Steady State
Air 52.09 ms +- 6.89 ms 29.59 ms +- 3.16 ms 2239.90 ms +- 54.60 ms
Basic 29.28 ms +- 1.46 ms 16.26 ms +- 0.66 ms 2025.15 ms +- 38.56 ms
[1]: https://bugs.webkit.org/show_bug.cgi?id=159281
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* builtins/GeneratorPrototype.js:
(globalPrivate.generatorResume):
* bytecode/BytecodeBasicBlock.cpp:
(JSC::BytecodeBasicBlock::shrinkToFit):
(JSC::BytecodeBasicBlock::computeImpl):
(JSC::BytecodeBasicBlock::compute):
(JSC::isBranch): Deleted.
(JSC::isUnconditionalBranch): Deleted.
(JSC::isTerminal): Deleted.
(JSC::isThrow): Deleted.
(JSC::linkBlocks): Deleted.
(JSC::computeBytecodeBasicBlocks): Deleted.
* bytecode/BytecodeBasicBlock.h:
(JSC::BytecodeBasicBlock::isEntryBlock):
(JSC::BytecodeBasicBlock::isExitBlock):
(JSC::BytecodeBasicBlock::leaderOffset):
(JSC::BytecodeBasicBlock::totalLength):
(JSC::BytecodeBasicBlock::offsets):
(JSC::BytecodeBasicBlock::successors):
(JSC::BytecodeBasicBlock::index):
(JSC::BytecodeBasicBlock::addSuccessor):
(JSC::BytecodeBasicBlock::BytecodeBasicBlock):
(JSC::BytecodeBasicBlock::addLength):
(JSC::BytecodeBasicBlock::leaderBytecodeOffset): Deleted.
(JSC::BytecodeBasicBlock::totalBytecodeLength): Deleted.
(JSC::BytecodeBasicBlock::bytecodeOffsets): Deleted.
(JSC::BytecodeBasicBlock::addBytecodeLength): Deleted.
* bytecode/BytecodeGeneratorification.cpp: Added.
(JSC::BytecodeGeneratorification::BytecodeGeneratorification):
(JSC::BytecodeGeneratorification::graph):
(JSC::BytecodeGeneratorification::yields):
(JSC::BytecodeGeneratorification::enterPoint):
(JSC::BytecodeGeneratorification::storageForGeneratorLocal):
(JSC::GeneratorLivenessAnalysis::GeneratorLivenessAnalysis):
(JSC::GeneratorLivenessAnalysis::computeDefsForBytecodeOffset):
(JSC::GeneratorLivenessAnalysis::computeUsesForBytecodeOffset):
(JSC::GeneratorLivenessAnalysis::run):
(JSC::BytecodeGeneratorification::run):
(JSC::performGeneratorification):
* bytecode/BytecodeGeneratorification.h: Copied from Source/JavaScriptCore/bytecode/BytecodeLivenessAnalysisInlines.h.
* bytecode/BytecodeGraph.h: Added.
(JSC::BytecodeGraph::codeBlock):
(JSC::BytecodeGraph::instructions):
(JSC::BytecodeGraph::basicBlocksInReverseOrder):
(JSC::BytecodeGraph::blockContainsBytecodeOffset):
(JSC::BytecodeGraph::findBasicBlockForBytecodeOffset):
(JSC::BytecodeGraph::findBasicBlockWithLeaderOffset):
(JSC::BytecodeGraph::size):
(JSC::BytecodeGraph::at):
(JSC::BytecodeGraph::operator[]):
(JSC::BytecodeGraph::begin):
(JSC::BytecodeGraph::end):
(JSC::BytecodeGraph::first):
(JSC::BytecodeGraph::last):
(JSC::BytecodeGraph<Block>::BytecodeGraph):
* bytecode/BytecodeList.json:
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::BytecodeLivenessAnalysis):
(JSC::BytecodeLivenessAnalysis::computeDefsForBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::computeUsesForBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::getLivenessInfoAtBytecodeOffset):
(JSC::BytecodeLivenessAnalysis::computeFullLiveness):
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
(JSC::BytecodeLivenessAnalysis::compute):
(JSC::isValidRegisterForLiveness): Deleted.
(JSC::getLeaderOffsetForBasicBlock): Deleted.
(JSC::findBasicBlockWithLeaderOffset): Deleted.
(JSC::blockContainsBytecodeOffset): Deleted.
(JSC::findBasicBlockForBytecodeOffset): Deleted.
(JSC::stepOverInstruction): Deleted.
(JSC::computeLocalLivenessForBytecodeOffset): Deleted.
(JSC::computeLocalLivenessForBlock): Deleted.
(JSC::BytecodeLivenessAnalysis::runLivenessFixpoint): Deleted.
* bytecode/BytecodeLivenessAnalysis.h:
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::isValidRegisterForLiveness):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::computeLocalLivenessForBytecodeOffset):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::computeLocalLivenessForBlock):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::getLivenessInfoAtBytecodeOffset):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/BytecodeRewriter.cpp: Added.
(JSC::BytecodeRewriter::applyModification):
(JSC::BytecodeRewriter::execute):
(JSC::BytecodeRewriter::adjustJumpTargetsInFragment):
(JSC::BytecodeRewriter::insertImpl):
(JSC::BytecodeRewriter::adjustJumpTarget):
* bytecode/BytecodeRewriter.h: Added.
(JSC::BytecodeRewriter::InsertionPoint::InsertionPoint):
(JSC::BytecodeRewriter::InsertionPoint::operator<):
(JSC::BytecodeRewriter::InsertionPoint::operator==):
(JSC::BytecodeRewriter::Insertion::length):
(JSC::BytecodeRewriter::Fragment::Fragment):
(JSC::BytecodeRewriter::Fragment::appendInstruction):
(JSC::BytecodeRewriter::BytecodeRewriter):
(JSC::BytecodeRewriter::insertFragmentBefore):
(JSC::BytecodeRewriter::insertFragmentAfter):
(JSC::BytecodeRewriter::removeBytecode):
(JSC::BytecodeRewriter::graph):
(JSC::BytecodeRewriter::adjustAbsoluteOffset):
(JSC::BytecodeRewriter::adjustJumpTarget):
(JSC::BytecodeRewriter::calculateDifference):
* bytecode/BytecodeUseDef.h:
(JSC::computeUsesForBytecodeOffset):
(JSC::computeDefsForBytecodeOffset):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::dumpBytecode):
(JSC::CodeBlock::finishCreation):
(JSC::CodeBlock::handlerForIndex):
(JSC::CodeBlock::shrinkToFit):
(JSC::CodeBlock::valueProfileForBytecodeOffset):
(JSC::CodeBlock::livenessAnalysisSlow):
* bytecode/CodeBlock.h:
(JSC::CodeBlock::isConstantRegisterIndex):
(JSC::CodeBlock::livenessAnalysis):
(JSC::CodeBlock::liveCalleeLocalsAtYield): Deleted.
* bytecode/HandlerInfo.h:
(JSC::HandlerInfoBase::handlerForIndex):
* bytecode/Opcode.h:
(JSC::isBranch):
(JSC::isUnconditionalBranch):
(JSC::isTerminal):
(JSC::isThrow):
* bytecode/PreciseJumpTargets.cpp:
(JSC::getJumpTargetsForBytecodeOffset):
(JSC::computePreciseJumpTargetsInternal):
(JSC::computePreciseJumpTargets):
(JSC::recomputePreciseJumpTargets):
(JSC::findJumpTargetsForBytecodeOffset):
* bytecode/PreciseJumpTargets.h:
* bytecode/PreciseJumpTargetsInlines.h: Added.
(JSC::extractStoredJumpTargetsForBytecodeOffset):
* bytecode/UnlinkedCodeBlock.cpp:
(JSC::UnlinkedCodeBlock::handlerForBytecodeOffset):
(JSC::UnlinkedCodeBlock::handlerForIndex):
(JSC::UnlinkedCodeBlock::applyModification):
* bytecode/UnlinkedCodeBlock.h:
(JSC::UnlinkedStringJumpTable::offsetForValue):
(JSC::UnlinkedCodeBlock::numCalleeLocals):
* bytecode/VirtualRegister.h:
* bytecompiler/BytecodeGenerator.cpp:
(JSC::BytecodeGenerator::generate):
(JSC::BytecodeGenerator::BytecodeGenerator):
(JSC::BytecodeGenerator::emitComplexPopScopes):
(JSC::prepareJumpTableForStringSwitch):
(JSC::BytecodeGenerator::emitYieldPoint):
(JSC::BytecodeGenerator::emitSave): Deleted.
(JSC::BytecodeGenerator::emitResume): Deleted.
(JSC::BytecodeGenerator::emitGeneratorStateLabel): Deleted.
(JSC::BytecodeGenerator::beginGenerator): Deleted.
(JSC::BytecodeGenerator::endGenerator): Deleted.
* bytecompiler/BytecodeGenerator.h:
(JSC::BytecodeGenerator::generatorStateRegister):
(JSC::BytecodeGenerator::generatorValueRegister):
(JSC::BytecodeGenerator::generatorResumeModeRegister):
(JSC::BytecodeGenerator::generatorFrameRegister):
* bytecompiler/NodesCodegen.cpp:
(JSC::FunctionNode::emitBytecode):
* dfg/DFGOperations.cpp:
* interpreter/Interpreter.cpp:
(JSC::findExceptionHandler):
(JSC::GetCatchHandlerFunctor::operator()):
(JSC::UnwindFunctor::operator()):
* interpreter/Interpreter.h:
* interpreter/InterpreterInlines.h: Copied from Source/JavaScriptCore/bytecode/PreciseJumpTargets.h.
(JSC::Interpreter::getOpcodeID):
* jit/JIT.cpp:
(JSC::JIT::privateCompileMainPass):
* jit/JIT.h:
* jit/JITOpcodes.cpp:
(JSC::JIT::emit_op_save): Deleted.
(JSC::JIT::emit_op_resume): Deleted.
* llint/LowLevelInterpreter.asm:
* parser/Parser.cpp:
(JSC::Parser<LexerType>::parseInner):
(JSC::Parser<LexerType>::parseGeneratorFunctionSourceElements):
(JSC::Parser<LexerType>::createGeneratorParameters):
* parser/Parser.h:
* runtime/CommonSlowPaths.cpp:
(JSC::SLOW_PATH_DECL): Deleted.
* runtime/CommonSlowPaths.h:
* runtime/GeneratorFrame.cpp: Removed.
(JSC::GeneratorFrame::GeneratorFrame): Deleted.
(JSC::GeneratorFrame::finishCreation): Deleted.
(JSC::GeneratorFrame::createStructure): Deleted.
(JSC::GeneratorFrame::create): Deleted.
(JSC::GeneratorFrame::save): Deleted.
(JSC::GeneratorFrame::resume): Deleted.
(JSC::GeneratorFrame::visitChildren): Deleted.
* runtime/GeneratorFrame.h: Removed.
(JSC::GeneratorFrame::locals): Deleted.
(JSC::GeneratorFrame::localAt): Deleted.
(JSC::GeneratorFrame::offsetOfLocals): Deleted.
(JSC::GeneratorFrame::allocationSizeForLocals): Deleted.
* runtime/JSGeneratorFunction.h:
* runtime/VM.cpp:
(JSC::VM::VM):
* runtime/VM.h:
Source/WTF:
* wtf/FastBitVector.h:
(WTF::FastBitVector::FastBitVector):
Canonical link: https://commits.webkit.org/179373@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@204994 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-08-25 22:55:10 +00:00
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
FastBitVectorWordOwner(const FastBitVectorWordOwner& other)
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
|
|
|
*this = other;
|
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
~FastBitVectorWordOwner()
|
2012-05-22 07:10:51 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
if (m_words)
|
Experiment: create lots of different malloc zones for easier accounting of memory use
https://bugs.webkit.org/show_bug.cgi?id=186422
Patch by Yusuke Suzuki <ysuzuki@apple.com> and Simon Fraser <simon.fraser@apple.com> on 2020-01-02
Reviewed by Saam Barati.
Source/bmalloc:
* bmalloc/BPlatform.h:
* bmalloc/Environment.cpp:
(bmalloc::Environment::computeIsDebugHeapEnabled):
* bmalloc/IsoHeap.h:
(bmalloc::api::IsoHeap::IsoHeap):
* bmalloc/IsoHeapInlines.h:
(bmalloc::api::IsoHeap<Type>::IsoHeap):
* bmalloc/IsoTLSInlines.h:
(bmalloc::IsoTLS::allocateSlow):
(bmalloc::IsoTLS::deallocateSlow):
Source/JavaScriptCore:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* assembler/AssemblerBuffer.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* assembler/AssemblerBuffer.h:
(JSC::AssemblerData::AssemblerData):
(JSC::AssemblerData::operator=):
(JSC::AssemblerData::~AssemblerData):
(JSC::AssemblerData::grow):
* bytecode/AccessCase.cpp:
* bytecode/AccessCase.h:
* bytecode/BytecodeBasicBlock.cpp:
* bytecode/BytecodeBasicBlock.h:
* bytecode/CodeBlock.cpp:
* bytecode/CodeBlock.h:
* bytecode/InstructionStream.cpp:
* bytecode/InstructionStream.h:
* bytecode/PolymorphicAccess.cpp:
* bytecode/PolymorphicAccess.h:
* bytecode/UnlinkedMetadataTable.cpp:
(JSC::UnlinkedMetadataTable::finalize):
* bytecode/UnlinkedMetadataTable.h:
* bytecode/UnlinkedMetadataTableInlines.h:
(JSC::UnlinkedMetadataTable::UnlinkedMetadataTable):
(JSC::UnlinkedMetadataTable::~UnlinkedMetadataTable):
(JSC::UnlinkedMetadataTable::link):
(JSC::UnlinkedMetadataTable::unlink):
* bytecode/ValueProfile.h:
(JSC::ValueProfileAndVirtualRegisterBuffer::ValueProfileAndVirtualRegisterBuffer):
* bytecode/Watchpoint.cpp:
* bytecode/Watchpoint.h:
* dfg/DFGBasicBlock.cpp:
* dfg/DFGBasicBlock.h:
* dfg/DFGNode.cpp:
* dfg/DFGNode.h:
* dfg/DFGSpeculativeJIT.cpp:
* dfg/DFGSpeculativeJIT.h:
* heap/BlockDirectory.cpp:
* heap/BlockDirectory.h:
* heap/FastMallocAlignedMemoryAllocator.cpp:
(JSC::FastMallocAlignedMemoryAllocator::FastMallocAlignedMemoryAllocator):
(JSC::FastMallocAlignedMemoryAllocator::tryAllocateAlignedMemory):
(JSC::FastMallocAlignedMemoryAllocator::freeAlignedMemory):
(JSC::FastMallocAlignedMemoryAllocator::tryAllocateMemory):
(JSC::FastMallocAlignedMemoryAllocator::freeMemory):
(JSC::FastMallocAlignedMemoryAllocator::tryReallocateMemory):
* heap/FastMallocAlignedMemoryAllocator.h:
* heap/GCSegmentedArray.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp.
* heap/GCSegmentedArray.h:
* heap/GCSegmentedArrayInlines.h:
(JSC::GCArraySegment<T>::create):
(JSC::GCArraySegment<T>::destroy):
* heap/GigacageAlignedMemoryAllocator.cpp:
(JSC::GigacageAlignedMemoryAllocator::GigacageAlignedMemoryAllocator):
(JSC::GigacageAlignedMemoryAllocator::tryAllocateAlignedMemory):
(JSC::GigacageAlignedMemoryAllocator::freeAlignedMemory):
(JSC::GigacageAlignedMemoryAllocator::tryAllocateMemory):
(JSC::GigacageAlignedMemoryAllocator::freeMemory):
(JSC::GigacageAlignedMemoryAllocator::tryReallocateMemory):
* heap/GigacageAlignedMemoryAllocator.h:
* heap/IsoAlignedMemoryAllocator.cpp:
(JSC::IsoAlignedMemoryAllocator::IsoAlignedMemoryAllocator):
(JSC::IsoAlignedMemoryAllocator::~IsoAlignedMemoryAllocator):
(JSC::IsoAlignedMemoryAllocator::tryAllocateAlignedMemory):
(JSC::IsoAlignedMemoryAllocator::freeAlignedMemory):
(JSC::IsoAlignedMemoryAllocator::tryAllocateMemory):
(JSC::IsoAlignedMemoryAllocator::freeMemory):
* heap/IsoAlignedMemoryAllocator.h:
* heap/IsoSubspace.cpp:
(JSC::IsoSubspace::IsoSubspace):
* heap/MarkedBlock.cpp:
* heap/MarkedBlock.h:
* heap/WeakBlock.cpp:
(JSC::WeakBlock::create):
(JSC::WeakBlock::destroy):
* heap/WeakBlock.h:
* jit/JITCode.cpp:
* jit/JITCode.h:
* jit/RegisterAtOffsetList.cpp:
* jit/RegisterAtOffsetList.h:
* parser/Nodes.cpp:
* parser/Nodes.h:
* parser/ParserArena.cpp:
(JSC::ParserArena::deallocateObjects):
(JSC::ParserArena::allocateFreeablePool):
* parser/ParserArena.h:
* parser/SourceProvider.cpp:
* parser/SourceProvider.h:
* parser/SourceProviderCache.cpp:
* parser/SourceProviderCache.h:
* parser/SourceProviderCacheItem.h:
(JSC::SourceProviderCacheItem::create):
* runtime/CachePayload.cpp:
(JSC::CachePayload::makeMallocPayload):
* runtime/CachePayload.h:
* runtime/CachedBytecode.h:
(JSC::CachedBytecode::create):
* runtime/CachedTypes.cpp:
(JSC::Encoder::release):
(JSC::Encoder::Page::Page):
(JSC::CachedVector::encode):
(JSC::CachedVector::decode const):
(JSC::CachedInstructionStream::decode const):
* runtime/PropertyMapHashTable.h:
(JSC::PropertyTable::rehash):
* runtime/PropertyTable.cpp:
(JSC::PropertyTable::PropertyTable):
(JSC::PropertyTable::~PropertyTable):
* runtime/SymbolTable.cpp:
* runtime/SymbolTable.h:
* runtime/VM.cpp:
(JSC::VM::~VM):
* runtime/VM.h:
(JSC::ScratchBuffer::create):
(JSC::VM::exceptionFuzzingBuffer):
* wasm/WasmInstance.cpp:
(JSC::Wasm::Instance::Instance):
* wasm/WasmInstance.h:
* wasm/WasmTable.cpp:
(JSC::Wasm::Table::Table):
(JSC::Wasm::FuncRefTable::FuncRefTable):
* wasm/WasmTable.h:
Source/WebCore:
* Sources.txt:
* WebCore.xcodeproj/project.pbxproj:
* bindings/js/SerializedScriptValue.cpp:
* bindings/js/SerializedScriptValue.h:
* css/CSSFontFace.cpp:
* css/CSSFontFace.h:
* css/CSSSelector.cpp:
* css/CSSSelector.h:
* css/CSSValue.cpp:
* css/CSSValue.h:
* css/StyleProperties.cpp:
(WebCore::ImmutableStyleProperties::create):
* css/StyleProperties.h:
* css/StyleRule.cpp:
* css/StyleRule.h:
* dom/ElementData.cpp:
(WebCore::ShareableElementData::createWithAttributes):
(WebCore::UniqueElementData::makeShareableCopy const):
* dom/ElementData.h:
* dom/NodeRareData.cpp:
* dom/NodeRareData.h:
* dom/QualifiedName.cpp:
* dom/QualifiedName.h:
* html/parser/HTMLDocumentParser.cpp:
* html/parser/HTMLDocumentParser.h:
* loader/DocumentLoader.cpp:
* loader/DocumentLoader.h:
* loader/ResourceLoader.cpp:
* loader/ResourceLoader.h:
* loader/cache/CachedResource.cpp:
* loader/cache/CachedResource.h:
* page/PerformanceEntry.cpp:
* page/PerformanceEntry.h:
* platform/graphics/Font.cpp:
* platform/graphics/Font.h:
* platform/graphics/FontCascadeFonts.cpp:
* platform/graphics/FontCascadeFonts.h:
* platform/graphics/Region.cpp:
* platform/graphics/Region.h:
* platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm:
(WebCore::releaseUint8Vector):
* platform/graphics/cg/ImageBufferCG.cpp:
(WebCore::ImageBuffer::ImageBuffer):
* platform/graphics/nicosia/NicosiaBuffer.cpp:
(Nicosia::Buffer::Buffer):
* platform/network/ResourceHandle.cpp:
* platform/network/ResourceHandleInternal.h:
* platform/network/cf/FormDataStreamCFNet.cpp:
(WebCore::closeCurrentStream):
(WebCore::advanceCurrentStream):
* rendering/RenderLayer.cpp:
* rendering/RenderLayer.h:
* rendering/TableLayout.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp.
* rendering/TableLayout.h:
* rendering/style/RenderStyle.cpp:
* rendering/style/RenderStyle.h:
* rendering/style/SVGRenderStyle.cpp:
* rendering/style/SVGRenderStyle.h:
* rendering/style/SVGRenderStyleDefs.cpp:
* rendering/style/SVGRenderStyleDefs.h:
* rendering/style/StyleBoxData.cpp:
* rendering/style/StyleBoxData.h:
* rendering/style/StyleInheritedData.cpp:
* rendering/style/StyleInheritedData.h:
* rendering/style/StyleRareInheritedData.cpp:
* rendering/style/StyleRareInheritedData.h:
* rendering/style/StyleRareNonInheritedData.cpp:
* rendering/style/StyleRareNonInheritedData.h:
* rendering/style/StyleSurroundData.cpp:
* rendering/style/StyleSurroundData.h:
* rendering/style/StyleTransformData.cpp:
* rendering/style/StyleTransformData.h:
* style/StyleTreeResolver.cpp:
* style/StyleTreeResolver.h:
* svg/animation/SMILTimeContainer.cpp:
* svg/animation/SMILTimeContainer.h:
Source/WebKit:
* Shared/ShareableBitmap.cpp:
(WebKit::ShareableBitmap::create):
(WebKit::ShareableBitmap::~ShareableBitmap):
* UIProcess/mac/LegacySessionStateCoding.cpp:
(WebKit::HistoryEntryDataEncoder::HistoryEntryDataEncoder):
(WebKit::HistoryEntryDataEncoder::finishEncoding):
(WebKit::encodeSessionHistoryEntryData):
(WebKit::encodeLegacySessionState):
Source/WTF:
This patch introduces ENABLE(MALLOC_HEAP_BREAKDOWN). If this is enabled, we allocate malloc_zone per malloc kind.
This offers the way to investigate the usage of memory per kind by using vmmap, like the following.
VIRTUAL RESIDENT DIRTY SWAPPED ALLOCATION BYTES DIRTY+SWAP REGION
MALLOC ZONE SIZE SIZE SIZE SIZE COUNT ALLOCATED FRAG SIZE % FRAG COUNT
=========== ======= ========= ========= ========= ========= ========= ========= ====== ======
StringImpl_0x116efd000 188.0M 69.3M 30.9M 0K 139456 18.0M 12.9M 42% 34
DefaultMallocZone_0x10f487000 176.0M 53.9M 14.1M 0K 115956 9955K 4497K 32% 22
Vector_0x116eff000 162.0M 56.3M 55.3M 0K 140715 17.3M 37.9M 69% 36
MetadataTable_0x11843b000 152.0M 17.5M 17.5M 0K 14200 2353K 15.2M 87% 26
WebKit Using System Malloc_0x114cbe000 150.0M 31.6M 21.8M 0K 87422 16.7M 5278K 24% 23
InstructionStream_0x118469000 150.0M 5764K 5764K 0K 14470 4688K 1076K 19% 24
AssemblerData_0x117ee6000 150.0M 1928K 1928K 0K 1 16 1928K 100% 24
To achieve this goal without making very large change, we put a template type in various containers.
For example, Vector will take Malloc parameter (the default one is FastMalloc allocator). If ENABLE(MALLOC_HEAP_BREAKDOWN) is enabled, we change this to
specific VectorMalloc allocator, and vmmap can show memory usage of this allocator. This patch also supports malloc_zone per IsoHeap. So we can see memory
allocation per IsoHeap in vmmap.
To use this feature, we need to flip two compile time flags, ENABLE(MALLOC_HEAP_BREAKDOWN) in WTF and BENABLE_MALLOC_HEAP_BREAKDOWN in bmalloc.
And use `vmmap $PID` to dump malloc zones. To allocate objects of a class with a specific malloc-zone, use WTF_MAKE_FAST_ALLOCATED_WITH_HEAP_IDENTIFIER(HeapIdentifier) for the class,
and define allocator by DECLARE_ALLOCATOR_WITH_HEAP_IDENTIFIER(HeapIdentifier) in a header and DEFINE_ALLOCATOR_WITH_HEAP_IDENTIFIER(HeapIdentifier) in a cpp file.
This patch also introduce callstack collector for malloc. Vector, HashMap etc. are used to allocate various things, but the above malloc_zone feature only tells thing like "Vector
takes XXX MB memory". But what we want to know in this case is what Vector is consuming memory. We collect StackShot for each malloc call, and combine these information to tell
which callsite is consuming much memory, which tell us that what Vector is consuming memory.
* WTF.xcodeproj/project.pbxproj:
* wtf/Bag.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp.
* wtf/Bag.h:
(WTF::Private::BagNode::BagNode): Deleted.
* wtf/BitVector.cpp:
(WTF::BitVector::OutOfLineBits::create):
(WTF::BitVector::OutOfLineBits::destroy):
* wtf/CMakeLists.txt:
* wtf/ConcurrentBuffer.cpp: Copied from Source/JavaScriptCore/parser/SourceProviderCache.cpp.
* wtf/ConcurrentBuffer.h:
* wtf/DebugHeap.cpp: Copied from Source/JavaScriptCore/runtime/CachePayload.cpp.
(WTF::DebugHeap::DebugHeap):
(WTF::DebugHeap::malloc):
(WTF::DebugHeap::calloc):
(WTF::DebugHeap::memalign):
(WTF::DebugHeap::realloc):
(WTF::DebugHeap::free):
* wtf/DebugHeap.h: Added.
* wtf/FastBitVector.cpp:
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
* wtf/FastMalloc.cpp:
(WTF::fastMallocDumpMallocStats):
(WTF::AvoidRecordingScope::AvoidRecordingScope):
(WTF::AvoidRecordingScope::~AvoidRecordingScope):
(WTF::MallocCallTracker::MallocSiteData::MallocSiteData):
(WTF::MallocCallTracker::singleton):
(WTF::MallocCallTracker::MallocCallTracker):
(WTF::MallocCallTracker::recordMalloc):
(WTF::MallocCallTracker::recordRealloc):
(WTF::MallocCallTracker::recordFree):
(WTF::MallocCallTracker::dumpStats):
(WTF::fastMalloc):
(WTF::fastRealloc):
(WTF::fastFree):
(WTF::fastAlignedMalloc):
(WTF::tryFastAlignedMalloc):
(WTF::fastAlignedFree):
* wtf/FastMalloc.h:
(WTF::FastMalloc::zeroedMalloc):
(WTF::FastMalloc::tryZeroedMalloc):
* wtf/Forward.h:
* wtf/HashTable.cpp:
* wtf/HashTable.h:
(WTF::KeyTraits>::allocateTable):
(WTF::KeyTraits>::deallocateTable):
(WTF::KeyTraits>::rehash):
* wtf/MallocPtr.h:
(WTF::MallocPtr::MallocPtr):
(WTF::MallocPtr::malloc):
(WTF::MallocPtr::zeroedMalloc):
(WTF::MallocPtr::tryMalloc):
(WTF::MallocPtr::tryZeroedMalloc):
(WTF::adoptMallocPtr):
* wtf/MetaAllocator.cpp:
(WTF::MetaAllocator::allocFreeSpaceNode):
(WTF::MetaAllocator::freeFreeSpaceNode):
* wtf/MetaAllocatorHandle.h:
* wtf/Platform.h:
* wtf/RefCountedArray.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/RefCountedArray.h:
(WTF::RefCountedArray::RefCountedArray):
(WTF::RefCountedArray::~RefCountedArray):
(WTF::RefCountedArray::assign):
* wtf/SegmentedVector.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/SegmentedVector.h:
* wtf/SmallPtrSet.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/SmallPtrSet.h:
(WTF::SmallPtrSet::~SmallPtrSet):
(WTF::SmallPtrSet::grow):
* wtf/UniqueArray.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/UniqueArray.h:
(WTF::UniqueArrayFree::operator() const):
(WTF::UniqueArrayFree<T::operator() const):
* wtf/Vector.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/Vector.h:
(WTF::VectorBufferBase::allocateBuffer):
(WTF::VectorBufferBase::tryAllocateBuffer):
(WTF::VectorBufferBase::reallocateBuffer):
(WTF::VectorBufferBase::deallocateBuffer):
(WTF::VectorBufferBase::releaseBuffer):
(WTF::VectorBuffer::releaseBuffer):
(WTF::Vector::swap):
(WTF::Malloc>::Vector):
(WTF::=):
(WTF::Malloc>::contains const):
(WTF::Malloc>::findMatching const):
(WTF::Malloc>::find const):
(WTF::Malloc>::reverseFind const):
(WTF::Malloc>::appendIfNotContains):
(WTF::Malloc>::fill):
(WTF::Malloc>::appendRange):
(WTF::Malloc>::expandCapacity):
(WTF::Malloc>::tryExpandCapacity):
(WTF::Malloc>::resize):
(WTF::Malloc>::resizeToFit):
(WTF::Malloc>::shrink):
(WTF::Malloc>::grow):
(WTF::Malloc>::asanSetInitialBufferSizeTo):
(WTF::Malloc>::asanSetBufferSizeToFullCapacity):
(WTF::Malloc>::asanBufferSizeWillChangeTo):
(WTF::Malloc>::reserveCapacity):
(WTF::Malloc>::tryReserveCapacity):
(WTF::Malloc>::reserveInitialCapacity):
(WTF::Malloc>::shrinkCapacity):
(WTF::Malloc>::append):
(WTF::Malloc>::tryAppend):
(WTF::Malloc>::constructAndAppend):
(WTF::Malloc>::tryConstructAndAppend):
(WTF::Malloc>::appendSlowCase):
(WTF::Malloc>::constructAndAppendSlowCase):
(WTF::Malloc>::tryConstructAndAppendSlowCase):
(WTF::Malloc>::uncheckedAppend):
(WTF::Malloc>::uncheckedConstructAndAppend):
(WTF::Malloc>::appendVector):
(WTF::Malloc>::insert):
(WTF::Malloc>::insertVector):
(WTF::Malloc>::remove):
(WTF::Malloc>::removeFirst):
(WTF::Malloc>::removeFirstMatching):
(WTF::Malloc>::removeAll):
(WTF::Malloc>::removeAllMatching):
(WTF::Malloc>::reverse):
(WTF::Malloc>::map const):
(WTF::Malloc>::releaseBuffer):
(WTF::Malloc>::checkConsistency):
(WTF::swap):
(WTF::operator==):
(WTF::operator!=):
(WTF::Malloc>::isolatedCopy const):
(WTF::removeRepeatedElements):
(WTF::minCapacity>::Vector): Deleted.
(WTF::minCapacity>::contains const): Deleted.
(WTF::minCapacity>::findMatching const): Deleted.
(WTF::minCapacity>::find const): Deleted.
(WTF::minCapacity>::reverseFind const): Deleted.
(WTF::minCapacity>::appendIfNotContains): Deleted.
(WTF::minCapacity>::fill): Deleted.
(WTF::minCapacity>::appendRange): Deleted.
(WTF::minCapacity>::expandCapacity): Deleted.
(WTF::minCapacity>::tryExpandCapacity): Deleted.
(WTF::minCapacity>::resize): Deleted.
(WTF::minCapacity>::resizeToFit): Deleted.
(WTF::minCapacity>::shrink): Deleted.
(WTF::minCapacity>::grow): Deleted.
(WTF::minCapacity>::asanSetInitialBufferSizeTo): Deleted.
(WTF::minCapacity>::asanSetBufferSizeToFullCapacity): Deleted.
(WTF::minCapacity>::asanBufferSizeWillChangeTo): Deleted.
(WTF::minCapacity>::reserveCapacity): Deleted.
(WTF::minCapacity>::tryReserveCapacity): Deleted.
(WTF::minCapacity>::reserveInitialCapacity): Deleted.
(WTF::minCapacity>::shrinkCapacity): Deleted.
(WTF::minCapacity>::append): Deleted.
(WTF::minCapacity>::tryAppend): Deleted.
(WTF::minCapacity>::constructAndAppend): Deleted.
(WTF::minCapacity>::tryConstructAndAppend): Deleted.
(WTF::minCapacity>::appendSlowCase): Deleted.
(WTF::minCapacity>::constructAndAppendSlowCase): Deleted.
(WTF::minCapacity>::tryConstructAndAppendSlowCase): Deleted.
(WTF::minCapacity>::uncheckedAppend): Deleted.
(WTF::minCapacity>::uncheckedConstructAndAppend): Deleted.
(WTF::minCapacity>::appendVector): Deleted.
(WTF::minCapacity>::insert): Deleted.
(WTF::minCapacity>::insertVector): Deleted.
(WTF::minCapacity>::remove): Deleted.
(WTF::minCapacity>::removeFirst): Deleted.
(WTF::minCapacity>::removeFirstMatching): Deleted.
(WTF::minCapacity>::removeAll): Deleted.
(WTF::minCapacity>::removeAllMatching): Deleted.
(WTF::minCapacity>::reverse): Deleted.
(WTF::minCapacity>::map const): Deleted.
(WTF::minCapacity>::releaseBuffer): Deleted.
(WTF::minCapacity>::checkConsistency): Deleted.
(WTF::minCapacity>::isolatedCopy const): Deleted.
* wtf/text/CString.cpp:
(WTF::CStringBuffer::createUninitialized):
* wtf/text/CString.h:
* wtf/text/StringBuffer.cpp: Copied from Source/JavaScriptCore/bytecode/InstructionStream.cpp.
* wtf/text/StringBuffer.h:
(WTF::StringBuffer::StringBuffer):
(WTF::StringBuffer::~StringBuffer):
(WTF::StringBuffer::resize):
(WTF::StringBuffer::release):
* wtf/text/StringImpl.cpp:
(WTF::StringImpl::~StringImpl):
(WTF::StringImpl::destroy):
(WTF::StringImpl::createUninitializedInternalNonEmpty):
(WTF::StringImpl::reallocateInternal):
* wtf/text/StringImpl.h:
(WTF::StringImpl::StringImpl):
(WTF::StringImpl::createSubstringSharingImpl):
(WTF::StringImpl::tryCreateUninitialized):
(WTF::StringImpl::adopt):
* wtf/text/cf/StringImplCF.cpp:
(WTF::StringWrapperCFAllocator::allocate):
(WTF::StringWrapperCFAllocator::reallocate):
(WTF::StringWrapperCFAllocator::deallocate):
Canonical link: https://commits.webkit.org/218863@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@253987 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2020-01-03 02:36:43 +00:00
|
|
|
FastBitVectorMalloc::free(m_words);
|
2012-05-22 07:10:51 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
FastBitVectorWordView view() const { return FastBitVectorWordView(m_words, m_numBits); }
|
|
|
|
|
|
|
|
FastBitVectorWordOwner& operator=(const FastBitVectorWordOwner& other)
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
Make MarkedBlock state tracking support overlapped allocation and marking state
https://bugs.webkit.org/show_bug.cgi?id=161581
Reviewed by Geoffrey Garen.
JSTests:
Add a microbenchmark for why we want to reclaim empty blocks from other allocators.
* microbenchmarks/switching-size-classes.js: Added.
Source/JavaScriptCore:
Concurrent GCs must allow for mutation and allocation during collection. We already know
how to mutate during collection. We have a write barrier for that. Allocation during
collection is more involved: the collector modifies the the mark bits, as well as other
kinds of MarkedBlock state, in-place during a collection. The allocator uses that same
MarkedBlock state to decide which regions of memory are free. This works if the allocator
never runs while the collector is running, but if we want to allow them to run at the same
time, then we need to have two versions of the state: one version built up by the
collector and another consumed by the allocator. We clear the collector state at the
beginning of collection, and splat the collector state onto the allocator state after
collection.
This could be super expensive, but we can make it cheap with some cleverness. The biggest
observation is just that most of the state is a handful of bits per block: is the block
free-listed? is it completely full? completely empty? in the incremental sweeper's
snapshot? is it retired? is it in eden? There is also state inside blocks, like the mark
bits, but I have a solid plan there and I'll save it for another patch. Once we view the
state of blocks as bits, we can put that state into bitvectors, so that if the collector
needs to transform the state of some blocks, it can do it with a single operation over
bitvectors. I like to think of this as 32-way parallelizing block operations, since
doing one operation on a 32-bit word in one of those bitvectors instantly affects 32
blocks.
This change converts all previous collections of MarkedBlocks, along with the MarkedBlock
state, into 8 bitvectors (live, empty, allocated, canAllocateButNotEmpty, eden, unswept,
markingNotEmpty, and markingRetired). The bitvectors separate allocator state (empty,
allocated, canAllocateButNotEmpty) from marking state (markingNotEmpty, markingRetired).
As a nice side-effect of switching to bitvectors, we get size class rebalancing for free.
It used to be that if a MarkedAllocator had an empty block, we would only allow that
memory to be reused by a different MarkedAllocator if we did an incremental sweep or a
full eager sweep. Now we hunt down all destructorless empty blocks before allocating new
MarkedBlocks. It would be relatively easy to also hunt down destructor empty blocks, but
the theory is that those might be expensive to sweep, so it might still be better to leave
those to the incremental sweeper.
This change is perf-neutral all around. I did some tests with two different kinds of
allocation strategies - something that is somewhat easier to do now that you can look for
blocks that are candidates for allocation by just scanning some bitvectors. I tried two
variants:
- Allocate out of non-empty blocks first, leaving empty blocks for last in case a
different allocator needed them. This is sort of a best-fit strategy. I tried this
first, and it can be expressed as:
m_allocationCursor = m_canAllocateButNotEmpty.findBit(m_allocationCursor, true)
- Allocate out of lower-indexed blocks first, treating empty and canAllocateButNotEmpty
blocks equally. This is sort of a first-fit strategy. This is what I ended up settling
on, and it can be expressed as:
m_allocationCursor = (m_canAllocateButNotEmpty | m_empty).findBit(m_allocationCursor, true)
The best-fit strategy meant 1% regressions in LongSpider and Octane overall, and a 11%
regression on Octane/earley. First-fit means perf-neutrality. Most great allocators skew
towards first-fit because it's empirically better, so this result is not surprising.
Overall, the performance of this patch on my machine is as follows, where "neutral" means
less than 1% and not statistically significant.
run-jsc-benchmarks:
SunSpider: neutral
LongSpider: 0.6% slower
V8Spider: neutral
Octane: neutral
Kraken: neutral
Microbenchmarks: 0.37% slower
AsmBench: neutral
CompressionBench: maybe 1% faster
For browser benchmarks, I report the ratio of means (bigger / smaller) along with a T-test
from Mathematica reported as % chance of not [sic] the null hypothesis. Note that we
normally consider anything less than 95% confidence to be inconclusive.
Browser benchmarks:
PLT3: 0.3% faster with 67% confidence
membuster:
Snap2FinishedLoadingPost: 0.68% more memory with 50% confidence
Snap3EndPost: 2.4% more memory with 61% confidence
JetStream: 0.2% slower with 32% confidence
Speedometer: 0.7% faster with 82% confidence
Additionally, Octane/splay's heap capacity goes down to ~180KB from ~200KB, so about a 10%
progression. This is due to the allocator rebalancing feature.
Finally, this breaks --useImmortalObjects. It was already broken as far as I can tell. I
filed a bug to reimplement it (bug 162296). Unless someone urgently needs this internal
tool, it's probably best to reimplement it after I'm done refactoring MarkedSpace.
* JavaScriptCore.xcodeproj/project.pbxproj:
* debugger/Debugger.cpp:
* heap/CellContainer.h:
* heap/CellContainerInlines.h:
(JSC::CellContainer::vm):
(JSC::CellContainer::heap):
(JSC::CellContainer::isMarkedOrNewlyAllocated):
(JSC::CellContainer::aboutToMark):
(JSC::CellContainer::isMarked): Deleted.
(JSC::CellContainer::flipIfNecessary): Deleted.
* heap/ConservativeRoots.cpp:
* heap/Heap.cpp:
(JSC::Heap::beginMarking):
(JSC::Heap::endMarking):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collectImpl):
(JSC::Heap::snapshotMarkedSpace):
(JSC::Heap::prepareForAllocation):
(JSC::Heap::zombifyDeadObjects):
(JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): Deleted.
(JSC::MarkedBlockSnapshotFunctor::operator()): Deleted.
(JSC::Heap::resetAllocators): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isMarked):
(JSC::Heap::isMarkedConcurrently):
(JSC::Heap::testAndSetMarked):
* heap/HeapStatistics.cpp:
* heap/HeapUtil.h:
(JSC::HeapUtil::findGCObjectPointersForMarking):
(JSC::HeapUtil::isPointerGCObjectJSCell):
* heap/HeapVerifier.cpp:
* heap/IncrementalSweeper.cpp:
(JSC::IncrementalSweeper::IncrementalSweeper):
(JSC::IncrementalSweeper::doSweep):
(JSC::IncrementalSweeper::sweepNextBlock):
(JSC::IncrementalSweeper::startSweeping):
(JSC::IncrementalSweeper::willFinishSweeping):
* heap/IncrementalSweeper.h:
* heap/LargeAllocation.h:
(JSC::LargeAllocation::isMarked):
(JSC::LargeAllocation::isMarkedConcurrently):
(JSC::LargeAllocation::isMarkedOrNewlyAllocated):
(JSC::LargeAllocation::aboutToMark):
(JSC::LargeAllocation::isMarkedDuringWeakVisiting): Deleted.
(JSC::LargeAllocation::flipIfNecessary): Deleted.
(JSC::LargeAllocation::flipIfNecessaryDuringMarking): Deleted.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::isPagedOut):
(JSC::MarkedAllocator::findEmptyBlock):
(JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
(JSC::MarkedAllocator::allocateIn):
(JSC::MarkedAllocator::tryAllocateIn):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
(JSC::MarkedAllocator::tryAllocateBlock):
(JSC::MarkedAllocator::addBlock):
(JSC::MarkedAllocator::removeBlock):
(JSC::MarkedAllocator::stopAllocating):
(JSC::MarkedAllocator::prepareForAllocation):
(JSC::MarkedAllocator::lastChanceToFinalize):
(JSC::MarkedAllocator::resumeAllocating):
(JSC::MarkedAllocator::beginMarkingForFullCollection):
(JSC::MarkedAllocator::endMarking):
(JSC::MarkedAllocator::snapshotForEdenCollection):
(JSC::MarkedAllocator::snapshotForFullCollection):
(JSC::MarkedAllocator::findBlockToSweep):
(JSC::MarkedAllocator::sweep):
(JSC::MarkedAllocator::shrink):
(JSC::MarkedAllocator::assertSnapshotEmpty):
(JSC::MarkedAllocator::dump):
(JSC::MarkedAllocator::dumpBits):
(JSC::MarkedAllocator::retire): Deleted.
(JSC::MarkedAllocator::filterNextBlock): Deleted.
(JSC::MarkedAllocator::setNextBlockToSweep): Deleted.
(JSC::MarkedAllocator::reset): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::forEachBitVector):
(JSC::MarkedAllocator::forEachBitVectorWithName):
(JSC::MarkedAllocator::nextAllocator):
(JSC::MarkedAllocator::setNextAllocator):
(JSC::MarkedAllocator::forEachBlock):
(JSC::MarkedAllocator::resumeAllocating): Deleted.
* heap/MarkedBlock.cpp:
(JSC::MarkedBlock::tryCreate):
(JSC::MarkedBlock::Handle::Handle):
(JSC::MarkedBlock::Handle::~Handle):
(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::Handle::specializedSweep):
(JSC::MarkedBlock::Handle::sweep):
(JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectEmptyMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectHasNewlyAllocated):
(JSC::MarkedBlock::Handle::sweepHelperSelectSweepMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectFlipMode):
(JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated):
(JSC::MarkedBlock::Handle::setIsFreeListed):
(JSC::MarkedBlock::Handle::stopAllocating):
(JSC::MarkedBlock::Handle::lastChanceToFinalize):
(JSC::MarkedBlock::Handle::resumeAllocating):
(JSC::MarkedBlock::aboutToMarkSlow):
(JSC::MarkedBlock::clearMarks):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::didConsumeFreeList):
(JSC::MarkedBlock::markCount):
(JSC::MarkedBlock::Handle::isEmpty):
(JSC::MarkedBlock::noteMarkedSlow):
(JSC::MarkedBlock::Handle::removeFromAllocator):
(JSC::MarkedBlock::Handle::didAddToAllocator):
(JSC::MarkedBlock::Handle::didRemoveFromAllocator):
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessarySlow): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarkingSlow): Deleted.
(JSC::MarkedBlock::Handle::willRemoveBlock): Deleted.
(WTF::printInternal): Deleted.
* heap/MarkedBlock.h:
(JSC::MarkedBlock::Handle::isFreeListed):
(JSC::MarkedBlock::Handle::index):
(JSC::MarkedBlock::aboutToMark):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::isMarkedConcurrently):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::isOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::state): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipForEdenCollection): Deleted.
(JSC::MarkedBlock::isMarkedDuringWeakVisiting): Deleted.
(JSC::MarkedBlock::Handle::isLive): Deleted.
(JSC::MarkedBlock::Handle::isLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachDeadCell): Deleted.
(JSC::MarkedBlock::Handle::needsSweeping): Deleted.
(JSC::MarkedBlock::Handle::isAllocated): Deleted.
(JSC::MarkedBlock::Handle::isMarked): Deleted.
* heap/MarkedBlockInlines.h: Added.
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::forEachLiveCell):
(JSC::MarkedBlock::Handle::forEachDeadCell):
(JSC::MarkedBlock::resetVersion):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::MarkedSpace):
(JSC::MarkedSpace::allocate):
(JSC::MarkedSpace::tryAllocate):
(JSC::MarkedSpace::sweep):
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::shrink):
(JSC::MarkedSpace::clearNewlyAllocated):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::endMarking):
(JSC::MarkedSpace::didAllocateInBlock):
(JSC::MarkedSpace::findEmptyBlock):
(JSC::MarkedSpace::snapshot):
(JSC::MarkedSpace::assertSnapshotEmpty):
(JSC::MarkedSpace::dumpBits):
(JSC::MarkedSpace::zombifySweep): Deleted.
(JSC::MarkedSpace::resetAllocators): Deleted.
(JSC::VerifyMarked::operator()): Deleted.
(JSC::MarkedSpace::flip): Deleted.
* heap/MarkedSpace.h:
(JSC::MarkedSpace::nextVersion):
(JSC::MarkedSpace::firstAllocator):
(JSC::MarkedSpace::allocatorForEmptyAllocation):
(JSC::MarkedSpace::forEachAllocator):
(JSC::MarkedSpace::blocksWithNewObjects): Deleted.
(JSC::MarkedSpace::setIsMarking): Deleted.
(JSC::MarkedSpace::forEachLiveCell): Deleted.
(JSC::MarkedSpace::forEachDeadCell): Deleted.
* heap/MarkedSpaceInlines.h: Added.
(JSC::MarkedSpace::forEachLiveCell):
(JSC::MarkedSpace::forEachDeadCell):
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::setMarkedAndAppendToMarkStack):
(JSC::SlotVisitor::markAuxiliary):
(JSC::SlotVisitor::visitChildren):
* heap/Weak.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue):
(WTF::HashTraits<JSC::Weak<T>>::peek):
* heap/WeakBlock.cpp:
(JSC::WeakBlock::specializedVisit):
(JSC::WeakBlock::reap):
* heap/WeakInlines.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue): Deleted.
(WTF::HashTraits<JSC::Weak<T>>::peek): Deleted.
* jit/JITThunks.h:
* runtime/JSGlobalObject.cpp:
* runtime/PrototypeMap.h:
* runtime/SamplingProfiler.cpp:
* runtime/WeakGCMap.h:
* tools/JSDollarVMPrototype.cpp:
Source/WTF:
The main change here is to bring back FastBitVector.cpp, so that I could outline some
large slow path functions. This also adds some utilities, like atomicSetAndCheck() and
isEmpty(). The GC uses these.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/FastBitVector.cpp: Added.
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorImpl::isEmpty):
(WTF::FastBitVector::atomicSetAndCheck):
(WTF::FastBitVector::operator[]): Deleted.
Tools:
Remove the always-trigger-copy-phase configuration.
* Scripts/run-jsc-stress-tests:
Canonical link: https://commits.webkit.org/180306@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206154 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-20 18:12:18 +00:00
|
|
|
if (arrayLength() != other.arrayLength())
|
|
|
|
setEqualsSlow(other);
|
|
|
|
else {
|
2018-02-16 21:38:53 +00:00
|
|
|
memcpy(m_words, other.m_words, arrayLength() * sizeof(uint32_t));
|
Make MarkedBlock state tracking support overlapped allocation and marking state
https://bugs.webkit.org/show_bug.cgi?id=161581
Reviewed by Geoffrey Garen.
JSTests:
Add a microbenchmark for why we want to reclaim empty blocks from other allocators.
* microbenchmarks/switching-size-classes.js: Added.
Source/JavaScriptCore:
Concurrent GCs must allow for mutation and allocation during collection. We already know
how to mutate during collection. We have a write barrier for that. Allocation during
collection is more involved: the collector modifies the the mark bits, as well as other
kinds of MarkedBlock state, in-place during a collection. The allocator uses that same
MarkedBlock state to decide which regions of memory are free. This works if the allocator
never runs while the collector is running, but if we want to allow them to run at the same
time, then we need to have two versions of the state: one version built up by the
collector and another consumed by the allocator. We clear the collector state at the
beginning of collection, and splat the collector state onto the allocator state after
collection.
This could be super expensive, but we can make it cheap with some cleverness. The biggest
observation is just that most of the state is a handful of bits per block: is the block
free-listed? is it completely full? completely empty? in the incremental sweeper's
snapshot? is it retired? is it in eden? There is also state inside blocks, like the mark
bits, but I have a solid plan there and I'll save it for another patch. Once we view the
state of blocks as bits, we can put that state into bitvectors, so that if the collector
needs to transform the state of some blocks, it can do it with a single operation over
bitvectors. I like to think of this as 32-way parallelizing block operations, since
doing one operation on a 32-bit word in one of those bitvectors instantly affects 32
blocks.
This change converts all previous collections of MarkedBlocks, along with the MarkedBlock
state, into 8 bitvectors (live, empty, allocated, canAllocateButNotEmpty, eden, unswept,
markingNotEmpty, and markingRetired). The bitvectors separate allocator state (empty,
allocated, canAllocateButNotEmpty) from marking state (markingNotEmpty, markingRetired).
As a nice side-effect of switching to bitvectors, we get size class rebalancing for free.
It used to be that if a MarkedAllocator had an empty block, we would only allow that
memory to be reused by a different MarkedAllocator if we did an incremental sweep or a
full eager sweep. Now we hunt down all destructorless empty blocks before allocating new
MarkedBlocks. It would be relatively easy to also hunt down destructor empty blocks, but
the theory is that those might be expensive to sweep, so it might still be better to leave
those to the incremental sweeper.
This change is perf-neutral all around. I did some tests with two different kinds of
allocation strategies - something that is somewhat easier to do now that you can look for
blocks that are candidates for allocation by just scanning some bitvectors. I tried two
variants:
- Allocate out of non-empty blocks first, leaving empty blocks for last in case a
different allocator needed them. This is sort of a best-fit strategy. I tried this
first, and it can be expressed as:
m_allocationCursor = m_canAllocateButNotEmpty.findBit(m_allocationCursor, true)
- Allocate out of lower-indexed blocks first, treating empty and canAllocateButNotEmpty
blocks equally. This is sort of a first-fit strategy. This is what I ended up settling
on, and it can be expressed as:
m_allocationCursor = (m_canAllocateButNotEmpty | m_empty).findBit(m_allocationCursor, true)
The best-fit strategy meant 1% regressions in LongSpider and Octane overall, and a 11%
regression on Octane/earley. First-fit means perf-neutrality. Most great allocators skew
towards first-fit because it's empirically better, so this result is not surprising.
Overall, the performance of this patch on my machine is as follows, where "neutral" means
less than 1% and not statistically significant.
run-jsc-benchmarks:
SunSpider: neutral
LongSpider: 0.6% slower
V8Spider: neutral
Octane: neutral
Kraken: neutral
Microbenchmarks: 0.37% slower
AsmBench: neutral
CompressionBench: maybe 1% faster
For browser benchmarks, I report the ratio of means (bigger / smaller) along with a T-test
from Mathematica reported as % chance of not [sic] the null hypothesis. Note that we
normally consider anything less than 95% confidence to be inconclusive.
Browser benchmarks:
PLT3: 0.3% faster with 67% confidence
membuster:
Snap2FinishedLoadingPost: 0.68% more memory with 50% confidence
Snap3EndPost: 2.4% more memory with 61% confidence
JetStream: 0.2% slower with 32% confidence
Speedometer: 0.7% faster with 82% confidence
Additionally, Octane/splay's heap capacity goes down to ~180KB from ~200KB, so about a 10%
progression. This is due to the allocator rebalancing feature.
Finally, this breaks --useImmortalObjects. It was already broken as far as I can tell. I
filed a bug to reimplement it (bug 162296). Unless someone urgently needs this internal
tool, it's probably best to reimplement it after I'm done refactoring MarkedSpace.
* JavaScriptCore.xcodeproj/project.pbxproj:
* debugger/Debugger.cpp:
* heap/CellContainer.h:
* heap/CellContainerInlines.h:
(JSC::CellContainer::vm):
(JSC::CellContainer::heap):
(JSC::CellContainer::isMarkedOrNewlyAllocated):
(JSC::CellContainer::aboutToMark):
(JSC::CellContainer::isMarked): Deleted.
(JSC::CellContainer::flipIfNecessary): Deleted.
* heap/ConservativeRoots.cpp:
* heap/Heap.cpp:
(JSC::Heap::beginMarking):
(JSC::Heap::endMarking):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collectImpl):
(JSC::Heap::snapshotMarkedSpace):
(JSC::Heap::prepareForAllocation):
(JSC::Heap::zombifyDeadObjects):
(JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): Deleted.
(JSC::MarkedBlockSnapshotFunctor::operator()): Deleted.
(JSC::Heap::resetAllocators): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isMarked):
(JSC::Heap::isMarkedConcurrently):
(JSC::Heap::testAndSetMarked):
* heap/HeapStatistics.cpp:
* heap/HeapUtil.h:
(JSC::HeapUtil::findGCObjectPointersForMarking):
(JSC::HeapUtil::isPointerGCObjectJSCell):
* heap/HeapVerifier.cpp:
* heap/IncrementalSweeper.cpp:
(JSC::IncrementalSweeper::IncrementalSweeper):
(JSC::IncrementalSweeper::doSweep):
(JSC::IncrementalSweeper::sweepNextBlock):
(JSC::IncrementalSweeper::startSweeping):
(JSC::IncrementalSweeper::willFinishSweeping):
* heap/IncrementalSweeper.h:
* heap/LargeAllocation.h:
(JSC::LargeAllocation::isMarked):
(JSC::LargeAllocation::isMarkedConcurrently):
(JSC::LargeAllocation::isMarkedOrNewlyAllocated):
(JSC::LargeAllocation::aboutToMark):
(JSC::LargeAllocation::isMarkedDuringWeakVisiting): Deleted.
(JSC::LargeAllocation::flipIfNecessary): Deleted.
(JSC::LargeAllocation::flipIfNecessaryDuringMarking): Deleted.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::isPagedOut):
(JSC::MarkedAllocator::findEmptyBlock):
(JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
(JSC::MarkedAllocator::allocateIn):
(JSC::MarkedAllocator::tryAllocateIn):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
(JSC::MarkedAllocator::tryAllocateBlock):
(JSC::MarkedAllocator::addBlock):
(JSC::MarkedAllocator::removeBlock):
(JSC::MarkedAllocator::stopAllocating):
(JSC::MarkedAllocator::prepareForAllocation):
(JSC::MarkedAllocator::lastChanceToFinalize):
(JSC::MarkedAllocator::resumeAllocating):
(JSC::MarkedAllocator::beginMarkingForFullCollection):
(JSC::MarkedAllocator::endMarking):
(JSC::MarkedAllocator::snapshotForEdenCollection):
(JSC::MarkedAllocator::snapshotForFullCollection):
(JSC::MarkedAllocator::findBlockToSweep):
(JSC::MarkedAllocator::sweep):
(JSC::MarkedAllocator::shrink):
(JSC::MarkedAllocator::assertSnapshotEmpty):
(JSC::MarkedAllocator::dump):
(JSC::MarkedAllocator::dumpBits):
(JSC::MarkedAllocator::retire): Deleted.
(JSC::MarkedAllocator::filterNextBlock): Deleted.
(JSC::MarkedAllocator::setNextBlockToSweep): Deleted.
(JSC::MarkedAllocator::reset): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::forEachBitVector):
(JSC::MarkedAllocator::forEachBitVectorWithName):
(JSC::MarkedAllocator::nextAllocator):
(JSC::MarkedAllocator::setNextAllocator):
(JSC::MarkedAllocator::forEachBlock):
(JSC::MarkedAllocator::resumeAllocating): Deleted.
* heap/MarkedBlock.cpp:
(JSC::MarkedBlock::tryCreate):
(JSC::MarkedBlock::Handle::Handle):
(JSC::MarkedBlock::Handle::~Handle):
(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::Handle::specializedSweep):
(JSC::MarkedBlock::Handle::sweep):
(JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectEmptyMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectHasNewlyAllocated):
(JSC::MarkedBlock::Handle::sweepHelperSelectSweepMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectFlipMode):
(JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated):
(JSC::MarkedBlock::Handle::setIsFreeListed):
(JSC::MarkedBlock::Handle::stopAllocating):
(JSC::MarkedBlock::Handle::lastChanceToFinalize):
(JSC::MarkedBlock::Handle::resumeAllocating):
(JSC::MarkedBlock::aboutToMarkSlow):
(JSC::MarkedBlock::clearMarks):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::didConsumeFreeList):
(JSC::MarkedBlock::markCount):
(JSC::MarkedBlock::Handle::isEmpty):
(JSC::MarkedBlock::noteMarkedSlow):
(JSC::MarkedBlock::Handle::removeFromAllocator):
(JSC::MarkedBlock::Handle::didAddToAllocator):
(JSC::MarkedBlock::Handle::didRemoveFromAllocator):
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessarySlow): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarkingSlow): Deleted.
(JSC::MarkedBlock::Handle::willRemoveBlock): Deleted.
(WTF::printInternal): Deleted.
* heap/MarkedBlock.h:
(JSC::MarkedBlock::Handle::isFreeListed):
(JSC::MarkedBlock::Handle::index):
(JSC::MarkedBlock::aboutToMark):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::isMarkedConcurrently):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::isOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::state): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipForEdenCollection): Deleted.
(JSC::MarkedBlock::isMarkedDuringWeakVisiting): Deleted.
(JSC::MarkedBlock::Handle::isLive): Deleted.
(JSC::MarkedBlock::Handle::isLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachDeadCell): Deleted.
(JSC::MarkedBlock::Handle::needsSweeping): Deleted.
(JSC::MarkedBlock::Handle::isAllocated): Deleted.
(JSC::MarkedBlock::Handle::isMarked): Deleted.
* heap/MarkedBlockInlines.h: Added.
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::forEachLiveCell):
(JSC::MarkedBlock::Handle::forEachDeadCell):
(JSC::MarkedBlock::resetVersion):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::MarkedSpace):
(JSC::MarkedSpace::allocate):
(JSC::MarkedSpace::tryAllocate):
(JSC::MarkedSpace::sweep):
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::shrink):
(JSC::MarkedSpace::clearNewlyAllocated):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::endMarking):
(JSC::MarkedSpace::didAllocateInBlock):
(JSC::MarkedSpace::findEmptyBlock):
(JSC::MarkedSpace::snapshot):
(JSC::MarkedSpace::assertSnapshotEmpty):
(JSC::MarkedSpace::dumpBits):
(JSC::MarkedSpace::zombifySweep): Deleted.
(JSC::MarkedSpace::resetAllocators): Deleted.
(JSC::VerifyMarked::operator()): Deleted.
(JSC::MarkedSpace::flip): Deleted.
* heap/MarkedSpace.h:
(JSC::MarkedSpace::nextVersion):
(JSC::MarkedSpace::firstAllocator):
(JSC::MarkedSpace::allocatorForEmptyAllocation):
(JSC::MarkedSpace::forEachAllocator):
(JSC::MarkedSpace::blocksWithNewObjects): Deleted.
(JSC::MarkedSpace::setIsMarking): Deleted.
(JSC::MarkedSpace::forEachLiveCell): Deleted.
(JSC::MarkedSpace::forEachDeadCell): Deleted.
* heap/MarkedSpaceInlines.h: Added.
(JSC::MarkedSpace::forEachLiveCell):
(JSC::MarkedSpace::forEachDeadCell):
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::setMarkedAndAppendToMarkStack):
(JSC::SlotVisitor::markAuxiliary):
(JSC::SlotVisitor::visitChildren):
* heap/Weak.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue):
(WTF::HashTraits<JSC::Weak<T>>::peek):
* heap/WeakBlock.cpp:
(JSC::WeakBlock::specializedVisit):
(JSC::WeakBlock::reap):
* heap/WeakInlines.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue): Deleted.
(WTF::HashTraits<JSC::Weak<T>>::peek): Deleted.
* jit/JITThunks.h:
* runtime/JSGlobalObject.cpp:
* runtime/PrototypeMap.h:
* runtime/SamplingProfiler.cpp:
* runtime/WeakGCMap.h:
* tools/JSDollarVMPrototype.cpp:
Source/WTF:
The main change here is to bring back FastBitVector.cpp, so that I could outline some
large slow path functions. This also adds some utilities, like atomicSetAndCheck() and
isEmpty(). The GC uses these.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/FastBitVector.cpp: Added.
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorImpl::isEmpty):
(WTF::FastBitVector::atomicSetAndCheck):
(WTF::FastBitVector::operator[]): Deleted.
Tools:
Remove the always-trigger-copy-phase configuration.
* Scripts/run-jsc-stress-tests:
Canonical link: https://commits.webkit.org/180306@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206154 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-20 18:12:18 +00:00
|
|
|
m_numBits = other.m_numBits;
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
}
|
2012-05-22 00:51:05 +00:00
|
|
|
return *this;
|
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
FastBitVectorWordOwner& operator=(FastBitVectorWordOwner&& other)
|
|
|
|
{
|
|
|
|
std::swap(m_words, other.m_words);
|
|
|
|
std::swap(m_numBits, other.m_numBits);
|
|
|
|
return *this;
|
|
|
|
}
|
|
|
|
|
|
|
|
void setAll()
|
|
|
|
{
|
|
|
|
memset(m_words, 255, arrayLength() * sizeof(uint32_t));
|
|
|
|
}
|
|
|
|
|
|
|
|
void clearAll()
|
|
|
|
{
|
2018-02-16 21:38:53 +00:00
|
|
|
memset(m_words, 0, arrayLength() * sizeof(uint32_t));
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
void set(const FastBitVectorWordOwner& other)
|
|
|
|
{
|
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(m_numBits == other.m_numBits);
|
2018-02-16 21:38:53 +00:00
|
|
|
memcpy(m_words, other.m_words, arrayLength() * sizeof(uint32_t));
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
size_t numBits() const
|
|
|
|
{
|
|
|
|
return m_numBits;
|
|
|
|
}
|
|
|
|
|
|
|
|
size_t arrayLength() const
|
|
|
|
{
|
|
|
|
return fastBitVectorArrayLength(numBits());
|
|
|
|
}
|
2012-05-22 00:51:05 +00:00
|
|
|
|
|
|
|
void resize(size_t numBits)
|
|
|
|
{
|
Make MarkedBlock state tracking support overlapped allocation and marking state
https://bugs.webkit.org/show_bug.cgi?id=161581
Reviewed by Geoffrey Garen.
JSTests:
Add a microbenchmark for why we want to reclaim empty blocks from other allocators.
* microbenchmarks/switching-size-classes.js: Added.
Source/JavaScriptCore:
Concurrent GCs must allow for mutation and allocation during collection. We already know
how to mutate during collection. We have a write barrier for that. Allocation during
collection is more involved: the collector modifies the the mark bits, as well as other
kinds of MarkedBlock state, in-place during a collection. The allocator uses that same
MarkedBlock state to decide which regions of memory are free. This works if the allocator
never runs while the collector is running, but if we want to allow them to run at the same
time, then we need to have two versions of the state: one version built up by the
collector and another consumed by the allocator. We clear the collector state at the
beginning of collection, and splat the collector state onto the allocator state after
collection.
This could be super expensive, but we can make it cheap with some cleverness. The biggest
observation is just that most of the state is a handful of bits per block: is the block
free-listed? is it completely full? completely empty? in the incremental sweeper's
snapshot? is it retired? is it in eden? There is also state inside blocks, like the mark
bits, but I have a solid plan there and I'll save it for another patch. Once we view the
state of blocks as bits, we can put that state into bitvectors, so that if the collector
needs to transform the state of some blocks, it can do it with a single operation over
bitvectors. I like to think of this as 32-way parallelizing block operations, since
doing one operation on a 32-bit word in one of those bitvectors instantly affects 32
blocks.
This change converts all previous collections of MarkedBlocks, along with the MarkedBlock
state, into 8 bitvectors (live, empty, allocated, canAllocateButNotEmpty, eden, unswept,
markingNotEmpty, and markingRetired). The bitvectors separate allocator state (empty,
allocated, canAllocateButNotEmpty) from marking state (markingNotEmpty, markingRetired).
As a nice side-effect of switching to bitvectors, we get size class rebalancing for free.
It used to be that if a MarkedAllocator had an empty block, we would only allow that
memory to be reused by a different MarkedAllocator if we did an incremental sweep or a
full eager sweep. Now we hunt down all destructorless empty blocks before allocating new
MarkedBlocks. It would be relatively easy to also hunt down destructor empty blocks, but
the theory is that those might be expensive to sweep, so it might still be better to leave
those to the incremental sweeper.
This change is perf-neutral all around. I did some tests with two different kinds of
allocation strategies - something that is somewhat easier to do now that you can look for
blocks that are candidates for allocation by just scanning some bitvectors. I tried two
variants:
- Allocate out of non-empty blocks first, leaving empty blocks for last in case a
different allocator needed them. This is sort of a best-fit strategy. I tried this
first, and it can be expressed as:
m_allocationCursor = m_canAllocateButNotEmpty.findBit(m_allocationCursor, true)
- Allocate out of lower-indexed blocks first, treating empty and canAllocateButNotEmpty
blocks equally. This is sort of a first-fit strategy. This is what I ended up settling
on, and it can be expressed as:
m_allocationCursor = (m_canAllocateButNotEmpty | m_empty).findBit(m_allocationCursor, true)
The best-fit strategy meant 1% regressions in LongSpider and Octane overall, and a 11%
regression on Octane/earley. First-fit means perf-neutrality. Most great allocators skew
towards first-fit because it's empirically better, so this result is not surprising.
Overall, the performance of this patch on my machine is as follows, where "neutral" means
less than 1% and not statistically significant.
run-jsc-benchmarks:
SunSpider: neutral
LongSpider: 0.6% slower
V8Spider: neutral
Octane: neutral
Kraken: neutral
Microbenchmarks: 0.37% slower
AsmBench: neutral
CompressionBench: maybe 1% faster
For browser benchmarks, I report the ratio of means (bigger / smaller) along with a T-test
from Mathematica reported as % chance of not [sic] the null hypothesis. Note that we
normally consider anything less than 95% confidence to be inconclusive.
Browser benchmarks:
PLT3: 0.3% faster with 67% confidence
membuster:
Snap2FinishedLoadingPost: 0.68% more memory with 50% confidence
Snap3EndPost: 2.4% more memory with 61% confidence
JetStream: 0.2% slower with 32% confidence
Speedometer: 0.7% faster with 82% confidence
Additionally, Octane/splay's heap capacity goes down to ~180KB from ~200KB, so about a 10%
progression. This is due to the allocator rebalancing feature.
Finally, this breaks --useImmortalObjects. It was already broken as far as I can tell. I
filed a bug to reimplement it (bug 162296). Unless someone urgently needs this internal
tool, it's probably best to reimplement it after I'm done refactoring MarkedSpace.
* JavaScriptCore.xcodeproj/project.pbxproj:
* debugger/Debugger.cpp:
* heap/CellContainer.h:
* heap/CellContainerInlines.h:
(JSC::CellContainer::vm):
(JSC::CellContainer::heap):
(JSC::CellContainer::isMarkedOrNewlyAllocated):
(JSC::CellContainer::aboutToMark):
(JSC::CellContainer::isMarked): Deleted.
(JSC::CellContainer::flipIfNecessary): Deleted.
* heap/ConservativeRoots.cpp:
* heap/Heap.cpp:
(JSC::Heap::beginMarking):
(JSC::Heap::endMarking):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collectImpl):
(JSC::Heap::snapshotMarkedSpace):
(JSC::Heap::prepareForAllocation):
(JSC::Heap::zombifyDeadObjects):
(JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): Deleted.
(JSC::MarkedBlockSnapshotFunctor::operator()): Deleted.
(JSC::Heap::resetAllocators): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isMarked):
(JSC::Heap::isMarkedConcurrently):
(JSC::Heap::testAndSetMarked):
* heap/HeapStatistics.cpp:
* heap/HeapUtil.h:
(JSC::HeapUtil::findGCObjectPointersForMarking):
(JSC::HeapUtil::isPointerGCObjectJSCell):
* heap/HeapVerifier.cpp:
* heap/IncrementalSweeper.cpp:
(JSC::IncrementalSweeper::IncrementalSweeper):
(JSC::IncrementalSweeper::doSweep):
(JSC::IncrementalSweeper::sweepNextBlock):
(JSC::IncrementalSweeper::startSweeping):
(JSC::IncrementalSweeper::willFinishSweeping):
* heap/IncrementalSweeper.h:
* heap/LargeAllocation.h:
(JSC::LargeAllocation::isMarked):
(JSC::LargeAllocation::isMarkedConcurrently):
(JSC::LargeAllocation::isMarkedOrNewlyAllocated):
(JSC::LargeAllocation::aboutToMark):
(JSC::LargeAllocation::isMarkedDuringWeakVisiting): Deleted.
(JSC::LargeAllocation::flipIfNecessary): Deleted.
(JSC::LargeAllocation::flipIfNecessaryDuringMarking): Deleted.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::isPagedOut):
(JSC::MarkedAllocator::findEmptyBlock):
(JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
(JSC::MarkedAllocator::allocateIn):
(JSC::MarkedAllocator::tryAllocateIn):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
(JSC::MarkedAllocator::tryAllocateBlock):
(JSC::MarkedAllocator::addBlock):
(JSC::MarkedAllocator::removeBlock):
(JSC::MarkedAllocator::stopAllocating):
(JSC::MarkedAllocator::prepareForAllocation):
(JSC::MarkedAllocator::lastChanceToFinalize):
(JSC::MarkedAllocator::resumeAllocating):
(JSC::MarkedAllocator::beginMarkingForFullCollection):
(JSC::MarkedAllocator::endMarking):
(JSC::MarkedAllocator::snapshotForEdenCollection):
(JSC::MarkedAllocator::snapshotForFullCollection):
(JSC::MarkedAllocator::findBlockToSweep):
(JSC::MarkedAllocator::sweep):
(JSC::MarkedAllocator::shrink):
(JSC::MarkedAllocator::assertSnapshotEmpty):
(JSC::MarkedAllocator::dump):
(JSC::MarkedAllocator::dumpBits):
(JSC::MarkedAllocator::retire): Deleted.
(JSC::MarkedAllocator::filterNextBlock): Deleted.
(JSC::MarkedAllocator::setNextBlockToSweep): Deleted.
(JSC::MarkedAllocator::reset): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::forEachBitVector):
(JSC::MarkedAllocator::forEachBitVectorWithName):
(JSC::MarkedAllocator::nextAllocator):
(JSC::MarkedAllocator::setNextAllocator):
(JSC::MarkedAllocator::forEachBlock):
(JSC::MarkedAllocator::resumeAllocating): Deleted.
* heap/MarkedBlock.cpp:
(JSC::MarkedBlock::tryCreate):
(JSC::MarkedBlock::Handle::Handle):
(JSC::MarkedBlock::Handle::~Handle):
(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::Handle::specializedSweep):
(JSC::MarkedBlock::Handle::sweep):
(JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectEmptyMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectHasNewlyAllocated):
(JSC::MarkedBlock::Handle::sweepHelperSelectSweepMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectFlipMode):
(JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated):
(JSC::MarkedBlock::Handle::setIsFreeListed):
(JSC::MarkedBlock::Handle::stopAllocating):
(JSC::MarkedBlock::Handle::lastChanceToFinalize):
(JSC::MarkedBlock::Handle::resumeAllocating):
(JSC::MarkedBlock::aboutToMarkSlow):
(JSC::MarkedBlock::clearMarks):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::didConsumeFreeList):
(JSC::MarkedBlock::markCount):
(JSC::MarkedBlock::Handle::isEmpty):
(JSC::MarkedBlock::noteMarkedSlow):
(JSC::MarkedBlock::Handle::removeFromAllocator):
(JSC::MarkedBlock::Handle::didAddToAllocator):
(JSC::MarkedBlock::Handle::didRemoveFromAllocator):
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessarySlow): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarkingSlow): Deleted.
(JSC::MarkedBlock::Handle::willRemoveBlock): Deleted.
(WTF::printInternal): Deleted.
* heap/MarkedBlock.h:
(JSC::MarkedBlock::Handle::isFreeListed):
(JSC::MarkedBlock::Handle::index):
(JSC::MarkedBlock::aboutToMark):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::isMarkedConcurrently):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::isOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::state): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipForEdenCollection): Deleted.
(JSC::MarkedBlock::isMarkedDuringWeakVisiting): Deleted.
(JSC::MarkedBlock::Handle::isLive): Deleted.
(JSC::MarkedBlock::Handle::isLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachDeadCell): Deleted.
(JSC::MarkedBlock::Handle::needsSweeping): Deleted.
(JSC::MarkedBlock::Handle::isAllocated): Deleted.
(JSC::MarkedBlock::Handle::isMarked): Deleted.
* heap/MarkedBlockInlines.h: Added.
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::forEachLiveCell):
(JSC::MarkedBlock::Handle::forEachDeadCell):
(JSC::MarkedBlock::resetVersion):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::MarkedSpace):
(JSC::MarkedSpace::allocate):
(JSC::MarkedSpace::tryAllocate):
(JSC::MarkedSpace::sweep):
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::shrink):
(JSC::MarkedSpace::clearNewlyAllocated):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::endMarking):
(JSC::MarkedSpace::didAllocateInBlock):
(JSC::MarkedSpace::findEmptyBlock):
(JSC::MarkedSpace::snapshot):
(JSC::MarkedSpace::assertSnapshotEmpty):
(JSC::MarkedSpace::dumpBits):
(JSC::MarkedSpace::zombifySweep): Deleted.
(JSC::MarkedSpace::resetAllocators): Deleted.
(JSC::VerifyMarked::operator()): Deleted.
(JSC::MarkedSpace::flip): Deleted.
* heap/MarkedSpace.h:
(JSC::MarkedSpace::nextVersion):
(JSC::MarkedSpace::firstAllocator):
(JSC::MarkedSpace::allocatorForEmptyAllocation):
(JSC::MarkedSpace::forEachAllocator):
(JSC::MarkedSpace::blocksWithNewObjects): Deleted.
(JSC::MarkedSpace::setIsMarking): Deleted.
(JSC::MarkedSpace::forEachLiveCell): Deleted.
(JSC::MarkedSpace::forEachDeadCell): Deleted.
* heap/MarkedSpaceInlines.h: Added.
(JSC::MarkedSpace::forEachLiveCell):
(JSC::MarkedSpace::forEachDeadCell):
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::setMarkedAndAppendToMarkStack):
(JSC::SlotVisitor::markAuxiliary):
(JSC::SlotVisitor::visitChildren):
* heap/Weak.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue):
(WTF::HashTraits<JSC::Weak<T>>::peek):
* heap/WeakBlock.cpp:
(JSC::WeakBlock::specializedVisit):
(JSC::WeakBlock::reap):
* heap/WeakInlines.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue): Deleted.
(WTF::HashTraits<JSC::Weak<T>>::peek): Deleted.
* jit/JITThunks.h:
* runtime/JSGlobalObject.cpp:
* runtime/PrototypeMap.h:
* runtime/SamplingProfiler.cpp:
* runtime/WeakGCMap.h:
* tools/JSDollarVMPrototype.cpp:
Source/WTF:
The main change here is to bring back FastBitVector.cpp, so that I could outline some
large slow path functions. This also adds some utilities, like atomicSetAndCheck() and
isEmpty(). The GC uses these.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/FastBitVector.cpp: Added.
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorImpl::isEmpty):
(WTF::FastBitVector::atomicSetAndCheck):
(WTF::FastBitVector::operator[]): Deleted.
Tools:
Remove the always-trigger-copy-phase configuration.
* Scripts/run-jsc-stress-tests:
Canonical link: https://commits.webkit.org/180306@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206154 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-20 18:12:18 +00:00
|
|
|
if (arrayLength() != fastBitVectorArrayLength(numBits))
|
|
|
|
resizeSlow(numBits);
|
2012-05-22 00:51:05 +00:00
|
|
|
m_numBits = numBits;
|
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
uint32_t word(size_t index) const
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(index < arrayLength());
|
|
|
|
return m_words[index];
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
uint32_t& word(size_t index)
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(index < arrayLength());
|
|
|
|
return m_words[index];
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
const uint32_t* words() const { return m_words; }
|
|
|
|
uint32_t* words() { return m_words; }
|
|
|
|
|
|
|
|
private:
|
Make MarkedBlock state tracking support overlapped allocation and marking state
https://bugs.webkit.org/show_bug.cgi?id=161581
Reviewed by Geoffrey Garen.
JSTests:
Add a microbenchmark for why we want to reclaim empty blocks from other allocators.
* microbenchmarks/switching-size-classes.js: Added.
Source/JavaScriptCore:
Concurrent GCs must allow for mutation and allocation during collection. We already know
how to mutate during collection. We have a write barrier for that. Allocation during
collection is more involved: the collector modifies the the mark bits, as well as other
kinds of MarkedBlock state, in-place during a collection. The allocator uses that same
MarkedBlock state to decide which regions of memory are free. This works if the allocator
never runs while the collector is running, but if we want to allow them to run at the same
time, then we need to have two versions of the state: one version built up by the
collector and another consumed by the allocator. We clear the collector state at the
beginning of collection, and splat the collector state onto the allocator state after
collection.
This could be super expensive, but we can make it cheap with some cleverness. The biggest
observation is just that most of the state is a handful of bits per block: is the block
free-listed? is it completely full? completely empty? in the incremental sweeper's
snapshot? is it retired? is it in eden? There is also state inside blocks, like the mark
bits, but I have a solid plan there and I'll save it for another patch. Once we view the
state of blocks as bits, we can put that state into bitvectors, so that if the collector
needs to transform the state of some blocks, it can do it with a single operation over
bitvectors. I like to think of this as 32-way parallelizing block operations, since
doing one operation on a 32-bit word in one of those bitvectors instantly affects 32
blocks.
This change converts all previous collections of MarkedBlocks, along with the MarkedBlock
state, into 8 bitvectors (live, empty, allocated, canAllocateButNotEmpty, eden, unswept,
markingNotEmpty, and markingRetired). The bitvectors separate allocator state (empty,
allocated, canAllocateButNotEmpty) from marking state (markingNotEmpty, markingRetired).
As a nice side-effect of switching to bitvectors, we get size class rebalancing for free.
It used to be that if a MarkedAllocator had an empty block, we would only allow that
memory to be reused by a different MarkedAllocator if we did an incremental sweep or a
full eager sweep. Now we hunt down all destructorless empty blocks before allocating new
MarkedBlocks. It would be relatively easy to also hunt down destructor empty blocks, but
the theory is that those might be expensive to sweep, so it might still be better to leave
those to the incremental sweeper.
This change is perf-neutral all around. I did some tests with two different kinds of
allocation strategies - something that is somewhat easier to do now that you can look for
blocks that are candidates for allocation by just scanning some bitvectors. I tried two
variants:
- Allocate out of non-empty blocks first, leaving empty blocks for last in case a
different allocator needed them. This is sort of a best-fit strategy. I tried this
first, and it can be expressed as:
m_allocationCursor = m_canAllocateButNotEmpty.findBit(m_allocationCursor, true)
- Allocate out of lower-indexed blocks first, treating empty and canAllocateButNotEmpty
blocks equally. This is sort of a first-fit strategy. This is what I ended up settling
on, and it can be expressed as:
m_allocationCursor = (m_canAllocateButNotEmpty | m_empty).findBit(m_allocationCursor, true)
The best-fit strategy meant 1% regressions in LongSpider and Octane overall, and a 11%
regression on Octane/earley. First-fit means perf-neutrality. Most great allocators skew
towards first-fit because it's empirically better, so this result is not surprising.
Overall, the performance of this patch on my machine is as follows, where "neutral" means
less than 1% and not statistically significant.
run-jsc-benchmarks:
SunSpider: neutral
LongSpider: 0.6% slower
V8Spider: neutral
Octane: neutral
Kraken: neutral
Microbenchmarks: 0.37% slower
AsmBench: neutral
CompressionBench: maybe 1% faster
For browser benchmarks, I report the ratio of means (bigger / smaller) along with a T-test
from Mathematica reported as % chance of not [sic] the null hypothesis. Note that we
normally consider anything less than 95% confidence to be inconclusive.
Browser benchmarks:
PLT3: 0.3% faster with 67% confidence
membuster:
Snap2FinishedLoadingPost: 0.68% more memory with 50% confidence
Snap3EndPost: 2.4% more memory with 61% confidence
JetStream: 0.2% slower with 32% confidence
Speedometer: 0.7% faster with 82% confidence
Additionally, Octane/splay's heap capacity goes down to ~180KB from ~200KB, so about a 10%
progression. This is due to the allocator rebalancing feature.
Finally, this breaks --useImmortalObjects. It was already broken as far as I can tell. I
filed a bug to reimplement it (bug 162296). Unless someone urgently needs this internal
tool, it's probably best to reimplement it after I'm done refactoring MarkedSpace.
* JavaScriptCore.xcodeproj/project.pbxproj:
* debugger/Debugger.cpp:
* heap/CellContainer.h:
* heap/CellContainerInlines.h:
(JSC::CellContainer::vm):
(JSC::CellContainer::heap):
(JSC::CellContainer::isMarkedOrNewlyAllocated):
(JSC::CellContainer::aboutToMark):
(JSC::CellContainer::isMarked): Deleted.
(JSC::CellContainer::flipIfNecessary): Deleted.
* heap/ConservativeRoots.cpp:
* heap/Heap.cpp:
(JSC::Heap::beginMarking):
(JSC::Heap::endMarking):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collectImpl):
(JSC::Heap::snapshotMarkedSpace):
(JSC::Heap::prepareForAllocation):
(JSC::Heap::zombifyDeadObjects):
(JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): Deleted.
(JSC::MarkedBlockSnapshotFunctor::operator()): Deleted.
(JSC::Heap::resetAllocators): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isMarked):
(JSC::Heap::isMarkedConcurrently):
(JSC::Heap::testAndSetMarked):
* heap/HeapStatistics.cpp:
* heap/HeapUtil.h:
(JSC::HeapUtil::findGCObjectPointersForMarking):
(JSC::HeapUtil::isPointerGCObjectJSCell):
* heap/HeapVerifier.cpp:
* heap/IncrementalSweeper.cpp:
(JSC::IncrementalSweeper::IncrementalSweeper):
(JSC::IncrementalSweeper::doSweep):
(JSC::IncrementalSweeper::sweepNextBlock):
(JSC::IncrementalSweeper::startSweeping):
(JSC::IncrementalSweeper::willFinishSweeping):
* heap/IncrementalSweeper.h:
* heap/LargeAllocation.h:
(JSC::LargeAllocation::isMarked):
(JSC::LargeAllocation::isMarkedConcurrently):
(JSC::LargeAllocation::isMarkedOrNewlyAllocated):
(JSC::LargeAllocation::aboutToMark):
(JSC::LargeAllocation::isMarkedDuringWeakVisiting): Deleted.
(JSC::LargeAllocation::flipIfNecessary): Deleted.
(JSC::LargeAllocation::flipIfNecessaryDuringMarking): Deleted.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::isPagedOut):
(JSC::MarkedAllocator::findEmptyBlock):
(JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
(JSC::MarkedAllocator::allocateIn):
(JSC::MarkedAllocator::tryAllocateIn):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
(JSC::MarkedAllocator::tryAllocateBlock):
(JSC::MarkedAllocator::addBlock):
(JSC::MarkedAllocator::removeBlock):
(JSC::MarkedAllocator::stopAllocating):
(JSC::MarkedAllocator::prepareForAllocation):
(JSC::MarkedAllocator::lastChanceToFinalize):
(JSC::MarkedAllocator::resumeAllocating):
(JSC::MarkedAllocator::beginMarkingForFullCollection):
(JSC::MarkedAllocator::endMarking):
(JSC::MarkedAllocator::snapshotForEdenCollection):
(JSC::MarkedAllocator::snapshotForFullCollection):
(JSC::MarkedAllocator::findBlockToSweep):
(JSC::MarkedAllocator::sweep):
(JSC::MarkedAllocator::shrink):
(JSC::MarkedAllocator::assertSnapshotEmpty):
(JSC::MarkedAllocator::dump):
(JSC::MarkedAllocator::dumpBits):
(JSC::MarkedAllocator::retire): Deleted.
(JSC::MarkedAllocator::filterNextBlock): Deleted.
(JSC::MarkedAllocator::setNextBlockToSweep): Deleted.
(JSC::MarkedAllocator::reset): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::forEachBitVector):
(JSC::MarkedAllocator::forEachBitVectorWithName):
(JSC::MarkedAllocator::nextAllocator):
(JSC::MarkedAllocator::setNextAllocator):
(JSC::MarkedAllocator::forEachBlock):
(JSC::MarkedAllocator::resumeAllocating): Deleted.
* heap/MarkedBlock.cpp:
(JSC::MarkedBlock::tryCreate):
(JSC::MarkedBlock::Handle::Handle):
(JSC::MarkedBlock::Handle::~Handle):
(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::Handle::specializedSweep):
(JSC::MarkedBlock::Handle::sweep):
(JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectEmptyMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectHasNewlyAllocated):
(JSC::MarkedBlock::Handle::sweepHelperSelectSweepMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectFlipMode):
(JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated):
(JSC::MarkedBlock::Handle::setIsFreeListed):
(JSC::MarkedBlock::Handle::stopAllocating):
(JSC::MarkedBlock::Handle::lastChanceToFinalize):
(JSC::MarkedBlock::Handle::resumeAllocating):
(JSC::MarkedBlock::aboutToMarkSlow):
(JSC::MarkedBlock::clearMarks):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::didConsumeFreeList):
(JSC::MarkedBlock::markCount):
(JSC::MarkedBlock::Handle::isEmpty):
(JSC::MarkedBlock::noteMarkedSlow):
(JSC::MarkedBlock::Handle::removeFromAllocator):
(JSC::MarkedBlock::Handle::didAddToAllocator):
(JSC::MarkedBlock::Handle::didRemoveFromAllocator):
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessarySlow): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarkingSlow): Deleted.
(JSC::MarkedBlock::Handle::willRemoveBlock): Deleted.
(WTF::printInternal): Deleted.
* heap/MarkedBlock.h:
(JSC::MarkedBlock::Handle::isFreeListed):
(JSC::MarkedBlock::Handle::index):
(JSC::MarkedBlock::aboutToMark):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::isMarkedConcurrently):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::isOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::state): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipForEdenCollection): Deleted.
(JSC::MarkedBlock::isMarkedDuringWeakVisiting): Deleted.
(JSC::MarkedBlock::Handle::isLive): Deleted.
(JSC::MarkedBlock::Handle::isLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachDeadCell): Deleted.
(JSC::MarkedBlock::Handle::needsSweeping): Deleted.
(JSC::MarkedBlock::Handle::isAllocated): Deleted.
(JSC::MarkedBlock::Handle::isMarked): Deleted.
* heap/MarkedBlockInlines.h: Added.
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::forEachLiveCell):
(JSC::MarkedBlock::Handle::forEachDeadCell):
(JSC::MarkedBlock::resetVersion):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::MarkedSpace):
(JSC::MarkedSpace::allocate):
(JSC::MarkedSpace::tryAllocate):
(JSC::MarkedSpace::sweep):
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::shrink):
(JSC::MarkedSpace::clearNewlyAllocated):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::endMarking):
(JSC::MarkedSpace::didAllocateInBlock):
(JSC::MarkedSpace::findEmptyBlock):
(JSC::MarkedSpace::snapshot):
(JSC::MarkedSpace::assertSnapshotEmpty):
(JSC::MarkedSpace::dumpBits):
(JSC::MarkedSpace::zombifySweep): Deleted.
(JSC::MarkedSpace::resetAllocators): Deleted.
(JSC::VerifyMarked::operator()): Deleted.
(JSC::MarkedSpace::flip): Deleted.
* heap/MarkedSpace.h:
(JSC::MarkedSpace::nextVersion):
(JSC::MarkedSpace::firstAllocator):
(JSC::MarkedSpace::allocatorForEmptyAllocation):
(JSC::MarkedSpace::forEachAllocator):
(JSC::MarkedSpace::blocksWithNewObjects): Deleted.
(JSC::MarkedSpace::setIsMarking): Deleted.
(JSC::MarkedSpace::forEachLiveCell): Deleted.
(JSC::MarkedSpace::forEachDeadCell): Deleted.
* heap/MarkedSpaceInlines.h: Added.
(JSC::MarkedSpace::forEachLiveCell):
(JSC::MarkedSpace::forEachDeadCell):
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::setMarkedAndAppendToMarkStack):
(JSC::SlotVisitor::markAuxiliary):
(JSC::SlotVisitor::visitChildren):
* heap/Weak.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue):
(WTF::HashTraits<JSC::Weak<T>>::peek):
* heap/WeakBlock.cpp:
(JSC::WeakBlock::specializedVisit):
(JSC::WeakBlock::reap):
* heap/WeakInlines.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue): Deleted.
(WTF::HashTraits<JSC::Weak<T>>::peek): Deleted.
* jit/JITThunks.h:
* runtime/JSGlobalObject.cpp:
* runtime/PrototypeMap.h:
* runtime/SamplingProfiler.cpp:
* runtime/WeakGCMap.h:
* tools/JSDollarVMPrototype.cpp:
Source/WTF:
The main change here is to bring back FastBitVector.cpp, so that I could outline some
large slow path functions. This also adds some utilities, like atomicSetAndCheck() and
isEmpty(). The GC uses these.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/FastBitVector.cpp: Added.
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorImpl::isEmpty):
(WTF::FastBitVector::atomicSetAndCheck):
(WTF::FastBitVector::operator[]): Deleted.
Tools:
Remove the always-trigger-copy-phase configuration.
* Scripts/run-jsc-stress-tests:
Canonical link: https://commits.webkit.org/180306@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206154 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-20 18:12:18 +00:00
|
|
|
WTF_EXPORT_PRIVATE void setEqualsSlow(const FastBitVectorWordOwner& other);
|
|
|
|
WTF_EXPORT_PRIVATE void resizeSlow(size_t numBits);
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
uint32_t* m_words { nullptr };
|
|
|
|
size_t m_numBits { 0 };
|
|
|
|
};
|
|
|
|
|
|
|
|
template<typename Left, typename Right>
|
|
|
|
class FastBitVectorAndWords {
|
2019-08-12 20:57:15 +00:00
|
|
|
WTF_MAKE_FAST_ALLOCATED;
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
public:
|
|
|
|
typedef FastBitVectorAndWords ViewType;
|
|
|
|
|
|
|
|
FastBitVectorAndWords(const Left& left, const Right& right)
|
|
|
|
: m_left(left)
|
|
|
|
, m_right(right)
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(m_left.numBits() == m_right.numBits());
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
FastBitVectorAndWords view() const { return *this; }
|
|
|
|
|
|
|
|
size_t numBits() const
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
return m_left.numBits();
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
uint32_t word(size_t index) const
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
return m_left.word(index) & m_right.word(index);
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
private:
|
|
|
|
Left m_left;
|
|
|
|
Right m_right;
|
|
|
|
};
|
|
|
|
|
|
|
|
template<typename Left, typename Right>
|
|
|
|
class FastBitVectorOrWords {
|
2019-08-12 20:57:15 +00:00
|
|
|
WTF_MAKE_FAST_ALLOCATED;
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
public:
|
|
|
|
typedef FastBitVectorOrWords ViewType;
|
|
|
|
|
|
|
|
FastBitVectorOrWords(const Left& left, const Right& right)
|
|
|
|
: m_left(left)
|
|
|
|
, m_right(right)
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(m_left.numBits() == m_right.numBits());
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
FastBitVectorOrWords view() const { return *this; }
|
|
|
|
|
|
|
|
size_t numBits() const
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
return m_left.numBits();
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
uint32_t word(size_t index) const
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
return m_left.word(index) | m_right.word(index);
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
private:
|
|
|
|
Left m_left;
|
|
|
|
Right m_right;
|
|
|
|
};
|
|
|
|
|
|
|
|
template<typename View>
|
|
|
|
class FastBitVectorNotWords {
|
2019-08-12 20:57:15 +00:00
|
|
|
WTF_MAKE_FAST_ALLOCATED;
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
public:
|
|
|
|
typedef FastBitVectorNotWords ViewType;
|
|
|
|
|
|
|
|
FastBitVectorNotWords(const View& view)
|
|
|
|
: m_view(view)
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
FastBitVectorNotWords view() const { return *this; }
|
|
|
|
|
|
|
|
size_t numBits() const
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
return m_view.numBits();
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
uint32_t word(size_t index) const
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
return ~m_view.word(index);
|
|
|
|
}
|
|
|
|
|
|
|
|
private:
|
|
|
|
View m_view;
|
|
|
|
};
|
2016-09-12 16:08:52 +00:00
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
class FastBitVector;
|
|
|
|
|
|
|
|
template<typename Words>
|
|
|
|
class FastBitVectorImpl {
|
2019-08-12 20:57:15 +00:00
|
|
|
WTF_MAKE_FAST_ALLOCATED;
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
public:
|
|
|
|
FastBitVectorImpl()
|
|
|
|
: m_words()
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
FastBitVectorImpl(const Words& words)
|
|
|
|
: m_words(words)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
FastBitVectorImpl(Words&& words)
|
|
|
|
: m_words(WTFMove(words))
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
size_t numBits() const { return m_words.numBits(); }
|
|
|
|
size_t size() const { return numBits(); }
|
|
|
|
|
|
|
|
size_t arrayLength() const { return fastBitVectorArrayLength(numBits()); }
|
|
|
|
|
|
|
|
template<typename Other>
|
|
|
|
bool operator==(const Other& other) const
|
|
|
|
{
|
|
|
|
if (numBits() != other.numBits())
|
|
|
|
return false;
|
|
|
|
for (size_t i = arrayLength(); i--;) {
|
|
|
|
if (m_words.word(i) != other.m_words.word(i))
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
return true;
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
template<typename Other>
|
|
|
|
bool operator!=(const Other& other) const
|
2012-05-22 00:51:05 +00:00
|
|
|
{
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
return !(*this == other);
|
|
|
|
}
|
|
|
|
|
|
|
|
bool at(size_t index) const
|
|
|
|
{
|
|
|
|
return atImpl(index);
|
|
|
|
}
|
|
|
|
|
|
|
|
bool operator[](size_t index) const
|
|
|
|
{
|
|
|
|
return atImpl(index);
|
2012-05-22 00:51:05 +00:00
|
|
|
}
|
2013-11-27 23:15:48 +00:00
|
|
|
|
|
|
|
size_t bitCount() const
|
|
|
|
{
|
|
|
|
size_t result = 0;
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
for (size_t index = arrayLength(); index--;)
|
|
|
|
result += WTF::bitCount(m_words.word(index));
|
2013-11-27 23:15:48 +00:00
|
|
|
return result;
|
|
|
|
}
|
|
|
|
|
Make MarkedBlock state tracking support overlapped allocation and marking state
https://bugs.webkit.org/show_bug.cgi?id=161581
Reviewed by Geoffrey Garen.
JSTests:
Add a microbenchmark for why we want to reclaim empty blocks from other allocators.
* microbenchmarks/switching-size-classes.js: Added.
Source/JavaScriptCore:
Concurrent GCs must allow for mutation and allocation during collection. We already know
how to mutate during collection. We have a write barrier for that. Allocation during
collection is more involved: the collector modifies the the mark bits, as well as other
kinds of MarkedBlock state, in-place during a collection. The allocator uses that same
MarkedBlock state to decide which regions of memory are free. This works if the allocator
never runs while the collector is running, but if we want to allow them to run at the same
time, then we need to have two versions of the state: one version built up by the
collector and another consumed by the allocator. We clear the collector state at the
beginning of collection, and splat the collector state onto the allocator state after
collection.
This could be super expensive, but we can make it cheap with some cleverness. The biggest
observation is just that most of the state is a handful of bits per block: is the block
free-listed? is it completely full? completely empty? in the incremental sweeper's
snapshot? is it retired? is it in eden? There is also state inside blocks, like the mark
bits, but I have a solid plan there and I'll save it for another patch. Once we view the
state of blocks as bits, we can put that state into bitvectors, so that if the collector
needs to transform the state of some blocks, it can do it with a single operation over
bitvectors. I like to think of this as 32-way parallelizing block operations, since
doing one operation on a 32-bit word in one of those bitvectors instantly affects 32
blocks.
This change converts all previous collections of MarkedBlocks, along with the MarkedBlock
state, into 8 bitvectors (live, empty, allocated, canAllocateButNotEmpty, eden, unswept,
markingNotEmpty, and markingRetired). The bitvectors separate allocator state (empty,
allocated, canAllocateButNotEmpty) from marking state (markingNotEmpty, markingRetired).
As a nice side-effect of switching to bitvectors, we get size class rebalancing for free.
It used to be that if a MarkedAllocator had an empty block, we would only allow that
memory to be reused by a different MarkedAllocator if we did an incremental sweep or a
full eager sweep. Now we hunt down all destructorless empty blocks before allocating new
MarkedBlocks. It would be relatively easy to also hunt down destructor empty blocks, but
the theory is that those might be expensive to sweep, so it might still be better to leave
those to the incremental sweeper.
This change is perf-neutral all around. I did some tests with two different kinds of
allocation strategies - something that is somewhat easier to do now that you can look for
blocks that are candidates for allocation by just scanning some bitvectors. I tried two
variants:
- Allocate out of non-empty blocks first, leaving empty blocks for last in case a
different allocator needed them. This is sort of a best-fit strategy. I tried this
first, and it can be expressed as:
m_allocationCursor = m_canAllocateButNotEmpty.findBit(m_allocationCursor, true)
- Allocate out of lower-indexed blocks first, treating empty and canAllocateButNotEmpty
blocks equally. This is sort of a first-fit strategy. This is what I ended up settling
on, and it can be expressed as:
m_allocationCursor = (m_canAllocateButNotEmpty | m_empty).findBit(m_allocationCursor, true)
The best-fit strategy meant 1% regressions in LongSpider and Octane overall, and a 11%
regression on Octane/earley. First-fit means perf-neutrality. Most great allocators skew
towards first-fit because it's empirically better, so this result is not surprising.
Overall, the performance of this patch on my machine is as follows, where "neutral" means
less than 1% and not statistically significant.
run-jsc-benchmarks:
SunSpider: neutral
LongSpider: 0.6% slower
V8Spider: neutral
Octane: neutral
Kraken: neutral
Microbenchmarks: 0.37% slower
AsmBench: neutral
CompressionBench: maybe 1% faster
For browser benchmarks, I report the ratio of means (bigger / smaller) along with a T-test
from Mathematica reported as % chance of not [sic] the null hypothesis. Note that we
normally consider anything less than 95% confidence to be inconclusive.
Browser benchmarks:
PLT3: 0.3% faster with 67% confidence
membuster:
Snap2FinishedLoadingPost: 0.68% more memory with 50% confidence
Snap3EndPost: 2.4% more memory with 61% confidence
JetStream: 0.2% slower with 32% confidence
Speedometer: 0.7% faster with 82% confidence
Additionally, Octane/splay's heap capacity goes down to ~180KB from ~200KB, so about a 10%
progression. This is due to the allocator rebalancing feature.
Finally, this breaks --useImmortalObjects. It was already broken as far as I can tell. I
filed a bug to reimplement it (bug 162296). Unless someone urgently needs this internal
tool, it's probably best to reimplement it after I'm done refactoring MarkedSpace.
* JavaScriptCore.xcodeproj/project.pbxproj:
* debugger/Debugger.cpp:
* heap/CellContainer.h:
* heap/CellContainerInlines.h:
(JSC::CellContainer::vm):
(JSC::CellContainer::heap):
(JSC::CellContainer::isMarkedOrNewlyAllocated):
(JSC::CellContainer::aboutToMark):
(JSC::CellContainer::isMarked): Deleted.
(JSC::CellContainer::flipIfNecessary): Deleted.
* heap/ConservativeRoots.cpp:
* heap/Heap.cpp:
(JSC::Heap::beginMarking):
(JSC::Heap::endMarking):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collectImpl):
(JSC::Heap::snapshotMarkedSpace):
(JSC::Heap::prepareForAllocation):
(JSC::Heap::zombifyDeadObjects):
(JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): Deleted.
(JSC::MarkedBlockSnapshotFunctor::operator()): Deleted.
(JSC::Heap::resetAllocators): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isMarked):
(JSC::Heap::isMarkedConcurrently):
(JSC::Heap::testAndSetMarked):
* heap/HeapStatistics.cpp:
* heap/HeapUtil.h:
(JSC::HeapUtil::findGCObjectPointersForMarking):
(JSC::HeapUtil::isPointerGCObjectJSCell):
* heap/HeapVerifier.cpp:
* heap/IncrementalSweeper.cpp:
(JSC::IncrementalSweeper::IncrementalSweeper):
(JSC::IncrementalSweeper::doSweep):
(JSC::IncrementalSweeper::sweepNextBlock):
(JSC::IncrementalSweeper::startSweeping):
(JSC::IncrementalSweeper::willFinishSweeping):
* heap/IncrementalSweeper.h:
* heap/LargeAllocation.h:
(JSC::LargeAllocation::isMarked):
(JSC::LargeAllocation::isMarkedConcurrently):
(JSC::LargeAllocation::isMarkedOrNewlyAllocated):
(JSC::LargeAllocation::aboutToMark):
(JSC::LargeAllocation::isMarkedDuringWeakVisiting): Deleted.
(JSC::LargeAllocation::flipIfNecessary): Deleted.
(JSC::LargeAllocation::flipIfNecessaryDuringMarking): Deleted.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::isPagedOut):
(JSC::MarkedAllocator::findEmptyBlock):
(JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
(JSC::MarkedAllocator::allocateIn):
(JSC::MarkedAllocator::tryAllocateIn):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
(JSC::MarkedAllocator::tryAllocateBlock):
(JSC::MarkedAllocator::addBlock):
(JSC::MarkedAllocator::removeBlock):
(JSC::MarkedAllocator::stopAllocating):
(JSC::MarkedAllocator::prepareForAllocation):
(JSC::MarkedAllocator::lastChanceToFinalize):
(JSC::MarkedAllocator::resumeAllocating):
(JSC::MarkedAllocator::beginMarkingForFullCollection):
(JSC::MarkedAllocator::endMarking):
(JSC::MarkedAllocator::snapshotForEdenCollection):
(JSC::MarkedAllocator::snapshotForFullCollection):
(JSC::MarkedAllocator::findBlockToSweep):
(JSC::MarkedAllocator::sweep):
(JSC::MarkedAllocator::shrink):
(JSC::MarkedAllocator::assertSnapshotEmpty):
(JSC::MarkedAllocator::dump):
(JSC::MarkedAllocator::dumpBits):
(JSC::MarkedAllocator::retire): Deleted.
(JSC::MarkedAllocator::filterNextBlock): Deleted.
(JSC::MarkedAllocator::setNextBlockToSweep): Deleted.
(JSC::MarkedAllocator::reset): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::forEachBitVector):
(JSC::MarkedAllocator::forEachBitVectorWithName):
(JSC::MarkedAllocator::nextAllocator):
(JSC::MarkedAllocator::setNextAllocator):
(JSC::MarkedAllocator::forEachBlock):
(JSC::MarkedAllocator::resumeAllocating): Deleted.
* heap/MarkedBlock.cpp:
(JSC::MarkedBlock::tryCreate):
(JSC::MarkedBlock::Handle::Handle):
(JSC::MarkedBlock::Handle::~Handle):
(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::Handle::specializedSweep):
(JSC::MarkedBlock::Handle::sweep):
(JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectEmptyMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectHasNewlyAllocated):
(JSC::MarkedBlock::Handle::sweepHelperSelectSweepMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectFlipMode):
(JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated):
(JSC::MarkedBlock::Handle::setIsFreeListed):
(JSC::MarkedBlock::Handle::stopAllocating):
(JSC::MarkedBlock::Handle::lastChanceToFinalize):
(JSC::MarkedBlock::Handle::resumeAllocating):
(JSC::MarkedBlock::aboutToMarkSlow):
(JSC::MarkedBlock::clearMarks):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::didConsumeFreeList):
(JSC::MarkedBlock::markCount):
(JSC::MarkedBlock::Handle::isEmpty):
(JSC::MarkedBlock::noteMarkedSlow):
(JSC::MarkedBlock::Handle::removeFromAllocator):
(JSC::MarkedBlock::Handle::didAddToAllocator):
(JSC::MarkedBlock::Handle::didRemoveFromAllocator):
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessarySlow): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarkingSlow): Deleted.
(JSC::MarkedBlock::Handle::willRemoveBlock): Deleted.
(WTF::printInternal): Deleted.
* heap/MarkedBlock.h:
(JSC::MarkedBlock::Handle::isFreeListed):
(JSC::MarkedBlock::Handle::index):
(JSC::MarkedBlock::aboutToMark):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::isMarkedConcurrently):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::isOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::state): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipForEdenCollection): Deleted.
(JSC::MarkedBlock::isMarkedDuringWeakVisiting): Deleted.
(JSC::MarkedBlock::Handle::isLive): Deleted.
(JSC::MarkedBlock::Handle::isLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachDeadCell): Deleted.
(JSC::MarkedBlock::Handle::needsSweeping): Deleted.
(JSC::MarkedBlock::Handle::isAllocated): Deleted.
(JSC::MarkedBlock::Handle::isMarked): Deleted.
* heap/MarkedBlockInlines.h: Added.
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::forEachLiveCell):
(JSC::MarkedBlock::Handle::forEachDeadCell):
(JSC::MarkedBlock::resetVersion):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::MarkedSpace):
(JSC::MarkedSpace::allocate):
(JSC::MarkedSpace::tryAllocate):
(JSC::MarkedSpace::sweep):
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::shrink):
(JSC::MarkedSpace::clearNewlyAllocated):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::endMarking):
(JSC::MarkedSpace::didAllocateInBlock):
(JSC::MarkedSpace::findEmptyBlock):
(JSC::MarkedSpace::snapshot):
(JSC::MarkedSpace::assertSnapshotEmpty):
(JSC::MarkedSpace::dumpBits):
(JSC::MarkedSpace::zombifySweep): Deleted.
(JSC::MarkedSpace::resetAllocators): Deleted.
(JSC::VerifyMarked::operator()): Deleted.
(JSC::MarkedSpace::flip): Deleted.
* heap/MarkedSpace.h:
(JSC::MarkedSpace::nextVersion):
(JSC::MarkedSpace::firstAllocator):
(JSC::MarkedSpace::allocatorForEmptyAllocation):
(JSC::MarkedSpace::forEachAllocator):
(JSC::MarkedSpace::blocksWithNewObjects): Deleted.
(JSC::MarkedSpace::setIsMarking): Deleted.
(JSC::MarkedSpace::forEachLiveCell): Deleted.
(JSC::MarkedSpace::forEachDeadCell): Deleted.
* heap/MarkedSpaceInlines.h: Added.
(JSC::MarkedSpace::forEachLiveCell):
(JSC::MarkedSpace::forEachDeadCell):
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::setMarkedAndAppendToMarkStack):
(JSC::SlotVisitor::markAuxiliary):
(JSC::SlotVisitor::visitChildren):
* heap/Weak.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue):
(WTF::HashTraits<JSC::Weak<T>>::peek):
* heap/WeakBlock.cpp:
(JSC::WeakBlock::specializedVisit):
(JSC::WeakBlock::reap):
* heap/WeakInlines.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue): Deleted.
(WTF::HashTraits<JSC::Weak<T>>::peek): Deleted.
* jit/JITThunks.h:
* runtime/JSGlobalObject.cpp:
* runtime/PrototypeMap.h:
* runtime/SamplingProfiler.cpp:
* runtime/WeakGCMap.h:
* tools/JSDollarVMPrototype.cpp:
Source/WTF:
The main change here is to bring back FastBitVector.cpp, so that I could outline some
large slow path functions. This also adds some utilities, like atomicSetAndCheck() and
isEmpty(). The GC uses these.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/FastBitVector.cpp: Added.
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorImpl::isEmpty):
(WTF::FastBitVector::atomicSetAndCheck):
(WTF::FastBitVector::operator[]): Deleted.
Tools:
Remove the always-trigger-copy-phase configuration.
* Scripts/run-jsc-stress-tests:
Canonical link: https://commits.webkit.org/180306@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206154 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-20 18:12:18 +00:00
|
|
|
bool isEmpty() const
|
|
|
|
{
|
|
|
|
for (size_t index = arrayLength(); index--;) {
|
|
|
|
if (m_words.word(index))
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
template<typename OtherWords>
|
|
|
|
FastBitVectorImpl<FastBitVectorAndWords<typename Words::ViewType, typename OtherWords::ViewType>> operator&(const FastBitVectorImpl<OtherWords>& other) const
|
2015-03-13 01:57:59 +00:00
|
|
|
{
|
2016-09-12 16:08:52 +00:00
|
|
|
return FastBitVectorImpl<FastBitVectorAndWords<typename Words::ViewType, typename OtherWords::ViewType>>(FastBitVectorAndWords<typename Words::ViewType, typename OtherWords::ViewType>(wordView(), other.wordView()));
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
template<typename OtherWords>
|
|
|
|
FastBitVectorImpl<FastBitVectorOrWords<typename Words::ViewType, typename OtherWords::ViewType>> operator|(const FastBitVectorImpl<OtherWords>& other) const
|
|
|
|
{
|
2016-09-12 16:08:52 +00:00
|
|
|
return FastBitVectorImpl<FastBitVectorOrWords<typename Words::ViewType, typename OtherWords::ViewType>>(FastBitVectorOrWords<typename Words::ViewType, typename OtherWords::ViewType>(wordView(), other.wordView()));
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
FastBitVectorImpl<FastBitVectorNotWords<typename Words::ViewType>> operator~() const
|
|
|
|
{
|
2016-09-12 16:08:52 +00:00
|
|
|
return FastBitVectorImpl<FastBitVectorNotWords<typename Words::ViewType>>(FastBitVectorNotWords<typename Words::ViewType>(wordView()));
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
template<typename Func>
|
|
|
|
ALWAYS_INLINE void forEachSetBit(const Func& func) const
|
|
|
|
{
|
2016-09-12 16:08:52 +00:00
|
|
|
size_t n = arrayLength();
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
for (size_t i = 0; i < n; ++i) {
|
|
|
|
uint32_t word = m_words.word(i);
|
|
|
|
size_t j = i * 32;
|
2015-03-13 01:57:59 +00:00
|
|
|
while (word) {
|
|
|
|
if (word & 1)
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
func(j);
|
2015-03-13 01:57:59 +00:00
|
|
|
word >>= 1;
|
|
|
|
j++;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
|
|
|
|
template<typename Func>
|
|
|
|
ALWAYS_INLINE void forEachClearBit(const Func& func) const
|
|
|
|
{
|
|
|
|
(~*this).forEachSetBit(func);
|
|
|
|
}
|
|
|
|
|
|
|
|
template<typename Func>
|
|
|
|
void forEachBit(bool value, const Func& func) const
|
|
|
|
{
|
|
|
|
if (value)
|
|
|
|
forEachSetBit(func);
|
|
|
|
else
|
|
|
|
forEachClearBit(func);
|
|
|
|
}
|
|
|
|
|
|
|
|
// Starts looking for bits at the index you pass. If that index contains the value you want,
|
|
|
|
// then it will return that index. Returns numBits when we get to the end. For example, you
|
|
|
|
// can write a loop to iterate over all set bits like this:
|
|
|
|
//
|
|
|
|
// for (size_t i = 0; i < bits.numBits(); i = bits.findBit(i + 1, true))
|
|
|
|
// ...
|
|
|
|
ALWAYS_INLINE size_t findBit(size_t startIndex, bool value) const
|
|
|
|
{
|
|
|
|
// If value is true, this produces 0. If value is false, this produces UINT_MAX. It's
|
|
|
|
// written this way so that it performs well regardless of whether value is a constant.
|
|
|
|
uint32_t skipValue = -(static_cast<uint32_t>(value) ^ 1);
|
|
|
|
|
Make MarkedBlock state tracking support overlapped allocation and marking state
https://bugs.webkit.org/show_bug.cgi?id=161581
Reviewed by Geoffrey Garen.
JSTests:
Add a microbenchmark for why we want to reclaim empty blocks from other allocators.
* microbenchmarks/switching-size-classes.js: Added.
Source/JavaScriptCore:
Concurrent GCs must allow for mutation and allocation during collection. We already know
how to mutate during collection. We have a write barrier for that. Allocation during
collection is more involved: the collector modifies the the mark bits, as well as other
kinds of MarkedBlock state, in-place during a collection. The allocator uses that same
MarkedBlock state to decide which regions of memory are free. This works if the allocator
never runs while the collector is running, but if we want to allow them to run at the same
time, then we need to have two versions of the state: one version built up by the
collector and another consumed by the allocator. We clear the collector state at the
beginning of collection, and splat the collector state onto the allocator state after
collection.
This could be super expensive, but we can make it cheap with some cleverness. The biggest
observation is just that most of the state is a handful of bits per block: is the block
free-listed? is it completely full? completely empty? in the incremental sweeper's
snapshot? is it retired? is it in eden? There is also state inside blocks, like the mark
bits, but I have a solid plan there and I'll save it for another patch. Once we view the
state of blocks as bits, we can put that state into bitvectors, so that if the collector
needs to transform the state of some blocks, it can do it with a single operation over
bitvectors. I like to think of this as 32-way parallelizing block operations, since
doing one operation on a 32-bit word in one of those bitvectors instantly affects 32
blocks.
This change converts all previous collections of MarkedBlocks, along with the MarkedBlock
state, into 8 bitvectors (live, empty, allocated, canAllocateButNotEmpty, eden, unswept,
markingNotEmpty, and markingRetired). The bitvectors separate allocator state (empty,
allocated, canAllocateButNotEmpty) from marking state (markingNotEmpty, markingRetired).
As a nice side-effect of switching to bitvectors, we get size class rebalancing for free.
It used to be that if a MarkedAllocator had an empty block, we would only allow that
memory to be reused by a different MarkedAllocator if we did an incremental sweep or a
full eager sweep. Now we hunt down all destructorless empty blocks before allocating new
MarkedBlocks. It would be relatively easy to also hunt down destructor empty blocks, but
the theory is that those might be expensive to sweep, so it might still be better to leave
those to the incremental sweeper.
This change is perf-neutral all around. I did some tests with two different kinds of
allocation strategies - something that is somewhat easier to do now that you can look for
blocks that are candidates for allocation by just scanning some bitvectors. I tried two
variants:
- Allocate out of non-empty blocks first, leaving empty blocks for last in case a
different allocator needed them. This is sort of a best-fit strategy. I tried this
first, and it can be expressed as:
m_allocationCursor = m_canAllocateButNotEmpty.findBit(m_allocationCursor, true)
- Allocate out of lower-indexed blocks first, treating empty and canAllocateButNotEmpty
blocks equally. This is sort of a first-fit strategy. This is what I ended up settling
on, and it can be expressed as:
m_allocationCursor = (m_canAllocateButNotEmpty | m_empty).findBit(m_allocationCursor, true)
The best-fit strategy meant 1% regressions in LongSpider and Octane overall, and a 11%
regression on Octane/earley. First-fit means perf-neutrality. Most great allocators skew
towards first-fit because it's empirically better, so this result is not surprising.
Overall, the performance of this patch on my machine is as follows, where "neutral" means
less than 1% and not statistically significant.
run-jsc-benchmarks:
SunSpider: neutral
LongSpider: 0.6% slower
V8Spider: neutral
Octane: neutral
Kraken: neutral
Microbenchmarks: 0.37% slower
AsmBench: neutral
CompressionBench: maybe 1% faster
For browser benchmarks, I report the ratio of means (bigger / smaller) along with a T-test
from Mathematica reported as % chance of not [sic] the null hypothesis. Note that we
normally consider anything less than 95% confidence to be inconclusive.
Browser benchmarks:
PLT3: 0.3% faster with 67% confidence
membuster:
Snap2FinishedLoadingPost: 0.68% more memory with 50% confidence
Snap3EndPost: 2.4% more memory with 61% confidence
JetStream: 0.2% slower with 32% confidence
Speedometer: 0.7% faster with 82% confidence
Additionally, Octane/splay's heap capacity goes down to ~180KB from ~200KB, so about a 10%
progression. This is due to the allocator rebalancing feature.
Finally, this breaks --useImmortalObjects. It was already broken as far as I can tell. I
filed a bug to reimplement it (bug 162296). Unless someone urgently needs this internal
tool, it's probably best to reimplement it after I'm done refactoring MarkedSpace.
* JavaScriptCore.xcodeproj/project.pbxproj:
* debugger/Debugger.cpp:
* heap/CellContainer.h:
* heap/CellContainerInlines.h:
(JSC::CellContainer::vm):
(JSC::CellContainer::heap):
(JSC::CellContainer::isMarkedOrNewlyAllocated):
(JSC::CellContainer::aboutToMark):
(JSC::CellContainer::isMarked): Deleted.
(JSC::CellContainer::flipIfNecessary): Deleted.
* heap/ConservativeRoots.cpp:
* heap/Heap.cpp:
(JSC::Heap::beginMarking):
(JSC::Heap::endMarking):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collectImpl):
(JSC::Heap::snapshotMarkedSpace):
(JSC::Heap::prepareForAllocation):
(JSC::Heap::zombifyDeadObjects):
(JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): Deleted.
(JSC::MarkedBlockSnapshotFunctor::operator()): Deleted.
(JSC::Heap::resetAllocators): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isMarked):
(JSC::Heap::isMarkedConcurrently):
(JSC::Heap::testAndSetMarked):
* heap/HeapStatistics.cpp:
* heap/HeapUtil.h:
(JSC::HeapUtil::findGCObjectPointersForMarking):
(JSC::HeapUtil::isPointerGCObjectJSCell):
* heap/HeapVerifier.cpp:
* heap/IncrementalSweeper.cpp:
(JSC::IncrementalSweeper::IncrementalSweeper):
(JSC::IncrementalSweeper::doSweep):
(JSC::IncrementalSweeper::sweepNextBlock):
(JSC::IncrementalSweeper::startSweeping):
(JSC::IncrementalSweeper::willFinishSweeping):
* heap/IncrementalSweeper.h:
* heap/LargeAllocation.h:
(JSC::LargeAllocation::isMarked):
(JSC::LargeAllocation::isMarkedConcurrently):
(JSC::LargeAllocation::isMarkedOrNewlyAllocated):
(JSC::LargeAllocation::aboutToMark):
(JSC::LargeAllocation::isMarkedDuringWeakVisiting): Deleted.
(JSC::LargeAllocation::flipIfNecessary): Deleted.
(JSC::LargeAllocation::flipIfNecessaryDuringMarking): Deleted.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::isPagedOut):
(JSC::MarkedAllocator::findEmptyBlock):
(JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
(JSC::MarkedAllocator::allocateIn):
(JSC::MarkedAllocator::tryAllocateIn):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
(JSC::MarkedAllocator::tryAllocateBlock):
(JSC::MarkedAllocator::addBlock):
(JSC::MarkedAllocator::removeBlock):
(JSC::MarkedAllocator::stopAllocating):
(JSC::MarkedAllocator::prepareForAllocation):
(JSC::MarkedAllocator::lastChanceToFinalize):
(JSC::MarkedAllocator::resumeAllocating):
(JSC::MarkedAllocator::beginMarkingForFullCollection):
(JSC::MarkedAllocator::endMarking):
(JSC::MarkedAllocator::snapshotForEdenCollection):
(JSC::MarkedAllocator::snapshotForFullCollection):
(JSC::MarkedAllocator::findBlockToSweep):
(JSC::MarkedAllocator::sweep):
(JSC::MarkedAllocator::shrink):
(JSC::MarkedAllocator::assertSnapshotEmpty):
(JSC::MarkedAllocator::dump):
(JSC::MarkedAllocator::dumpBits):
(JSC::MarkedAllocator::retire): Deleted.
(JSC::MarkedAllocator::filterNextBlock): Deleted.
(JSC::MarkedAllocator::setNextBlockToSweep): Deleted.
(JSC::MarkedAllocator::reset): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::forEachBitVector):
(JSC::MarkedAllocator::forEachBitVectorWithName):
(JSC::MarkedAllocator::nextAllocator):
(JSC::MarkedAllocator::setNextAllocator):
(JSC::MarkedAllocator::forEachBlock):
(JSC::MarkedAllocator::resumeAllocating): Deleted.
* heap/MarkedBlock.cpp:
(JSC::MarkedBlock::tryCreate):
(JSC::MarkedBlock::Handle::Handle):
(JSC::MarkedBlock::Handle::~Handle):
(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::Handle::specializedSweep):
(JSC::MarkedBlock::Handle::sweep):
(JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectEmptyMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectHasNewlyAllocated):
(JSC::MarkedBlock::Handle::sweepHelperSelectSweepMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectFlipMode):
(JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated):
(JSC::MarkedBlock::Handle::setIsFreeListed):
(JSC::MarkedBlock::Handle::stopAllocating):
(JSC::MarkedBlock::Handle::lastChanceToFinalize):
(JSC::MarkedBlock::Handle::resumeAllocating):
(JSC::MarkedBlock::aboutToMarkSlow):
(JSC::MarkedBlock::clearMarks):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::didConsumeFreeList):
(JSC::MarkedBlock::markCount):
(JSC::MarkedBlock::Handle::isEmpty):
(JSC::MarkedBlock::noteMarkedSlow):
(JSC::MarkedBlock::Handle::removeFromAllocator):
(JSC::MarkedBlock::Handle::didAddToAllocator):
(JSC::MarkedBlock::Handle::didRemoveFromAllocator):
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessarySlow): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarkingSlow): Deleted.
(JSC::MarkedBlock::Handle::willRemoveBlock): Deleted.
(WTF::printInternal): Deleted.
* heap/MarkedBlock.h:
(JSC::MarkedBlock::Handle::isFreeListed):
(JSC::MarkedBlock::Handle::index):
(JSC::MarkedBlock::aboutToMark):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::isMarkedConcurrently):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::isOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::state): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipForEdenCollection): Deleted.
(JSC::MarkedBlock::isMarkedDuringWeakVisiting): Deleted.
(JSC::MarkedBlock::Handle::isLive): Deleted.
(JSC::MarkedBlock::Handle::isLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachDeadCell): Deleted.
(JSC::MarkedBlock::Handle::needsSweeping): Deleted.
(JSC::MarkedBlock::Handle::isAllocated): Deleted.
(JSC::MarkedBlock::Handle::isMarked): Deleted.
* heap/MarkedBlockInlines.h: Added.
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::forEachLiveCell):
(JSC::MarkedBlock::Handle::forEachDeadCell):
(JSC::MarkedBlock::resetVersion):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::MarkedSpace):
(JSC::MarkedSpace::allocate):
(JSC::MarkedSpace::tryAllocate):
(JSC::MarkedSpace::sweep):
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::shrink):
(JSC::MarkedSpace::clearNewlyAllocated):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::endMarking):
(JSC::MarkedSpace::didAllocateInBlock):
(JSC::MarkedSpace::findEmptyBlock):
(JSC::MarkedSpace::snapshot):
(JSC::MarkedSpace::assertSnapshotEmpty):
(JSC::MarkedSpace::dumpBits):
(JSC::MarkedSpace::zombifySweep): Deleted.
(JSC::MarkedSpace::resetAllocators): Deleted.
(JSC::VerifyMarked::operator()): Deleted.
(JSC::MarkedSpace::flip): Deleted.
* heap/MarkedSpace.h:
(JSC::MarkedSpace::nextVersion):
(JSC::MarkedSpace::firstAllocator):
(JSC::MarkedSpace::allocatorForEmptyAllocation):
(JSC::MarkedSpace::forEachAllocator):
(JSC::MarkedSpace::blocksWithNewObjects): Deleted.
(JSC::MarkedSpace::setIsMarking): Deleted.
(JSC::MarkedSpace::forEachLiveCell): Deleted.
(JSC::MarkedSpace::forEachDeadCell): Deleted.
* heap/MarkedSpaceInlines.h: Added.
(JSC::MarkedSpace::forEachLiveCell):
(JSC::MarkedSpace::forEachDeadCell):
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::setMarkedAndAppendToMarkStack):
(JSC::SlotVisitor::markAuxiliary):
(JSC::SlotVisitor::visitChildren):
* heap/Weak.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue):
(WTF::HashTraits<JSC::Weak<T>>::peek):
* heap/WeakBlock.cpp:
(JSC::WeakBlock::specializedVisit):
(JSC::WeakBlock::reap):
* heap/WeakInlines.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue): Deleted.
(WTF::HashTraits<JSC::Weak<T>>::peek): Deleted.
* jit/JITThunks.h:
* runtime/JSGlobalObject.cpp:
* runtime/PrototypeMap.h:
* runtime/SamplingProfiler.cpp:
* runtime/WeakGCMap.h:
* tools/JSDollarVMPrototype.cpp:
Source/WTF:
The main change here is to bring back FastBitVector.cpp, so that I could outline some
large slow path functions. This also adds some utilities, like atomicSetAndCheck() and
isEmpty(). The GC uses these.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/FastBitVector.cpp: Added.
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorImpl::isEmpty):
(WTF::FastBitVector::atomicSetAndCheck):
(WTF::FastBitVector::operator[]): Deleted.
Tools:
Remove the always-trigger-copy-phase configuration.
* Scripts/run-jsc-stress-tests:
Canonical link: https://commits.webkit.org/180306@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206154 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-20 18:12:18 +00:00
|
|
|
size_t numWords = fastBitVectorArrayLength(m_words.numBits());
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
|
|
|
|
size_t wordIndex = startIndex / 32;
|
|
|
|
size_t startIndexInWord = startIndex - wordIndex * 32;
|
|
|
|
|
|
|
|
while (wordIndex < numWords) {
|
|
|
|
uint32_t word = m_words.word(wordIndex);
|
|
|
|
if (word != skipValue) {
|
|
|
|
size_t index = startIndexInWord;
|
|
|
|
if (findBitInWord(word, index, 32, value))
|
|
|
|
return wordIndex * 32 + index;
|
|
|
|
}
|
|
|
|
|
|
|
|
wordIndex++;
|
|
|
|
startIndexInWord = 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
return numBits();
|
|
|
|
}
|
|
|
|
|
|
|
|
ALWAYS_INLINE size_t findSetBit(size_t index) const
|
|
|
|
{
|
|
|
|
return findBit(index, true);
|
|
|
|
}
|
|
|
|
|
|
|
|
ALWAYS_INLINE size_t findClearBit(size_t index) const
|
|
|
|
{
|
|
|
|
return findBit(index, false);
|
|
|
|
}
|
|
|
|
|
|
|
|
void dump(PrintStream& out) const
|
|
|
|
{
|
|
|
|
for (size_t i = 0; i < numBits(); ++i)
|
|
|
|
out.print((*this)[i] ? "1" : "-");
|
|
|
|
}
|
2013-11-27 23:15:48 +00:00
|
|
|
|
2016-09-12 16:08:52 +00:00
|
|
|
typename Words::ViewType wordView() const { return m_words.view(); }
|
2019-11-14 09:37:12 +00:00
|
|
|
|
|
|
|
Words& unsafeWords() { return m_words; }
|
|
|
|
const Words& unsafeWords() const { return m_words; }
|
2016-09-12 16:08:52 +00:00
|
|
|
|
2012-05-22 00:51:05 +00:00
|
|
|
private:
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
// You'd think that we could remove this friend if we used protected, but you'd be wrong,
|
|
|
|
// because templates.
|
|
|
|
friend class FastBitVector;
|
2012-05-22 00:51:05 +00:00
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
bool atImpl(size_t index) const
|
|
|
|
{
|
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(index < numBits());
|
|
|
|
return !!(m_words.word(index >> 5) & (1 << (index & 31)));
|
|
|
|
}
|
|
|
|
|
|
|
|
Words m_words;
|
|
|
|
};
|
|
|
|
|
2019-11-14 09:37:12 +00:00
|
|
|
class FastBitReference {
|
|
|
|
WTF_MAKE_FAST_ALLOCATED;
|
|
|
|
public:
|
|
|
|
FastBitReference() = default;
|
|
|
|
|
|
|
|
FastBitReference(uint32_t* word, uint32_t mask)
|
|
|
|
: m_word(word)
|
|
|
|
, m_mask(mask)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2020-01-17 04:09:32 +00:00
|
|
|
operator bool() const
|
2019-11-14 09:37:12 +00:00
|
|
|
{
|
|
|
|
return !!(*m_word & m_mask);
|
|
|
|
}
|
|
|
|
|
|
|
|
FastBitReference& operator=(bool value)
|
|
|
|
{
|
|
|
|
if (value)
|
|
|
|
*m_word |= m_mask;
|
|
|
|
else
|
|
|
|
*m_word &= ~m_mask;
|
|
|
|
return *this;
|
|
|
|
}
|
|
|
|
|
2020-01-17 04:09:32 +00:00
|
|
|
FastBitReference& operator|=(bool value) { return value ? *this = value : *this; }
|
|
|
|
FastBitReference& operator&=(bool value) { return value ? *this : *this = value; }
|
|
|
|
|
2019-11-14 09:37:12 +00:00
|
|
|
private:
|
|
|
|
uint32_t* m_word { nullptr };
|
|
|
|
uint32_t m_mask { 0 };
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
|
|
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
class FastBitVector : public FastBitVectorImpl<FastBitVectorWordOwner> {
|
|
|
|
public:
|
|
|
|
FastBitVector() { }
|
2021-04-06 19:47:47 +00:00
|
|
|
explicit FastBitVector(size_t numBits)
|
|
|
|
{
|
|
|
|
grow(numBits);
|
|
|
|
}
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
|
|
|
|
FastBitVector(const FastBitVector&) = default;
|
|
|
|
FastBitVector& operator=(const FastBitVector&) = default;
|
|
|
|
|
|
|
|
template<typename OtherWords>
|
|
|
|
FastBitVector(const FastBitVectorImpl<OtherWords>& other)
|
|
|
|
{
|
|
|
|
*this = other;
|
|
|
|
}
|
|
|
|
|
|
|
|
template<typename OtherWords>
|
|
|
|
FastBitVector& operator=(const FastBitVectorImpl<OtherWords>& other)
|
|
|
|
{
|
|
|
|
if (UNLIKELY(numBits() != other.numBits()))
|
|
|
|
resize(other.numBits());
|
|
|
|
|
|
|
|
for (unsigned i = arrayLength(); i--;)
|
|
|
|
m_words.word(i) = other.m_words.word(i);
|
|
|
|
return *this;
|
|
|
|
}
|
|
|
|
|
|
|
|
void resize(size_t numBits)
|
|
|
|
{
|
|
|
|
m_words.resize(numBits);
|
|
|
|
}
|
|
|
|
|
|
|
|
void setAll()
|
|
|
|
{
|
|
|
|
m_words.setAll();
|
|
|
|
}
|
|
|
|
|
|
|
|
void clearAll()
|
|
|
|
{
|
|
|
|
m_words.clearAll();
|
|
|
|
}
|
2020-01-17 04:09:32 +00:00
|
|
|
|
|
|
|
// For templating as Vector<bool>
|
|
|
|
void fill(bool value) { value ? setAll() : clearAll(); }
|
|
|
|
void grow(size_t newSize) { resize(newSize); }
|
|
|
|
|
The write barrier should be down with TSO
https://bugs.webkit.org/show_bug.cgi?id=162316
Reviewed by Geoffrey Garen.
Source/JavaScriptCore:
This makes our write barrier behave correctly when it races with the collector. The
collector wants to do this when visiting:
object->cellState = black
visit(object)
The mutator wants to do this when storing:
object->property = newValue
if (object->cellState == black)
remember(object)
Prior to this change, this didn't work right because the compiler would sometimes place
barriers before the store to the property and because the mutator did not have adequate
fences.
Prior to this change, the DFG and FTL would emit this:
if (object->cellState == black)
remember(object)
object->property = newValue
Which is wrong, because the object could start being scanned just after the cellState
check, at which point the store would be lost. We need to confirm that the state was not
black *after* the store! This change was harder than you'd expect: placing the barrier
after the store broke B3's ability to do its super crazy ninja CSE on some store-load
redundancies. Because the B3 CSE has some moves that the DFG CSE lacks, the DFG CSE's
ability to ignore barriers didn't help. I fixed this by having the FTL convey precise
heap ranges for the patchpoint corresponding to the barrier slow path. It reads the world
(because of the store-load fence) and it writes only cellState (because the B3 heap ranges
don't have any way to represent any of the GC's other state, which means that B3 does not
have to worry about aliasing with any of that).
The collector already uses a store-load fence on x86 just after setting the cellState and
before visiting the object. The mutator needs to do the same. But we cannot put a
store-load fence of any kind before store barriers, because that causes enormous slow
downs. In the worst case, Octane/richards slowed down by 90%! That's crazy! However, the
overall slow downs were small enough (0-15% on benchmark suite aggregates) that it would be
reasonable if the slow down only happened while the GC was running. Then, the concurrent GC
would lift throughput-while-collecting from 0% of peak to 85% of peak. This changes the
barrier so that it looks like this:
if (object->cellState <= heap.sneakyBlackThreshold)
slowPath(object)
Where sneakyBlackThreshold is the normal blackThreshold when we're not collecting, or a
tautoligical threshold (that makes everything look black) when we are collecting. This
turns out to not be any more expensive than the barrier in tip of tree when the GC is not
running, or a 0-15% slow-down when it is "running". (Of course we don't run the GC
concurrently yet. I still have more work to do.) The slowPath() does some extra work to
check if we are concurrently collecting; if so, it does a fence and rechecks if the object
really did need that barrier.
This also reintroduces elimination of redundant store barriers, which was lost in the last
store barrier change. We can only do it when there is no possibility of GC, exit, or
exceptions between the two store barriers. We could remove the exit/exception limitation if
we taught OSR exit how to buffer store barriers, which is an insane thing to do considering
that I've never been able to detect a win from redundant store barrier elimination. I just
want us to have it for stupidly obvious situations, like a tight sequence of stores to the
same object. This same optimization also sometimes strength-reduces the store barrier so
that it uses a constant black threshold rather than the sneaky one, thereby saving one
load.
Even with all of those optimizations, I still had problems with barrier cost. I found that one
of the benchmarks that was being hit particularly hard was JetStream/regexp-2010. Fortunately
that benchmark does most of its barriers in a tight C++ loop in RegExpMatchesArray.h. When we
know what we're doing, we can defer GC around a bunch of object initializations and then remove
all of the barriers between any of the objects allocated within the deferral. Unfortunately,
our GC deferral mechanism isn't really performant enough to make this be a worthwhile
optimization. The most efficient version of such an optimization that I could come up with was
to have a DeferralContext object that houses a boolean that is false by default, but the GC
writes true into it if it would have wanted to GC. You thread a pointer to the deferralContext
through all of your allocations. This kind of mechanism has the overhead of a zero
initialization on the stack on entry and a zero check on exit. This is probably even efficient
enough that we could start thinking about having the DFG use it, for example if we found a
bounded-time section of code with a lot of barriers and entry/exit sites that aren't totally
wacky. This optimization took this patch from 0.68% JetStream regressed to neutral, according
to my latest data.
Finally, an earlier version of this change put the store-load fence in B3 IR, so I ended up
adding FTLOutput support for it and AbstractHeapRepository magic for decorating the heaps.
I think we might as well keep that, it'll be useful.
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* assembler/MacroAssembler.h:
(JSC::MacroAssembler::branch32):
* assembler/MacroAssemblerX86_64.h:
(JSC::MacroAssemblerX86_64::branch32):
(JSC::MacroAssemblerX86_64::branch64): Deleted.
* bytecode/PolymorphicAccess.cpp:
(JSC::AccessCase::generateImpl):
* dfg/DFGAbstractHeap.h:
* dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
* dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* dfg/DFGClobbersExitState.cpp:
(JSC::DFG::clobbersExitState):
* dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC):
* dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* dfg/DFGMayExit.cpp:
* dfg/DFGNode.h:
(JSC::DFG::Node::isStoreBarrier):
* dfg/DFGNodeType.h:
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::compileInThreadImpl):
* dfg/DFGPredictionPropagationPhase.cpp:
* dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::compileStoreBarrier):
(JSC::DFG::SpeculativeJIT::storeToWriteBarrierBuffer): Deleted.
(JSC::DFG::SpeculativeJIT::writeBarrier): Deleted.
* dfg/DFGSpeculativeJIT.h:
* dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
(JSC::DFG::SpeculativeJIT::compileBaseValueStoreBarrier): Deleted.
(JSC::DFG::SpeculativeJIT::writeBarrier): Deleted.
* dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
(JSC::DFG::SpeculativeJIT::compileBaseValueStoreBarrier): Deleted.
(JSC::DFG::SpeculativeJIT::writeBarrier): Deleted.
* dfg/DFGStoreBarrierClusteringPhase.cpp: Added.
(JSC::DFG::performStoreBarrierClustering):
* dfg/DFGStoreBarrierClusteringPhase.h: Added.
* dfg/DFGStoreBarrierInsertionPhase.cpp:
* dfg/DFGStoreBarrierInsertionPhase.h:
* ftl/FTLAbstractHeap.h:
(JSC::FTL::AbsoluteAbstractHeap::at):
(JSC::FTL::AbsoluteAbstractHeap::operator[]):
* ftl/FTLAbstractHeapRepository.cpp:
(JSC::FTL::AbstractHeapRepository::decorateFenceRead):
(JSC::FTL::AbstractHeapRepository::decorateFenceWrite):
(JSC::FTL::AbstractHeapRepository::computeRangesAndDecorateInstructions):
* ftl/FTLAbstractHeapRepository.h:
* ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileStoreBarrier):
(JSC::FTL::DFG::LowerDFGToB3::storageForTransition):
(JSC::FTL::DFG::LowerDFGToB3::lazySlowPath):
(JSC::FTL::DFG::LowerDFGToB3::emitStoreBarrier):
* ftl/FTLOutput.cpp:
(JSC::FTL::Output::fence):
(JSC::FTL::Output::absolute):
* ftl/FTLOutput.h:
* heap/CellState.h:
(JSC::isWithinThreshold):
(JSC::isBlack):
* heap/Heap.cpp:
(JSC::Heap::writeBarrierSlowPath):
* heap/Heap.h:
(JSC::Heap::barrierShouldBeFenced):
(JSC::Heap::addressOfBarrierShouldBeFenced):
(JSC::Heap::sneakyBlackThreshold):
(JSC::Heap::addressOfSneakyBlackThreshold):
* heap/HeapInlines.h:
(JSC::Heap::writeBarrier):
(JSC::Heap::writeBarrierWithoutFence):
* jit/AssemblyHelpers.h:
(JSC::AssemblyHelpers::jumpIfIsRememberedOrInEdenWithoutFence):
(JSC::AssemblyHelpers::sneakyJumpIfIsRememberedOrInEden):
(JSC::AssemblyHelpers::jumpIfIsRememberedOrInEden):
(JSC::AssemblyHelpers::storeBarrierStoreLoadFence):
(JSC::AssemblyHelpers::jumpIfStoreBarrierStoreLoadFenceNotNeeded):
* jit/JITOperations.cpp:
* jit/JITOperations.h:
* jit/JITPropertyAccess.cpp:
(JSC::JIT::emit_op_put_by_id):
(JSC::JIT::emitWriteBarrier):
(JSC::JIT::privateCompilePutByVal):
* jit/JITPropertyAccess32_64.cpp:
(JSC::JIT::emit_op_put_by_id):
* llint/LowLevelInterpreter.asm:
* offlineasm/x86.rb:
* runtime/Options.h:
Source/WTF:
Added clearRange(), which quickly clears a range of bits. This turned out to be useful for
a DFG optimization pass.
* wtf/FastBitVector.cpp:
(WTF::FastBitVector::clearRange):
* wtf/FastBitVector.h:
Canonical link: https://commits.webkit.org/180647@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206555 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-28 21:55:53 +00:00
|
|
|
WTF_EXPORT_PRIVATE void clearRange(size_t begin, size_t end);
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
|
Make MarkedBlock state tracking support overlapped allocation and marking state
https://bugs.webkit.org/show_bug.cgi?id=161581
Reviewed by Geoffrey Garen.
JSTests:
Add a microbenchmark for why we want to reclaim empty blocks from other allocators.
* microbenchmarks/switching-size-classes.js: Added.
Source/JavaScriptCore:
Concurrent GCs must allow for mutation and allocation during collection. We already know
how to mutate during collection. We have a write barrier for that. Allocation during
collection is more involved: the collector modifies the the mark bits, as well as other
kinds of MarkedBlock state, in-place during a collection. The allocator uses that same
MarkedBlock state to decide which regions of memory are free. This works if the allocator
never runs while the collector is running, but if we want to allow them to run at the same
time, then we need to have two versions of the state: one version built up by the
collector and another consumed by the allocator. We clear the collector state at the
beginning of collection, and splat the collector state onto the allocator state after
collection.
This could be super expensive, but we can make it cheap with some cleverness. The biggest
observation is just that most of the state is a handful of bits per block: is the block
free-listed? is it completely full? completely empty? in the incremental sweeper's
snapshot? is it retired? is it in eden? There is also state inside blocks, like the mark
bits, but I have a solid plan there and I'll save it for another patch. Once we view the
state of blocks as bits, we can put that state into bitvectors, so that if the collector
needs to transform the state of some blocks, it can do it with a single operation over
bitvectors. I like to think of this as 32-way parallelizing block operations, since
doing one operation on a 32-bit word in one of those bitvectors instantly affects 32
blocks.
This change converts all previous collections of MarkedBlocks, along with the MarkedBlock
state, into 8 bitvectors (live, empty, allocated, canAllocateButNotEmpty, eden, unswept,
markingNotEmpty, and markingRetired). The bitvectors separate allocator state (empty,
allocated, canAllocateButNotEmpty) from marking state (markingNotEmpty, markingRetired).
As a nice side-effect of switching to bitvectors, we get size class rebalancing for free.
It used to be that if a MarkedAllocator had an empty block, we would only allow that
memory to be reused by a different MarkedAllocator if we did an incremental sweep or a
full eager sweep. Now we hunt down all destructorless empty blocks before allocating new
MarkedBlocks. It would be relatively easy to also hunt down destructor empty blocks, but
the theory is that those might be expensive to sweep, so it might still be better to leave
those to the incremental sweeper.
This change is perf-neutral all around. I did some tests with two different kinds of
allocation strategies - something that is somewhat easier to do now that you can look for
blocks that are candidates for allocation by just scanning some bitvectors. I tried two
variants:
- Allocate out of non-empty blocks first, leaving empty blocks for last in case a
different allocator needed them. This is sort of a best-fit strategy. I tried this
first, and it can be expressed as:
m_allocationCursor = m_canAllocateButNotEmpty.findBit(m_allocationCursor, true)
- Allocate out of lower-indexed blocks first, treating empty and canAllocateButNotEmpty
blocks equally. This is sort of a first-fit strategy. This is what I ended up settling
on, and it can be expressed as:
m_allocationCursor = (m_canAllocateButNotEmpty | m_empty).findBit(m_allocationCursor, true)
The best-fit strategy meant 1% regressions in LongSpider and Octane overall, and a 11%
regression on Octane/earley. First-fit means perf-neutrality. Most great allocators skew
towards first-fit because it's empirically better, so this result is not surprising.
Overall, the performance of this patch on my machine is as follows, where "neutral" means
less than 1% and not statistically significant.
run-jsc-benchmarks:
SunSpider: neutral
LongSpider: 0.6% slower
V8Spider: neutral
Octane: neutral
Kraken: neutral
Microbenchmarks: 0.37% slower
AsmBench: neutral
CompressionBench: maybe 1% faster
For browser benchmarks, I report the ratio of means (bigger / smaller) along with a T-test
from Mathematica reported as % chance of not [sic] the null hypothesis. Note that we
normally consider anything less than 95% confidence to be inconclusive.
Browser benchmarks:
PLT3: 0.3% faster with 67% confidence
membuster:
Snap2FinishedLoadingPost: 0.68% more memory with 50% confidence
Snap3EndPost: 2.4% more memory with 61% confidence
JetStream: 0.2% slower with 32% confidence
Speedometer: 0.7% faster with 82% confidence
Additionally, Octane/splay's heap capacity goes down to ~180KB from ~200KB, so about a 10%
progression. This is due to the allocator rebalancing feature.
Finally, this breaks --useImmortalObjects. It was already broken as far as I can tell. I
filed a bug to reimplement it (bug 162296). Unless someone urgently needs this internal
tool, it's probably best to reimplement it after I'm done refactoring MarkedSpace.
* JavaScriptCore.xcodeproj/project.pbxproj:
* debugger/Debugger.cpp:
* heap/CellContainer.h:
* heap/CellContainerInlines.h:
(JSC::CellContainer::vm):
(JSC::CellContainer::heap):
(JSC::CellContainer::isMarkedOrNewlyAllocated):
(JSC::CellContainer::aboutToMark):
(JSC::CellContainer::isMarked): Deleted.
(JSC::CellContainer::flipIfNecessary): Deleted.
* heap/ConservativeRoots.cpp:
* heap/Heap.cpp:
(JSC::Heap::beginMarking):
(JSC::Heap::endMarking):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collectImpl):
(JSC::Heap::snapshotMarkedSpace):
(JSC::Heap::prepareForAllocation):
(JSC::Heap::zombifyDeadObjects):
(JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): Deleted.
(JSC::MarkedBlockSnapshotFunctor::operator()): Deleted.
(JSC::Heap::resetAllocators): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isMarked):
(JSC::Heap::isMarkedConcurrently):
(JSC::Heap::testAndSetMarked):
* heap/HeapStatistics.cpp:
* heap/HeapUtil.h:
(JSC::HeapUtil::findGCObjectPointersForMarking):
(JSC::HeapUtil::isPointerGCObjectJSCell):
* heap/HeapVerifier.cpp:
* heap/IncrementalSweeper.cpp:
(JSC::IncrementalSweeper::IncrementalSweeper):
(JSC::IncrementalSweeper::doSweep):
(JSC::IncrementalSweeper::sweepNextBlock):
(JSC::IncrementalSweeper::startSweeping):
(JSC::IncrementalSweeper::willFinishSweeping):
* heap/IncrementalSweeper.h:
* heap/LargeAllocation.h:
(JSC::LargeAllocation::isMarked):
(JSC::LargeAllocation::isMarkedConcurrently):
(JSC::LargeAllocation::isMarkedOrNewlyAllocated):
(JSC::LargeAllocation::aboutToMark):
(JSC::LargeAllocation::isMarkedDuringWeakVisiting): Deleted.
(JSC::LargeAllocation::flipIfNecessary): Deleted.
(JSC::LargeAllocation::flipIfNecessaryDuringMarking): Deleted.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::isPagedOut):
(JSC::MarkedAllocator::findEmptyBlock):
(JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
(JSC::MarkedAllocator::allocateIn):
(JSC::MarkedAllocator::tryAllocateIn):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
(JSC::MarkedAllocator::tryAllocateBlock):
(JSC::MarkedAllocator::addBlock):
(JSC::MarkedAllocator::removeBlock):
(JSC::MarkedAllocator::stopAllocating):
(JSC::MarkedAllocator::prepareForAllocation):
(JSC::MarkedAllocator::lastChanceToFinalize):
(JSC::MarkedAllocator::resumeAllocating):
(JSC::MarkedAllocator::beginMarkingForFullCollection):
(JSC::MarkedAllocator::endMarking):
(JSC::MarkedAllocator::snapshotForEdenCollection):
(JSC::MarkedAllocator::snapshotForFullCollection):
(JSC::MarkedAllocator::findBlockToSweep):
(JSC::MarkedAllocator::sweep):
(JSC::MarkedAllocator::shrink):
(JSC::MarkedAllocator::assertSnapshotEmpty):
(JSC::MarkedAllocator::dump):
(JSC::MarkedAllocator::dumpBits):
(JSC::MarkedAllocator::retire): Deleted.
(JSC::MarkedAllocator::filterNextBlock): Deleted.
(JSC::MarkedAllocator::setNextBlockToSweep): Deleted.
(JSC::MarkedAllocator::reset): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::forEachBitVector):
(JSC::MarkedAllocator::forEachBitVectorWithName):
(JSC::MarkedAllocator::nextAllocator):
(JSC::MarkedAllocator::setNextAllocator):
(JSC::MarkedAllocator::forEachBlock):
(JSC::MarkedAllocator::resumeAllocating): Deleted.
* heap/MarkedBlock.cpp:
(JSC::MarkedBlock::tryCreate):
(JSC::MarkedBlock::Handle::Handle):
(JSC::MarkedBlock::Handle::~Handle):
(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::Handle::specializedSweep):
(JSC::MarkedBlock::Handle::sweep):
(JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectEmptyMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectHasNewlyAllocated):
(JSC::MarkedBlock::Handle::sweepHelperSelectSweepMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectFlipMode):
(JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated):
(JSC::MarkedBlock::Handle::setIsFreeListed):
(JSC::MarkedBlock::Handle::stopAllocating):
(JSC::MarkedBlock::Handle::lastChanceToFinalize):
(JSC::MarkedBlock::Handle::resumeAllocating):
(JSC::MarkedBlock::aboutToMarkSlow):
(JSC::MarkedBlock::clearMarks):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::didConsumeFreeList):
(JSC::MarkedBlock::markCount):
(JSC::MarkedBlock::Handle::isEmpty):
(JSC::MarkedBlock::noteMarkedSlow):
(JSC::MarkedBlock::Handle::removeFromAllocator):
(JSC::MarkedBlock::Handle::didAddToAllocator):
(JSC::MarkedBlock::Handle::didRemoveFromAllocator):
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessarySlow): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarkingSlow): Deleted.
(JSC::MarkedBlock::Handle::willRemoveBlock): Deleted.
(WTF::printInternal): Deleted.
* heap/MarkedBlock.h:
(JSC::MarkedBlock::Handle::isFreeListed):
(JSC::MarkedBlock::Handle::index):
(JSC::MarkedBlock::aboutToMark):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::isMarkedConcurrently):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::isOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::state): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipForEdenCollection): Deleted.
(JSC::MarkedBlock::isMarkedDuringWeakVisiting): Deleted.
(JSC::MarkedBlock::Handle::isLive): Deleted.
(JSC::MarkedBlock::Handle::isLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachDeadCell): Deleted.
(JSC::MarkedBlock::Handle::needsSweeping): Deleted.
(JSC::MarkedBlock::Handle::isAllocated): Deleted.
(JSC::MarkedBlock::Handle::isMarked): Deleted.
* heap/MarkedBlockInlines.h: Added.
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::forEachLiveCell):
(JSC::MarkedBlock::Handle::forEachDeadCell):
(JSC::MarkedBlock::resetVersion):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::MarkedSpace):
(JSC::MarkedSpace::allocate):
(JSC::MarkedSpace::tryAllocate):
(JSC::MarkedSpace::sweep):
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::shrink):
(JSC::MarkedSpace::clearNewlyAllocated):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::endMarking):
(JSC::MarkedSpace::didAllocateInBlock):
(JSC::MarkedSpace::findEmptyBlock):
(JSC::MarkedSpace::snapshot):
(JSC::MarkedSpace::assertSnapshotEmpty):
(JSC::MarkedSpace::dumpBits):
(JSC::MarkedSpace::zombifySweep): Deleted.
(JSC::MarkedSpace::resetAllocators): Deleted.
(JSC::VerifyMarked::operator()): Deleted.
(JSC::MarkedSpace::flip): Deleted.
* heap/MarkedSpace.h:
(JSC::MarkedSpace::nextVersion):
(JSC::MarkedSpace::firstAllocator):
(JSC::MarkedSpace::allocatorForEmptyAllocation):
(JSC::MarkedSpace::forEachAllocator):
(JSC::MarkedSpace::blocksWithNewObjects): Deleted.
(JSC::MarkedSpace::setIsMarking): Deleted.
(JSC::MarkedSpace::forEachLiveCell): Deleted.
(JSC::MarkedSpace::forEachDeadCell): Deleted.
* heap/MarkedSpaceInlines.h: Added.
(JSC::MarkedSpace::forEachLiveCell):
(JSC::MarkedSpace::forEachDeadCell):
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::setMarkedAndAppendToMarkStack):
(JSC::SlotVisitor::markAuxiliary):
(JSC::SlotVisitor::visitChildren):
* heap/Weak.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue):
(WTF::HashTraits<JSC::Weak<T>>::peek):
* heap/WeakBlock.cpp:
(JSC::WeakBlock::specializedVisit):
(JSC::WeakBlock::reap):
* heap/WeakInlines.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue): Deleted.
(WTF::HashTraits<JSC::Weak<T>>::peek): Deleted.
* jit/JITThunks.h:
* runtime/JSGlobalObject.cpp:
* runtime/PrototypeMap.h:
* runtime/SamplingProfiler.cpp:
* runtime/WeakGCMap.h:
* tools/JSDollarVMPrototype.cpp:
Source/WTF:
The main change here is to bring back FastBitVector.cpp, so that I could outline some
large slow path functions. This also adds some utilities, like atomicSetAndCheck() and
isEmpty(). The GC uses these.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/FastBitVector.cpp: Added.
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorImpl::isEmpty):
(WTF::FastBitVector::atomicSetAndCheck):
(WTF::FastBitVector::operator[]): Deleted.
Tools:
Remove the always-trigger-copy-phase configuration.
* Scripts/run-jsc-stress-tests:
Canonical link: https://commits.webkit.org/180306@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206154 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-20 18:12:18 +00:00
|
|
|
// Returns true if the contents of this bitvector changed.
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
template<typename OtherWords>
|
|
|
|
bool setAndCheck(const FastBitVectorImpl<OtherWords>& other)
|
|
|
|
{
|
|
|
|
bool changed = false;
|
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(numBits() == other.numBits());
|
|
|
|
for (unsigned i = arrayLength(); i--;) {
|
|
|
|
changed |= m_words.word(i) != other.m_words.word(i);
|
|
|
|
m_words.word(i) = other.m_words.word(i);
|
|
|
|
}
|
|
|
|
return changed;
|
|
|
|
}
|
|
|
|
|
|
|
|
template<typename OtherWords>
|
|
|
|
FastBitVector& operator|=(const FastBitVectorImpl<OtherWords>& other)
|
|
|
|
{
|
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(numBits() == other.numBits());
|
|
|
|
for (unsigned i = arrayLength(); i--;)
|
|
|
|
m_words.word(i) |= other.m_words.word(i);
|
|
|
|
return *this;
|
|
|
|
}
|
|
|
|
|
|
|
|
template<typename OtherWords>
|
|
|
|
FastBitVector& operator&=(const FastBitVectorImpl<OtherWords>& other)
|
|
|
|
{
|
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(numBits() == other.numBits());
|
|
|
|
for (unsigned i = arrayLength(); i--;)
|
|
|
|
m_words.word(i) &= other.m_words.word(i);
|
|
|
|
return *this;
|
|
|
|
}
|
|
|
|
|
|
|
|
bool at(size_t index) const
|
|
|
|
{
|
|
|
|
return atImpl(index);
|
|
|
|
}
|
|
|
|
|
|
|
|
bool operator[](size_t index) const
|
|
|
|
{
|
|
|
|
return atImpl(index);
|
|
|
|
}
|
|
|
|
|
2019-11-14 09:37:12 +00:00
|
|
|
FastBitReference at(size_t index)
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
{
|
|
|
|
ASSERT_WITH_SECURITY_IMPLICATION(index < numBits());
|
2019-11-14 09:37:12 +00:00
|
|
|
return FastBitReference(&m_words.word(index >> 5), 1 << (index & 31));
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
}
|
|
|
|
|
2019-11-14 09:37:12 +00:00
|
|
|
FastBitReference operator[](size_t index)
|
FastBitVector should have efficient and easy-to-use vector-vector operations
https://bugs.webkit.org/show_bug.cgi?id=161847
Reviewed by Saam Barati.
Source/JavaScriptCore:
Adapt existing users of FastBitVector to the new API.
* bytecode/BytecodeLivenessAnalysis.cpp:
(JSC::BytecodeLivenessAnalysis::computeKills):
(JSC::BytecodeLivenessAnalysis::dumpResults):
* bytecode/BytecodeLivenessAnalysisInlines.h:
(JSC::operandThatIsNotAlwaysLiveIsLive):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::stepOverInstruction):
(JSC::BytecodeLivenessPropagation<DerivedAnalysis>::runLivenessFixpoint):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::validate):
* dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::flushForTerminal):
* dfg/DFGForAllKills.h:
(JSC::DFG::forAllKilledOperands):
* dfg/DFGGraph.h:
(JSC::DFG::Graph::forAllLocalsLiveInBytecode):
* dfg/DFGLiveCatchVariablePreservationPhase.cpp:
(JSC::DFG::LiveCatchVariablePreservationPhase::willCatchException):
(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):
* dfg/DFGNaturalLoops.cpp:
(JSC::DFG::NaturalLoops::NaturalLoops):
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::cleanMustHandleValuesIfNecessary):
Source/WTF:
FastBitVector is a bitvector representation that supports manual dynamic resizing and is
optimized for speed, not space. (BitVector supports automatic dynamic resizing and is
optimized for space, while Bitmap is sized statically and is optimized for both speed and
space.) This change greatly increases the power of FastBitVector. We will use these new
powers for changing the JSC GC to use FastBitVectors to track sets of MarkedBlocks (bug
161581) instead of using a combination of Vectors and doubly-linked lists.
This change splits FastBitVector into two parts:
- A thing that manages the storage of a bitvector: a uint32_t array and a size_t numBits.
We call this the word view.
- A thing that takes some kind of abstract array of uint32_t's and does bitvector
operations to it. We call this the FastBitVectorImpl.
FastBitVectorImpl and word views are immutable. The FastBitVector class is a subclass of
FastBitVectorImpl specialized on a word view that owns its words and has additional
support for mutable operations.
Doing this allows us to efficiently support things like this without any unnecessary
memory allocation or copying:
FastBitVector a, b, c; // Assume that there is code to initialize these.
a &= b | ~c;
Previously, this kind of operation would not be efficient, because "~c" would have to
create a whole new FastBitVector. But now, it just returns a FastBitVectorImpl whose
underlying word view bitnots (~) its words on the fly. Using template magic, this can get
pretty complex. For example "b | ~c" returns a FastBitVectorImpl that wraps a word view
whose implementation of WordView::word(size_t index) is something like:
uint32_t word(size_t index) { return b.m_words.word(index) | ~c.m_words.word(index); }
FastBitVectorImpl supports all of the fast bulk bitvector operations, like
forEachSetBit(), bitCount(), etc. So, when you say "a &= b | ~c", the actual
implementation is going to run these bit operations on word granularity directly over the
storage inside a, b, c.
The use of operator overloading is worth explaining a bit. Previously, FastBitVector
avoided operator overloading. For example, the &= operation was called filter(). I think
that this was a pretty good approach at the time. I tried using non-operator methods in
this FastBitVector rewrite, but I found it very odd to say things like:
a.filter(b.bitOr(c.bitNot()));
I think that it's harder to see what is going on here, then using operators, because infix
notation is always better.
* WTF.xcodeproj/project.pbxproj:
* wtf/BitVector.h:
(WTF::BitVector::findBitInWord): Deleted.
* wtf/CMakeLists.txt:
* wtf/Dominators.h:
(WTF::Dominators::NaiveDominators::NaiveDominators):
(WTF::Dominators::NaiveDominators::dominates):
(WTF::Dominators::NaiveDominators::pruneDominators):
* wtf/FastBitVector.cpp: Removed.
* wtf/FastBitVector.h:
(WTF::fastBitVectorArrayLength):
(WTF::FastBitVectorWordView::FastBitVectorWordView):
(WTF::FastBitVectorWordView::numBits):
(WTF::FastBitVectorWordView::word):
(WTF::FastBitVectorWordOwner::FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::~FastBitVectorWordOwner):
(WTF::FastBitVectorWordOwner::view):
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::setAll):
(WTF::FastBitVectorWordOwner::clearAll):
(WTF::FastBitVectorWordOwner::set):
(WTF::FastBitVectorWordOwner::numBits):
(WTF::FastBitVectorWordOwner::arrayLength):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorWordOwner::word):
(WTF::FastBitVectorWordOwner::words):
(WTF::FastBitVectorAndWords::FastBitVectorAndWords):
(WTF::FastBitVectorAndWords::view):
(WTF::FastBitVectorAndWords::numBits):
(WTF::FastBitVectorAndWords::word):
(WTF::FastBitVectorOrWords::FastBitVectorOrWords):
(WTF::FastBitVectorOrWords::view):
(WTF::FastBitVectorOrWords::numBits):
(WTF::FastBitVectorOrWords::word):
(WTF::FastBitVectorNotWords::FastBitVectorNotWords):
(WTF::FastBitVectorNotWords::view):
(WTF::FastBitVectorNotWords::numBits):
(WTF::FastBitVectorNotWords::word):
(WTF::FastBitVectorImpl::FastBitVectorImpl):
(WTF::FastBitVectorImpl::numBits):
(WTF::FastBitVectorImpl::size):
(WTF::FastBitVectorImpl::arrayLength):
(WTF::FastBitVectorImpl::operator==):
(WTF::FastBitVectorImpl::operator!=):
(WTF::FastBitVectorImpl::at):
(WTF::FastBitVectorImpl::operator[]):
(WTF::FastBitVectorImpl::bitCount):
(WTF::FastBitVectorImpl::operator&):
(WTF::FastBitVectorImpl::operator|):
(WTF::FastBitVectorImpl::operator~):
(WTF::FastBitVectorImpl::forEachSetBit):
(WTF::FastBitVectorImpl::forEachClearBit):
(WTF::FastBitVectorImpl::forEachBit):
(WTF::FastBitVectorImpl::findBit):
(WTF::FastBitVectorImpl::findSetBit):
(WTF::FastBitVectorImpl::findClearBit):
(WTF::FastBitVectorImpl::dump):
(WTF::FastBitVectorImpl::atImpl):
(WTF::FastBitVector::FastBitVector):
(WTF::FastBitVector::operator=):
(WTF::FastBitVector::resize):
(WTF::FastBitVector::setAll):
(WTF::FastBitVector::clearAll):
(WTF::FastBitVector::setAndCheck):
(WTF::FastBitVector::operator|=):
(WTF::FastBitVector::operator&=):
(WTF::FastBitVector::at):
(WTF::FastBitVector::operator[]):
(WTF::FastBitVector::BitReference::BitReference):
(WTF::FastBitVector::BitReference::operator bool):
(WTF::FastBitVector::BitReference::operator=):
(WTF::FastBitVector::~FastBitVector): Deleted.
(WTF::FastBitVector::numBits): Deleted.
(WTF::FastBitVector::set): Deleted.
(WTF::FastBitVector::equals): Deleted.
(WTF::FastBitVector::merge): Deleted.
(WTF::FastBitVector::filter): Deleted.
(WTF::FastBitVector::exclude): Deleted.
(WTF::FastBitVector::clear): Deleted.
(WTF::FastBitVector::get): Deleted.
(WTF::FastBitVector::bitCount): Deleted.
(WTF::FastBitVector::forEachSetBit): Deleted.
(WTF::FastBitVector::arrayLength): Deleted.
* wtf/StdLibExtras.h:
(WTF::findBitInWord):
Canonical link: https://commits.webkit.org/180007@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205794 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-12 04:03:36 +00:00
|
|
|
{
|
|
|
|
return at(index);
|
|
|
|
}
|
Make MarkedBlock state tracking support overlapped allocation and marking state
https://bugs.webkit.org/show_bug.cgi?id=161581
Reviewed by Geoffrey Garen.
JSTests:
Add a microbenchmark for why we want to reclaim empty blocks from other allocators.
* microbenchmarks/switching-size-classes.js: Added.
Source/JavaScriptCore:
Concurrent GCs must allow for mutation and allocation during collection. We already know
how to mutate during collection. We have a write barrier for that. Allocation during
collection is more involved: the collector modifies the the mark bits, as well as other
kinds of MarkedBlock state, in-place during a collection. The allocator uses that same
MarkedBlock state to decide which regions of memory are free. This works if the allocator
never runs while the collector is running, but if we want to allow them to run at the same
time, then we need to have two versions of the state: one version built up by the
collector and another consumed by the allocator. We clear the collector state at the
beginning of collection, and splat the collector state onto the allocator state after
collection.
This could be super expensive, but we can make it cheap with some cleverness. The biggest
observation is just that most of the state is a handful of bits per block: is the block
free-listed? is it completely full? completely empty? in the incremental sweeper's
snapshot? is it retired? is it in eden? There is also state inside blocks, like the mark
bits, but I have a solid plan there and I'll save it for another patch. Once we view the
state of blocks as bits, we can put that state into bitvectors, so that if the collector
needs to transform the state of some blocks, it can do it with a single operation over
bitvectors. I like to think of this as 32-way parallelizing block operations, since
doing one operation on a 32-bit word in one of those bitvectors instantly affects 32
blocks.
This change converts all previous collections of MarkedBlocks, along with the MarkedBlock
state, into 8 bitvectors (live, empty, allocated, canAllocateButNotEmpty, eden, unswept,
markingNotEmpty, and markingRetired). The bitvectors separate allocator state (empty,
allocated, canAllocateButNotEmpty) from marking state (markingNotEmpty, markingRetired).
As a nice side-effect of switching to bitvectors, we get size class rebalancing for free.
It used to be that if a MarkedAllocator had an empty block, we would only allow that
memory to be reused by a different MarkedAllocator if we did an incremental sweep or a
full eager sweep. Now we hunt down all destructorless empty blocks before allocating new
MarkedBlocks. It would be relatively easy to also hunt down destructor empty blocks, but
the theory is that those might be expensive to sweep, so it might still be better to leave
those to the incremental sweeper.
This change is perf-neutral all around. I did some tests with two different kinds of
allocation strategies - something that is somewhat easier to do now that you can look for
blocks that are candidates for allocation by just scanning some bitvectors. I tried two
variants:
- Allocate out of non-empty blocks first, leaving empty blocks for last in case a
different allocator needed them. This is sort of a best-fit strategy. I tried this
first, and it can be expressed as:
m_allocationCursor = m_canAllocateButNotEmpty.findBit(m_allocationCursor, true)
- Allocate out of lower-indexed blocks first, treating empty and canAllocateButNotEmpty
blocks equally. This is sort of a first-fit strategy. This is what I ended up settling
on, and it can be expressed as:
m_allocationCursor = (m_canAllocateButNotEmpty | m_empty).findBit(m_allocationCursor, true)
The best-fit strategy meant 1% regressions in LongSpider and Octane overall, and a 11%
regression on Octane/earley. First-fit means perf-neutrality. Most great allocators skew
towards first-fit because it's empirically better, so this result is not surprising.
Overall, the performance of this patch on my machine is as follows, where "neutral" means
less than 1% and not statistically significant.
run-jsc-benchmarks:
SunSpider: neutral
LongSpider: 0.6% slower
V8Spider: neutral
Octane: neutral
Kraken: neutral
Microbenchmarks: 0.37% slower
AsmBench: neutral
CompressionBench: maybe 1% faster
For browser benchmarks, I report the ratio of means (bigger / smaller) along with a T-test
from Mathematica reported as % chance of not [sic] the null hypothesis. Note that we
normally consider anything less than 95% confidence to be inconclusive.
Browser benchmarks:
PLT3: 0.3% faster with 67% confidence
membuster:
Snap2FinishedLoadingPost: 0.68% more memory with 50% confidence
Snap3EndPost: 2.4% more memory with 61% confidence
JetStream: 0.2% slower with 32% confidence
Speedometer: 0.7% faster with 82% confidence
Additionally, Octane/splay's heap capacity goes down to ~180KB from ~200KB, so about a 10%
progression. This is due to the allocator rebalancing feature.
Finally, this breaks --useImmortalObjects. It was already broken as far as I can tell. I
filed a bug to reimplement it (bug 162296). Unless someone urgently needs this internal
tool, it's probably best to reimplement it after I'm done refactoring MarkedSpace.
* JavaScriptCore.xcodeproj/project.pbxproj:
* debugger/Debugger.cpp:
* heap/CellContainer.h:
* heap/CellContainerInlines.h:
(JSC::CellContainer::vm):
(JSC::CellContainer::heap):
(JSC::CellContainer::isMarkedOrNewlyAllocated):
(JSC::CellContainer::aboutToMark):
(JSC::CellContainer::isMarked): Deleted.
(JSC::CellContainer::flipIfNecessary): Deleted.
* heap/ConservativeRoots.cpp:
* heap/Heap.cpp:
(JSC::Heap::beginMarking):
(JSC::Heap::endMarking):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collectImpl):
(JSC::Heap::snapshotMarkedSpace):
(JSC::Heap::prepareForAllocation):
(JSC::Heap::zombifyDeadObjects):
(JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): Deleted.
(JSC::MarkedBlockSnapshotFunctor::operator()): Deleted.
(JSC::Heap::resetAllocators): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isMarked):
(JSC::Heap::isMarkedConcurrently):
(JSC::Heap::testAndSetMarked):
* heap/HeapStatistics.cpp:
* heap/HeapUtil.h:
(JSC::HeapUtil::findGCObjectPointersForMarking):
(JSC::HeapUtil::isPointerGCObjectJSCell):
* heap/HeapVerifier.cpp:
* heap/IncrementalSweeper.cpp:
(JSC::IncrementalSweeper::IncrementalSweeper):
(JSC::IncrementalSweeper::doSweep):
(JSC::IncrementalSweeper::sweepNextBlock):
(JSC::IncrementalSweeper::startSweeping):
(JSC::IncrementalSweeper::willFinishSweeping):
* heap/IncrementalSweeper.h:
* heap/LargeAllocation.h:
(JSC::LargeAllocation::isMarked):
(JSC::LargeAllocation::isMarkedConcurrently):
(JSC::LargeAllocation::isMarkedOrNewlyAllocated):
(JSC::LargeAllocation::aboutToMark):
(JSC::LargeAllocation::isMarkedDuringWeakVisiting): Deleted.
(JSC::LargeAllocation::flipIfNecessary): Deleted.
(JSC::LargeAllocation::flipIfNecessaryDuringMarking): Deleted.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::isPagedOut):
(JSC::MarkedAllocator::findEmptyBlock):
(JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
(JSC::MarkedAllocator::allocateIn):
(JSC::MarkedAllocator::tryAllocateIn):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
(JSC::MarkedAllocator::tryAllocateBlock):
(JSC::MarkedAllocator::addBlock):
(JSC::MarkedAllocator::removeBlock):
(JSC::MarkedAllocator::stopAllocating):
(JSC::MarkedAllocator::prepareForAllocation):
(JSC::MarkedAllocator::lastChanceToFinalize):
(JSC::MarkedAllocator::resumeAllocating):
(JSC::MarkedAllocator::beginMarkingForFullCollection):
(JSC::MarkedAllocator::endMarking):
(JSC::MarkedAllocator::snapshotForEdenCollection):
(JSC::MarkedAllocator::snapshotForFullCollection):
(JSC::MarkedAllocator::findBlockToSweep):
(JSC::MarkedAllocator::sweep):
(JSC::MarkedAllocator::shrink):
(JSC::MarkedAllocator::assertSnapshotEmpty):
(JSC::MarkedAllocator::dump):
(JSC::MarkedAllocator::dumpBits):
(JSC::MarkedAllocator::retire): Deleted.
(JSC::MarkedAllocator::filterNextBlock): Deleted.
(JSC::MarkedAllocator::setNextBlockToSweep): Deleted.
(JSC::MarkedAllocator::reset): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::forEachBitVector):
(JSC::MarkedAllocator::forEachBitVectorWithName):
(JSC::MarkedAllocator::nextAllocator):
(JSC::MarkedAllocator::setNextAllocator):
(JSC::MarkedAllocator::forEachBlock):
(JSC::MarkedAllocator::resumeAllocating): Deleted.
* heap/MarkedBlock.cpp:
(JSC::MarkedBlock::tryCreate):
(JSC::MarkedBlock::Handle::Handle):
(JSC::MarkedBlock::Handle::~Handle):
(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::Handle::specializedSweep):
(JSC::MarkedBlock::Handle::sweep):
(JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectEmptyMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectHasNewlyAllocated):
(JSC::MarkedBlock::Handle::sweepHelperSelectSweepMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectFlipMode):
(JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated):
(JSC::MarkedBlock::Handle::setIsFreeListed):
(JSC::MarkedBlock::Handle::stopAllocating):
(JSC::MarkedBlock::Handle::lastChanceToFinalize):
(JSC::MarkedBlock::Handle::resumeAllocating):
(JSC::MarkedBlock::aboutToMarkSlow):
(JSC::MarkedBlock::clearMarks):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::didConsumeFreeList):
(JSC::MarkedBlock::markCount):
(JSC::MarkedBlock::Handle::isEmpty):
(JSC::MarkedBlock::noteMarkedSlow):
(JSC::MarkedBlock::Handle::removeFromAllocator):
(JSC::MarkedBlock::Handle::didAddToAllocator):
(JSC::MarkedBlock::Handle::didRemoveFromAllocator):
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessarySlow): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarkingSlow): Deleted.
(JSC::MarkedBlock::Handle::willRemoveBlock): Deleted.
(WTF::printInternal): Deleted.
* heap/MarkedBlock.h:
(JSC::MarkedBlock::Handle::isFreeListed):
(JSC::MarkedBlock::Handle::index):
(JSC::MarkedBlock::aboutToMark):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::isMarkedConcurrently):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::isOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::state): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipForEdenCollection): Deleted.
(JSC::MarkedBlock::isMarkedDuringWeakVisiting): Deleted.
(JSC::MarkedBlock::Handle::isLive): Deleted.
(JSC::MarkedBlock::Handle::isLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachDeadCell): Deleted.
(JSC::MarkedBlock::Handle::needsSweeping): Deleted.
(JSC::MarkedBlock::Handle::isAllocated): Deleted.
(JSC::MarkedBlock::Handle::isMarked): Deleted.
* heap/MarkedBlockInlines.h: Added.
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::forEachLiveCell):
(JSC::MarkedBlock::Handle::forEachDeadCell):
(JSC::MarkedBlock::resetVersion):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::MarkedSpace):
(JSC::MarkedSpace::allocate):
(JSC::MarkedSpace::tryAllocate):
(JSC::MarkedSpace::sweep):
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::shrink):
(JSC::MarkedSpace::clearNewlyAllocated):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::endMarking):
(JSC::MarkedSpace::didAllocateInBlock):
(JSC::MarkedSpace::findEmptyBlock):
(JSC::MarkedSpace::snapshot):
(JSC::MarkedSpace::assertSnapshotEmpty):
(JSC::MarkedSpace::dumpBits):
(JSC::MarkedSpace::zombifySweep): Deleted.
(JSC::MarkedSpace::resetAllocators): Deleted.
(JSC::VerifyMarked::operator()): Deleted.
(JSC::MarkedSpace::flip): Deleted.
* heap/MarkedSpace.h:
(JSC::MarkedSpace::nextVersion):
(JSC::MarkedSpace::firstAllocator):
(JSC::MarkedSpace::allocatorForEmptyAllocation):
(JSC::MarkedSpace::forEachAllocator):
(JSC::MarkedSpace::blocksWithNewObjects): Deleted.
(JSC::MarkedSpace::setIsMarking): Deleted.
(JSC::MarkedSpace::forEachLiveCell): Deleted.
(JSC::MarkedSpace::forEachDeadCell): Deleted.
* heap/MarkedSpaceInlines.h: Added.
(JSC::MarkedSpace::forEachLiveCell):
(JSC::MarkedSpace::forEachDeadCell):
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::setMarkedAndAppendToMarkStack):
(JSC::SlotVisitor::markAuxiliary):
(JSC::SlotVisitor::visitChildren):
* heap/Weak.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue):
(WTF::HashTraits<JSC::Weak<T>>::peek):
* heap/WeakBlock.cpp:
(JSC::WeakBlock::specializedVisit):
(JSC::WeakBlock::reap):
* heap/WeakInlines.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue): Deleted.
(WTF::HashTraits<JSC::Weak<T>>::peek): Deleted.
* jit/JITThunks.h:
* runtime/JSGlobalObject.cpp:
* runtime/PrototypeMap.h:
* runtime/SamplingProfiler.cpp:
* runtime/WeakGCMap.h:
* tools/JSDollarVMPrototype.cpp:
Source/WTF:
The main change here is to bring back FastBitVector.cpp, so that I could outline some
large slow path functions. This also adds some utilities, like atomicSetAndCheck() and
isEmpty(). The GC uses these.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/FastBitVector.cpp: Added.
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorImpl::isEmpty):
(WTF::FastBitVector::atomicSetAndCheck):
(WTF::FastBitVector::operator[]): Deleted.
Tools:
Remove the always-trigger-copy-phase configuration.
* Scripts/run-jsc-stress-tests:
Canonical link: https://commits.webkit.org/180306@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206154 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-20 18:12:18 +00:00
|
|
|
|
|
|
|
// Returns true if the contents changed.
|
|
|
|
ALWAYS_INLINE bool atomicSetAndCheck(size_t index, bool value)
|
|
|
|
{
|
|
|
|
uint32_t* pointer = &m_words.word(index >> 5);
|
|
|
|
uint32_t mask = 1 << (index & 31);
|
|
|
|
for (;;) {
|
|
|
|
uint32_t oldValue = *pointer;
|
|
|
|
uint32_t newValue;
|
|
|
|
if (value) {
|
|
|
|
if (oldValue & mask)
|
|
|
|
return false;
|
|
|
|
newValue = oldValue | mask;
|
|
|
|
} else {
|
|
|
|
if (!(oldValue & mask))
|
|
|
|
return false;
|
|
|
|
newValue = oldValue & ~mask;
|
|
|
|
}
|
JSC should support SharedArrayBuffer
https://bugs.webkit.org/show_bug.cgi?id=163986
Reviewed by Keith Miller.
JSTests:
This adds our own test for the various corner cases of SharedArrayBuffer. This test is meant to
check all of the things that don't require concurrency.
* stress/SharedArrayBuffer.js: Added.
(checkAtomics):
(shouldFail):
(Symbol):
(runAtomic):
Source/JavaScriptCore:
This implements https://tc39.github.io/ecmascript_sharedmem/shmem.html.
There is now a new SharedArrayBuffer type. In the JS runtime, which includes typed array
types, the SharedArrayBuffer is a drop-in replacement for ArrayBuffer, even though they are
distinct types (new SharedArrayBuffer() instanceof ArrayBuffer == false and vice versa). The
DOM will not recognize SharedArrayBuffer, or any typed array that wraps it, to ensure safety.
This matches what other browsers intend to do, see
https://github.com/tc39/ecmascript_sharedmem/issues/38. API is provided for the DOM to opt
into SharedArrayBuffer. One notable place is postMessage, which will share the
SharedArrayBuffer's underlying data storage with other workers. This creates a pool of shared
memory that the workers can use to talk to each other.
There is also an Atomics object in global scope, which exposes sequentially consistent atomic
operations: add, and, compareExchange, exchange, load, or, store, sub, and xor. Additionally
it exposes a Atomics.isLockFree utility, which takes a byte amount and returns true or false.
Also there is Atomics.wake/wait, which neatly map to ParkingLot.
Accesses to typed arrays that wrap SharedArrayBuffer are optimized by JSC the same way as
always. I believe that DFG and B3 already obey the following memory model, which I believe is
a bit weaker than Cambridge and a bit stronger than what is being proposed for
SharedArrayBuffer. To predict a program's behavior under the B3 memory model, imagine the
space of all possible programs that would result from running an optimizer that adversarially
follows B3's transformation rules. B3 transformations are correct if the newly created
program is equivalent to the old one, assuming that any opaque effect in IR (like the reads
and writes of a patchpoint/call/fence) could perform any load/store that satisfies the
B3::Effects summary. Opaque effects are a way of describing an infinite set of programs: any
program that only does the effects summarized in B3::Effects belongs to the set. For example,
this prevents motion of operations across fences since fences are summarized as opaque
effects that could read or write memory. This rule alone is not enough, because it leaves the
door open for turning an atomic operation (like a load) into a non-atomic one (like a load
followed by a store of the same value back to the same location or multiple loads). This is
not an optimization that either our compiler or the CPU would want to do. One way to think of
what exactly is forbidden is that B3 transformations that mess with memory accesses can only
reorder them or remove them. This means that for any execution of the untransformed program,
the corresponding execution of the transformed program (i.e. with the same input arguments
and the same programs filled in for the opaque effects) must have the same loads and stores,
with some removed and some reordered. This is a fairly simple mental model that B3 and DFG
already follow and it's based on existing abstractions for the infinite set of programs
inside an opaque effect (DFG's AbstractHeaps and B3's Effects).
This patch makes all atomics operations intrinsic, but the DFG doesn't know about any of them
yet. That's covered by bug 164108.
This ought to be perf-neutral, but I am still running tests to confirm this. I'm also still
writing new tests to cover all of the Atomics functionality and the behavior of SAB objects.
* API/JSTypedArray.cpp:
(JSObjectGetTypedArrayBytesPtr):
(JSObjectGetTypedArrayBuffer):
(JSObjectMakeArrayBufferWithBytesNoCopy):
* API/tests/CompareAndSwapTest.cpp:
(Bitmap::concurrentTestAndSet):
* CMakeLists.txt:
* JavaScriptCore.xcodeproj/project.pbxproj:
* dfg/DFGDesiredWatchpoints.cpp:
(JSC::DFG::ArrayBufferViewWatchpointAdaptor::add):
* heap/Heap.cpp:
(JSC::Heap::reportExtraMemoryVisited):
(JSC::Heap::reportExternalMemoryVisited):
* jsc.cpp:
(functionTransferArrayBuffer):
* runtime/ArrayBuffer.cpp:
(JSC::SharedArrayBufferContents::SharedArrayBufferContents):
(JSC::SharedArrayBufferContents::~SharedArrayBufferContents):
(JSC::ArrayBufferContents::ArrayBufferContents):
(JSC::ArrayBufferContents::operator=):
(JSC::ArrayBufferContents::~ArrayBufferContents):
(JSC::ArrayBufferContents::clear):
(JSC::ArrayBufferContents::destroy):
(JSC::ArrayBufferContents::reset):
(JSC::ArrayBufferContents::tryAllocate):
(JSC::ArrayBufferContents::makeShared):
(JSC::ArrayBufferContents::transferTo):
(JSC::ArrayBufferContents::copyTo):
(JSC::ArrayBufferContents::shareWith):
(JSC::ArrayBuffer::create):
(JSC::ArrayBuffer::createAdopted):
(JSC::ArrayBuffer::createFromBytes):
(JSC::ArrayBuffer::tryCreate):
(JSC::ArrayBuffer::createUninitialized):
(JSC::ArrayBuffer::tryCreateUninitialized):
(JSC::ArrayBuffer::createInternal):
(JSC::ArrayBuffer::ArrayBuffer):
(JSC::ArrayBuffer::slice):
(JSC::ArrayBuffer::sliceImpl):
(JSC::ArrayBuffer::makeShared):
(JSC::ArrayBuffer::setSharingMode):
(JSC::ArrayBuffer::transferTo):
(JSC::ArrayBuffer::transfer): Deleted.
* runtime/ArrayBuffer.h:
(JSC::arrayBufferSharingModeName):
(JSC::SharedArrayBufferContents::data):
(JSC::ArrayBufferContents::data):
(JSC::ArrayBufferContents::sizeInBytes):
(JSC::ArrayBufferContents::isShared):
(JSC::ArrayBuffer::sharingMode):
(JSC::ArrayBuffer::isShared):
(JSC::ArrayBuffer::gcSizeEstimateInBytes):
(JSC::arrayBufferDestructorNull): Deleted.
(JSC::arrayBufferDestructorDefault): Deleted.
(JSC::ArrayBufferContents::ArrayBufferContents): Deleted.
(JSC::ArrayBufferContents::transfer): Deleted.
(JSC::ArrayBufferContents::copyTo): Deleted.
(JSC::ArrayBuffer::create): Deleted.
(JSC::ArrayBuffer::createAdopted): Deleted.
(JSC::ArrayBuffer::createFromBytes): Deleted.
(JSC::ArrayBuffer::tryCreate): Deleted.
(JSC::ArrayBuffer::createUninitialized): Deleted.
(JSC::ArrayBuffer::tryCreateUninitialized): Deleted.
(JSC::ArrayBuffer::createInternal): Deleted.
(JSC::ArrayBuffer::ArrayBuffer): Deleted.
(JSC::ArrayBuffer::slice): Deleted.
(JSC::ArrayBuffer::sliceImpl): Deleted.
(JSC::ArrayBufferContents::tryAllocate): Deleted.
(JSC::ArrayBufferContents::~ArrayBufferContents): Deleted.
* runtime/ArrayBufferSharingMode.h: Added.
* runtime/ArrayBufferView.h:
(JSC::ArrayBufferView::possiblySharedBuffer):
(JSC::ArrayBufferView::unsharedBuffer):
(JSC::ArrayBufferView::isShared):
(JSC::ArrayBufferView::buffer): Deleted.
* runtime/AtomicsObject.cpp: Added.
(JSC::AtomicsObject::AtomicsObject):
(JSC::AtomicsObject::create):
(JSC::AtomicsObject::createStructure):
(JSC::AtomicsObject::finishCreation):
(JSC::atomicsFuncAdd):
(JSC::atomicsFuncAnd):
(JSC::atomicsFuncCompareExchange):
(JSC::atomicsFuncExchange):
(JSC::atomicsFuncIsLockFree):
(JSC::atomicsFuncLoad):
(JSC::atomicsFuncOr):
(JSC::atomicsFuncStore):
(JSC::atomicsFuncSub):
(JSC::atomicsFuncWait):
(JSC::atomicsFuncWake):
(JSC::atomicsFuncXor):
* runtime/AtomicsObject.h: Added.
* runtime/CommonIdentifiers.h:
* runtime/DataView.cpp:
(JSC::DataView::wrap):
* runtime/GenericTypedArrayViewInlines.h:
(JSC::GenericTypedArrayView<Adaptor>::subarray):
* runtime/Intrinsic.h:
* runtime/JSArrayBuffer.cpp:
(JSC::JSArrayBuffer::finishCreation):
(JSC::JSArrayBuffer::isShared):
(JSC::JSArrayBuffer::sharingMode):
* runtime/JSArrayBuffer.h:
(JSC::toPossiblySharedArrayBuffer):
(JSC::toUnsharedArrayBuffer):
(JSC::JSArrayBuffer::toWrapped):
(JSC::toArrayBuffer): Deleted.
* runtime/JSArrayBufferConstructor.cpp:
(JSC::JSArrayBufferConstructor::JSArrayBufferConstructor):
(JSC::JSArrayBufferConstructor::finishCreation):
(JSC::JSArrayBufferConstructor::create):
(JSC::constructArrayBuffer):
* runtime/JSArrayBufferConstructor.h:
(JSC::JSArrayBufferConstructor::sharingMode):
* runtime/JSArrayBufferPrototype.cpp:
(JSC::arrayBufferProtoFuncSlice):
(JSC::JSArrayBufferPrototype::JSArrayBufferPrototype):
(JSC::JSArrayBufferPrototype::finishCreation):
(JSC::JSArrayBufferPrototype::create):
* runtime/JSArrayBufferPrototype.h:
* runtime/JSArrayBufferView.cpp:
(JSC::JSArrayBufferView::finishCreation):
(JSC::JSArrayBufferView::visitChildren):
(JSC::JSArrayBufferView::unsharedBuffer):
(JSC::JSArrayBufferView::unsharedJSBuffer):
(JSC::JSArrayBufferView::possiblySharedJSBuffer):
(JSC::JSArrayBufferView::neuter):
(JSC::JSArrayBufferView::toWrapped): Deleted.
* runtime/JSArrayBufferView.h:
(JSC::JSArrayBufferView::jsBuffer): Deleted.
* runtime/JSArrayBufferViewInlines.h:
(JSC::JSArrayBufferView::isShared):
(JSC::JSArrayBufferView::possiblySharedBuffer):
(JSC::JSArrayBufferView::possiblySharedImpl):
(JSC::JSArrayBufferView::unsharedImpl):
(JSC::JSArrayBufferView::byteOffset):
(JSC::JSArrayBufferView::toWrapped):
(JSC::JSArrayBufferView::buffer): Deleted.
(JSC::JSArrayBufferView::impl): Deleted.
(JSC::JSArrayBufferView::neuter): Deleted.
* runtime/JSDataView.cpp:
(JSC::JSDataView::possiblySharedTypedImpl):
(JSC::JSDataView::unsharedTypedImpl):
(JSC::JSDataView::getTypedArrayImpl):
(JSC::JSDataView::typedImpl): Deleted.
* runtime/JSDataView.h:
(JSC::JSDataView::possiblySharedBuffer):
(JSC::JSDataView::unsharedBuffer):
(JSC::JSDataView::buffer): Deleted.
* runtime/JSDataViewPrototype.cpp:
(JSC::dataViewProtoGetterBuffer):
* runtime/JSGenericTypedArrayView.h:
(JSC::toPossiblySharedNativeTypedView):
(JSC::toUnsharedNativeTypedView):
(JSC::JSGenericTypedArrayView<Adaptor>::toWrapped):
(JSC::JSGenericTypedArrayView::typedImpl): Deleted.
(JSC::toNativeTypedView): Deleted.
* runtime/JSGenericTypedArrayViewInlines.h:
(JSC::JSGenericTypedArrayView<Adaptor>::create):
(JSC::JSGenericTypedArrayView<Adaptor>::possiblySharedTypedImpl):
(JSC::JSGenericTypedArrayView<Adaptor>::unsharedTypedImpl):
(JSC::JSGenericTypedArrayView<Adaptor>::getTypedArrayImpl):
* runtime/JSGenericTypedArrayViewPrototypeFunctions.h:
(JSC::genericTypedArrayViewProtoGetterFuncBuffer):
(JSC::genericTypedArrayViewPrivateFuncSubarrayCreate):
* runtime/JSGlobalObject.cpp:
(JSC::createAtomicsProperty):
(JSC::JSGlobalObject::init):
(JSC::JSGlobalObject::visitChildren):
* runtime/JSGlobalObject.h:
(JSC::JSGlobalObject::arrayBufferPrototype):
(JSC::JSGlobalObject::arrayBufferStructure):
* runtime/MathObject.cpp:
* runtime/RuntimeFlags.h:
* runtime/SimpleTypedArrayController.cpp:
(JSC::SimpleTypedArrayController::toJS):
* runtime/TypedArrayType.h:
(JSC::typedArrayTypeForType):
Source/WebCore:
New tests added in the LayoutTests/workers/sab directory.
This teaches WebCore that a typed array could be shared or not. By default, WebCore will
reject shared typed arrays as if they were not typed arrays. This ensures that we don't get
race conditions in code that can't handle it.
If you postMessage a SharedArrayBuffer or something that wraps it, you will send the shared
memory to the other worker.
* Modules/encryptedmedia/CDMSessionClearKey.cpp:
(WebCore::CDMSessionClearKey::cachedKeyForKeyID):
* Modules/fetch/FetchBody.cpp:
(WebCore::FetchBody::extract):
* Modules/mediastream/RTCDataChannel.cpp:
(WebCore::RTCDataChannel::send):
* Modules/webaudio/AudioBuffer.cpp:
(WebCore::AudioBuffer::getChannelData):
* Modules/websockets/WebSocket.cpp:
(WebCore::WebSocket::send):
* bindings/js/JSBlobCustom.cpp:
(WebCore::constructJSBlob):
* bindings/js/JSCryptoAlgorithmDictionary.cpp:
(WebCore::createRsaKeyGenParams):
* bindings/js/JSCryptoCustom.cpp:
(WebCore::JSCrypto::getRandomValues):
* bindings/js/JSCryptoOperationData.cpp:
(WebCore::cryptoOperationDataFromJSValue):
* bindings/js/JSDOMBinding.h:
(WebCore::toJS):
(WebCore::toPossiblySharedArrayBufferView):
(WebCore::toUnsharedArrayBufferView):
(WebCore::toPossiblySharedInt8Array):
(WebCore::toPossiblySharedInt16Array):
(WebCore::toPossiblySharedInt32Array):
(WebCore::toPossiblySharedUint8Array):
(WebCore::toPossiblySharedUint8ClampedArray):
(WebCore::toPossiblySharedUint16Array):
(WebCore::toPossiblySharedUint32Array):
(WebCore::toPossiblySharedFloat32Array):
(WebCore::toPossiblySharedFloat64Array):
(WebCore::toUnsharedInt8Array):
(WebCore::toUnsharedInt16Array):
(WebCore::toUnsharedInt32Array):
(WebCore::toUnsharedUint8Array):
(WebCore::toUnsharedUint8ClampedArray):
(WebCore::toUnsharedUint16Array):
(WebCore::toUnsharedUint32Array):
(WebCore::toUnsharedFloat32Array):
(WebCore::toUnsharedFloat64Array):
(WebCore::toArrayBufferView): Deleted.
(WebCore::toInt8Array): Deleted.
(WebCore::toInt16Array): Deleted.
(WebCore::toInt32Array): Deleted.
(WebCore::toUint8Array): Deleted.
(WebCore::toUint8ClampedArray): Deleted.
(WebCore::toUint16Array): Deleted.
(WebCore::toUint32Array): Deleted.
(WebCore::toFloat32Array): Deleted.
(WebCore::toFloat64Array): Deleted.
* bindings/js/JSDataCueCustom.cpp:
(WebCore::constructJSDataCue):
* bindings/js/JSDictionary.cpp:
(WebCore::JSDictionary::convertValue):
* bindings/js/JSFileCustom.cpp:
(WebCore::constructJSFile):
* bindings/js/JSMessagePortCustom.cpp:
(WebCore::extractTransferables):
* bindings/js/JSWebGLRenderingContextBaseCustom.cpp:
(WebCore::dataFunctionf):
(WebCore::dataFunctioni):
(WebCore::dataFunctionMatrix):
* bindings/js/JSXMLHttpRequestCustom.cpp:
(WebCore::JSXMLHttpRequest::send):
* bindings/js/SerializedScriptValue.cpp:
(WebCore::CloneSerializer::dumpArrayBufferView):
(WebCore::CloneSerializer::dumpIfTerminal):
(WebCore::CloneDeserializer::readArrayBufferView):
(WebCore::CloneDeserializer::readTerminal):
(WebCore::SerializedScriptValue::transferArrayBuffers):
* bindings/js/StructuredClone.cpp:
(WebCore::structuredCloneArrayBuffer):
(WebCore::structuredCloneArrayBufferView):
* bindings/scripts/CodeGeneratorJS.pm:
(JSValueToNative):
* css/FontFace.cpp:
(WebCore::FontFace::create):
* html/canvas/WebGL2RenderingContext.cpp:
(WebCore::WebGL2RenderingContext::bufferData):
(WebCore::WebGL2RenderingContext::bufferSubData):
* platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp:
(WebCore::MediaPlayerPrivateAVFoundation::extractKeyURIKeyIDAndCertificateFromInitData):
Source/WebKit/mac:
Support the RuntimeFlag.
* WebView/WebPreferencesPrivate.h:
Source/WebKit/win:
Support the RuntimeFlag.
* Interfaces/IWebPreferencesPrivate.idl:
Source/WebKit2:
Adds some small things we need for SharedArrayBuffer.
* UIProcess/API/C/WKPreferencesRefPrivate.h:
* UIProcess/API/Cocoa/WKPreferencesPrivate.h:
* WebProcess/InjectedBundle/InjectedBundle.cpp:
(WebKit::InjectedBundle::createWebDataFromUint8Array):
Source/WTF:
Adds some small things we need for SharedArrayBuffer.
* wtf/Atomics.h:
(WTF::Atomic::compareExchangeWeakRelaxed):
(WTF::Atomic::exchangeAdd):
(WTF::Atomic::exchangeAnd):
(WTF::Atomic::exchangeOr):
(WTF::Atomic::exchangeSub):
(WTF::Atomic::exchangeXor):
(WTF::atomicLoad):
(WTF::atomicStore):
(WTF::atomicCompareExchangeWeak):
(WTF::atomicCompareExchangeWeakRelaxed):
(WTF::atomicCompareExchangeStrong):
(WTF::atomicExchangeAdd):
(WTF::atomicExchangeAnd):
(WTF::atomicExchangeOr):
(WTF::atomicExchangeSub):
(WTF::atomicExchangeXor):
(WTF::atomicExchange):
(WTF::Atomic::exchangeAndAdd): Deleted.
(WTF::weakCompareAndSwap): Deleted.
We need to be able to do atomics operations on naked pointers. We also need to be able to do
all of the things that std::atomic does. This adds those things and renames
weakCompareAndSwap to atomicCompareExchangeWeakRelaxed so that we're using consistent
terminology.
* wtf/Bitmap.h:
(WTF::WordType>::concurrentTestAndSet): Renamed weakCompareAndSwap.
(WTF::WordType>::concurrentTestAndClear): Renamed weakCompareAndSwap.
* wtf/FastBitVector.h:
(WTF::FastBitVector::atomicSetAndCheck): Renamed weakCompareAndSwap.
* wtf/ParkingLot.cpp:
(WTF::ParkingLot::unparkOne):
(WTF::ParkingLot::unparkCount):
* wtf/ParkingLot.h:
Added unparkCount(), which lets you unpark some bounded number of threads and returns the
number of threads unparked. This is just a modest extension of unparkAll(). unparkAll() now
just calls unparkCount(ptr, UINT_MAX).
Tools:
Use the right kind of typed array API.
* DumpRenderTree/TestRunner.cpp:
(setAudioResultCallback):
LayoutTests:
Adding tests. This is a work in progress.
* workers/sab: Added.
* workers/sab/simple-worker-1.js: Added.
(onmessage):
* workers/sab/simple-worker-2.js: Added.
(onmessage):
* workers/sab/simple.html: Added.
Canonical link: https://commits.webkit.org/181984@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@208209 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-11-01 03:10:00 +00:00
|
|
|
if (atomicCompareExchangeWeakRelaxed(pointer, oldValue, newValue))
|
Make MarkedBlock state tracking support overlapped allocation and marking state
https://bugs.webkit.org/show_bug.cgi?id=161581
Reviewed by Geoffrey Garen.
JSTests:
Add a microbenchmark for why we want to reclaim empty blocks from other allocators.
* microbenchmarks/switching-size-classes.js: Added.
Source/JavaScriptCore:
Concurrent GCs must allow for mutation and allocation during collection. We already know
how to mutate during collection. We have a write barrier for that. Allocation during
collection is more involved: the collector modifies the the mark bits, as well as other
kinds of MarkedBlock state, in-place during a collection. The allocator uses that same
MarkedBlock state to decide which regions of memory are free. This works if the allocator
never runs while the collector is running, but if we want to allow them to run at the same
time, then we need to have two versions of the state: one version built up by the
collector and another consumed by the allocator. We clear the collector state at the
beginning of collection, and splat the collector state onto the allocator state after
collection.
This could be super expensive, but we can make it cheap with some cleverness. The biggest
observation is just that most of the state is a handful of bits per block: is the block
free-listed? is it completely full? completely empty? in the incremental sweeper's
snapshot? is it retired? is it in eden? There is also state inside blocks, like the mark
bits, but I have a solid plan there and I'll save it for another patch. Once we view the
state of blocks as bits, we can put that state into bitvectors, so that if the collector
needs to transform the state of some blocks, it can do it with a single operation over
bitvectors. I like to think of this as 32-way parallelizing block operations, since
doing one operation on a 32-bit word in one of those bitvectors instantly affects 32
blocks.
This change converts all previous collections of MarkedBlocks, along with the MarkedBlock
state, into 8 bitvectors (live, empty, allocated, canAllocateButNotEmpty, eden, unswept,
markingNotEmpty, and markingRetired). The bitvectors separate allocator state (empty,
allocated, canAllocateButNotEmpty) from marking state (markingNotEmpty, markingRetired).
As a nice side-effect of switching to bitvectors, we get size class rebalancing for free.
It used to be that if a MarkedAllocator had an empty block, we would only allow that
memory to be reused by a different MarkedAllocator if we did an incremental sweep or a
full eager sweep. Now we hunt down all destructorless empty blocks before allocating new
MarkedBlocks. It would be relatively easy to also hunt down destructor empty blocks, but
the theory is that those might be expensive to sweep, so it might still be better to leave
those to the incremental sweeper.
This change is perf-neutral all around. I did some tests with two different kinds of
allocation strategies - something that is somewhat easier to do now that you can look for
blocks that are candidates for allocation by just scanning some bitvectors. I tried two
variants:
- Allocate out of non-empty blocks first, leaving empty blocks for last in case a
different allocator needed them. This is sort of a best-fit strategy. I tried this
first, and it can be expressed as:
m_allocationCursor = m_canAllocateButNotEmpty.findBit(m_allocationCursor, true)
- Allocate out of lower-indexed blocks first, treating empty and canAllocateButNotEmpty
blocks equally. This is sort of a first-fit strategy. This is what I ended up settling
on, and it can be expressed as:
m_allocationCursor = (m_canAllocateButNotEmpty | m_empty).findBit(m_allocationCursor, true)
The best-fit strategy meant 1% regressions in LongSpider and Octane overall, and a 11%
regression on Octane/earley. First-fit means perf-neutrality. Most great allocators skew
towards first-fit because it's empirically better, so this result is not surprising.
Overall, the performance of this patch on my machine is as follows, where "neutral" means
less than 1% and not statistically significant.
run-jsc-benchmarks:
SunSpider: neutral
LongSpider: 0.6% slower
V8Spider: neutral
Octane: neutral
Kraken: neutral
Microbenchmarks: 0.37% slower
AsmBench: neutral
CompressionBench: maybe 1% faster
For browser benchmarks, I report the ratio of means (bigger / smaller) along with a T-test
from Mathematica reported as % chance of not [sic] the null hypothesis. Note that we
normally consider anything less than 95% confidence to be inconclusive.
Browser benchmarks:
PLT3: 0.3% faster with 67% confidence
membuster:
Snap2FinishedLoadingPost: 0.68% more memory with 50% confidence
Snap3EndPost: 2.4% more memory with 61% confidence
JetStream: 0.2% slower with 32% confidence
Speedometer: 0.7% faster with 82% confidence
Additionally, Octane/splay's heap capacity goes down to ~180KB from ~200KB, so about a 10%
progression. This is due to the allocator rebalancing feature.
Finally, this breaks --useImmortalObjects. It was already broken as far as I can tell. I
filed a bug to reimplement it (bug 162296). Unless someone urgently needs this internal
tool, it's probably best to reimplement it after I'm done refactoring MarkedSpace.
* JavaScriptCore.xcodeproj/project.pbxproj:
* debugger/Debugger.cpp:
* heap/CellContainer.h:
* heap/CellContainerInlines.h:
(JSC::CellContainer::vm):
(JSC::CellContainer::heap):
(JSC::CellContainer::isMarkedOrNewlyAllocated):
(JSC::CellContainer::aboutToMark):
(JSC::CellContainer::isMarked): Deleted.
(JSC::CellContainer::flipIfNecessary): Deleted.
* heap/ConservativeRoots.cpp:
* heap/Heap.cpp:
(JSC::Heap::beginMarking):
(JSC::Heap::endMarking):
(JSC::Heap::collectAllGarbage):
(JSC::Heap::collectImpl):
(JSC::Heap::snapshotMarkedSpace):
(JSC::Heap::prepareForAllocation):
(JSC::Heap::zombifyDeadObjects):
(JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): Deleted.
(JSC::MarkedBlockSnapshotFunctor::operator()): Deleted.
(JSC::Heap::resetAllocators): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isMarked):
(JSC::Heap::isMarkedConcurrently):
(JSC::Heap::testAndSetMarked):
* heap/HeapStatistics.cpp:
* heap/HeapUtil.h:
(JSC::HeapUtil::findGCObjectPointersForMarking):
(JSC::HeapUtil::isPointerGCObjectJSCell):
* heap/HeapVerifier.cpp:
* heap/IncrementalSweeper.cpp:
(JSC::IncrementalSweeper::IncrementalSweeper):
(JSC::IncrementalSweeper::doSweep):
(JSC::IncrementalSweeper::sweepNextBlock):
(JSC::IncrementalSweeper::startSweeping):
(JSC::IncrementalSweeper::willFinishSweeping):
* heap/IncrementalSweeper.h:
* heap/LargeAllocation.h:
(JSC::LargeAllocation::isMarked):
(JSC::LargeAllocation::isMarkedConcurrently):
(JSC::LargeAllocation::isMarkedOrNewlyAllocated):
(JSC::LargeAllocation::aboutToMark):
(JSC::LargeAllocation::isMarkedDuringWeakVisiting): Deleted.
(JSC::LargeAllocation::flipIfNecessary): Deleted.
(JSC::LargeAllocation::flipIfNecessaryDuringMarking): Deleted.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::isPagedOut):
(JSC::MarkedAllocator::findEmptyBlock):
(JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
(JSC::MarkedAllocator::allocateIn):
(JSC::MarkedAllocator::tryAllocateIn):
(JSC::MarkedAllocator::allocateSlowCaseImpl):
(JSC::MarkedAllocator::tryAllocateBlock):
(JSC::MarkedAllocator::addBlock):
(JSC::MarkedAllocator::removeBlock):
(JSC::MarkedAllocator::stopAllocating):
(JSC::MarkedAllocator::prepareForAllocation):
(JSC::MarkedAllocator::lastChanceToFinalize):
(JSC::MarkedAllocator::resumeAllocating):
(JSC::MarkedAllocator::beginMarkingForFullCollection):
(JSC::MarkedAllocator::endMarking):
(JSC::MarkedAllocator::snapshotForEdenCollection):
(JSC::MarkedAllocator::snapshotForFullCollection):
(JSC::MarkedAllocator::findBlockToSweep):
(JSC::MarkedAllocator::sweep):
(JSC::MarkedAllocator::shrink):
(JSC::MarkedAllocator::assertSnapshotEmpty):
(JSC::MarkedAllocator::dump):
(JSC::MarkedAllocator::dumpBits):
(JSC::MarkedAllocator::retire): Deleted.
(JSC::MarkedAllocator::filterNextBlock): Deleted.
(JSC::MarkedAllocator::setNextBlockToSweep): Deleted.
(JSC::MarkedAllocator::reset): Deleted.
* heap/MarkedAllocator.h:
(JSC::MarkedAllocator::forEachBitVector):
(JSC::MarkedAllocator::forEachBitVectorWithName):
(JSC::MarkedAllocator::nextAllocator):
(JSC::MarkedAllocator::setNextAllocator):
(JSC::MarkedAllocator::forEachBlock):
(JSC::MarkedAllocator::resumeAllocating): Deleted.
* heap/MarkedBlock.cpp:
(JSC::MarkedBlock::tryCreate):
(JSC::MarkedBlock::Handle::Handle):
(JSC::MarkedBlock::Handle::~Handle):
(JSC::MarkedBlock::MarkedBlock):
(JSC::MarkedBlock::Handle::specializedSweep):
(JSC::MarkedBlock::Handle::sweep):
(JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectEmptyMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectHasNewlyAllocated):
(JSC::MarkedBlock::Handle::sweepHelperSelectSweepMode):
(JSC::MarkedBlock::Handle::sweepHelperSelectFlipMode):
(JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated):
(JSC::MarkedBlock::Handle::setIsFreeListed):
(JSC::MarkedBlock::Handle::stopAllocating):
(JSC::MarkedBlock::Handle::lastChanceToFinalize):
(JSC::MarkedBlock::Handle::resumeAllocating):
(JSC::MarkedBlock::aboutToMarkSlow):
(JSC::MarkedBlock::clearMarks):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::didConsumeFreeList):
(JSC::MarkedBlock::markCount):
(JSC::MarkedBlock::Handle::isEmpty):
(JSC::MarkedBlock::noteMarkedSlow):
(JSC::MarkedBlock::Handle::removeFromAllocator):
(JSC::MarkedBlock::Handle::didAddToAllocator):
(JSC::MarkedBlock::Handle::didRemoveFromAllocator):
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessarySlow): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarkingSlow): Deleted.
(JSC::MarkedBlock::Handle::willRemoveBlock): Deleted.
(WTF::printInternal): Deleted.
* heap/MarkedBlock.h:
(JSC::MarkedBlock::Handle::isFreeListed):
(JSC::MarkedBlock::Handle::index):
(JSC::MarkedBlock::aboutToMark):
(JSC::MarkedBlock::isMarked):
(JSC::MarkedBlock::isMarkedConcurrently):
(JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::isMarkedOrNewlyAllocated):
(JSC::MarkedBlock::Handle::isOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): Deleted.
(JSC::MarkedBlock::Handle::state): Deleted.
(JSC::MarkedBlock::flipIfNecessary): Deleted.
(JSC::MarkedBlock::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessary): Deleted.
(JSC::MarkedBlock::Handle::flipIfNecessaryDuringMarking): Deleted.
(JSC::MarkedBlock::Handle::flipForEdenCollection): Deleted.
(JSC::MarkedBlock::isMarkedDuringWeakVisiting): Deleted.
(JSC::MarkedBlock::Handle::isLive): Deleted.
(JSC::MarkedBlock::Handle::isLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachLiveCell): Deleted.
(JSC::MarkedBlock::Handle::forEachDeadCell): Deleted.
(JSC::MarkedBlock::Handle::needsSweeping): Deleted.
(JSC::MarkedBlock::Handle::isAllocated): Deleted.
(JSC::MarkedBlock::Handle::isMarked): Deleted.
* heap/MarkedBlockInlines.h: Added.
(JSC::MarkedBlock::Handle::isLive):
(JSC::MarkedBlock::Handle::isLiveCell):
(JSC::MarkedBlock::Handle::forEachLiveCell):
(JSC::MarkedBlock::Handle::forEachDeadCell):
(JSC::MarkedBlock::resetVersion):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::MarkedSpace):
(JSC::MarkedSpace::allocate):
(JSC::MarkedSpace::tryAllocate):
(JSC::MarkedSpace::sweep):
(JSC::MarkedSpace::prepareForAllocation):
(JSC::MarkedSpace::shrink):
(JSC::MarkedSpace::clearNewlyAllocated):
(JSC::MarkedSpace::beginMarking):
(JSC::MarkedSpace::endMarking):
(JSC::MarkedSpace::didAllocateInBlock):
(JSC::MarkedSpace::findEmptyBlock):
(JSC::MarkedSpace::snapshot):
(JSC::MarkedSpace::assertSnapshotEmpty):
(JSC::MarkedSpace::dumpBits):
(JSC::MarkedSpace::zombifySweep): Deleted.
(JSC::MarkedSpace::resetAllocators): Deleted.
(JSC::VerifyMarked::operator()): Deleted.
(JSC::MarkedSpace::flip): Deleted.
* heap/MarkedSpace.h:
(JSC::MarkedSpace::nextVersion):
(JSC::MarkedSpace::firstAllocator):
(JSC::MarkedSpace::allocatorForEmptyAllocation):
(JSC::MarkedSpace::forEachAllocator):
(JSC::MarkedSpace::blocksWithNewObjects): Deleted.
(JSC::MarkedSpace::setIsMarking): Deleted.
(JSC::MarkedSpace::forEachLiveCell): Deleted.
(JSC::MarkedSpace::forEachDeadCell): Deleted.
* heap/MarkedSpaceInlines.h: Added.
(JSC::MarkedSpace::forEachLiveCell):
(JSC::MarkedSpace::forEachDeadCell):
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::setMarkedAndAppendToMarkStack):
(JSC::SlotVisitor::markAuxiliary):
(JSC::SlotVisitor::visitChildren):
* heap/Weak.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue):
(WTF::HashTraits<JSC::Weak<T>>::peek):
* heap/WeakBlock.cpp:
(JSC::WeakBlock::specializedVisit):
(JSC::WeakBlock::reap):
* heap/WeakInlines.h:
(WTF::HashTraits<JSC::Weak<T>>::emptyValue): Deleted.
(WTF::HashTraits<JSC::Weak<T>>::peek): Deleted.
* jit/JITThunks.h:
* runtime/JSGlobalObject.cpp:
* runtime/PrototypeMap.h:
* runtime/SamplingProfiler.cpp:
* runtime/WeakGCMap.h:
* tools/JSDollarVMPrototype.cpp:
Source/WTF:
The main change here is to bring back FastBitVector.cpp, so that I could outline some
large slow path functions. This also adds some utilities, like atomicSetAndCheck() and
isEmpty(). The GC uses these.
* WTF.xcodeproj/project.pbxproj:
* wtf/CMakeLists.txt:
* wtf/FastBitVector.cpp: Added.
(WTF::FastBitVectorWordOwner::setEqualsSlow):
(WTF::FastBitVectorWordOwner::resizeSlow):
* wtf/FastBitVector.h:
(WTF::FastBitVectorWordOwner::operator=):
(WTF::FastBitVectorWordOwner::resize):
(WTF::FastBitVectorImpl::isEmpty):
(WTF::FastBitVector::atomicSetAndCheck):
(WTF::FastBitVector::operator[]): Deleted.
Tools:
Remove the always-trigger-copy-phase configuration.
* Scripts/run-jsc-stress-tests:
Canonical link: https://commits.webkit.org/180306@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@206154 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-20 18:12:18 +00:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
}
|
2012-05-22 00:51:05 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
} // namespace WTF
|
|
|
|
|
2020-01-17 04:09:32 +00:00
|
|
|
using WTF::FastBitReference;
|
2012-05-22 00:51:05 +00:00
|
|
|
using WTF::FastBitVector;
|