haikuwebkit/Source/WTF/wtf/ScopedLambda.h

197 lines
7.0 KiB
C
Raw Permalink Normal View History

Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
/*
ScopedLambda should have a lifetime story that makes sense to the compiler https://bugs.webkit.org/show_bug.cgi?id=158118 Reviewed by Mark Lam. Source/WTF: Prior to this change, there were two lifetime bugs in ScopedLambda: - scopedLambda(Functor&&) would bind Functor to const lambda&, so the resulting ScopedLambdaFunctor would hold a reference to the original lambda. This would have surprising behavior; for example it meant that this code was wrong: auto l = scopedLambda<things>([&] ...); The solution is to have explicit copy/move versions of scopedLambda() rather than rely on perfect forwarding. - ScopedLambdaFunctor did not override its copy or move operations, so if the compiler did not RVO scopedLambda(), it would return a ScopedLambdaFunctor whose m_arg points to a dead temporary ScopedLambdaFunctor instance. The solution is to have explicit copy/move constructors and operators, which preserve the invariant that ScopedLambda::m_arg points to this. One nice side-effect of all of these constructors and operators being explicit is that we can rely on WTFMove's excellent assertions, which helped catch the first issue. This reverts ParkingLot to use ScopedLambda again. * wtf/ParkingLot.cpp: (WTF::ParkingLot::parkConditionallyImpl): (WTF::ParkingLot::unparkOne): (WTF::ParkingLot::unparkOneImpl): * wtf/ParkingLot.h: (WTF::ParkingLot::parkConditionally): (WTF::ParkingLot::unparkOne): * wtf/ScopedLambda.h: (WTF::scopedLambda): Tools: Added a test case. This test crashes before the fix and now it passes. * TestWebKitAPI/CMakeLists.txt: * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj: * TestWebKitAPI/Tests/WTF/ScopedLambda.cpp: Added. (TestWebKitAPI::TEST): Canonical link: https://commits.webkit.org/176236@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@201433 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-05-26 21:58:42 +00:00
* Copyright (C) 2015-2016 Apple Inc. All rights reserved.
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
Use pragma once in WTF https://bugs.webkit.org/show_bug.cgi?id=190527 Reviewed by Chris Dumez. Source/WTF: We also need to consistently include wtf headers from within wtf so we can build wtf without symbol redefinition errors from including the copy in Source and the copy in the build directory. * wtf/ASCIICType.h: * wtf/Assertions.cpp: * wtf/Assertions.h: * wtf/Atomics.h: * wtf/AutomaticThread.cpp: * wtf/AutomaticThread.h: * wtf/BackwardsGraph.h: * wtf/Bag.h: * wtf/BagToHashMap.h: * wtf/BitVector.cpp: * wtf/BitVector.h: * wtf/Bitmap.h: * wtf/BloomFilter.h: * wtf/Box.h: * wtf/BubbleSort.h: * wtf/BumpPointerAllocator.h: * wtf/ByteOrder.h: * wtf/CPUTime.cpp: * wtf/CallbackAggregator.h: * wtf/CheckedArithmetic.h: * wtf/CheckedBoolean.h: * wtf/ClockType.cpp: * wtf/ClockType.h: * wtf/CommaPrinter.h: * wtf/CompilationThread.cpp: * wtf/CompilationThread.h: * wtf/Compiler.h: * wtf/ConcurrentPtrHashSet.cpp: * wtf/ConcurrentVector.h: * wtf/Condition.h: * wtf/CountingLock.cpp: * wtf/CrossThreadTaskHandler.cpp: * wtf/CryptographicUtilities.cpp: * wtf/CryptographicUtilities.h: * wtf/CryptographicallyRandomNumber.cpp: * wtf/CryptographicallyRandomNumber.h: * wtf/CurrentTime.cpp: * wtf/DataLog.cpp: * wtf/DataLog.h: * wtf/DateMath.cpp: * wtf/DateMath.h: * wtf/DecimalNumber.cpp: * wtf/DecimalNumber.h: * wtf/Deque.h: * wtf/DisallowCType.h: * wtf/Dominators.h: * wtf/DoublyLinkedList.h: * wtf/FastBitVector.cpp: * wtf/FastMalloc.cpp: * wtf/FastMalloc.h: * wtf/FeatureDefines.h: * wtf/FilePrintStream.cpp: * wtf/FilePrintStream.h: * wtf/FlipBytes.h: * wtf/FunctionDispatcher.cpp: * wtf/FunctionDispatcher.h: * wtf/GetPtr.h: * wtf/Gigacage.cpp: * wtf/GlobalVersion.cpp: * wtf/GraphNodeWorklist.h: * wtf/GregorianDateTime.cpp: * wtf/GregorianDateTime.h: * wtf/HashFunctions.h: * wtf/HashMap.h: * wtf/HashMethod.h: * wtf/HashSet.h: * wtf/HashTable.cpp: * wtf/HashTraits.h: * wtf/Indenter.h: * wtf/IndexSparseSet.h: * wtf/InlineASM.h: * wtf/Insertion.h: * wtf/IteratorAdaptors.h: * wtf/IteratorRange.h: * wtf/JSONValues.cpp: * wtf/JSValueMalloc.cpp: * wtf/LEBDecoder.h: * wtf/Language.cpp: * wtf/ListDump.h: * wtf/Lock.cpp: * wtf/Lock.h: * wtf/LockAlgorithm.h: * wtf/LockedPrintStream.cpp: * wtf/Locker.h: * wtf/MD5.cpp: * wtf/MD5.h: * wtf/MainThread.cpp: * wtf/MainThread.h: * wtf/MallocPtr.h: * wtf/MathExtras.h: * wtf/MediaTime.cpp: * wtf/MediaTime.h: * wtf/MemoryPressureHandler.cpp: * wtf/MessageQueue.h: * wtf/MetaAllocator.cpp: * wtf/MetaAllocator.h: * wtf/MetaAllocatorHandle.h: * wtf/MonotonicTime.cpp: * wtf/MonotonicTime.h: * wtf/NakedPtr.h: * wtf/NoLock.h: * wtf/NoTailCalls.h: * wtf/Noncopyable.h: * wtf/NumberOfCores.cpp: * wtf/NumberOfCores.h: * wtf/OSAllocator.h: * wtf/OSAllocatorPosix.cpp: * wtf/OSRandomSource.cpp: * wtf/OSRandomSource.h: * wtf/ObjcRuntimeExtras.h: * wtf/OrderMaker.h: * wtf/PackedIntVector.h: * wtf/PageAllocation.h: * wtf/PageBlock.cpp: * wtf/PageBlock.h: * wtf/PageReservation.h: * wtf/ParallelHelperPool.cpp: * wtf/ParallelHelperPool.h: * wtf/ParallelJobs.h: * wtf/ParallelJobsLibdispatch.h: * wtf/ParallelVectorIterator.h: * wtf/ParkingLot.cpp: * wtf/ParkingLot.h: * wtf/Platform.h: * wtf/PointerComparison.h: * wtf/Poisoned.cpp: * wtf/PrintStream.cpp: * wtf/PrintStream.h: * wtf/ProcessID.h: * wtf/ProcessPrivilege.cpp: * wtf/RAMSize.cpp: * wtf/RAMSize.h: * wtf/RandomDevice.cpp: * wtf/RandomNumber.cpp: * wtf/RandomNumber.h: * wtf/RandomNumberSeed.h: * wtf/RangeSet.h: * wtf/RawPointer.h: * wtf/ReadWriteLock.cpp: * wtf/RedBlackTree.h: * wtf/Ref.h: * wtf/RefCountedArray.h: * wtf/RefCountedLeakCounter.cpp: * wtf/RefCountedLeakCounter.h: * wtf/RefCounter.h: * wtf/RefPtr.h: * wtf/RetainPtr.h: * wtf/RunLoop.cpp: * wtf/RunLoop.h: * wtf/RunLoopTimer.h: * wtf/RunLoopTimerCF.cpp: * wtf/SHA1.cpp: * wtf/SHA1.h: * wtf/SaturatedArithmetic.h: (saturatedSubtraction): * wtf/SchedulePair.h: * wtf/SchedulePairCF.cpp: * wtf/SchedulePairMac.mm: * wtf/ScopedLambda.h: * wtf/Seconds.cpp: * wtf/Seconds.h: * wtf/SegmentedVector.h: * wtf/SentinelLinkedList.h: * wtf/SharedTask.h: * wtf/SimpleStats.h: * wtf/SingleRootGraph.h: * wtf/SinglyLinkedList.h: * wtf/SixCharacterHash.cpp: * wtf/SixCharacterHash.h: * wtf/SmallPtrSet.h: * wtf/Spectrum.h: * wtf/StackBounds.cpp: * wtf/StackBounds.h: * wtf/StackStats.cpp: * wtf/StackStats.h: * wtf/StackTrace.cpp: * wtf/StdLibExtras.h: * wtf/StreamBuffer.h: * wtf/StringHashDumpContext.h: * wtf/StringPrintStream.cpp: * wtf/StringPrintStream.h: * wtf/ThreadGroup.cpp: * wtf/ThreadMessage.cpp: * wtf/ThreadSpecific.h: * wtf/Threading.cpp: * wtf/Threading.h: * wtf/ThreadingPrimitives.h: * wtf/ThreadingPthreads.cpp: * wtf/TimeWithDynamicClockType.cpp: * wtf/TimeWithDynamicClockType.h: * wtf/TimingScope.cpp: * wtf/TinyLRUCache.h: * wtf/TinyPtrSet.h: * wtf/TriState.h: * wtf/TypeCasts.h: * wtf/UUID.cpp: * wtf/UnionFind.h: * wtf/VMTags.h: * wtf/ValueCheck.h: * wtf/Vector.h: * wtf/VectorTraits.h: * wtf/WallTime.cpp: * wtf/WallTime.h: * wtf/WeakPtr.h: * wtf/WeakRandom.h: * wtf/WordLock.cpp: * wtf/WordLock.h: * wtf/WorkQueue.cpp: * wtf/WorkQueue.h: * wtf/WorkerPool.cpp: * wtf/cf/LanguageCF.cpp: * wtf/cf/RunLoopCF.cpp: * wtf/cocoa/Entitlements.mm: * wtf/cocoa/MachSendRight.cpp: * wtf/cocoa/MainThreadCocoa.mm: * wtf/cocoa/MemoryFootprintCocoa.cpp: * wtf/cocoa/WorkQueueCocoa.cpp: * wtf/dtoa.cpp: * wtf/dtoa.h: * wtf/ios/WebCoreThread.cpp: * wtf/ios/WebCoreThread.h: * wtf/mac/AppKitCompatibilityDeclarations.h: * wtf/mac/DeprecatedSymbolsUsedBySafari.mm: * wtf/mbmalloc.cpp: * wtf/persistence/PersistentCoders.cpp: * wtf/persistence/PersistentDecoder.cpp: * wtf/persistence/PersistentEncoder.cpp: * wtf/spi/cf/CFBundleSPI.h: * wtf/spi/darwin/CommonCryptoSPI.h: * wtf/text/ASCIIFastPath.h: * wtf/text/ASCIILiteral.cpp: * wtf/text/AtomicString.cpp: * wtf/text/AtomicString.h: * wtf/text/AtomicStringHash.h: * wtf/text/AtomicStringImpl.cpp: * wtf/text/AtomicStringImpl.h: * wtf/text/AtomicStringTable.cpp: * wtf/text/AtomicStringTable.h: * wtf/text/Base64.cpp: * wtf/text/CString.cpp: * wtf/text/CString.h: * wtf/text/ConversionMode.h: * wtf/text/ExternalStringImpl.cpp: * wtf/text/IntegerToStringConversion.h: * wtf/text/LChar.h: * wtf/text/LineEnding.cpp: * wtf/text/StringBuffer.h: * wtf/text/StringBuilder.cpp: * wtf/text/StringBuilder.h: * wtf/text/StringBuilderJSON.cpp: * wtf/text/StringCommon.h: * wtf/text/StringConcatenate.h: * wtf/text/StringHash.h: * wtf/text/StringImpl.cpp: * wtf/text/StringImpl.h: * wtf/text/StringOperators.h: * wtf/text/StringView.cpp: * wtf/text/StringView.h: * wtf/text/SymbolImpl.cpp: * wtf/text/SymbolRegistry.cpp: * wtf/text/SymbolRegistry.h: * wtf/text/TextBreakIterator.cpp: * wtf/text/TextBreakIterator.h: * wtf/text/TextBreakIteratorInternalICU.h: * wtf/text/TextPosition.h: * wtf/text/TextStream.cpp: * wtf/text/UniquedStringImpl.h: * wtf/text/WTFString.cpp: * wtf/text/WTFString.h: * wtf/text/cocoa/StringCocoa.mm: * wtf/text/cocoa/StringViewCocoa.mm: * wtf/text/cocoa/TextBreakIteratorInternalICUCocoa.cpp: * wtf/text/icu/UTextProvider.cpp: * wtf/text/icu/UTextProvider.h: * wtf/text/icu/UTextProviderLatin1.cpp: * wtf/text/icu/UTextProviderLatin1.h: * wtf/text/icu/UTextProviderUTF16.cpp: * wtf/text/icu/UTextProviderUTF16.h: * wtf/threads/BinarySemaphore.cpp: * wtf/threads/BinarySemaphore.h: * wtf/threads/Signals.cpp: * wtf/unicode/CharacterNames.h: * wtf/unicode/Collator.h: * wtf/unicode/CollatorDefault.cpp: * wtf/unicode/UTF8.cpp: * wtf/unicode/UTF8.h: Tools: Put WorkQueue in namespace DRT so it does not conflict with WTF::WorkQueue. * DumpRenderTree/TestRunner.cpp: (TestRunner::queueLoadHTMLString): (TestRunner::queueLoadAlternateHTMLString): (TestRunner::queueBackNavigation): (TestRunner::queueForwardNavigation): (TestRunner::queueLoadingScript): (TestRunner::queueNonLoadingScript): (TestRunner::queueReload): * DumpRenderTree/WorkQueue.cpp: (WorkQueue::singleton): Deleted. (WorkQueue::WorkQueue): Deleted. (WorkQueue::queue): Deleted. (WorkQueue::dequeue): Deleted. (WorkQueue::count): Deleted. (WorkQueue::clear): Deleted. (WorkQueue::processWork): Deleted. * DumpRenderTree/WorkQueue.h: (WorkQueue::setFrozen): Deleted. * DumpRenderTree/WorkQueueItem.h: * DumpRenderTree/mac/DumpRenderTree.mm: (runTest): * DumpRenderTree/mac/FrameLoadDelegate.mm: (-[FrameLoadDelegate processWork:]): (-[FrameLoadDelegate webView:locationChangeDone:forDataSource:]): * DumpRenderTree/mac/TestRunnerMac.mm: (TestRunner::notifyDone): (TestRunner::forceImmediateCompletion): (TestRunner::queueLoad): * DumpRenderTree/win/DumpRenderTree.cpp: (runTest): * DumpRenderTree/win/FrameLoadDelegate.cpp: (FrameLoadDelegate::processWork): (FrameLoadDelegate::locationChangeDone): * DumpRenderTree/win/TestRunnerWin.cpp: (TestRunner::notifyDone): (TestRunner::forceImmediateCompletion): (TestRunner::queueLoad): Canonical link: https://commits.webkit.org/205473@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@237099 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-10-15 14:24:49 +00:00
#pragma once
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
[JSC] Use Function / ScopedLambda / RecursableLambda instead of std::function https://bugs.webkit.org/show_bug.cgi?id=187472 Reviewed by Mark Lam. Source/JavaScriptCore: std::function allocates memory from standard malloc instead of bmalloc. Instead of using that, we should use WTF::{Function,ScopedLambda,RecursableLambda}. This patch attempts to replace std::function with the above WTF function types. If the function's lifetime can be the same to the stack, we can use ScopedLambda, which is really efficient. Otherwise, we should use WTF::Function. For recurring use cases, we can use RecursableLambda. * assembler/MacroAssembler.cpp: (JSC::stdFunctionCallback): (JSC::MacroAssembler::probe): * assembler/MacroAssembler.h: * b3/air/AirDisassembler.cpp: (JSC::B3::Air::Disassembler::dump): * b3/air/AirDisassembler.h: * bytecompiler/BytecodeGenerator.cpp: (JSC::BytecodeGenerator::BytecodeGenerator): (JSC::BytecodeGenerator::initializeDefaultParameterValuesAndSetupFunctionScopeStack): (JSC::BytecodeGenerator::emitEnumeration): * bytecompiler/BytecodeGenerator.h: * bytecompiler/NodesCodegen.cpp: (JSC::ArrayNode::emitBytecode): (JSC::ApplyFunctionCallDotNode::emitBytecode): (JSC::ForOfNode::emitBytecode): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::addSlowPathGeneratorLambda): (JSC::DFG::SpeculativeJIT::compileMathIC): * dfg/DFGSpeculativeJIT.h: * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGValidate.cpp: * ftl/FTLCompile.cpp: (JSC::FTL::compile): * heap/HeapSnapshotBuilder.cpp: (JSC::HeapSnapshotBuilder::json): * heap/HeapSnapshotBuilder.h: * interpreter/StackVisitor.cpp: (JSC::StackVisitor::Frame::dump const): * interpreter/StackVisitor.h: * runtime/PromiseDeferredTimer.h: * runtime/VM.cpp: (JSC::VM::whenIdle): (JSC::enableProfilerWithRespectToCount): (JSC::disableProfilerWithRespectToCount): * runtime/VM.h: * runtime/VMEntryScope.cpp: (JSC::VMEntryScope::addDidPopListener): * runtime/VMEntryScope.h: * tools/HeapVerifier.cpp: (JSC::HeapVerifier::verifyCellList): (JSC::HeapVerifier::validateCell): (JSC::HeapVerifier::validateJSCell): * tools/HeapVerifier.h: Source/WTF: * wtf/ScopedLambda.h: (WTF::ScopedLambda<ResultType): Canonical link: https://commits.webkit.org/203065@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234082 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-21 13:10:43 +00:00
#include <wtf/ForbidHeapAllocation.h>
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
namespace WTF {
// You can use ScopedLambda to efficiently pass lambdas without allocating memory or requiring
// template specialization of the callee. The callee should be declared as:
//
// void foo(const ScopedLambda<MyThings* (int, Stuff&)>&);
//
// The caller just does:
//
// void foo(scopedLambda<MyThings* (int, Stuff&)>([&] (int x, Stuff& y) -> MyThings* { blah }));
//
// Note that this relies on foo() not escaping the lambda. The lambda is only valid while foo() is
// on the stack - hence the name ScopedLambda.
template<typename FunctionType> class ScopedLambda;
template<typename ResultType, typename... ArgumentTypes>
class ScopedLambda<ResultType (ArgumentTypes...)> {
[JSC] Use Function / ScopedLambda / RecursableLambda instead of std::function https://bugs.webkit.org/show_bug.cgi?id=187472 Reviewed by Mark Lam. Source/JavaScriptCore: std::function allocates memory from standard malloc instead of bmalloc. Instead of using that, we should use WTF::{Function,ScopedLambda,RecursableLambda}. This patch attempts to replace std::function with the above WTF function types. If the function's lifetime can be the same to the stack, we can use ScopedLambda, which is really efficient. Otherwise, we should use WTF::Function. For recurring use cases, we can use RecursableLambda. * assembler/MacroAssembler.cpp: (JSC::stdFunctionCallback): (JSC::MacroAssembler::probe): * assembler/MacroAssembler.h: * b3/air/AirDisassembler.cpp: (JSC::B3::Air::Disassembler::dump): * b3/air/AirDisassembler.h: * bytecompiler/BytecodeGenerator.cpp: (JSC::BytecodeGenerator::BytecodeGenerator): (JSC::BytecodeGenerator::initializeDefaultParameterValuesAndSetupFunctionScopeStack): (JSC::BytecodeGenerator::emitEnumeration): * bytecompiler/BytecodeGenerator.h: * bytecompiler/NodesCodegen.cpp: (JSC::ArrayNode::emitBytecode): (JSC::ApplyFunctionCallDotNode::emitBytecode): (JSC::ForOfNode::emitBytecode): * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::addSlowPathGeneratorLambda): (JSC::DFG::SpeculativeJIT::compileMathIC): * dfg/DFGSpeculativeJIT.h: * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::compile): * dfg/DFGValidate.cpp: * ftl/FTLCompile.cpp: (JSC::FTL::compile): * heap/HeapSnapshotBuilder.cpp: (JSC::HeapSnapshotBuilder::json): * heap/HeapSnapshotBuilder.h: * interpreter/StackVisitor.cpp: (JSC::StackVisitor::Frame::dump const): * interpreter/StackVisitor.h: * runtime/PromiseDeferredTimer.h: * runtime/VM.cpp: (JSC::VM::whenIdle): (JSC::enableProfilerWithRespectToCount): (JSC::disableProfilerWithRespectToCount): * runtime/VM.h: * runtime/VMEntryScope.cpp: (JSC::VMEntryScope::addDidPopListener): * runtime/VMEntryScope.h: * tools/HeapVerifier.cpp: (JSC::HeapVerifier::verifyCellList): (JSC::HeapVerifier::validateCell): (JSC::HeapVerifier::validateJSCell): * tools/HeapVerifier.h: Source/WTF: * wtf/ScopedLambda.h: (WTF::ScopedLambda<ResultType): Canonical link: https://commits.webkit.org/203065@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@234082 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2018-07-21 13:10:43 +00:00
WTF_FORBID_HEAP_ALLOCATION;
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
public:
CheckAdd/Mul should have commutativity optimizations in B3->Air lowering https://bugs.webkit.org/show_bug.cgi?id=151214 Reviewed by Geoffrey Garen. Source/JavaScriptCore: This is an overhaul of CheckAdd/CheckSub/CheckMul that fixes bugs, improves codegen, and simplifies the contract between B3 and its client. Previously, the idea was that the operands to the Air BranchAdd/Sub/Mul matched the children of the B3 CheckAdd/Sub/Mul, or at least, that's what the B3::CheckSpecial would make you believe. This meant that B3/Air had to avoid optimizations that would break this protocol. This prevented commutativity optimizations on CheckAdd/Mul and it also prevented strength reduction from CheckMul(x, 2) to CheckAdd(x, x), for example. Those optimizations would break things because the client's Stackmap generation callback was allowed to assume that the first entry in params.reps was the first child. Also, there was a contract between B3 and its client that for CheckAdd/Sub, the client would undo the operation by doing the opposite operation with the params.reps[0] as the source and params.reps[1] as the destination. This not only prevented commutativity optimizations, it also led to bugs. Here are two bugs that we had: - Add(x, x) would not work. The client would be told to undo the add using %x as both the source and destination. The client would use a sub() instruction. The result would not be x - it would be zero. - Mul where the result got coalesced with one of the sources. You can't undo a multiplication, so you need to keep the inputs alive until after the result is computed - i.e. the inputs are late uses. I initially thought I worked around this by using a three-operand form of Mul, but of course that doesn't actually fix the issue. This patch fixes these issues comprehensively by introducing the following changes: The params.reps corresponding to the first two children of CheckAdd/Sub/Mul and the first child of Check are always garbage: if you want to know the values of those children in the slow path, pass them as additional stackmap children. This makes it clear to the compiler whose values you actually care about, and frees the compiler to reorder and commute the non-stackmap children. The "undo" of an Add or Sub is handled internally by B3: the client doesn't have to know anything about undoing. B3 does it. The contract is simply that if a B3::Value* is a stackmap child of a CheckXYZ, then the corresponding entry in params.reps inside the stackmap generator callback will contain the value of that Value*. For Add and Sub, B3 undoes the operation. For Add(x, x), the undo uses the carry flag and some shift magic. For Mul, B3 uses LateUse: A new kind of Arg::Role called Arg::LateUse is introduced: This kind of use means that the use happens at the start of the execution of the next instruction. None of the built-in Air opcodes use LateUse, but it is used by B3::CheckSpecial. We use it to implement CheckMul. Finally, this change fixes testComplex to actually create many live variables. This revealed a really dumb pathology in allocateStack(), and this patch fixes it. Basically, it's a bad idea to build interference graphs by repeatedly recreating the same cliques. * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::test32): (JSC::MacroAssemblerX86Common::setCarry): (JSC::MacroAssemblerX86Common::invert): * b3/B3CheckSpecial.cpp: (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::dump): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::operator==): (JSC::B3::CheckSpecial::Key::operator!=): (JSC::B3::CheckSpecial::Key::opcode): (JSC::B3::CheckSpecial::Key::numArgs): (JSC::B3::CheckSpecial::Key::stackmapRole): (JSC::B3::CheckSpecial::Key::hash): * b3/B3CheckValue.cpp: (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::convertToAdd): (JSC::B3::CheckValue::CheckValue): * b3/B3CheckValue.h: * b3/B3Const32Value.cpp: (JSC::B3::Const32Value::checkMulConstant): (JSC::B3::Const32Value::checkNegConstant): (JSC::B3::Const32Value::divConstant): * b3/B3Const32Value.h: * b3/B3Const64Value.cpp: (JSC::B3::Const64Value::checkMulConstant): (JSC::B3::Const64Value::checkNegConstant): (JSC::B3::Const64Value::divConstant): * b3/B3Const64Value.h: * b3/B3LowerToAir.cpp: (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::lower): * b3/B3Opcode.h: * b3/B3PatchpointSpecial.cpp: (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): * b3/B3ReduceStrength.cpp: * b3/B3StackmapSpecial.cpp: (JSC::B3::StackmapSpecial::forEachArgImpl): * b3/B3StackmapSpecial.h: * b3/B3StackmapValue.cpp: (JSC::B3::StackmapValue::append): (JSC::B3::StackmapValue::appendSomeRegister): (JSC::B3::StackmapValue::setConstrainedChild): * b3/B3StackmapValue.h: * b3/B3Validate.cpp: * b3/B3Value.cpp: (JSC::B3::Value::checkMulConstant): (JSC::B3::Value::checkNegConstant): (JSC::B3::Value::divConstant): * b3/B3Value.h: * b3/air/AirAllocateStack.cpp: (JSC::B3::Air::allocateStack): * b3/air/AirArg.cpp: (WTF::printInternal): * b3/air/AirArg.h: (JSC::B3::Air::Arg::isAnyUse): (JSC::B3::Air::Arg::isEarlyUse): (JSC::B3::Air::Arg::isLateUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::isUse): Deleted. * b3/air/AirGenerate.cpp: (JSC::B3::Air::generate): * b3/air/AirIteratedRegisterCoalescing.cpp: (JSC::B3::Air::IteratedRegisterCoalescingAllocator::build): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::allocate): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::hash): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::dump): (JSC::B3::Air::addSpillAndFillToProgram): (JSC::B3::Air::iteratedRegisterCoalescingOnType): (JSC::B3::Air::iteratedRegisterCoalescing): * b3/air/AirLiveness.h: (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: * b3/air/AirReportUsedRegisters.cpp: (JSC::B3::Air::reportUsedRegisters): * b3/air/AirSpillEverything.cpp: (JSC::B3::Air::spillEverything): * b3/testb3.cpp: (JSC::B3::testMulArg): (JSC::B3::testMulArgStore): (JSC::B3::testMulAddArg): (JSC::B3::testMulArgs): (JSC::B3::testComplex): (JSC::B3::testSimpleCheck): (JSC::B3::testCheckLessThan): (JSC::B3::testCheckMegaCombo): (JSC::B3::testCheckAddImm): (JSC::B3::testCheckAddImmCommute): (JSC::B3::testCheckAddImmSomeRegister): (JSC::B3::testCheckAdd): (JSC::B3::testCheckAdd64): (JSC::B3::testCheckSubImm): (JSC::B3::testCheckSubBadImm): (JSC::B3::testCheckSub): (JSC::B3::testCheckSub64): (JSC::B3::testCheckNeg): (JSC::B3::testCheckNeg64): (JSC::B3::testCheckMul): (JSC::B3::testCheckMulMemory): (JSC::B3::testCheckMul2): (JSC::B3::testCheckMul64): (JSC::B3::run): Source/WTF: Disable my failed attempts at perfect forwarding, since they were incorrect, and caused compile errors if you tried to pass an argument that bound to lvalue. This shouldn't affect performance of anything we care about, and performance tests seem to confirm that it's all good. * wtf/ScopedLambda.h: Canonical link: https://commits.webkit.org/169519@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@192540 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-11-17 22:31:40 +00:00
ScopedLambda(ResultType (*impl)(void* arg, ArgumentTypes...) = nullptr, void* arg = nullptr)
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
: m_impl(impl)
, m_arg(arg)
{
}
CheckAdd/Mul should have commutativity optimizations in B3->Air lowering https://bugs.webkit.org/show_bug.cgi?id=151214 Reviewed by Geoffrey Garen. Source/JavaScriptCore: This is an overhaul of CheckAdd/CheckSub/CheckMul that fixes bugs, improves codegen, and simplifies the contract between B3 and its client. Previously, the idea was that the operands to the Air BranchAdd/Sub/Mul matched the children of the B3 CheckAdd/Sub/Mul, or at least, that's what the B3::CheckSpecial would make you believe. This meant that B3/Air had to avoid optimizations that would break this protocol. This prevented commutativity optimizations on CheckAdd/Mul and it also prevented strength reduction from CheckMul(x, 2) to CheckAdd(x, x), for example. Those optimizations would break things because the client's Stackmap generation callback was allowed to assume that the first entry in params.reps was the first child. Also, there was a contract between B3 and its client that for CheckAdd/Sub, the client would undo the operation by doing the opposite operation with the params.reps[0] as the source and params.reps[1] as the destination. This not only prevented commutativity optimizations, it also led to bugs. Here are two bugs that we had: - Add(x, x) would not work. The client would be told to undo the add using %x as both the source and destination. The client would use a sub() instruction. The result would not be x - it would be zero. - Mul where the result got coalesced with one of the sources. You can't undo a multiplication, so you need to keep the inputs alive until after the result is computed - i.e. the inputs are late uses. I initially thought I worked around this by using a three-operand form of Mul, but of course that doesn't actually fix the issue. This patch fixes these issues comprehensively by introducing the following changes: The params.reps corresponding to the first two children of CheckAdd/Sub/Mul and the first child of Check are always garbage: if you want to know the values of those children in the slow path, pass them as additional stackmap children. This makes it clear to the compiler whose values you actually care about, and frees the compiler to reorder and commute the non-stackmap children. The "undo" of an Add or Sub is handled internally by B3: the client doesn't have to know anything about undoing. B3 does it. The contract is simply that if a B3::Value* is a stackmap child of a CheckXYZ, then the corresponding entry in params.reps inside the stackmap generator callback will contain the value of that Value*. For Add and Sub, B3 undoes the operation. For Add(x, x), the undo uses the carry flag and some shift magic. For Mul, B3 uses LateUse: A new kind of Arg::Role called Arg::LateUse is introduced: This kind of use means that the use happens at the start of the execution of the next instruction. None of the built-in Air opcodes use LateUse, but it is used by B3::CheckSpecial. We use it to implement CheckMul. Finally, this change fixes testComplex to actually create many live variables. This revealed a really dumb pathology in allocateStack(), and this patch fixes it. Basically, it's a bad idea to build interference graphs by repeatedly recreating the same cliques. * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::test32): (JSC::MacroAssemblerX86Common::setCarry): (JSC::MacroAssemblerX86Common::invert): * b3/B3CheckSpecial.cpp: (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::dump): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::operator==): (JSC::B3::CheckSpecial::Key::operator!=): (JSC::B3::CheckSpecial::Key::opcode): (JSC::B3::CheckSpecial::Key::numArgs): (JSC::B3::CheckSpecial::Key::stackmapRole): (JSC::B3::CheckSpecial::Key::hash): * b3/B3CheckValue.cpp: (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::convertToAdd): (JSC::B3::CheckValue::CheckValue): * b3/B3CheckValue.h: * b3/B3Const32Value.cpp: (JSC::B3::Const32Value::checkMulConstant): (JSC::B3::Const32Value::checkNegConstant): (JSC::B3::Const32Value::divConstant): * b3/B3Const32Value.h: * b3/B3Const64Value.cpp: (JSC::B3::Const64Value::checkMulConstant): (JSC::B3::Const64Value::checkNegConstant): (JSC::B3::Const64Value::divConstant): * b3/B3Const64Value.h: * b3/B3LowerToAir.cpp: (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::lower): * b3/B3Opcode.h: * b3/B3PatchpointSpecial.cpp: (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): * b3/B3ReduceStrength.cpp: * b3/B3StackmapSpecial.cpp: (JSC::B3::StackmapSpecial::forEachArgImpl): * b3/B3StackmapSpecial.h: * b3/B3StackmapValue.cpp: (JSC::B3::StackmapValue::append): (JSC::B3::StackmapValue::appendSomeRegister): (JSC::B3::StackmapValue::setConstrainedChild): * b3/B3StackmapValue.h: * b3/B3Validate.cpp: * b3/B3Value.cpp: (JSC::B3::Value::checkMulConstant): (JSC::B3::Value::checkNegConstant): (JSC::B3::Value::divConstant): * b3/B3Value.h: * b3/air/AirAllocateStack.cpp: (JSC::B3::Air::allocateStack): * b3/air/AirArg.cpp: (WTF::printInternal): * b3/air/AirArg.h: (JSC::B3::Air::Arg::isAnyUse): (JSC::B3::Air::Arg::isEarlyUse): (JSC::B3::Air::Arg::isLateUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::isUse): Deleted. * b3/air/AirGenerate.cpp: (JSC::B3::Air::generate): * b3/air/AirIteratedRegisterCoalescing.cpp: (JSC::B3::Air::IteratedRegisterCoalescingAllocator::build): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::allocate): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::hash): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::dump): (JSC::B3::Air::addSpillAndFillToProgram): (JSC::B3::Air::iteratedRegisterCoalescingOnType): (JSC::B3::Air::iteratedRegisterCoalescing): * b3/air/AirLiveness.h: (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: * b3/air/AirReportUsedRegisters.cpp: (JSC::B3::Air::reportUsedRegisters): * b3/air/AirSpillEverything.cpp: (JSC::B3::Air::spillEverything): * b3/testb3.cpp: (JSC::B3::testMulArg): (JSC::B3::testMulArgStore): (JSC::B3::testMulAddArg): (JSC::B3::testMulArgs): (JSC::B3::testComplex): (JSC::B3::testSimpleCheck): (JSC::B3::testCheckLessThan): (JSC::B3::testCheckMegaCombo): (JSC::B3::testCheckAddImm): (JSC::B3::testCheckAddImmCommute): (JSC::B3::testCheckAddImmSomeRegister): (JSC::B3::testCheckAdd): (JSC::B3::testCheckAdd64): (JSC::B3::testCheckSubImm): (JSC::B3::testCheckSubBadImm): (JSC::B3::testCheckSub): (JSC::B3::testCheckSub64): (JSC::B3::testCheckNeg): (JSC::B3::testCheckNeg64): (JSC::B3::testCheckMul): (JSC::B3::testCheckMulMemory): (JSC::B3::testCheckMul2): (JSC::B3::testCheckMul64): (JSC::B3::run): Source/WTF: Disable my failed attempts at perfect forwarding, since they were incorrect, and caused compile errors if you tried to pass an argument that bound to lvalue. This shouldn't affect performance of anything we care about, and performance tests seem to confirm that it's all good. * wtf/ScopedLambda.h: Canonical link: https://commits.webkit.org/169519@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@192540 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-11-17 22:31:40 +00:00
template<typename... PassedArgumentTypes>
ResultType operator()(PassedArgumentTypes&&... arguments) const
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
{
CheckAdd/Mul should have commutativity optimizations in B3->Air lowering https://bugs.webkit.org/show_bug.cgi?id=151214 Reviewed by Geoffrey Garen. Source/JavaScriptCore: This is an overhaul of CheckAdd/CheckSub/CheckMul that fixes bugs, improves codegen, and simplifies the contract between B3 and its client. Previously, the idea was that the operands to the Air BranchAdd/Sub/Mul matched the children of the B3 CheckAdd/Sub/Mul, or at least, that's what the B3::CheckSpecial would make you believe. This meant that B3/Air had to avoid optimizations that would break this protocol. This prevented commutativity optimizations on CheckAdd/Mul and it also prevented strength reduction from CheckMul(x, 2) to CheckAdd(x, x), for example. Those optimizations would break things because the client's Stackmap generation callback was allowed to assume that the first entry in params.reps was the first child. Also, there was a contract between B3 and its client that for CheckAdd/Sub, the client would undo the operation by doing the opposite operation with the params.reps[0] as the source and params.reps[1] as the destination. This not only prevented commutativity optimizations, it also led to bugs. Here are two bugs that we had: - Add(x, x) would not work. The client would be told to undo the add using %x as both the source and destination. The client would use a sub() instruction. The result would not be x - it would be zero. - Mul where the result got coalesced with one of the sources. You can't undo a multiplication, so you need to keep the inputs alive until after the result is computed - i.e. the inputs are late uses. I initially thought I worked around this by using a three-operand form of Mul, but of course that doesn't actually fix the issue. This patch fixes these issues comprehensively by introducing the following changes: The params.reps corresponding to the first two children of CheckAdd/Sub/Mul and the first child of Check are always garbage: if you want to know the values of those children in the slow path, pass them as additional stackmap children. This makes it clear to the compiler whose values you actually care about, and frees the compiler to reorder and commute the non-stackmap children. The "undo" of an Add or Sub is handled internally by B3: the client doesn't have to know anything about undoing. B3 does it. The contract is simply that if a B3::Value* is a stackmap child of a CheckXYZ, then the corresponding entry in params.reps inside the stackmap generator callback will contain the value of that Value*. For Add and Sub, B3 undoes the operation. For Add(x, x), the undo uses the carry flag and some shift magic. For Mul, B3 uses LateUse: A new kind of Arg::Role called Arg::LateUse is introduced: This kind of use means that the use happens at the start of the execution of the next instruction. None of the built-in Air opcodes use LateUse, but it is used by B3::CheckSpecial. We use it to implement CheckMul. Finally, this change fixes testComplex to actually create many live variables. This revealed a really dumb pathology in allocateStack(), and this patch fixes it. Basically, it's a bad idea to build interference graphs by repeatedly recreating the same cliques. * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::test32): (JSC::MacroAssemblerX86Common::setCarry): (JSC::MacroAssemblerX86Common::invert): * b3/B3CheckSpecial.cpp: (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::dump): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::operator==): (JSC::B3::CheckSpecial::Key::operator!=): (JSC::B3::CheckSpecial::Key::opcode): (JSC::B3::CheckSpecial::Key::numArgs): (JSC::B3::CheckSpecial::Key::stackmapRole): (JSC::B3::CheckSpecial::Key::hash): * b3/B3CheckValue.cpp: (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::convertToAdd): (JSC::B3::CheckValue::CheckValue): * b3/B3CheckValue.h: * b3/B3Const32Value.cpp: (JSC::B3::Const32Value::checkMulConstant): (JSC::B3::Const32Value::checkNegConstant): (JSC::B3::Const32Value::divConstant): * b3/B3Const32Value.h: * b3/B3Const64Value.cpp: (JSC::B3::Const64Value::checkMulConstant): (JSC::B3::Const64Value::checkNegConstant): (JSC::B3::Const64Value::divConstant): * b3/B3Const64Value.h: * b3/B3LowerToAir.cpp: (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::lower): * b3/B3Opcode.h: * b3/B3PatchpointSpecial.cpp: (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): * b3/B3ReduceStrength.cpp: * b3/B3StackmapSpecial.cpp: (JSC::B3::StackmapSpecial::forEachArgImpl): * b3/B3StackmapSpecial.h: * b3/B3StackmapValue.cpp: (JSC::B3::StackmapValue::append): (JSC::B3::StackmapValue::appendSomeRegister): (JSC::B3::StackmapValue::setConstrainedChild): * b3/B3StackmapValue.h: * b3/B3Validate.cpp: * b3/B3Value.cpp: (JSC::B3::Value::checkMulConstant): (JSC::B3::Value::checkNegConstant): (JSC::B3::Value::divConstant): * b3/B3Value.h: * b3/air/AirAllocateStack.cpp: (JSC::B3::Air::allocateStack): * b3/air/AirArg.cpp: (WTF::printInternal): * b3/air/AirArg.h: (JSC::B3::Air::Arg::isAnyUse): (JSC::B3::Air::Arg::isEarlyUse): (JSC::B3::Air::Arg::isLateUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::isUse): Deleted. * b3/air/AirGenerate.cpp: (JSC::B3::Air::generate): * b3/air/AirIteratedRegisterCoalescing.cpp: (JSC::B3::Air::IteratedRegisterCoalescingAllocator::build): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::allocate): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::hash): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::dump): (JSC::B3::Air::addSpillAndFillToProgram): (JSC::B3::Air::iteratedRegisterCoalescingOnType): (JSC::B3::Air::iteratedRegisterCoalescing): * b3/air/AirLiveness.h: (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: * b3/air/AirReportUsedRegisters.cpp: (JSC::B3::Air::reportUsedRegisters): * b3/air/AirSpillEverything.cpp: (JSC::B3::Air::spillEverything): * b3/testb3.cpp: (JSC::B3::testMulArg): (JSC::B3::testMulArgStore): (JSC::B3::testMulAddArg): (JSC::B3::testMulArgs): (JSC::B3::testComplex): (JSC::B3::testSimpleCheck): (JSC::B3::testCheckLessThan): (JSC::B3::testCheckMegaCombo): (JSC::B3::testCheckAddImm): (JSC::B3::testCheckAddImmCommute): (JSC::B3::testCheckAddImmSomeRegister): (JSC::B3::testCheckAdd): (JSC::B3::testCheckAdd64): (JSC::B3::testCheckSubImm): (JSC::B3::testCheckSubBadImm): (JSC::B3::testCheckSub): (JSC::B3::testCheckSub64): (JSC::B3::testCheckNeg): (JSC::B3::testCheckNeg64): (JSC::B3::testCheckMul): (JSC::B3::testCheckMulMemory): (JSC::B3::testCheckMul2): (JSC::B3::testCheckMul64): (JSC::B3::run): Source/WTF: Disable my failed attempts at perfect forwarding, since they were incorrect, and caused compile errors if you tried to pass an argument that bound to lvalue. This shouldn't affect performance of anything we care about, and performance tests seem to confirm that it's all good. * wtf/ScopedLambda.h: Canonical link: https://commits.webkit.org/169519@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@192540 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-11-17 22:31:40 +00:00
return m_impl(m_arg, std::forward<PassedArgumentTypes>(arguments)...);
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
}
private:
CheckAdd/Mul should have commutativity optimizations in B3->Air lowering https://bugs.webkit.org/show_bug.cgi?id=151214 Reviewed by Geoffrey Garen. Source/JavaScriptCore: This is an overhaul of CheckAdd/CheckSub/CheckMul that fixes bugs, improves codegen, and simplifies the contract between B3 and its client. Previously, the idea was that the operands to the Air BranchAdd/Sub/Mul matched the children of the B3 CheckAdd/Sub/Mul, or at least, that's what the B3::CheckSpecial would make you believe. This meant that B3/Air had to avoid optimizations that would break this protocol. This prevented commutativity optimizations on CheckAdd/Mul and it also prevented strength reduction from CheckMul(x, 2) to CheckAdd(x, x), for example. Those optimizations would break things because the client's Stackmap generation callback was allowed to assume that the first entry in params.reps was the first child. Also, there was a contract between B3 and its client that for CheckAdd/Sub, the client would undo the operation by doing the opposite operation with the params.reps[0] as the source and params.reps[1] as the destination. This not only prevented commutativity optimizations, it also led to bugs. Here are two bugs that we had: - Add(x, x) would not work. The client would be told to undo the add using %x as both the source and destination. The client would use a sub() instruction. The result would not be x - it would be zero. - Mul where the result got coalesced with one of the sources. You can't undo a multiplication, so you need to keep the inputs alive until after the result is computed - i.e. the inputs are late uses. I initially thought I worked around this by using a three-operand form of Mul, but of course that doesn't actually fix the issue. This patch fixes these issues comprehensively by introducing the following changes: The params.reps corresponding to the first two children of CheckAdd/Sub/Mul and the first child of Check are always garbage: if you want to know the values of those children in the slow path, pass them as additional stackmap children. This makes it clear to the compiler whose values you actually care about, and frees the compiler to reorder and commute the non-stackmap children. The "undo" of an Add or Sub is handled internally by B3: the client doesn't have to know anything about undoing. B3 does it. The contract is simply that if a B3::Value* is a stackmap child of a CheckXYZ, then the corresponding entry in params.reps inside the stackmap generator callback will contain the value of that Value*. For Add and Sub, B3 undoes the operation. For Add(x, x), the undo uses the carry flag and some shift magic. For Mul, B3 uses LateUse: A new kind of Arg::Role called Arg::LateUse is introduced: This kind of use means that the use happens at the start of the execution of the next instruction. None of the built-in Air opcodes use LateUse, but it is used by B3::CheckSpecial. We use it to implement CheckMul. Finally, this change fixes testComplex to actually create many live variables. This revealed a really dumb pathology in allocateStack(), and this patch fixes it. Basically, it's a bad idea to build interference graphs by repeatedly recreating the same cliques. * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::test32): (JSC::MacroAssemblerX86Common::setCarry): (JSC::MacroAssemblerX86Common::invert): * b3/B3CheckSpecial.cpp: (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::dump): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::operator==): (JSC::B3::CheckSpecial::Key::operator!=): (JSC::B3::CheckSpecial::Key::opcode): (JSC::B3::CheckSpecial::Key::numArgs): (JSC::B3::CheckSpecial::Key::stackmapRole): (JSC::B3::CheckSpecial::Key::hash): * b3/B3CheckValue.cpp: (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::convertToAdd): (JSC::B3::CheckValue::CheckValue): * b3/B3CheckValue.h: * b3/B3Const32Value.cpp: (JSC::B3::Const32Value::checkMulConstant): (JSC::B3::Const32Value::checkNegConstant): (JSC::B3::Const32Value::divConstant): * b3/B3Const32Value.h: * b3/B3Const64Value.cpp: (JSC::B3::Const64Value::checkMulConstant): (JSC::B3::Const64Value::checkNegConstant): (JSC::B3::Const64Value::divConstant): * b3/B3Const64Value.h: * b3/B3LowerToAir.cpp: (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::lower): * b3/B3Opcode.h: * b3/B3PatchpointSpecial.cpp: (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): * b3/B3ReduceStrength.cpp: * b3/B3StackmapSpecial.cpp: (JSC::B3::StackmapSpecial::forEachArgImpl): * b3/B3StackmapSpecial.h: * b3/B3StackmapValue.cpp: (JSC::B3::StackmapValue::append): (JSC::B3::StackmapValue::appendSomeRegister): (JSC::B3::StackmapValue::setConstrainedChild): * b3/B3StackmapValue.h: * b3/B3Validate.cpp: * b3/B3Value.cpp: (JSC::B3::Value::checkMulConstant): (JSC::B3::Value::checkNegConstant): (JSC::B3::Value::divConstant): * b3/B3Value.h: * b3/air/AirAllocateStack.cpp: (JSC::B3::Air::allocateStack): * b3/air/AirArg.cpp: (WTF::printInternal): * b3/air/AirArg.h: (JSC::B3::Air::Arg::isAnyUse): (JSC::B3::Air::Arg::isEarlyUse): (JSC::B3::Air::Arg::isLateUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::isUse): Deleted. * b3/air/AirGenerate.cpp: (JSC::B3::Air::generate): * b3/air/AirIteratedRegisterCoalescing.cpp: (JSC::B3::Air::IteratedRegisterCoalescingAllocator::build): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::allocate): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::hash): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::dump): (JSC::B3::Air::addSpillAndFillToProgram): (JSC::B3::Air::iteratedRegisterCoalescingOnType): (JSC::B3::Air::iteratedRegisterCoalescing): * b3/air/AirLiveness.h: (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: * b3/air/AirReportUsedRegisters.cpp: (JSC::B3::Air::reportUsedRegisters): * b3/air/AirSpillEverything.cpp: (JSC::B3::Air::spillEverything): * b3/testb3.cpp: (JSC::B3::testMulArg): (JSC::B3::testMulArgStore): (JSC::B3::testMulAddArg): (JSC::B3::testMulArgs): (JSC::B3::testComplex): (JSC::B3::testSimpleCheck): (JSC::B3::testCheckLessThan): (JSC::B3::testCheckMegaCombo): (JSC::B3::testCheckAddImm): (JSC::B3::testCheckAddImmCommute): (JSC::B3::testCheckAddImmSomeRegister): (JSC::B3::testCheckAdd): (JSC::B3::testCheckAdd64): (JSC::B3::testCheckSubImm): (JSC::B3::testCheckSubBadImm): (JSC::B3::testCheckSub): (JSC::B3::testCheckSub64): (JSC::B3::testCheckNeg): (JSC::B3::testCheckNeg64): (JSC::B3::testCheckMul): (JSC::B3::testCheckMulMemory): (JSC::B3::testCheckMul2): (JSC::B3::testCheckMul64): (JSC::B3::run): Source/WTF: Disable my failed attempts at perfect forwarding, since they were incorrect, and caused compile errors if you tried to pass an argument that bound to lvalue. This shouldn't affect performance of anything we care about, and performance tests seem to confirm that it's all good. * wtf/ScopedLambda.h: Canonical link: https://commits.webkit.org/169519@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@192540 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-11-17 22:31:40 +00:00
ResultType (*m_impl)(void* arg, ArgumentTypes...);
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
void *m_arg;
};
template<typename FunctionType, typename Functor> class ScopedLambdaFunctor;
template<typename ResultType, typename... ArgumentTypes, typename Functor>
class ScopedLambdaFunctor<ResultType (ArgumentTypes...), Functor> : public ScopedLambda<ResultType (ArgumentTypes...)> {
public:
template<typename PassedFunctor>
ScopedLambdaFunctor(PassedFunctor&& functor)
: ScopedLambda<ResultType (ArgumentTypes...)>(implFunction, this)
, m_functor(std::forward<PassedFunctor>(functor))
{
}
ScopedLambda should have a lifetime story that makes sense to the compiler https://bugs.webkit.org/show_bug.cgi?id=158118 Reviewed by Mark Lam. Source/WTF: Prior to this change, there were two lifetime bugs in ScopedLambda: - scopedLambda(Functor&&) would bind Functor to const lambda&, so the resulting ScopedLambdaFunctor would hold a reference to the original lambda. This would have surprising behavior; for example it meant that this code was wrong: auto l = scopedLambda<things>([&] ...); The solution is to have explicit copy/move versions of scopedLambda() rather than rely on perfect forwarding. - ScopedLambdaFunctor did not override its copy or move operations, so if the compiler did not RVO scopedLambda(), it would return a ScopedLambdaFunctor whose m_arg points to a dead temporary ScopedLambdaFunctor instance. The solution is to have explicit copy/move constructors and operators, which preserve the invariant that ScopedLambda::m_arg points to this. One nice side-effect of all of these constructors and operators being explicit is that we can rely on WTFMove's excellent assertions, which helped catch the first issue. This reverts ParkingLot to use ScopedLambda again. * wtf/ParkingLot.cpp: (WTF::ParkingLot::parkConditionallyImpl): (WTF::ParkingLot::unparkOne): (WTF::ParkingLot::unparkOneImpl): * wtf/ParkingLot.h: (WTF::ParkingLot::parkConditionally): (WTF::ParkingLot::unparkOne): * wtf/ScopedLambda.h: (WTF::scopedLambda): Tools: Added a test case. This test crashes before the fix and now it passes. * TestWebKitAPI/CMakeLists.txt: * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj: * TestWebKitAPI/Tests/WTF/ScopedLambda.cpp: Added. (TestWebKitAPI::TEST): Canonical link: https://commits.webkit.org/176236@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@201433 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-05-26 21:58:42 +00:00
// We need to make sure that copying and moving ScopedLambdaFunctor results in a ScopedLambdaFunctor
// whose ScopedLambda supertype still points to this rather than other.
ScopedLambdaFunctor(const ScopedLambdaFunctor& other)
: ScopedLambda<ResultType (ArgumentTypes...)>(implFunction, this)
, m_functor(other.m_functor)
{
}
ScopedLambdaFunctor(ScopedLambdaFunctor&& other)
: ScopedLambda<ResultType (ArgumentTypes...)>(implFunction, this)
, m_functor(WTFMove(other.m_functor))
{
}
ScopedLambdaFunctor& operator=(const ScopedLambdaFunctor& other)
{
m_functor = other.m_functor;
return *this;
}
ScopedLambdaFunctor& operator=(ScopedLambdaFunctor&& other)
{
m_functor = WTFMove(other.m_functor);
return *this;
}
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
private:
CheckAdd/Mul should have commutativity optimizations in B3->Air lowering https://bugs.webkit.org/show_bug.cgi?id=151214 Reviewed by Geoffrey Garen. Source/JavaScriptCore: This is an overhaul of CheckAdd/CheckSub/CheckMul that fixes bugs, improves codegen, and simplifies the contract between B3 and its client. Previously, the idea was that the operands to the Air BranchAdd/Sub/Mul matched the children of the B3 CheckAdd/Sub/Mul, or at least, that's what the B3::CheckSpecial would make you believe. This meant that B3/Air had to avoid optimizations that would break this protocol. This prevented commutativity optimizations on CheckAdd/Mul and it also prevented strength reduction from CheckMul(x, 2) to CheckAdd(x, x), for example. Those optimizations would break things because the client's Stackmap generation callback was allowed to assume that the first entry in params.reps was the first child. Also, there was a contract between B3 and its client that for CheckAdd/Sub, the client would undo the operation by doing the opposite operation with the params.reps[0] as the source and params.reps[1] as the destination. This not only prevented commutativity optimizations, it also led to bugs. Here are two bugs that we had: - Add(x, x) would not work. The client would be told to undo the add using %x as both the source and destination. The client would use a sub() instruction. The result would not be x - it would be zero. - Mul where the result got coalesced with one of the sources. You can't undo a multiplication, so you need to keep the inputs alive until after the result is computed - i.e. the inputs are late uses. I initially thought I worked around this by using a three-operand form of Mul, but of course that doesn't actually fix the issue. This patch fixes these issues comprehensively by introducing the following changes: The params.reps corresponding to the first two children of CheckAdd/Sub/Mul and the first child of Check are always garbage: if you want to know the values of those children in the slow path, pass them as additional stackmap children. This makes it clear to the compiler whose values you actually care about, and frees the compiler to reorder and commute the non-stackmap children. The "undo" of an Add or Sub is handled internally by B3: the client doesn't have to know anything about undoing. B3 does it. The contract is simply that if a B3::Value* is a stackmap child of a CheckXYZ, then the corresponding entry in params.reps inside the stackmap generator callback will contain the value of that Value*. For Add and Sub, B3 undoes the operation. For Add(x, x), the undo uses the carry flag and some shift magic. For Mul, B3 uses LateUse: A new kind of Arg::Role called Arg::LateUse is introduced: This kind of use means that the use happens at the start of the execution of the next instruction. None of the built-in Air opcodes use LateUse, but it is used by B3::CheckSpecial. We use it to implement CheckMul. Finally, this change fixes testComplex to actually create many live variables. This revealed a really dumb pathology in allocateStack(), and this patch fixes it. Basically, it's a bad idea to build interference graphs by repeatedly recreating the same cliques. * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::test32): (JSC::MacroAssemblerX86Common::setCarry): (JSC::MacroAssemblerX86Common::invert): * b3/B3CheckSpecial.cpp: (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::dump): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::operator==): (JSC::B3::CheckSpecial::Key::operator!=): (JSC::B3::CheckSpecial::Key::opcode): (JSC::B3::CheckSpecial::Key::numArgs): (JSC::B3::CheckSpecial::Key::stackmapRole): (JSC::B3::CheckSpecial::Key::hash): * b3/B3CheckValue.cpp: (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::convertToAdd): (JSC::B3::CheckValue::CheckValue): * b3/B3CheckValue.h: * b3/B3Const32Value.cpp: (JSC::B3::Const32Value::checkMulConstant): (JSC::B3::Const32Value::checkNegConstant): (JSC::B3::Const32Value::divConstant): * b3/B3Const32Value.h: * b3/B3Const64Value.cpp: (JSC::B3::Const64Value::checkMulConstant): (JSC::B3::Const64Value::checkNegConstant): (JSC::B3::Const64Value::divConstant): * b3/B3Const64Value.h: * b3/B3LowerToAir.cpp: (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::lower): * b3/B3Opcode.h: * b3/B3PatchpointSpecial.cpp: (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): * b3/B3ReduceStrength.cpp: * b3/B3StackmapSpecial.cpp: (JSC::B3::StackmapSpecial::forEachArgImpl): * b3/B3StackmapSpecial.h: * b3/B3StackmapValue.cpp: (JSC::B3::StackmapValue::append): (JSC::B3::StackmapValue::appendSomeRegister): (JSC::B3::StackmapValue::setConstrainedChild): * b3/B3StackmapValue.h: * b3/B3Validate.cpp: * b3/B3Value.cpp: (JSC::B3::Value::checkMulConstant): (JSC::B3::Value::checkNegConstant): (JSC::B3::Value::divConstant): * b3/B3Value.h: * b3/air/AirAllocateStack.cpp: (JSC::B3::Air::allocateStack): * b3/air/AirArg.cpp: (WTF::printInternal): * b3/air/AirArg.h: (JSC::B3::Air::Arg::isAnyUse): (JSC::B3::Air::Arg::isEarlyUse): (JSC::B3::Air::Arg::isLateUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::isUse): Deleted. * b3/air/AirGenerate.cpp: (JSC::B3::Air::generate): * b3/air/AirIteratedRegisterCoalescing.cpp: (JSC::B3::Air::IteratedRegisterCoalescingAllocator::build): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::allocate): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::hash): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::dump): (JSC::B3::Air::addSpillAndFillToProgram): (JSC::B3::Air::iteratedRegisterCoalescingOnType): (JSC::B3::Air::iteratedRegisterCoalescing): * b3/air/AirLiveness.h: (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: * b3/air/AirReportUsedRegisters.cpp: (JSC::B3::Air::reportUsedRegisters): * b3/air/AirSpillEverything.cpp: (JSC::B3::Air::spillEverything): * b3/testb3.cpp: (JSC::B3::testMulArg): (JSC::B3::testMulArgStore): (JSC::B3::testMulAddArg): (JSC::B3::testMulArgs): (JSC::B3::testComplex): (JSC::B3::testSimpleCheck): (JSC::B3::testCheckLessThan): (JSC::B3::testCheckMegaCombo): (JSC::B3::testCheckAddImm): (JSC::B3::testCheckAddImmCommute): (JSC::B3::testCheckAddImmSomeRegister): (JSC::B3::testCheckAdd): (JSC::B3::testCheckAdd64): (JSC::B3::testCheckSubImm): (JSC::B3::testCheckSubBadImm): (JSC::B3::testCheckSub): (JSC::B3::testCheckSub64): (JSC::B3::testCheckNeg): (JSC::B3::testCheckNeg64): (JSC::B3::testCheckMul): (JSC::B3::testCheckMulMemory): (JSC::B3::testCheckMul2): (JSC::B3::testCheckMul64): (JSC::B3::run): Source/WTF: Disable my failed attempts at perfect forwarding, since they were incorrect, and caused compile errors if you tried to pass an argument that bound to lvalue. This shouldn't affect performance of anything we care about, and performance tests seem to confirm that it's all good. * wtf/ScopedLambda.h: Canonical link: https://commits.webkit.org/169519@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@192540 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-11-17 22:31:40 +00:00
static ResultType implFunction(void* argument, ArgumentTypes... arguments)
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
{
CheckAdd/Mul should have commutativity optimizations in B3->Air lowering https://bugs.webkit.org/show_bug.cgi?id=151214 Reviewed by Geoffrey Garen. Source/JavaScriptCore: This is an overhaul of CheckAdd/CheckSub/CheckMul that fixes bugs, improves codegen, and simplifies the contract between B3 and its client. Previously, the idea was that the operands to the Air BranchAdd/Sub/Mul matched the children of the B3 CheckAdd/Sub/Mul, or at least, that's what the B3::CheckSpecial would make you believe. This meant that B3/Air had to avoid optimizations that would break this protocol. This prevented commutativity optimizations on CheckAdd/Mul and it also prevented strength reduction from CheckMul(x, 2) to CheckAdd(x, x), for example. Those optimizations would break things because the client's Stackmap generation callback was allowed to assume that the first entry in params.reps was the first child. Also, there was a contract between B3 and its client that for CheckAdd/Sub, the client would undo the operation by doing the opposite operation with the params.reps[0] as the source and params.reps[1] as the destination. This not only prevented commutativity optimizations, it also led to bugs. Here are two bugs that we had: - Add(x, x) would not work. The client would be told to undo the add using %x as both the source and destination. The client would use a sub() instruction. The result would not be x - it would be zero. - Mul where the result got coalesced with one of the sources. You can't undo a multiplication, so you need to keep the inputs alive until after the result is computed - i.e. the inputs are late uses. I initially thought I worked around this by using a three-operand form of Mul, but of course that doesn't actually fix the issue. This patch fixes these issues comprehensively by introducing the following changes: The params.reps corresponding to the first two children of CheckAdd/Sub/Mul and the first child of Check are always garbage: if you want to know the values of those children in the slow path, pass them as additional stackmap children. This makes it clear to the compiler whose values you actually care about, and frees the compiler to reorder and commute the non-stackmap children. The "undo" of an Add or Sub is handled internally by B3: the client doesn't have to know anything about undoing. B3 does it. The contract is simply that if a B3::Value* is a stackmap child of a CheckXYZ, then the corresponding entry in params.reps inside the stackmap generator callback will contain the value of that Value*. For Add and Sub, B3 undoes the operation. For Add(x, x), the undo uses the carry flag and some shift magic. For Mul, B3 uses LateUse: A new kind of Arg::Role called Arg::LateUse is introduced: This kind of use means that the use happens at the start of the execution of the next instruction. None of the built-in Air opcodes use LateUse, but it is used by B3::CheckSpecial. We use it to implement CheckMul. Finally, this change fixes testComplex to actually create many live variables. This revealed a really dumb pathology in allocateStack(), and this patch fixes it. Basically, it's a bad idea to build interference graphs by repeatedly recreating the same cliques. * assembler/MacroAssemblerX86Common.h: (JSC::MacroAssemblerX86Common::test32): (JSC::MacroAssemblerX86Common::setCarry): (JSC::MacroAssemblerX86Common::invert): * b3/B3CheckSpecial.cpp: (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::dump): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: (JSC::B3::CheckSpecial::Key::Key): (JSC::B3::CheckSpecial::Key::operator==): (JSC::B3::CheckSpecial::Key::operator!=): (JSC::B3::CheckSpecial::Key::opcode): (JSC::B3::CheckSpecial::Key::numArgs): (JSC::B3::CheckSpecial::Key::stackmapRole): (JSC::B3::CheckSpecial::Key::hash): * b3/B3CheckValue.cpp: (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::convertToAdd): (JSC::B3::CheckValue::CheckValue): * b3/B3CheckValue.h: * b3/B3Const32Value.cpp: (JSC::B3::Const32Value::checkMulConstant): (JSC::B3::Const32Value::checkNegConstant): (JSC::B3::Const32Value::divConstant): * b3/B3Const32Value.h: * b3/B3Const64Value.cpp: (JSC::B3::Const64Value::checkMulConstant): (JSC::B3::Const64Value::checkNegConstant): (JSC::B3::Const64Value::divConstant): * b3/B3Const64Value.h: * b3/B3LowerToAir.cpp: (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::lower): * b3/B3Opcode.h: * b3/B3PatchpointSpecial.cpp: (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): * b3/B3ReduceStrength.cpp: * b3/B3StackmapSpecial.cpp: (JSC::B3::StackmapSpecial::forEachArgImpl): * b3/B3StackmapSpecial.h: * b3/B3StackmapValue.cpp: (JSC::B3::StackmapValue::append): (JSC::B3::StackmapValue::appendSomeRegister): (JSC::B3::StackmapValue::setConstrainedChild): * b3/B3StackmapValue.h: * b3/B3Validate.cpp: * b3/B3Value.cpp: (JSC::B3::Value::checkMulConstant): (JSC::B3::Value::checkNegConstant): (JSC::B3::Value::divConstant): * b3/B3Value.h: * b3/air/AirAllocateStack.cpp: (JSC::B3::Air::allocateStack): * b3/air/AirArg.cpp: (WTF::printInternal): * b3/air/AirArg.h: (JSC::B3::Air::Arg::isAnyUse): (JSC::B3::Air::Arg::isEarlyUse): (JSC::B3::Air::Arg::isLateUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::isUse): Deleted. * b3/air/AirGenerate.cpp: (JSC::B3::Air::generate): * b3/air/AirIteratedRegisterCoalescing.cpp: (JSC::B3::Air::IteratedRegisterCoalescingAllocator::build): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::allocate): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::hash): (JSC::B3::Air::IteratedRegisterCoalescingAllocator::InterferenceEdge::dump): (JSC::B3::Air::addSpillAndFillToProgram): (JSC::B3::Air::iteratedRegisterCoalescingOnType): (JSC::B3::Air::iteratedRegisterCoalescing): * b3/air/AirLiveness.h: (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: * b3/air/AirReportUsedRegisters.cpp: (JSC::B3::Air::reportUsedRegisters): * b3/air/AirSpillEverything.cpp: (JSC::B3::Air::spillEverything): * b3/testb3.cpp: (JSC::B3::testMulArg): (JSC::B3::testMulArgStore): (JSC::B3::testMulAddArg): (JSC::B3::testMulArgs): (JSC::B3::testComplex): (JSC::B3::testSimpleCheck): (JSC::B3::testCheckLessThan): (JSC::B3::testCheckMegaCombo): (JSC::B3::testCheckAddImm): (JSC::B3::testCheckAddImmCommute): (JSC::B3::testCheckAddImmSomeRegister): (JSC::B3::testCheckAdd): (JSC::B3::testCheckAdd64): (JSC::B3::testCheckSubImm): (JSC::B3::testCheckSubBadImm): (JSC::B3::testCheckSub): (JSC::B3::testCheckSub64): (JSC::B3::testCheckNeg): (JSC::B3::testCheckNeg64): (JSC::B3::testCheckMul): (JSC::B3::testCheckMulMemory): (JSC::B3::testCheckMul2): (JSC::B3::testCheckMul64): (JSC::B3::run): Source/WTF: Disable my failed attempts at perfect forwarding, since they were incorrect, and caused compile errors if you tried to pass an argument that bound to lvalue. This shouldn't affect performance of anything we care about, and performance tests seem to confirm that it's all good. * wtf/ScopedLambda.h: Canonical link: https://commits.webkit.org/169519@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@192540 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-11-17 22:31:40 +00:00
return static_cast<ScopedLambdaFunctor*>(argument)->m_functor(arguments...);
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
}
Functor m_functor;
};
ScopedLambda should have a lifetime story that makes sense to the compiler https://bugs.webkit.org/show_bug.cgi?id=158118 Reviewed by Mark Lam. Source/WTF: Prior to this change, there were two lifetime bugs in ScopedLambda: - scopedLambda(Functor&&) would bind Functor to const lambda&, so the resulting ScopedLambdaFunctor would hold a reference to the original lambda. This would have surprising behavior; for example it meant that this code was wrong: auto l = scopedLambda<things>([&] ...); The solution is to have explicit copy/move versions of scopedLambda() rather than rely on perfect forwarding. - ScopedLambdaFunctor did not override its copy or move operations, so if the compiler did not RVO scopedLambda(), it would return a ScopedLambdaFunctor whose m_arg points to a dead temporary ScopedLambdaFunctor instance. The solution is to have explicit copy/move constructors and operators, which preserve the invariant that ScopedLambda::m_arg points to this. One nice side-effect of all of these constructors and operators being explicit is that we can rely on WTFMove's excellent assertions, which helped catch the first issue. This reverts ParkingLot to use ScopedLambda again. * wtf/ParkingLot.cpp: (WTF::ParkingLot::parkConditionallyImpl): (WTF::ParkingLot::unparkOne): (WTF::ParkingLot::unparkOneImpl): * wtf/ParkingLot.h: (WTF::ParkingLot::parkConditionally): (WTF::ParkingLot::unparkOne): * wtf/ScopedLambda.h: (WTF::scopedLambda): Tools: Added a test case. This test crashes before the fix and now it passes. * TestWebKitAPI/CMakeLists.txt: * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj: * TestWebKitAPI/Tests/WTF/ScopedLambda.cpp: Added. (TestWebKitAPI::TEST): Canonical link: https://commits.webkit.org/176236@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@201433 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-05-26 21:58:42 +00:00
// Can't simply rely on perfect forwarding because then the ScopedLambdaFunctor would point to the functor
// by const reference. This would be surprising in situations like:
//
// auto scopedLambda = scopedLambda<Foo(Bar)>([&] (Bar) -> Foo { ... });
//
// We expected scopedLambda to be valid for its entire lifetime, but if it computed the lambda by reference
// then it would be immediately invalid.
template<typename FunctionType, typename Functor>
ScopedLambdaFunctor<FunctionType, Functor> scopedLambda(const Functor& functor)
{
return ScopedLambdaFunctor<FunctionType, Functor>(functor);
}
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
template<typename FunctionType, typename Functor>
ScopedLambdaFunctor<FunctionType, Functor> scopedLambda(Functor&& functor)
{
return ScopedLambdaFunctor<FunctionType, Functor>(std::forward<Functor>(functor));
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
}
Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace and we should rewrite as much of the GC as needed to make this not a regression https://bugs.webkit.org/show_bug.cgi?id=160125 Reviewed by Geoffrey Garen and Keith Miller. JSTests: Most of the things I did properly covered by existing tests, but I found some simple cases of unshifting that had sketchy coverage. * stress/array-storage-array-unshift.js: Added. * stress/contiguous-array-unshift.js: Added. * stress/double-array-unshift.js: Added. * stress/int32-array-unshift.js: Added. Source/bmalloc: I needed to tryMemalign, so I added such a thing. * bmalloc/Allocator.cpp: (bmalloc::Allocator::allocate): (bmalloc::Allocator::tryAllocate): (bmalloc::Allocator::allocateImpl): * bmalloc/Allocator.h: * bmalloc/Cache.h: (bmalloc::Cache::tryAllocate): * bmalloc/bmalloc.h: (bmalloc::api::tryMemalign): Source/JavaScriptCore: In order to make the GC concurrent (bug 149432), we would either need to enable concurrent copying or we would need to not copy. Concurrent copying carries a 1-2% throughput overhead from the barriers alone. Considering that MarkedSpace does a decent job of avoiding fragmentation, it's unlikely that it's worth paying 1-2% throughput for copying. So, we want to get rid of copied space. This change moves copied space's biggest client over to marked space. Moving butterflies to marked space means having them use the new Auxiliary HeapCell allocation path. This is a fairly mechanical change, but it caused performance regressions everywhere, so this change also fixes MarkedSpace's performance issues. At a high level the mechanical changes are: - We use AuxiliaryBarrier instead of CopyBarrier. - We use tryAllocateAuxiliary instead of tryAllocateStorage. I got rid of the silly CheckedBoolean stuff, since it's so much more trouble than it's worth. - The JITs have to emit inlined marked space allocations instead of inline copy space allocations. - Everyone has to get used to zeroing their butterflies after allocation instead of relying on them being pre-zeroed by the GC. Copied space would zero things for you, while marked space doesn't. That's about 1/3 of this change. But this led to performance problems, which I fixed with optimizations that amounted to a major MarkedSpace rewrite: - MarkedSpace always causes internal fragmentation for array allocations because the vector length we choose when we resize usually leads to a cell size that doesn't correspond to any size class. I got around this by making array allocations usually round up vectorLength to the maximum allowed by the size class that we would have allocated in. Also, ensureLengthSlow() and friends first make sure that the requested length can't just be fulfilled with the current allocation size. This safeguard means that not every array allocation has to do size class queries. For example, the fast path of new Array(length) never does any size class queries, under the assumption that (1) the speed gained from avoiding an ensureLengthSlow() call, which then just changes the vectorLength by doing the size class query, is too small to offset the speed lost by doing the query on every allocation and (2) new Array(length) is a pretty good hint that resizing is not very likely. - Size classes in MarkedSpace were way too precise, which led to external fragmentation. This changes MarkedSpace size classes to use a linear progression for very small sizes followed by a geometric progression that naturally transitions to a hyperbolic progression. We want hyperbolic sizes when we get close to blockSize: for example the largest size we want is payloadSize / 2 rounded down, to ensure we get exactly two cells with minimal slop. The next size down should be payloadSize / 3 rounded down, and so on. After the last precise size (80 bytes), we proceed using a geometric progression, but round up each size to minimize slop at the end of the block. This naturally causes the geometric progression to turn hyperbolic for large sizes. The size class configuration happens at VM start-up, so it can be controlled with runtime options. I found that a base of 1.4 works pretty well. - Large allocations caused massive internal fragmentation, since the smallest large allocation had to use exactly blockSize, and the largest small allocation used blockSize / 2. The next size up - the first large allocation size to require two blocks - also had 50% internal fragmentation. This is because we required large allocations to be blockSize aligned, so that MarkedBlock::blockFor() would work. I decided to rewrite all of that. Cells no longer have to be owned by a MarkedBlock. They can now alternatively be owned by a LargeAllocation. These two things are abstracted as CellContainer. You know that a cell is owned by a LargeAllocation if the MarkedBlock::atomSize / 2 bit is set. Basically, large allocations are deliberately misaligned by 8 bytes. This actually works out great since (1) typed arrays won't use large allocations anyway since they have their own malloc fallback and (2) large array butterflies already have a 8 byte header, which means that the 8 byte base misalignment aligns the large array payload on a 16 byte boundary. I took extreme care to make sure that the isLargeAllocation bit checks are as rare as possible; for example, ExecState::vm() skips the check because we know that callees must be small allocations. It's also possible to use template tricks to do one check for cell container kind, and then invoke a function specialized for MarkedBlock or a function specialized for LargeAllocation. LargeAllocation includes stubs for all MarkedBlock methods that get used from functions that are template-specialized like this. That's mostly to speed up the GC marking code. Most other code can use CellContainer API or HeapCell API directly. That's another thing: HeapCell, the common base of JSCell and auxiliary allocations, is now smart enough to do a lot of things for you, like HeapCell::vm(), HeapCell::heap(), HeapCell::isLargeAllocation(), and HeapCell::cellContainer(). The size cutoff for large allocations is runtime-configurable, so long as you don't choose something so small that callees end up large. I found that 400 bytes is roughly optimal. This means that the MarkedBlock size classes end up being: 16, 32, 48, 64, 80, 112, 160, 224, 320 The next size class would have been 432, but that's above the 400 byte cutoff. All of this is configurable with --sizeClassProgression and --largeAllocationCutoff. You can see what size classes you end up with by doing --dumpSizeClasses=true. - Copied space uses 64KB blocks, while marked space used to use 16KB blocks. Allocating a lot of stuff in 16KB blocks was slower than allocating it in 64KB blocks because the GC had a lot of per-block overhead. I removed this overhead: It's now 2x faster to scan all MarkedBlocks because the list that contains the interesting meta-data is allocated on the side, for better locality during a sequential walk. It's no longer necessary to scan MarkedBlocks to find WeakSets, since the sets of WeakSets for eden scan and full scan are maintained on-the-fly. It's no longer necessary to scan all MarkedBlocks to clear mark bits because we now use versioned mark bits: to clear then, just increment the 64-bit heap version. It's no longer necessary to scan retired MarkedBlocks while allocating because marking retires them on-the-fly. It's no longer necessary to sort all blocks in the IncrementalSweeper's snapshot because blocks now know if they are in the snapshot. Put together, these optimizations allowed me to reduce block size to 16KB without losing much performance. There is some small perf loss on JetStream/splay, but not enough to hurt JetStream overall. I tried reducing block sizes further, to 4KB, since that is a progression on membuster. That's not possible yet, since there is still enough per-block overhead yet that such a reduction hurts JetStream too much. I filed a bug about improving this further: https://bugs.webkit.org/show_bug.cgi?id=161581. - Even after all of that, copying butterflies was still faster because it allowed us to skip sweeping dead space. A good GC allocates over dead bytes without explicitly freeing them, so the GC pause is O(size of live), not O(size of live + dead). O(dead) is usually much larger than O(live), especially in an eden collection. Copying satisfies this premise while mark+sweep does not. So, I invented a new kind of allocator: bump'n'pop. Previously, our MarkedSpace allocator was a freelist pop. That's simple and easy to inline but requires that we walk the block to build a free list. This means walking dead space. The new allocator allows totally free MarkedBlocks to simply set up a bump-pointer arena instead. The allocator is a hybrid of bump-pointer and freelist pop. It tries bump first. The bump pointer always bumps by cellSize, so the result of filling a block with bumping looks as if we had used freelist popping to fill it. Additionally, each MarkedBlock now has a bit to quickly tell if the block is entirely free. This makes sweeping O(1) whenever a MarkedBlock is completely empty, which is the common case because of the generational hypothesis: the number of objects that survive an eden collection is a tiny fraction of the number of objects that had been allocated, and this fraction is so small that there are typically fewer than one survivors per MarkedBlock. This change was enough to make this change a net win over tip-of-tree. - FTL now shares the same allocation fast paths as everything else, which is great, because bump'n'pop has gnarly control flow. We don't really want B3 to have to think about that control flow, since it won't be able to improve the machine code we write ourselves. GC fast paths are best written in assembly. So, I've empowered B3 to have even better support for Patchpoint terminals. It's now totally fine for a Patchpoint terminal to be non-Void. So, the new FTL allocation fast paths are just Patchpoint terminals that call through to AssemblyHelpers::emitAllocate(). B3 still reasons about things like constant-folding the size class calculation and constant-hoisting the allocator. Also, I gave the FTL the ability to constant-fold some allocator logic (in case we first assume that we're doing a variable-length allocation but then realize that the length is known). I think it makes sense to have constant folding rules in FTL::Output, or whatever the B3 IR builder is, since this makes lowering easier (you can constant fold during lowering more easily) and it reduces the amount of malloc traffic. In the future, we could teach B3 how to better constant-fold this code. That would require allowing loads to be constant-folded, which is doable but hella tricky. - It used to be that if a logical object allocation required two physical allocations (first the butterfly and then the cell), then the JIT would emit the code in such a way that a failure in the second fast path would cause us to forget the successful first physical allocation. This was pointlessly wasteful. It turns out that it's very cheap to devote a register to storing either the butterfly or null, because the butterfly register is anyway going to be free inside the first allocation. The only overhead here is zeroing the butterfly register. With that in place, we can just pass the butterfly-or-null to the slow path, which can then either allocate a butterfly or not. So now we never waste a successful allocation. This patch implements such a solution both in DFG (where it's easy to do this since we control registers already) and in FTL (where it's annoying, because mutable "butterfly-or-null" variables are hard to say in SSA; also I realized that we had code duplicated the JSArray allocation utility, so I deduplicated it). This came up because in one version of this patch, this wastage would resonate with some Kraken benchmark: the benchmark would always allocate N small things followed by one bigger thing. The problem was I accidentally adjusted the various fixed overheads in MarkedBlock in such a way that the JSObject size class, which both the small and big thing shared for their cell, could hold exactly N cells per MarkedBlock. Then the benchmark would always call slow path when it allocated the big thing. So, it would end up having to allocate the big thing's large butterfly twice, every single time! Ouch! - It used to be that we zeroed CopiedBlocks using memset, and so array allocations enjoyed amortization of the cost of zeroing. This doesn't work anymore - it's now up to the client of the allocator to initialize the object to whatever state they need. It used to be that we would just use a dumb loop. I initially changed this so that we would end up in memset for large allocations, but this didn't actually help performance that much. I got a much better result by playing with different memsets written in assembly. First I wrote one using non-temporal stores. That was a small speed-up over memset. Then I tried the classic "rep stos" approach, and holy cow that version was fast. It's a ~20% speed-up on array allocation microbenchmarks. So, this patch adds code paths to do "rep stos" on x86_64, or memset, or use a loop, as appropriate, for both "contiguous" arrays (holes are zero) and double arrays (holes are PNaN). Note that the JIT always emits either a loop or a flat slab of stores (if the size is known), but those paths in the JIT won't trigger for NewArrayWithSize() if the size is large, since that takes us to the operationNewArrayWithSize() slow path, which calls into JSArray::create(). That's why the optimizations here are all in JSArray::create() - that's the hot place for large arrays that need to be filled with holes. All of this put together gives us neutral perf on JetStream, membuster, and PLT3, a ~1% regression on Speedometer, and up to a 4% regression Kraken. The Kraken regression is because Kraken was allocating exactly 1024 element arrays at a rate of 400MB/sec. This is a best-case scenario for bump allocation. I think that we should fix bmalloc to make up the difference, but take the hit for now because it's a crazy corner case. By comparison, the alternative approach of using a copy barrier would have cost us 1-2%. That's the real apples-to-apples comparison if your premise is that we should have a concurrent GC. After we finish removing copied space, we will be barrier-ready for concurrent GC: we already have a marking barrier and we simply won't need a copying barrier. This change gets us there for the purposes of our benchmarks, since the remaining clients of copied space are not very important. On the other hand, if we keep copying, then getting barrier-ready would mean adding back the copy barrier, which costs more perf. We might get bigger speed-ups once we remove CopiedSpace altogether. That requires moving typed arrays and a few other weird things over to Aux MarkedSpace. This also includes some header sanitization. The introduction of AuxiliaryBarrier, HeapCell, and CellContainer meant that I had to include those files from everywhere. Fortunately, just including JSCInlines.h (instead of manually including the files that includes) is usually enough. So, I made most of JSC's cpp files include JSCInlines.h, which is something that we were already basically doing. In places where JSCInlines.h would be too much, I just included HeapInlines.h. This got weird, because we previously included HeapInlines.h from JSObject.h. That's bad because it led to some circular dependencies, so I fixed it - but that meant having to manually include HeapInlines.h from the places that previously got it implicitly via JSObject.h. But that led to more problems for some reason: I started getting build errors because non-JSC files were having trouble including Opcode.h. That's just silly, since Opcode.h is meant to be an internal JSC header. So, I made it an internal header and made it impossible to include it from outside JSC. This was a lot of work, but it was necessary to get the patch to build on all ports. It's also a net win. There were many places in WebCore that were transitively including a *ton* of JSC headers just because of the JSObject.h->HeapInlines.h edge and a bunch of dependency edges that arose from some public (for WebCore) JSC headers needing Interpreter.h or Opcode.h for bad reasons. * API/JSManagedValue.mm: (-[JSManagedValue initWithValue:]): * API/JSTypedArray.cpp: * API/ObjCCallbackFunction.mm: * API/tests/testapi.mm: (testObjectiveCAPI): (testWeakValue): Deleted. * CMakeLists.txt: * JavaScriptCore.xcodeproj/project.pbxproj: * Scripts/builtins/builtins_generate_combined_implementation.py: (BuiltinsCombinedImplementationGenerator.generate_secondary_header_includes): * Scripts/builtins/builtins_generate_internals_wrapper_implementation.py: (BuiltinsInternalsWrapperImplementationGenerator.generate_secondary_header_includes): * Scripts/builtins/builtins_generate_separate_implementation.py: (BuiltinsSeparateImplementationGenerator.generate_secondary_header_includes): * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::JumpList::link): (JSC::AbstractMacroAssembler::JumpList::linkTo): * assembler/MacroAssembler.h: * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::add32): * assembler/MacroAssemblerCodeRef.cpp: Added. (JSC::MacroAssemblerCodePtr::createLLIntCodePtr): (JSC::MacroAssemblerCodePtr::dumpWithName): (JSC::MacroAssemblerCodePtr::dump): (JSC::MacroAssemblerCodeRef::createLLIntCodeRef): (JSC::MacroAssemblerCodeRef::dump): * assembler/MacroAssemblerCodeRef.h: (JSC::MacroAssemblerCodePtr::createLLIntCodePtr): Deleted. (JSC::MacroAssemblerCodePtr::dumpWithName): Deleted. (JSC::MacroAssemblerCodePtr::dump): Deleted. (JSC::MacroAssemblerCodeRef::createLLIntCodeRef): Deleted. (JSC::MacroAssemblerCodeRef::dump): Deleted. * b3/B3BasicBlock.cpp: (JSC::B3::BasicBlock::appendBoolConstant): * b3/B3BasicBlock.h: * b3/B3DuplicateTails.cpp: * b3/B3StackmapGenerationParams.h: * b3/testb3.cpp: (JSC::B3::testPatchpointTerminalReturnValue): (JSC::B3::run): * bindings/ScriptValue.cpp: * bytecode/AdaptiveInferredPropertyValueWatchpointBase.cpp: * bytecode/BytecodeBasicBlock.cpp: * bytecode/BytecodeLivenessAnalysis.cpp: * bytecode/BytecodeUseDef.h: * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::callTypeFor): * bytecode/CallLinkInfo.h: (JSC::CallLinkInfo::callTypeFor): Deleted. * bytecode/CallLinkStatus.cpp: * bytecode/CodeBlock.cpp: (JSC::CodeBlock::finishCreation): (JSC::CodeBlock::clearLLIntGetByIdCache): (JSC::CodeBlock::predictedMachineCodeSize): * bytecode/CodeBlock.h: (JSC::CodeBlock::jitCodeMap): Deleted. (JSC::clearLLIntGetByIdCache): Deleted. * bytecode/ExecutionCounter.h: * bytecode/Instruction.h: * bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp: (JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal): * bytecode/ObjectAllocationProfile.h: (JSC::ObjectAllocationProfile::isNull): (JSC::ObjectAllocationProfile::initialize): * bytecode/Opcode.h: (JSC::padOpcodeName): * bytecode/PolymorphicAccess.cpp: (JSC::AccessCase::generateImpl): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: * bytecode/PreciseJumpTargets.cpp: * bytecode/StructureStubInfo.cpp: * bytecode/StructureStubInfo.h: * bytecode/UnlinkedCodeBlock.cpp: (JSC::UnlinkedCodeBlock::vm): Deleted. * bytecode/UnlinkedCodeBlock.h: * bytecode/UnlinkedInstructionStream.cpp: * bytecode/UnlinkedInstructionStream.h: * dfg/DFGOperations.cpp: * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::emitAllocateRawObject): (JSC::DFG::SpeculativeJIT::compileMakeRope): (JSC::DFG::SpeculativeJIT::compileAllocatePropertyStorage): (JSC::DFG::SpeculativeJIT::compileReallocatePropertyStorage): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::emitAllocateJSCell): (JSC::DFG::SpeculativeJIT::emitAllocateJSObject): * dfg/DFGSpeculativeJIT32_64.cpp: (JSC::DFG::SpeculativeJIT::compile): (JSC::DFG::SpeculativeJIT::compileAllocateNewArrayWithSize): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::compile): (JSC::DFG::SpeculativeJIT::compileAllocateNewArrayWithSize): * dfg/DFGStrengthReductionPhase.cpp: (JSC::DFG::StrengthReductionPhase::handleNode): * ftl/FTLAbstractHeapRepository.h: * ftl/FTLCompile.cpp: * ftl/FTLJITFinalizer.cpp: * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToB3::compileCreateRest): (JSC::FTL::DFG::LowerDFGToB3::allocateArrayWithSize): (JSC::FTL::DFG::LowerDFGToB3::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToB3::compileMakeRope): (JSC::FTL::DFG::LowerDFGToB3::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToB3::initializeArrayElements): (JSC::FTL::DFG::LowerDFGToB3::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToB3::allocateHeapCell): (JSC::FTL::DFG::LowerDFGToB3::allocateCell): (JSC::FTL::DFG::LowerDFGToB3::allocateObject): (JSC::FTL::DFG::LowerDFGToB3::allocatorForSize): (JSC::FTL::DFG::LowerDFGToB3::allocateVariableSizedObject): (JSC::FTL::DFG::LowerDFGToB3::allocateJSArray): (JSC::FTL::DFG::LowerDFGToB3::compileAllocateArrayWithSize): Deleted. * ftl/FTLOutput.cpp: (JSC::FTL::Output::constBool): (JSC::FTL::Output::add): (JSC::FTL::Output::shl): (JSC::FTL::Output::aShr): (JSC::FTL::Output::lShr): (JSC::FTL::Output::zeroExt): (JSC::FTL::Output::equal): (JSC::FTL::Output::notEqual): (JSC::FTL::Output::above): (JSC::FTL::Output::aboveOrEqual): (JSC::FTL::Output::below): (JSC::FTL::Output::belowOrEqual): (JSC::FTL::Output::greaterThan): (JSC::FTL::Output::greaterThanOrEqual): (JSC::FTL::Output::lessThan): (JSC::FTL::Output::lessThanOrEqual): (JSC::FTL::Output::select): (JSC::FTL::Output::appendSuccessor): (JSC::FTL::Output::addIncomingToPhi): * ftl/FTLOutput.h: * ftl/FTLValueFromBlock.h: (JSC::FTL::ValueFromBlock::operator bool): (JSC::FTL::ValueFromBlock::ValueFromBlock): Deleted. * ftl/FTLWeightedTarget.h: (JSC::FTL::WeightedTarget::frequentedBlock): * heap/CellContainer.h: Added. (JSC::CellContainer::CellContainer): (JSC::CellContainer::operator bool): (JSC::CellContainer::isMarkedBlock): (JSC::CellContainer::isLargeAllocation): (JSC::CellContainer::markedBlock): (JSC::CellContainer::largeAllocation): * heap/CellContainerInlines.h: Added. (JSC::CellContainer::isMarked): (JSC::CellContainer::isMarkedOrNewlyAllocated): (JSC::CellContainer::noteMarked): (JSC::CellContainer::cellSize): (JSC::CellContainer::weakSet): (JSC::CellContainer::flipIfNecessary): * heap/ConservativeRoots.cpp: (JSC::ConservativeRoots::ConservativeRoots): (JSC::ConservativeRoots::~ConservativeRoots): (JSC::ConservativeRoots::grow): (JSC::ConservativeRoots::genericAddPointer): (JSC::ConservativeRoots::genericAddSpan): * heap/ConservativeRoots.h: (JSC::ConservativeRoots::roots): * heap/CopyToken.h: * heap/FreeList.cpp: Added. (JSC::FreeList::dump): * heap/FreeList.h: Added. (JSC::FreeList::FreeList): (JSC::FreeList::list): (JSC::FreeList::bump): (JSC::FreeList::operator==): (JSC::FreeList::operator!=): (JSC::FreeList::operator bool): (JSC::FreeList::allocationWillFail): (JSC::FreeList::allocationWillSucceed): * heap/GCTypeMap.h: Added. (JSC::GCTypeMap::operator[]): * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::lastChanceToFinalize): (JSC::Heap::finalizeUnconditionalFinalizers): (JSC::Heap::markRoots): (JSC::Heap::copyBackingStores): (JSC::Heap::gatherStackRoots): (JSC::Heap::gatherJSStackRoots): (JSC::Heap::gatherScratchBufferRoots): (JSC::Heap::clearLivenessData): (JSC::Heap::visitSmallStrings): (JSC::Heap::visitConservativeRoots): (JSC::Heap::removeDeadCompilerWorklistEntries): (JSC::Heap::gatherExtraHeapSnapshotData): (JSC::Heap::removeDeadHeapSnapshotNodes): (JSC::Heap::visitProtectedObjects): (JSC::Heap::visitArgumentBuffers): (JSC::Heap::visitException): (JSC::Heap::visitStrongHandles): (JSC::Heap::visitHandleStack): (JSC::Heap::visitSamplingProfiler): (JSC::Heap::traceCodeBlocksAndJITStubRoutines): (JSC::Heap::converge): (JSC::Heap::visitWeakHandles): (JSC::Heap::updateObjectCounts): (JSC::Heap::clearUnmarkedExecutables): (JSC::Heap::deleteUnmarkedCompiledCode): (JSC::Heap::collectAllGarbage): (JSC::Heap::collect): (JSC::Heap::collectWithoutAnySweep): (JSC::Heap::collectImpl): (JSC::Heap::suspendCompilerThreads): (JSC::Heap::willStartCollection): (JSC::Heap::flushOldStructureIDTables): (JSC::Heap::flushWriteBarrierBuffer): (JSC::Heap::stopAllocation): (JSC::Heap::prepareForMarking): (JSC::Heap::reapWeakHandles): (JSC::Heap::pruneStaleEntriesFromWeakGCMaps): (JSC::Heap::sweepArrayBuffers): (JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): (JSC::MarkedBlockSnapshotFunctor::operator()): (JSC::Heap::snapshotMarkedSpace): (JSC::Heap::deleteSourceProviderCaches): (JSC::Heap::notifyIncrementalSweeper): (JSC::Heap::writeBarrierCurrentlyExecutingCodeBlocks): (JSC::Heap::resetAllocators): (JSC::Heap::updateAllocationLimits): (JSC::Heap::didFinishCollection): (JSC::Heap::resumeCompilerThreads): (JSC::Zombify::visit): (JSC::Heap::forEachCodeBlockImpl): * heap/Heap.h: (JSC::Heap::allocatorForObjectWithoutDestructor): (JSC::Heap::allocatorForObjectWithDestructor): (JSC::Heap::allocatorForAuxiliaryData): (JSC::Heap::jitStubRoutines): (JSC::Heap::codeBlockSet): (JSC::Heap::storageAllocator): Deleted. * heap/HeapCell.h: (JSC::HeapCell::isZapped): Deleted. * heap/HeapCellInlines.h: Added. (JSC::HeapCell::isLargeAllocation): (JSC::HeapCell::cellContainer): (JSC::HeapCell::markedBlock): (JSC::HeapCell::largeAllocation): (JSC::HeapCell::heap): (JSC::HeapCell::vm): (JSC::HeapCell::cellSize): (JSC::HeapCell::allocatorAttributes): (JSC::HeapCell::destructionMode): (JSC::HeapCell::cellKind): * heap/HeapInlines.h: (JSC::Heap::heap): (JSC::Heap::isLive): (JSC::Heap::isMarked): (JSC::Heap::testAndSetMarked): (JSC::Heap::setMarked): (JSC::Heap::cellSize): (JSC::Heap::forEachCodeBlock): (JSC::Heap::allocateObjectOfType): (JSC::Heap::subspaceForObjectOfType): (JSC::Heap::allocatorForObjectOfType): (JSC::Heap::allocateAuxiliary): (JSC::Heap::tryAllocateAuxiliary): (JSC::Heap::tryReallocateAuxiliary): (JSC::Heap::isPointerGCObject): Deleted. (JSC::Heap::isValueGCObject): Deleted. * heap/HeapOperation.cpp: Added. (WTF::printInternal): * heap/HeapOperation.h: * heap/HeapUtil.h: Added. (JSC::HeapUtil::findGCObjectPointersForMarking): (JSC::HeapUtil::isPointerGCObjectJSCell): (JSC::HeapUtil::isValueGCObject): * heap/IncrementalSweeper.cpp: (JSC::IncrementalSweeper::sweepNextBlock): * heap/IncrementalSweeper.h: * heap/LargeAllocation.cpp: Added. (JSC::LargeAllocation::tryCreate): (JSC::LargeAllocation::LargeAllocation): (JSC::LargeAllocation::lastChanceToFinalize): (JSC::LargeAllocation::shrink): (JSC::LargeAllocation::visitWeakSet): (JSC::LargeAllocation::reapWeakSet): (JSC::LargeAllocation::flip): (JSC::LargeAllocation::isEmpty): (JSC::LargeAllocation::sweep): (JSC::LargeAllocation::destroy): (JSC::LargeAllocation::dump): * heap/LargeAllocation.h: Added. (JSC::LargeAllocation::fromCell): (JSC::LargeAllocation::cell): (JSC::LargeAllocation::isLargeAllocation): (JSC::LargeAllocation::heap): (JSC::LargeAllocation::vm): (JSC::LargeAllocation::weakSet): (JSC::LargeAllocation::clearNewlyAllocated): (JSC::LargeAllocation::isNewlyAllocated): (JSC::LargeAllocation::isMarked): (JSC::LargeAllocation::isMarkedOrNewlyAllocated): (JSC::LargeAllocation::isLive): (JSC::LargeAllocation::hasValidCell): (JSC::LargeAllocation::cellSize): (JSC::LargeAllocation::aboveLowerBound): (JSC::LargeAllocation::belowUpperBound): (JSC::LargeAllocation::contains): (JSC::LargeAllocation::attributes): (JSC::LargeAllocation::flipIfNecessary): (JSC::LargeAllocation::flipIfNecessaryConcurrently): (JSC::LargeAllocation::testAndSetMarked): (JSC::LargeAllocation::setMarked): (JSC::LargeAllocation::clearMarked): (JSC::LargeAllocation::noteMarked): (JSC::LargeAllocation::headerSize): * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::MarkedAllocator): (JSC::MarkedAllocator::isPagedOut): (JSC::MarkedAllocator::retire): (JSC::MarkedAllocator::filterNextBlock): (JSC::MarkedAllocator::setNextBlockToSweep): (JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl): (JSC::MarkedAllocator::tryAllocateWithoutCollecting): (JSC::MarkedAllocator::allocateSlowCase): (JSC::MarkedAllocator::tryAllocateSlowCase): (JSC::MarkedAllocator::allocateSlowCaseImpl): (JSC::blockHeaderSize): (JSC::MarkedAllocator::blockSizeForBytes): (JSC::MarkedAllocator::tryAllocateBlock): (JSC::MarkedAllocator::addBlock): (JSC::MarkedAllocator::removeBlock): (JSC::MarkedAllocator::stopAllocating): (JSC::MarkedAllocator::reset): (JSC::MarkedAllocator::lastChanceToFinalize): (JSC::MarkedAllocator::setFreeList): (JSC::isListPagedOut): Deleted. (JSC::MarkedAllocator::tryAllocateHelper): Deleted. (JSC::MarkedAllocator::tryPopFreeList): Deleted. (JSC::MarkedAllocator::tryAllocate): Deleted. (JSC::MarkedAllocator::allocateBlock): Deleted. * heap/MarkedAllocator.h: (JSC::MarkedAllocator::takeLastActiveBlock): (JSC::MarkedAllocator::offsetOfFreeList): (JSC::MarkedAllocator::offsetOfCellSize): (JSC::MarkedAllocator::tryAllocate): (JSC::MarkedAllocator::allocate): (JSC::MarkedAllocator::forEachBlock): (JSC::MarkedAllocator::offsetOfFreeListHead): Deleted. (JSC::MarkedAllocator::MarkedAllocator): Deleted. (JSC::MarkedAllocator::init): Deleted. (JSC::MarkedAllocator::stopAllocating): Deleted. * heap/MarkedBlock.cpp: (JSC::MarkedBlock::tryCreate): (JSC::MarkedBlock::Handle::Handle): (JSC::MarkedBlock::Handle::~Handle): (JSC::MarkedBlock::MarkedBlock): (JSC::MarkedBlock::Handle::specializedSweep): (JSC::MarkedBlock::Handle::sweep): (JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode): (JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): (JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated): (JSC::SetNewlyAllocatedFunctor::SetNewlyAllocatedFunctor): (JSC::SetNewlyAllocatedFunctor::operator()): (JSC::MarkedBlock::Handle::stopAllocating): (JSC::MarkedBlock::Handle::lastChanceToFinalize): (JSC::MarkedBlock::Handle::resumeAllocating): (JSC::MarkedBlock::Handle::zap): (JSC::MarkedBlock::Handle::forEachFreeCell): (JSC::MarkedBlock::flipIfNecessary): (JSC::MarkedBlock::Handle::flipIfNecessary): (JSC::MarkedBlock::flipIfNecessarySlow): (JSC::MarkedBlock::flipIfNecessaryConcurrentlySlow): (JSC::MarkedBlock::clearMarks): (JSC::MarkedBlock::assertFlipped): (JSC::MarkedBlock::needsFlip): (JSC::MarkedBlock::Handle::needsFlip): (JSC::MarkedBlock::Handle::willRemoveBlock): (JSC::MarkedBlock::Handle::didConsumeFreeList): (JSC::MarkedBlock::markCount): (JSC::MarkedBlock::Handle::isEmpty): (JSC::MarkedBlock::clearHasAnyMarked): (JSC::MarkedBlock::noteMarkedSlow): (WTF::printInternal): (JSC::MarkedBlock::create): Deleted. (JSC::MarkedBlock::destroy): Deleted. (JSC::MarkedBlock::callDestructor): Deleted. (JSC::MarkedBlock::specializedSweep): Deleted. (JSC::MarkedBlock::sweep): Deleted. (JSC::MarkedBlock::sweepHelper): Deleted. (JSC::MarkedBlock::stopAllocating): Deleted. (JSC::MarkedBlock::clearMarksWithCollectionType): Deleted. (JSC::MarkedBlock::lastChanceToFinalize): Deleted. (JSC::MarkedBlock::resumeAllocating): Deleted. (JSC::MarkedBlock::didRetireBlock): Deleted. * heap/MarkedBlock.h: (JSC::MarkedBlock::VoidFunctor::returnValue): (JSC::MarkedBlock::CountFunctor::CountFunctor): (JSC::MarkedBlock::CountFunctor::count): (JSC::MarkedBlock::CountFunctor::returnValue): (JSC::MarkedBlock::Handle::hasAnyNewlyAllocated): (JSC::MarkedBlock::Handle::isOnBlocksToSweep): (JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): (JSC::MarkedBlock::Handle::state): (JSC::MarkedBlock::needsDestruction): (JSC::MarkedBlock::handle): (JSC::MarkedBlock::Handle::block): (JSC::MarkedBlock::firstAtom): (JSC::MarkedBlock::atoms): (JSC::MarkedBlock::isAtomAligned): (JSC::MarkedBlock::Handle::cellAlign): (JSC::MarkedBlock::blockFor): (JSC::MarkedBlock::Handle::allocator): (JSC::MarkedBlock::Handle::heap): (JSC::MarkedBlock::Handle::vm): (JSC::MarkedBlock::vm): (JSC::MarkedBlock::Handle::weakSet): (JSC::MarkedBlock::weakSet): (JSC::MarkedBlock::Handle::shrink): (JSC::MarkedBlock::Handle::visitWeakSet): (JSC::MarkedBlock::Handle::reapWeakSet): (JSC::MarkedBlock::Handle::cellSize): (JSC::MarkedBlock::cellSize): (JSC::MarkedBlock::Handle::attributes): (JSC::MarkedBlock::attributes): (JSC::MarkedBlock::Handle::needsDestruction): (JSC::MarkedBlock::Handle::destruction): (JSC::MarkedBlock::Handle::cellKind): (JSC::MarkedBlock::Handle::markCount): (JSC::MarkedBlock::Handle::size): (JSC::MarkedBlock::atomNumber): (JSC::MarkedBlock::flipIfNecessary): (JSC::MarkedBlock::flipIfNecessaryConcurrently): (JSC::MarkedBlock::Handle::flipIfNecessary): (JSC::MarkedBlock::Handle::flipIfNecessaryConcurrently): (JSC::MarkedBlock::Handle::flipForEdenCollection): (JSC::MarkedBlock::assertFlipped): (JSC::MarkedBlock::Handle::assertFlipped): (JSC::MarkedBlock::isMarked): (JSC::MarkedBlock::testAndSetMarked): (JSC::MarkedBlock::Handle::isNewlyAllocated): (JSC::MarkedBlock::Handle::setNewlyAllocated): (JSC::MarkedBlock::Handle::clearNewlyAllocated): (JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated): (JSC::MarkedBlock::isMarkedOrNewlyAllocated): (JSC::MarkedBlock::Handle::isLive): (JSC::MarkedBlock::isAtom): (JSC::MarkedBlock::Handle::isLiveCell): (JSC::MarkedBlock::Handle::forEachCell): (JSC::MarkedBlock::Handle::forEachLiveCell): (JSC::MarkedBlock::Handle::forEachDeadCell): (JSC::MarkedBlock::Handle::needsSweeping): (JSC::MarkedBlock::Handle::isAllocated): (JSC::MarkedBlock::Handle::isMarked): (JSC::MarkedBlock::Handle::isFreeListed): (JSC::MarkedBlock::hasAnyMarked): (JSC::MarkedBlock::noteMarked): (WTF::MarkedBlockHash::hash): (JSC::MarkedBlock::FreeList::FreeList): Deleted. (JSC::MarkedBlock::allocator): Deleted. (JSC::MarkedBlock::heap): Deleted. (JSC::MarkedBlock::shrink): Deleted. (JSC::MarkedBlock::visitWeakSet): Deleted. (JSC::MarkedBlock::reapWeakSet): Deleted. (JSC::MarkedBlock::willRemoveBlock): Deleted. (JSC::MarkedBlock::didConsumeFreeList): Deleted. (JSC::MarkedBlock::markCount): Deleted. (JSC::MarkedBlock::isEmpty): Deleted. (JSC::MarkedBlock::destruction): Deleted. (JSC::MarkedBlock::cellKind): Deleted. (JSC::MarkedBlock::size): Deleted. (JSC::MarkedBlock::capacity): Deleted. (JSC::MarkedBlock::setMarked): Deleted. (JSC::MarkedBlock::clearMarked): Deleted. (JSC::MarkedBlock::isNewlyAllocated): Deleted. (JSC::MarkedBlock::setNewlyAllocated): Deleted. (JSC::MarkedBlock::clearNewlyAllocated): Deleted. (JSC::MarkedBlock::isLive): Deleted. (JSC::MarkedBlock::isLiveCell): Deleted. (JSC::MarkedBlock::forEachCell): Deleted. (JSC::MarkedBlock::forEachLiveCell): Deleted. (JSC::MarkedBlock::forEachDeadCell): Deleted. (JSC::MarkedBlock::needsSweeping): Deleted. (JSC::MarkedBlock::isAllocated): Deleted. (JSC::MarkedBlock::isMarkedOrRetired): Deleted. * heap/MarkedSpace.cpp: (JSC::MarkedSpace::initializeSizeClassForStepSize): (JSC::MarkedSpace::MarkedSpace): (JSC::MarkedSpace::~MarkedSpace): (JSC::MarkedSpace::lastChanceToFinalize): (JSC::MarkedSpace::allocate): (JSC::MarkedSpace::tryAllocate): (JSC::MarkedSpace::allocateLarge): (JSC::MarkedSpace::tryAllocateLarge): (JSC::MarkedSpace::sweep): (JSC::MarkedSpace::sweepLargeAllocations): (JSC::MarkedSpace::zombifySweep): (JSC::MarkedSpace::resetAllocators): (JSC::MarkedSpace::visitWeakSets): (JSC::MarkedSpace::reapWeakSets): (JSC::MarkedSpace::stopAllocating): (JSC::MarkedSpace::prepareForMarking): (JSC::MarkedSpace::resumeAllocating): (JSC::MarkedSpace::isPagedOut): (JSC::MarkedSpace::freeBlock): (JSC::MarkedSpace::freeOrShrinkBlock): (JSC::MarkedSpace::shrink): (JSC::MarkedSpace::clearNewlyAllocated): (JSC::VerifyMarked::operator()): (JSC::MarkedSpace::flip): (JSC::MarkedSpace::objectCount): (JSC::MarkedSpace::size): (JSC::MarkedSpace::capacity): (JSC::MarkedSpace::addActiveWeakSet): (JSC::MarkedSpace::didAddBlock): (JSC::MarkedSpace::didAllocateInBlock): (JSC::MarkedSpace::forEachAllocator): Deleted. (JSC::VerifyMarkedOrRetired::operator()): Deleted. (JSC::MarkedSpace::clearMarks): Deleted. * heap/MarkedSpace.h: (JSC::MarkedSpace::sizeClassToIndex): (JSC::MarkedSpace::indexToSizeClass): (JSC::MarkedSpace::version): (JSC::MarkedSpace::blocksWithNewObjects): (JSC::MarkedSpace::largeAllocations): (JSC::MarkedSpace::largeAllocationsNurseryOffset): (JSC::MarkedSpace::largeAllocationsOffsetForThisCollection): (JSC::MarkedSpace::largeAllocationsForThisCollectionBegin): (JSC::MarkedSpace::largeAllocationsForThisCollectionEnd): (JSC::MarkedSpace::largeAllocationsForThisCollectionSize): (JSC::MarkedSpace::forEachLiveCell): (JSC::MarkedSpace::forEachDeadCell): (JSC::MarkedSpace::allocatorFor): (JSC::MarkedSpace::destructorAllocatorFor): (JSC::MarkedSpace::auxiliaryAllocatorFor): (JSC::MarkedSpace::allocateWithoutDestructor): (JSC::MarkedSpace::allocateWithDestructor): (JSC::MarkedSpace::allocateAuxiliary): (JSC::MarkedSpace::tryAllocateAuxiliary): (JSC::MarkedSpace::forEachBlock): (JSC::MarkedSpace::forEachAllocator): (JSC::MarkedSpace::optimalSizeFor): (JSC::MarkedSpace::didAddBlock): Deleted. (JSC::MarkedSpace::didAllocateInBlock): Deleted. (JSC::MarkedSpace::objectCount): Deleted. (JSC::MarkedSpace::size): Deleted. (JSC::MarkedSpace::capacity): Deleted. * heap/SlotVisitor.cpp: (JSC::SlotVisitor::SlotVisitor): (JSC::SlotVisitor::didStartMarking): (JSC::SlotVisitor::reset): (JSC::SlotVisitor::append): (JSC::SlotVisitor::appendJSCellOrAuxiliary): (JSC::SlotVisitor::setMarkedAndAppendToMarkStack): (JSC::SlotVisitor::appendToMarkStack): (JSC::SlotVisitor::markAuxiliary): (JSC::SlotVisitor::noteLiveAuxiliaryCell): (JSC::SlotVisitor::visitChildren): * heap/SlotVisitor.h: * heap/WeakBlock.cpp: (JSC::WeakBlock::create): (JSC::WeakBlock::WeakBlock): (JSC::WeakBlock::visit): (JSC::WeakBlock::reap): * heap/WeakBlock.h: (JSC::WeakBlock::disconnectContainer): (JSC::WeakBlock::disconnectMarkedBlock): Deleted. * heap/WeakSet.cpp: (JSC::WeakSet::~WeakSet): (JSC::WeakSet::sweep): (JSC::WeakSet::shrink): (JSC::WeakSet::addAllocator): * heap/WeakSet.h: (JSC::WeakSet::container): (JSC::WeakSet::setContainer): (JSC::WeakSet::WeakSet): (JSC::WeakSet::visit): (JSC::WeakSet::shrink): Deleted. * heap/WeakSetInlines.h: (JSC::WeakSet::allocate): * inspector/InjectedScriptManager.cpp: * inspector/JSGlobalObjectInspectorController.cpp: * inspector/JSJavaScriptCallFrame.cpp: * inspector/ScriptDebugServer.cpp: * inspector/agents/InspectorDebuggerAgent.cpp: * interpreter/CachedCall.h: (JSC::CachedCall::CachedCall): * interpreter/Interpreter.cpp: (JSC::loadVarargs): (JSC::StackFrame::sourceID): Deleted. (JSC::StackFrame::sourceURL): Deleted. (JSC::StackFrame::functionName): Deleted. (JSC::StackFrame::computeLineAndColumn): Deleted. (JSC::StackFrame::toString): Deleted. * interpreter/Interpreter.h: (JSC::StackFrame::isNative): Deleted. * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::emitAllocateWithNonNullAllocator): (JSC::AssemblyHelpers::emitAllocate): (JSC::AssemblyHelpers::emitAllocateJSCell): (JSC::AssemblyHelpers::emitAllocateJSObject): (JSC::AssemblyHelpers::emitAllocateJSObjectWithKnownSize): (JSC::AssemblyHelpers::emitAllocateVariableSized): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): * jit/JIT.cpp: (JSC::JIT::compileCTINativeCall): (JSC::JIT::link): * jit/JIT.h: (JSC::JIT::compileCTINativeCall): Deleted. * jit/JITExceptions.cpp: (JSC::genericUnwind): * jit/JITExceptions.h: * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_new_object): (JSC::JIT::emitSlow_op_new_object): (JSC::JIT::emit_op_create_this): (JSC::JIT::emitSlow_op_create_this): * jit/JITOpcodes32_64.cpp: (JSC::JIT::emit_op_new_object): (JSC::JIT::emitSlow_op_new_object): (JSC::JIT::emit_op_create_this): (JSC::JIT::emitSlow_op_create_this): * jit/JITOperations.cpp: * jit/JITOperations.h: * jit/JITPropertyAccess.cpp: (JSC::JIT::emitWriteBarrier): * jit/JITThunks.cpp: * jit/JITThunks.h: * jsc.cpp: (functionDescribeArray): (main): * llint/LLIntData.cpp: (JSC::LLInt::Data::performAssertions): * llint/LLIntExceptions.cpp: * llint/LLIntThunks.cpp: * llint/LLIntThunks.h: * llint/LowLevelInterpreter.asm: * llint/LowLevelInterpreter.cpp: * llint/LowLevelInterpreter32_64.asm: * llint/LowLevelInterpreter64.asm: * parser/ModuleAnalyzer.cpp: * parser/NodeConstructors.h: * parser/Nodes.h: * profiler/ProfilerBytecode.cpp: * profiler/ProfilerBytecode.h: * profiler/ProfilerBytecodeSequence.cpp: * runtime/ArrayConventions.h: (JSC::indexingHeaderForArrayStorage): (JSC::baseIndexingHeaderForArrayStorage): (JSC::indexingHeaderForArray): Deleted. (JSC::baseIndexingHeaderForArray): Deleted. * runtime/ArrayPrototype.cpp: (JSC::arrayProtoFuncSplice): (JSC::concatAppendOne): (JSC::arrayProtoPrivateFuncConcatMemcpy): * runtime/ArrayStorage.h: (JSC::ArrayStorage::vectorLength): (JSC::ArrayStorage::totalSizeFor): (JSC::ArrayStorage::totalSize): (JSC::ArrayStorage::availableVectorLength): (JSC::ArrayStorage::optimalVectorLength): (JSC::ArrayStorage::sizeFor): Deleted. * runtime/AuxiliaryBarrier.h: Added. (JSC::AuxiliaryBarrier::AuxiliaryBarrier): (JSC::AuxiliaryBarrier::clear): (JSC::AuxiliaryBarrier::get): (JSC::AuxiliaryBarrier::slot): (JSC::AuxiliaryBarrier::operator bool): (JSC::AuxiliaryBarrier::setWithoutBarrier): * runtime/AuxiliaryBarrierInlines.h: Added. (JSC::AuxiliaryBarrier<T>::AuxiliaryBarrier): (JSC::AuxiliaryBarrier<T>::set): * runtime/Butterfly.h: * runtime/ButterflyInlines.h: (JSC::Butterfly::availableContiguousVectorLength): (JSC::Butterfly::optimalContiguousVectorLength): (JSC::Butterfly::createUninitialized): (JSC::Butterfly::growArrayRight): * runtime/ClonedArguments.cpp: (JSC::ClonedArguments::createEmpty): * runtime/CommonSlowPathsExceptions.cpp: * runtime/CommonSlowPathsExceptions.h: * runtime/DataView.cpp: * runtime/DirectArguments.h: * runtime/ECMAScriptSpecInternalFunctions.cpp: * runtime/Error.cpp: * runtime/Error.h: * runtime/ErrorInstance.cpp: * runtime/ErrorInstance.h: * runtime/Exception.cpp: * runtime/Exception.h: * runtime/GeneratorFrame.cpp: * runtime/GeneratorPrototype.cpp: * runtime/InternalFunction.cpp: (JSC::InternalFunction::InternalFunction): * runtime/IntlCollator.cpp: * runtime/IntlCollatorConstructor.cpp: * runtime/IntlCollatorPrototype.cpp: * runtime/IntlDateTimeFormat.cpp: * runtime/IntlDateTimeFormatConstructor.cpp: * runtime/IntlDateTimeFormatPrototype.cpp: * runtime/IntlNumberFormat.cpp: * runtime/IntlNumberFormatConstructor.cpp: * runtime/IntlNumberFormatPrototype.cpp: * runtime/IntlObject.cpp: * runtime/IteratorPrototype.cpp: * runtime/JSArray.cpp: (JSC::JSArray::tryCreateUninitialized): (JSC::JSArray::setLengthWritable): (JSC::JSArray::unshiftCountSlowCase): (JSC::JSArray::setLengthWithArrayStorage): (JSC::JSArray::appendMemcpy): (JSC::JSArray::setLength): (JSC::JSArray::pop): (JSC::JSArray::push): (JSC::JSArray::fastSlice): (JSC::JSArray::shiftCountWithArrayStorage): (JSC::JSArray::shiftCountWithAnyIndexingType): (JSC::JSArray::unshiftCountWithArrayStorage): (JSC::JSArray::fillArgList): (JSC::JSArray::copyToArguments): * runtime/JSArray.h: (JSC::createContiguousArrayButterfly): (JSC::createArrayButterfly): (JSC::JSArray::create): (JSC::JSArray::tryCreateUninitialized): Deleted. * runtime/JSArrayBufferView.h: * runtime/JSCInlines.h: * runtime/JSCJSValue.cpp: (JSC::JSValue::dumpInContextAssumingStructure): * runtime/JSCallee.cpp: (JSC::JSCallee::JSCallee): * runtime/JSCell.cpp: (JSC::JSCell::estimatedSize): * runtime/JSCell.h: (JSC::JSCell::cellStateOffset): Deleted. * runtime/JSCellInlines.h: (JSC::ExecState::vm): (JSC::JSCell::classInfo): (JSC::JSCell::callDestructor): (JSC::JSCell::vm): Deleted. * runtime/JSFunction.cpp: (JSC::JSFunction::create): (JSC::JSFunction::allocateAndInitializeRareData): (JSC::JSFunction::initializeRareData): (JSC::JSFunction::getOwnPropertySlot): (JSC::JSFunction::put): (JSC::JSFunction::deleteProperty): (JSC::JSFunction::defineOwnProperty): (JSC::JSFunction::setFunctionName): (JSC::JSFunction::reifyLength): (JSC::JSFunction::reifyName): (JSC::JSFunction::reifyLazyPropertyIfNeeded): (JSC::JSFunction::reifyBoundNameIfNeeded): * runtime/JSFunction.h: * runtime/JSFunctionInlines.h: (JSC::JSFunction::createWithInvalidatedReallocationWatchpoint): (JSC::JSFunction::JSFunction): * runtime/JSGenericTypedArrayViewInlines.h: (JSC::JSGenericTypedArrayView<Adaptor>::slowDownAndWasteMemory): * runtime/JSInternalPromise.cpp: * runtime/JSInternalPromiseConstructor.cpp: * runtime/JSInternalPromiseDeferred.cpp: * runtime/JSInternalPromisePrototype.cpp: * runtime/JSJob.cpp: * runtime/JSMapIterator.cpp: * runtime/JSModuleNamespaceObject.cpp: * runtime/JSModuleRecord.cpp: * runtime/JSObject.cpp: (JSC::JSObject::visitButterfly): (JSC::JSObject::notifyPresenceOfIndexedAccessors): (JSC::JSObject::createInitialIndexedStorage): (JSC::JSObject::createInitialUndecided): (JSC::JSObject::createInitialInt32): (JSC::JSObject::createInitialDouble): (JSC::JSObject::createInitialContiguous): (JSC::JSObject::createArrayStorage): (JSC::JSObject::createInitialArrayStorage): (JSC::JSObject::convertUndecidedToInt32): (JSC::JSObject::convertUndecidedToContiguous): (JSC::JSObject::convertUndecidedToArrayStorage): (JSC::JSObject::convertInt32ToDouble): (JSC::JSObject::convertInt32ToArrayStorage): (JSC::JSObject::convertDoubleToArrayStorage): (JSC::JSObject::convertContiguousToArrayStorage): (JSC::JSObject::putByIndexBeyondVectorLength): (JSC::JSObject::putDirectIndexBeyondVectorLength): (JSC::JSObject::getNewVectorLength): (JSC::JSObject::increaseVectorLength): (JSC::JSObject::ensureLengthSlow): (JSC::JSObject::growOutOfLineStorage): (JSC::JSObject::copyButterfly): Deleted. (JSC::JSObject::copyBackingStore): Deleted. * runtime/JSObject.h: (JSC::JSObject::globalObject): (JSC::JSObject::putDirectInternal): (JSC::JSObject::setStructureAndReallocateStorageIfNecessary): Deleted. * runtime/JSObjectInlines.h: * runtime/JSPromise.cpp: * runtime/JSPromiseConstructor.cpp: * runtime/JSPromiseDeferred.cpp: * runtime/JSPromisePrototype.cpp: * runtime/JSPropertyNameIterator.cpp: * runtime/JSScope.cpp: (JSC::JSScope::resolve): * runtime/JSScope.h: (JSC::JSScope::globalObject): (JSC::JSScope::vm): Deleted. * runtime/JSSetIterator.cpp: * runtime/JSStringIterator.cpp: * runtime/JSTemplateRegistryKey.cpp: * runtime/JSTypedArrayViewConstructor.cpp: * runtime/JSTypedArrayViewPrototype.cpp: * runtime/JSWeakMap.cpp: * runtime/JSWeakSet.cpp: * runtime/MapConstructor.cpp: * runtime/MapIteratorPrototype.cpp: * runtime/MapPrototype.cpp: * runtime/NativeErrorConstructor.cpp: * runtime/NativeStdFunctionCell.cpp: * runtime/Operations.h: (JSC::scribbleFreeCells): (JSC::scribble): * runtime/Options.h: * runtime/PropertyTable.cpp: * runtime/ProxyConstructor.cpp: * runtime/ProxyObject.cpp: * runtime/ProxyRevoke.cpp: * runtime/RegExp.cpp: (JSC::RegExp::match): (JSC::RegExp::matchConcurrently): (JSC::RegExp::matchCompareWithInterpreter): * runtime/RegExp.h: * runtime/RegExpConstructor.h: * runtime/RegExpInlines.h: (JSC::RegExp::matchInline): * runtime/RegExpMatchesArray.h: (JSC::tryCreateUninitializedRegExpMatchesArray): (JSC::createRegExpMatchesArray): * runtime/RegExpPrototype.cpp: (JSC::genericSplit): * runtime/RuntimeType.cpp: * runtime/SamplingProfiler.cpp: (JSC::SamplingProfiler::processUnverifiedStackTraces): * runtime/SetConstructor.cpp: * runtime/SetIteratorPrototype.cpp: * runtime/SetPrototype.cpp: * runtime/StackFrame.cpp: Added. (JSC::StackFrame::sourceID): (JSC::StackFrame::sourceURL): (JSC::StackFrame::functionName): (JSC::StackFrame::computeLineAndColumn): (JSC::StackFrame::toString): * runtime/StackFrame.h: Added. (JSC::StackFrame::isNative): * runtime/StringConstructor.cpp: * runtime/StringIteratorPrototype.cpp: * runtime/StructureInlines.h: (JSC::Structure::propertyTable): * runtime/TemplateRegistry.cpp: * runtime/TestRunnerUtils.cpp: (JSC::finalizeStatsAtEndOfTesting): * runtime/TestRunnerUtils.h: * runtime/TypeProfilerLog.cpp: * runtime/TypeSet.cpp: * runtime/VM.cpp: (JSC::VM::VM): (JSC::VM::ensureStackCapacityForCLoop): (JSC::VM::isSafeToRecurseSoftCLoop): * runtime/VM.h: * runtime/VMEntryScope.h: * runtime/VMInlines.h: (JSC::VM::ensureStackCapacityFor): (JSC::VM::isSafeToRecurseSoft): * runtime/WeakMapConstructor.cpp: * runtime/WeakMapData.cpp: * runtime/WeakMapPrototype.cpp: * runtime/WeakSetConstructor.cpp: * runtime/WeakSetPrototype.cpp: * testRegExp.cpp: (testOneRegExp): * tools/JSDollarVM.cpp: * tools/JSDollarVMPrototype.cpp: (JSC::JSDollarVMPrototype::isInObjectSpace): Source/WebCore: No new tests because no new WebCore behavior. Just rewiring #includes. * ForwardingHeaders/heap/HeapInlines.h: Added. * ForwardingHeaders/interpreter/Interpreter.h: Removed. * ForwardingHeaders/runtime/AuxiliaryBarrierInlines.h: Added. * Modules/indexeddb/IDBCursorWithValue.cpp: * Modules/indexeddb/client/TransactionOperation.cpp: * Modules/indexeddb/server/SQLiteIDBBackingStore.cpp: * Modules/indexeddb/server/UniqueIDBDatabase.cpp: * bindings/js/JSApplePayPaymentAuthorizedEventCustom.cpp: * bindings/js/JSApplePayPaymentMethodSelectedEventCustom.cpp: * bindings/js/JSApplePayShippingContactSelectedEventCustom.cpp: * bindings/js/JSApplePayShippingMethodSelectedEventCustom.cpp: * bindings/js/JSClientRectCustom.cpp: * bindings/js/JSDOMBinding.cpp: * bindings/js/JSDOMBinding.h: * bindings/js/JSDeviceMotionEventCustom.cpp: * bindings/js/JSDeviceOrientationEventCustom.cpp: * bindings/js/JSErrorEventCustom.cpp: * bindings/js/JSIDBCursorWithValueCustom.cpp: * bindings/js/JSIDBIndexCustom.cpp: * bindings/js/JSPopStateEventCustom.cpp: * bindings/js/JSWebGL2RenderingContextCustom.cpp: * bindings/js/JSWorkerGlobalScopeCustom.cpp: * bindings/js/WorkerScriptController.cpp: * contentextensions/ContentExtensionParser.cpp: * dom/ErrorEvent.cpp: * html/HTMLCanvasElement.cpp: * html/MediaDocument.cpp: * inspector/CommandLineAPIModule.cpp: * loader/EmptyClients.cpp: * page/CaptionUserPreferences.cpp: * page/Frame.cpp: * page/PageGroup.cpp: * page/UserContentController.cpp: * platform/mock/mediasource/MockBox.cpp: * testing/GCObservation.cpp: Source/WebKit2: Just rewiring some #includes. * UIProcess/ViewGestureController.cpp: * UIProcess/WebPageProxy.cpp: * UIProcess/WebProcessPool.cpp: * UIProcess/WebProcessProxy.cpp: * WebProcess/InjectedBundle/DOM/InjectedBundleRangeHandle.cpp: * WebProcess/Plugins/Netscape/JSNPObject.cpp: Source/WTF: I needed tryFastAlignedMalloc() so I added it. * wtf/FastMalloc.cpp: (WTF::tryFastAlignedMalloc): * wtf/FastMalloc.h: * wtf/ParkingLot.cpp: (WTF::ParkingLot::forEachImpl): (WTF::ParkingLot::forEach): Deleted. * wtf/ParkingLot.h: (WTF::ParkingLot::parkConditionally): (WTF::ParkingLot::unparkOne): (WTF::ParkingLot::forEach): * wtf/ScopedLambda.h: (WTF::scopedLambdaRef): * wtf/SentinelLinkedList.h: (WTF::SentinelLinkedList::forEach): (WTF::RawNode>::takeFrom): * wtf/SimpleStats.h: (WTF::SimpleStats::operator bool): (WTF::SimpleStats::operator!): Deleted. Tools: * DumpRenderTree/TestRunner.cpp: * DumpRenderTree/mac/DumpRenderTree.mm: (DumpRenderTreeMain): * Scripts/run-jsc-stress-tests: * TestWebKitAPI/Tests/WTF/Vector.cpp: (TestWebKitAPI::TEST): Canonical link: https://commits.webkit.org/179778@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205462 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-06 01:02:22 +00:00
template<typename FunctionType, typename Functor> class ScopedLambdaRefFunctor;
template<typename ResultType, typename... ArgumentTypes, typename Functor>
class ScopedLambdaRefFunctor<ResultType (ArgumentTypes...), Functor> : public ScopedLambda<ResultType (ArgumentTypes...)> {
public:
ScopedLambdaRefFunctor(const Functor& functor)
: ScopedLambda<ResultType (ArgumentTypes...)>(implFunction, this)
, m_functor(&functor)
{
}
// We need to make sure that copying and moving ScopedLambdaRefFunctor results in a
// ScopedLambdaRefFunctor whose ScopedLambda supertype still points to this rather than
// other.
ScopedLambdaRefFunctor(const ScopedLambdaRefFunctor& other)
: ScopedLambda<ResultType (ArgumentTypes...)>(implFunction, this)
, m_functor(other.m_functor)
{
}
ScopedLambdaRefFunctor(ScopedLambdaRefFunctor&& other)
: ScopedLambda<ResultType (ArgumentTypes...)>(implFunction, this)
, m_functor(other.m_functor)
{
}
ScopedLambdaRefFunctor& operator=(const ScopedLambdaRefFunctor& other)
{
m_functor = other.m_functor;
return *this;
}
ScopedLambdaRefFunctor& operator=(ScopedLambdaRefFunctor&& other)
{
m_functor = other.m_functor;
return *this;
}
private:
static ResultType implFunction(void* argument, ArgumentTypes... arguments)
{
return (*static_cast<ScopedLambdaRefFunctor*>(argument)->m_functor)(arguments...);
}
const Functor* m_functor;
};
// This is for when you already refer to a functor by reference, and you know its lifetime is
// good. This just creates a ScopedLambda that points to your functor.
GC constraint solving should be parallel https://bugs.webkit.org/show_bug.cgi?id=179934 Reviewed by JF Bastien. PerformanceTests: Added a version of splay that measures latency in a way that run-jsc-benchmarks groks. * Octane/splay.js: Added. (this.Setup.setup.setup): (this.TearDown.tearDown.tearDown): (Benchmark): (BenchmarkResult): (BenchmarkResult.prototype.valueOf): (BenchmarkSuite): (alert): (Math.random): (BenchmarkSuite.ResetRNG): (RunStep): (BenchmarkSuite.RunSuites): (BenchmarkSuite.CountBenchmarks): (BenchmarkSuite.GeometricMean): (BenchmarkSuite.GeometricMeanTime): (BenchmarkSuite.AverageAbovePercentile): (BenchmarkSuite.GeometricMeanLatency): (BenchmarkSuite.FormatScore): (BenchmarkSuite.prototype.NotifyStep): (BenchmarkSuite.prototype.NotifyResult): (BenchmarkSuite.prototype.NotifyError): (BenchmarkSuite.prototype.RunSingleBenchmark): (RunNextSetup): (RunNextBenchmark): (RunNextTearDown): (BenchmarkSuite.prototype.RunStep): (GeneratePayloadTree): (GenerateKey): (SplayUpdateStats): (InsertNewNode): (SplaySetup): (SplayTearDown): (SplayRun): (SplayTree): (SplayTree.prototype.isEmpty): (SplayTree.prototype.insert): (SplayTree.prototype.remove): (SplayTree.prototype.find): (SplayTree.prototype.findMax): (SplayTree.prototype.findGreatestLessThan): (SplayTree.prototype.exportKeys): (SplayTree.prototype.splay_): (SplayTree.Node): (SplayTree.Node.prototype.traverse_): (report): (start): Source/JavaScriptCore: This makes it possible to do constraint solving in parallel. This looks like a 1% Speedometer speed-up. It's more than 1% on trunk-Speedometer. The constraint solver supports running constraints in parallel in two different ways: - Run multiple constraints in parallel to each other. This only works for constraints that can tolerate other constraints running concurrently to them (constraint.concurrency() == ConstraintConcurrency::Concurrent). This is the most basic kind of parallelism that the constraint solver supports. All constraints except the JSC SPI constraints are concurrent. We could probably make them concurrent, but I'm playing it safe for now. - A constraint can create parallel work for itself, which the constraint solver will interleave with other stuff. A constraint can report that it has parallel work by returning ConstraintParallelism::Parallel from its executeImpl() function. Then the solver will allow that constraint's doParallelWorkImpl() function to run on as many GC marker threads as are available, for as long as that function wants to run. It's not possible to have a non-concurrent constraint that creates parallel work. The parallelism is implemented in terms of the existing GC marker threads. This turns out to be most natural for two reasons: - No need to start any other threads. - The constraints all want to be passed a SlotVisitor. Running on the marker threads means having access to those threads' SlotVisitors. Also, it means less load balancing. The solver will create work on each marking thread's SlotVisitor. When the solver is done "stealing" a marker thread, that thread will have work it can start doing immediately. Before this change, we had to contribute the work found by the constraint solver to the global worklist so that it could be distributed to the marker threads by load balancing. This change probably helps to avoid that load balancing step. A lot of this change is about making it easy to iterate GC data structures in parallel. This change makes almost all constraints parallel-enabled, but only the DOM's output constraint uses the parallel work API. That constraint iterates the marked cells in two subspaces. This change makes it very easy to compose parallel iterators over subspaces, allocators, blocks, and cells. The marked cell parallel iterator is composed out of parallel iterators for the others. A parallel iterator is just an iterator that can do an atomic next() very quickly. We abstract them using RefPtr<SharedTask<...()>>, where ... is the type returned from the iterator. We know it's done when it returns a falsish version of ... (in the current code, that's always a pointer type, so done is indicated by null). * API/JSMarkingConstraintPrivate.cpp: (JSContextGroupAddMarkingConstraint): * API/JSVirtualMachine.mm: (scanExternalObjectGraph): (scanExternalRememberedSet): * JavaScriptCore.xcodeproj/project.pbxproj: * Sources.txt: * bytecode/AccessCase.cpp: (JSC::AccessCase::propagateTransitions const): * bytecode/CodeBlock.cpp: (JSC::CodeBlock::visitWeakly): (JSC::CodeBlock::shouldJettisonDueToOldAge): (JSC::shouldMarkTransition): (JSC::CodeBlock::propagateTransitions): (JSC::CodeBlock::determineLiveness): * dfg/DFGWorklist.cpp: * ftl/FTLCompile.cpp: (JSC::FTL::compile): * heap/ConstraintParallelism.h: Added. (WTF::printInternal): * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::addToRememberedSet): (JSC::Heap::runFixpointPhase): (JSC::Heap::stopThePeriphery): (JSC::Heap::resumeThePeriphery): (JSC::Heap::addCoreConstraints): (JSC::Heap::setBonusVisitorTask): (JSC::Heap::runTaskInParallel): (JSC::Heap::forEachSlotVisitor): Deleted. * heap/Heap.h: (JSC::Heap::worldIsRunning const): (JSC::Heap::runFunctionInParallel): * heap/HeapInlines.h: (JSC::Heap::worldIsStopped const): (JSC::Heap::isMarked): (JSC::Heap::incrementDeferralDepth): (JSC::Heap::decrementDeferralDepth): (JSC::Heap::decrementDeferralDepthAndGCIfNeeded): (JSC::Heap::forEachSlotVisitor): (JSC::Heap::collectorBelievesThatTheWorldIsStopped const): Deleted. (JSC::Heap::isMarkedConcurrently): Deleted. * heap/HeapSnapshotBuilder.cpp: (JSC::HeapSnapshotBuilder::appendNode): * heap/LargeAllocation.h: (JSC::LargeAllocation::isMarked): (JSC::LargeAllocation::isMarkedConcurrently): Deleted. * heap/LockDuringMarking.h: (JSC::lockDuringMarking): * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::parallelNotEmptyBlockSource): * heap/MarkedAllocator.h: * heap/MarkedBlock.h: (JSC::MarkedBlock::aboutToMark): (JSC::MarkedBlock::isMarked): (JSC::MarkedBlock::areMarksStaleWithDependency): Deleted. (JSC::MarkedBlock::isMarkedConcurrently): Deleted. * heap/MarkedSpace.h: (JSC::MarkedSpace::activeWeakSetsBegin): (JSC::MarkedSpace::activeWeakSetsEnd): (JSC::MarkedSpace::newActiveWeakSetsBegin): (JSC::MarkedSpace::newActiveWeakSetsEnd): * heap/MarkingConstraint.cpp: (JSC::MarkingConstraint::MarkingConstraint): (JSC::MarkingConstraint::execute): (JSC::MarkingConstraint::quickWorkEstimate): (JSC::MarkingConstraint::workEstimate): (JSC::MarkingConstraint::doParallelWork): (JSC::MarkingConstraint::finishParallelWork): (JSC::MarkingConstraint::doParallelWorkImpl): (JSC::MarkingConstraint::finishParallelWorkImpl): * heap/MarkingConstraint.h: (JSC::MarkingConstraint::lastExecuteParallelism const): (JSC::MarkingConstraint::parallelism const): (JSC::MarkingConstraint::quickWorkEstimate): Deleted. (JSC::MarkingConstraint::workEstimate): Deleted. * heap/MarkingConstraintSet.cpp: (JSC::MarkingConstraintSet::MarkingConstraintSet): (JSC::MarkingConstraintSet::add): (JSC::MarkingConstraintSet::executeConvergence): (JSC::MarkingConstraintSet::executeConvergenceImpl): (JSC::MarkingConstraintSet::executeAll): (JSC::MarkingConstraintSet::ExecutionContext::ExecutionContext): Deleted. (JSC::MarkingConstraintSet::ExecutionContext::didVisitSomething const): Deleted. (JSC::MarkingConstraintSet::ExecutionContext::shouldTimeOut const): Deleted. (JSC::MarkingConstraintSet::ExecutionContext::drain): Deleted. (JSC::MarkingConstraintSet::ExecutionContext::didExecute const): Deleted. (JSC::MarkingConstraintSet::ExecutionContext::execute): Deleted. (): Deleted. * heap/MarkingConstraintSet.h: * heap/MarkingConstraintSolver.cpp: Added. (JSC::MarkingConstraintSolver::MarkingConstraintSolver): (JSC::MarkingConstraintSolver::~MarkingConstraintSolver): (JSC::MarkingConstraintSolver::didVisitSomething const): (JSC::MarkingConstraintSolver::execute): (JSC::MarkingConstraintSolver::drain): (JSC::MarkingConstraintSolver::converge): (JSC::MarkingConstraintSolver::runExecutionThread): (JSC::MarkingConstraintSolver::didExecute): * heap/MarkingConstraintSolver.h: Added. * heap/OpaqueRootSet.h: Removed. * heap/ParallelSourceAdapter.h: Added. (JSC::ParallelSourceAdapter::ParallelSourceAdapter): (JSC::createParallelSourceAdapter): * heap/SimpleMarkingConstraint.cpp: Added. (JSC::SimpleMarkingConstraint::SimpleMarkingConstraint): (JSC::SimpleMarkingConstraint::~SimpleMarkingConstraint): (JSC::SimpleMarkingConstraint::quickWorkEstimate): (JSC::SimpleMarkingConstraint::executeImpl): * heap/SimpleMarkingConstraint.h: Added. * heap/SlotVisitor.cpp: (JSC::SlotVisitor::didStartMarking): (JSC::SlotVisitor::reset): (JSC::SlotVisitor::appendToMarkStack): (JSC::SlotVisitor::visitChildren): (JSC::SlotVisitor::updateMutatorIsStopped): (JSC::SlotVisitor::mutatorIsStoppedIsUpToDate const): (JSC::SlotVisitor::drain): (JSC::SlotVisitor::performIncrementOfDraining): (JSC::SlotVisitor::didReachTermination): (JSC::SlotVisitor::hasWork): (JSC::SlotVisitor::drainFromShared): (JSC::SlotVisitor::drainInParallelPassively): (JSC::SlotVisitor::waitForTermination): (JSC::SlotVisitor::addOpaqueRoot): Deleted. (JSC::SlotVisitor::containsOpaqueRoot const): Deleted. (JSC::SlotVisitor::containsOpaqueRootTriState const): Deleted. (JSC::SlotVisitor::mergeIfNecessary): Deleted. (JSC::SlotVisitor::mergeOpaqueRootsIfProfitable): Deleted. (JSC::SlotVisitor::mergeOpaqueRoots): Deleted. * heap/SlotVisitor.h: * heap/SlotVisitorInlines.h: (JSC::SlotVisitor::addOpaqueRoot): (JSC::SlotVisitor::containsOpaqueRoot const): (JSC::SlotVisitor::vm): (JSC::SlotVisitor::vm const): * heap/Subspace.cpp: (JSC::Subspace::parallelAllocatorSource): (JSC::Subspace::parallelNotEmptyMarkedBlockSource): * heap/Subspace.h: * heap/SubspaceInlines.h: (JSC::Subspace::forEachMarkedCellInParallel): * heap/VisitCounter.h: Added. (JSC::VisitCounter::VisitCounter): (JSC::VisitCounter::visitCount const): * heap/VisitingTimeout.h: Removed. * heap/WeakBlock.cpp: (JSC::WeakBlock::specializedVisit): * runtime/Structure.cpp: (JSC::Structure::isCheapDuringGC): (JSC::Structure::markIfCheap): Source/WebCore: No new tests because no change in behavior. This change is best tested using DOM-GC-intensive benchmarks like Speedometer and Dromaeo. This parallelizes the DOM's output constraint, and makes some small changes to make this more scalable. * ForwardingHeaders/heap/SimpleMarkingConstraint.h: Added. * ForwardingHeaders/heap/VisitingTimeout.h: Removed. * Sources.txt: * WebCore.xcodeproj/project.pbxproj: * bindings/js/DOMGCOutputConstraint.cpp: Added. (WebCore::DOMGCOutputConstraint::DOMGCOutputConstraint): (WebCore::DOMGCOutputConstraint::~DOMGCOutputConstraint): (WebCore::DOMGCOutputConstraint::executeImpl): (WebCore::DOMGCOutputConstraint::doParallelWorkImpl): (WebCore::DOMGCOutputConstraint::finishParallelWorkImpl): * bindings/js/DOMGCOutputConstraint.h: Added. * bindings/js/WebCoreJSClientData.cpp: (WebCore::JSVMClientData::initNormalWorld): * dom/Node.cpp: (WebCore::Node::eventTargetDataConcurrently): (WebCore::Node::ensureEventTargetData): (WebCore::Node::clearEventTargetData): Source/WTF: This does some changes to make it easier to do parallel constraint solving: - I finally removed dependencyWith. This was a silly construct whose only purpose is to confuse people about what it means to have a dependency chain. I took that as an opportunity to grealy simplify the GC's use of dependency chaining. - Added more logic to Deque<>, since I use it for part of the load balancer. - Made it possible to profile lock contention. See https://bugs.webkit.org/show_bug.cgi?id=180250#c0 for some preliminary measurements. - Introduced holdLockIf, which makes it easy to perform predicated lock acquisition. We use that to pick a lock in WebCore. - Introduced CountingLock. It's like WTF::Lock except it also enables optimistic read transactions sorta like Java's StampedLock. * WTF.xcodeproj/project.pbxproj: * wtf/Atomics.h: (WTF::dependency): (WTF::DependencyWith::DependencyWith): Deleted. (WTF::dependencyWith): Deleted. * wtf/BitVector.h: (WTF::BitVector::iterator::operator++): * wtf/CMakeLists.txt: * wtf/ConcurrentPtrHashSet.cpp: Added. (WTF::ConcurrentPtrHashSet::ConcurrentPtrHashSet): (WTF::ConcurrentPtrHashSet::~ConcurrentPtrHashSet): (WTF::ConcurrentPtrHashSet::deleteOldTables): (WTF::ConcurrentPtrHashSet::clear): (WTF::ConcurrentPtrHashSet::initialize): (WTF::ConcurrentPtrHashSet::addSlow): (WTF::ConcurrentPtrHashSet::resizeIfNecessary): (WTF::ConcurrentPtrHashSet::resizeAndAdd): (WTF::ConcurrentPtrHashSet::Table::create): * wtf/ConcurrentPtrHashSet.h: Added. (WTF::ConcurrentPtrHashSet::contains): (WTF::ConcurrentPtrHashSet::add): (WTF::ConcurrentPtrHashSet::size const): (WTF::ConcurrentPtrHashSet::Table::maxLoad const): (WTF::ConcurrentPtrHashSet::hash): (WTF::ConcurrentPtrHashSet::cast): (WTF::ConcurrentPtrHashSet::containsImpl const): (WTF::ConcurrentPtrHashSet::addImpl): * wtf/Deque.h: (WTF::inlineCapacity>::takeFirst): * wtf/FastMalloc.h: * wtf/Lock.cpp: (WTF::LockBase::lockSlow): * wtf/Locker.h: (WTF::holdLockIf): * wtf/ScopedLambda.h: * wtf/SharedTask.h: (WTF::SharedTask<PassedResultType): (WTF::SharedTask<ResultType): Deleted. * wtf/StackShot.h: Added. (WTF::StackShot::StackShot): (WTF::StackShot::operator=): (WTF::StackShot::array const): (WTF::StackShot::size const): (WTF::StackShot::operator bool const): (WTF::StackShot::operator== const): (WTF::StackShot::hash const): (WTF::StackShot::isHashTableDeletedValue const): (WTF::StackShot::operator> const): (WTF::StackShot::deletedValueArray): (WTF::StackShotHash::hash): (WTF::StackShotHash::equal): * wtf/StackShotProfiler.h: Added. (WTF::StackShotProfiler::StackShotProfiler): (WTF::StackShotProfiler::profile): (WTF::StackShotProfiler::run): Tools: * Scripts/run-jsc-benchmarks: Add splay-latency test, since this change needed to be carefully validated with that benchmark. * TestWebKitAPI/CMakeLists.txt: * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj: * TestWebKitAPI/Tests/WTF/ConcurrentPtrHashSet.cpp: Added. This has unit tests of the new concurrent data structure. The tests focus on correctness under serial execution, which appears to be enough for now (it's so easy to catch a concurrency bug by just running the GC). (TestWebKitAPI::TEST): Canonical link: https://commits.webkit.org/196360@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@225524 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2017-12-05 17:53:57 +00:00
//
// Note that this is always wrong:
//
// auto ref = scopedLambdaRef([...] (...) {...});
//
// Because the scopedLambdaRef will refer to the lambda by reference, and the lambda will die after the
// semicolon. Use scopedLambda() in that case.
Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace and we should rewrite as much of the GC as needed to make this not a regression https://bugs.webkit.org/show_bug.cgi?id=160125 Reviewed by Geoffrey Garen and Keith Miller. JSTests: Most of the things I did properly covered by existing tests, but I found some simple cases of unshifting that had sketchy coverage. * stress/array-storage-array-unshift.js: Added. * stress/contiguous-array-unshift.js: Added. * stress/double-array-unshift.js: Added. * stress/int32-array-unshift.js: Added. Source/bmalloc: I needed to tryMemalign, so I added such a thing. * bmalloc/Allocator.cpp: (bmalloc::Allocator::allocate): (bmalloc::Allocator::tryAllocate): (bmalloc::Allocator::allocateImpl): * bmalloc/Allocator.h: * bmalloc/Cache.h: (bmalloc::Cache::tryAllocate): * bmalloc/bmalloc.h: (bmalloc::api::tryMemalign): Source/JavaScriptCore: In order to make the GC concurrent (bug 149432), we would either need to enable concurrent copying or we would need to not copy. Concurrent copying carries a 1-2% throughput overhead from the barriers alone. Considering that MarkedSpace does a decent job of avoiding fragmentation, it's unlikely that it's worth paying 1-2% throughput for copying. So, we want to get rid of copied space. This change moves copied space's biggest client over to marked space. Moving butterflies to marked space means having them use the new Auxiliary HeapCell allocation path. This is a fairly mechanical change, but it caused performance regressions everywhere, so this change also fixes MarkedSpace's performance issues. At a high level the mechanical changes are: - We use AuxiliaryBarrier instead of CopyBarrier. - We use tryAllocateAuxiliary instead of tryAllocateStorage. I got rid of the silly CheckedBoolean stuff, since it's so much more trouble than it's worth. - The JITs have to emit inlined marked space allocations instead of inline copy space allocations. - Everyone has to get used to zeroing their butterflies after allocation instead of relying on them being pre-zeroed by the GC. Copied space would zero things for you, while marked space doesn't. That's about 1/3 of this change. But this led to performance problems, which I fixed with optimizations that amounted to a major MarkedSpace rewrite: - MarkedSpace always causes internal fragmentation for array allocations because the vector length we choose when we resize usually leads to a cell size that doesn't correspond to any size class. I got around this by making array allocations usually round up vectorLength to the maximum allowed by the size class that we would have allocated in. Also, ensureLengthSlow() and friends first make sure that the requested length can't just be fulfilled with the current allocation size. This safeguard means that not every array allocation has to do size class queries. For example, the fast path of new Array(length) never does any size class queries, under the assumption that (1) the speed gained from avoiding an ensureLengthSlow() call, which then just changes the vectorLength by doing the size class query, is too small to offset the speed lost by doing the query on every allocation and (2) new Array(length) is a pretty good hint that resizing is not very likely. - Size classes in MarkedSpace were way too precise, which led to external fragmentation. This changes MarkedSpace size classes to use a linear progression for very small sizes followed by a geometric progression that naturally transitions to a hyperbolic progression. We want hyperbolic sizes when we get close to blockSize: for example the largest size we want is payloadSize / 2 rounded down, to ensure we get exactly two cells with minimal slop. The next size down should be payloadSize / 3 rounded down, and so on. After the last precise size (80 bytes), we proceed using a geometric progression, but round up each size to minimize slop at the end of the block. This naturally causes the geometric progression to turn hyperbolic for large sizes. The size class configuration happens at VM start-up, so it can be controlled with runtime options. I found that a base of 1.4 works pretty well. - Large allocations caused massive internal fragmentation, since the smallest large allocation had to use exactly blockSize, and the largest small allocation used blockSize / 2. The next size up - the first large allocation size to require two blocks - also had 50% internal fragmentation. This is because we required large allocations to be blockSize aligned, so that MarkedBlock::blockFor() would work. I decided to rewrite all of that. Cells no longer have to be owned by a MarkedBlock. They can now alternatively be owned by a LargeAllocation. These two things are abstracted as CellContainer. You know that a cell is owned by a LargeAllocation if the MarkedBlock::atomSize / 2 bit is set. Basically, large allocations are deliberately misaligned by 8 bytes. This actually works out great since (1) typed arrays won't use large allocations anyway since they have their own malloc fallback and (2) large array butterflies already have a 8 byte header, which means that the 8 byte base misalignment aligns the large array payload on a 16 byte boundary. I took extreme care to make sure that the isLargeAllocation bit checks are as rare as possible; for example, ExecState::vm() skips the check because we know that callees must be small allocations. It's also possible to use template tricks to do one check for cell container kind, and then invoke a function specialized for MarkedBlock or a function specialized for LargeAllocation. LargeAllocation includes stubs for all MarkedBlock methods that get used from functions that are template-specialized like this. That's mostly to speed up the GC marking code. Most other code can use CellContainer API or HeapCell API directly. That's another thing: HeapCell, the common base of JSCell and auxiliary allocations, is now smart enough to do a lot of things for you, like HeapCell::vm(), HeapCell::heap(), HeapCell::isLargeAllocation(), and HeapCell::cellContainer(). The size cutoff for large allocations is runtime-configurable, so long as you don't choose something so small that callees end up large. I found that 400 bytes is roughly optimal. This means that the MarkedBlock size classes end up being: 16, 32, 48, 64, 80, 112, 160, 224, 320 The next size class would have been 432, but that's above the 400 byte cutoff. All of this is configurable with --sizeClassProgression and --largeAllocationCutoff. You can see what size classes you end up with by doing --dumpSizeClasses=true. - Copied space uses 64KB blocks, while marked space used to use 16KB blocks. Allocating a lot of stuff in 16KB blocks was slower than allocating it in 64KB blocks because the GC had a lot of per-block overhead. I removed this overhead: It's now 2x faster to scan all MarkedBlocks because the list that contains the interesting meta-data is allocated on the side, for better locality during a sequential walk. It's no longer necessary to scan MarkedBlocks to find WeakSets, since the sets of WeakSets for eden scan and full scan are maintained on-the-fly. It's no longer necessary to scan all MarkedBlocks to clear mark bits because we now use versioned mark bits: to clear then, just increment the 64-bit heap version. It's no longer necessary to scan retired MarkedBlocks while allocating because marking retires them on-the-fly. It's no longer necessary to sort all blocks in the IncrementalSweeper's snapshot because blocks now know if they are in the snapshot. Put together, these optimizations allowed me to reduce block size to 16KB without losing much performance. There is some small perf loss on JetStream/splay, but not enough to hurt JetStream overall. I tried reducing block sizes further, to 4KB, since that is a progression on membuster. That's not possible yet, since there is still enough per-block overhead yet that such a reduction hurts JetStream too much. I filed a bug about improving this further: https://bugs.webkit.org/show_bug.cgi?id=161581. - Even after all of that, copying butterflies was still faster because it allowed us to skip sweeping dead space. A good GC allocates over dead bytes without explicitly freeing them, so the GC pause is O(size of live), not O(size of live + dead). O(dead) is usually much larger than O(live), especially in an eden collection. Copying satisfies this premise while mark+sweep does not. So, I invented a new kind of allocator: bump'n'pop. Previously, our MarkedSpace allocator was a freelist pop. That's simple and easy to inline but requires that we walk the block to build a free list. This means walking dead space. The new allocator allows totally free MarkedBlocks to simply set up a bump-pointer arena instead. The allocator is a hybrid of bump-pointer and freelist pop. It tries bump first. The bump pointer always bumps by cellSize, so the result of filling a block with bumping looks as if we had used freelist popping to fill it. Additionally, each MarkedBlock now has a bit to quickly tell if the block is entirely free. This makes sweeping O(1) whenever a MarkedBlock is completely empty, which is the common case because of the generational hypothesis: the number of objects that survive an eden collection is a tiny fraction of the number of objects that had been allocated, and this fraction is so small that there are typically fewer than one survivors per MarkedBlock. This change was enough to make this change a net win over tip-of-tree. - FTL now shares the same allocation fast paths as everything else, which is great, because bump'n'pop has gnarly control flow. We don't really want B3 to have to think about that control flow, since it won't be able to improve the machine code we write ourselves. GC fast paths are best written in assembly. So, I've empowered B3 to have even better support for Patchpoint terminals. It's now totally fine for a Patchpoint terminal to be non-Void. So, the new FTL allocation fast paths are just Patchpoint terminals that call through to AssemblyHelpers::emitAllocate(). B3 still reasons about things like constant-folding the size class calculation and constant-hoisting the allocator. Also, I gave the FTL the ability to constant-fold some allocator logic (in case we first assume that we're doing a variable-length allocation but then realize that the length is known). I think it makes sense to have constant folding rules in FTL::Output, or whatever the B3 IR builder is, since this makes lowering easier (you can constant fold during lowering more easily) and it reduces the amount of malloc traffic. In the future, we could teach B3 how to better constant-fold this code. That would require allowing loads to be constant-folded, which is doable but hella tricky. - It used to be that if a logical object allocation required two physical allocations (first the butterfly and then the cell), then the JIT would emit the code in such a way that a failure in the second fast path would cause us to forget the successful first physical allocation. This was pointlessly wasteful. It turns out that it's very cheap to devote a register to storing either the butterfly or null, because the butterfly register is anyway going to be free inside the first allocation. The only overhead here is zeroing the butterfly register. With that in place, we can just pass the butterfly-or-null to the slow path, which can then either allocate a butterfly or not. So now we never waste a successful allocation. This patch implements such a solution both in DFG (where it's easy to do this since we control registers already) and in FTL (where it's annoying, because mutable "butterfly-or-null" variables are hard to say in SSA; also I realized that we had code duplicated the JSArray allocation utility, so I deduplicated it). This came up because in one version of this patch, this wastage would resonate with some Kraken benchmark: the benchmark would always allocate N small things followed by one bigger thing. The problem was I accidentally adjusted the various fixed overheads in MarkedBlock in such a way that the JSObject size class, which both the small and big thing shared for their cell, could hold exactly N cells per MarkedBlock. Then the benchmark would always call slow path when it allocated the big thing. So, it would end up having to allocate the big thing's large butterfly twice, every single time! Ouch! - It used to be that we zeroed CopiedBlocks using memset, and so array allocations enjoyed amortization of the cost of zeroing. This doesn't work anymore - it's now up to the client of the allocator to initialize the object to whatever state they need. It used to be that we would just use a dumb loop. I initially changed this so that we would end up in memset for large allocations, but this didn't actually help performance that much. I got a much better result by playing with different memsets written in assembly. First I wrote one using non-temporal stores. That was a small speed-up over memset. Then I tried the classic "rep stos" approach, and holy cow that version was fast. It's a ~20% speed-up on array allocation microbenchmarks. So, this patch adds code paths to do "rep stos" on x86_64, or memset, or use a loop, as appropriate, for both "contiguous" arrays (holes are zero) and double arrays (holes are PNaN). Note that the JIT always emits either a loop or a flat slab of stores (if the size is known), but those paths in the JIT won't trigger for NewArrayWithSize() if the size is large, since that takes us to the operationNewArrayWithSize() slow path, which calls into JSArray::create(). That's why the optimizations here are all in JSArray::create() - that's the hot place for large arrays that need to be filled with holes. All of this put together gives us neutral perf on JetStream, membuster, and PLT3, a ~1% regression on Speedometer, and up to a 4% regression Kraken. The Kraken regression is because Kraken was allocating exactly 1024 element arrays at a rate of 400MB/sec. This is a best-case scenario for bump allocation. I think that we should fix bmalloc to make up the difference, but take the hit for now because it's a crazy corner case. By comparison, the alternative approach of using a copy barrier would have cost us 1-2%. That's the real apples-to-apples comparison if your premise is that we should have a concurrent GC. After we finish removing copied space, we will be barrier-ready for concurrent GC: we already have a marking barrier and we simply won't need a copying barrier. This change gets us there for the purposes of our benchmarks, since the remaining clients of copied space are not very important. On the other hand, if we keep copying, then getting barrier-ready would mean adding back the copy barrier, which costs more perf. We might get bigger speed-ups once we remove CopiedSpace altogether. That requires moving typed arrays and a few other weird things over to Aux MarkedSpace. This also includes some header sanitization. The introduction of AuxiliaryBarrier, HeapCell, and CellContainer meant that I had to include those files from everywhere. Fortunately, just including JSCInlines.h (instead of manually including the files that includes) is usually enough. So, I made most of JSC's cpp files include JSCInlines.h, which is something that we were already basically doing. In places where JSCInlines.h would be too much, I just included HeapInlines.h. This got weird, because we previously included HeapInlines.h from JSObject.h. That's bad because it led to some circular dependencies, so I fixed it - but that meant having to manually include HeapInlines.h from the places that previously got it implicitly via JSObject.h. But that led to more problems for some reason: I started getting build errors because non-JSC files were having trouble including Opcode.h. That's just silly, since Opcode.h is meant to be an internal JSC header. So, I made it an internal header and made it impossible to include it from outside JSC. This was a lot of work, but it was necessary to get the patch to build on all ports. It's also a net win. There were many places in WebCore that were transitively including a *ton* of JSC headers just because of the JSObject.h->HeapInlines.h edge and a bunch of dependency edges that arose from some public (for WebCore) JSC headers needing Interpreter.h or Opcode.h for bad reasons. * API/JSManagedValue.mm: (-[JSManagedValue initWithValue:]): * API/JSTypedArray.cpp: * API/ObjCCallbackFunction.mm: * API/tests/testapi.mm: (testObjectiveCAPI): (testWeakValue): Deleted. * CMakeLists.txt: * JavaScriptCore.xcodeproj/project.pbxproj: * Scripts/builtins/builtins_generate_combined_implementation.py: (BuiltinsCombinedImplementationGenerator.generate_secondary_header_includes): * Scripts/builtins/builtins_generate_internals_wrapper_implementation.py: (BuiltinsInternalsWrapperImplementationGenerator.generate_secondary_header_includes): * Scripts/builtins/builtins_generate_separate_implementation.py: (BuiltinsSeparateImplementationGenerator.generate_secondary_header_includes): * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::JumpList::link): (JSC::AbstractMacroAssembler::JumpList::linkTo): * assembler/MacroAssembler.h: * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::add32): * assembler/MacroAssemblerCodeRef.cpp: Added. (JSC::MacroAssemblerCodePtr::createLLIntCodePtr): (JSC::MacroAssemblerCodePtr::dumpWithName): (JSC::MacroAssemblerCodePtr::dump): (JSC::MacroAssemblerCodeRef::createLLIntCodeRef): (JSC::MacroAssemblerCodeRef::dump): * assembler/MacroAssemblerCodeRef.h: (JSC::MacroAssemblerCodePtr::createLLIntCodePtr): Deleted. (JSC::MacroAssemblerCodePtr::dumpWithName): Deleted. (JSC::MacroAssemblerCodePtr::dump): Deleted. (JSC::MacroAssemblerCodeRef::createLLIntCodeRef): Deleted. (JSC::MacroAssemblerCodeRef::dump): Deleted. * b3/B3BasicBlock.cpp: (JSC::B3::BasicBlock::appendBoolConstant): * b3/B3BasicBlock.h: * b3/B3DuplicateTails.cpp: * b3/B3StackmapGenerationParams.h: * b3/testb3.cpp: (JSC::B3::testPatchpointTerminalReturnValue): (JSC::B3::run): * bindings/ScriptValue.cpp: * bytecode/AdaptiveInferredPropertyValueWatchpointBase.cpp: * bytecode/BytecodeBasicBlock.cpp: * bytecode/BytecodeLivenessAnalysis.cpp: * bytecode/BytecodeUseDef.h: * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::callTypeFor): * bytecode/CallLinkInfo.h: (JSC::CallLinkInfo::callTypeFor): Deleted. * bytecode/CallLinkStatus.cpp: * bytecode/CodeBlock.cpp: (JSC::CodeBlock::finishCreation): (JSC::CodeBlock::clearLLIntGetByIdCache): (JSC::CodeBlock::predictedMachineCodeSize): * bytecode/CodeBlock.h: (JSC::CodeBlock::jitCodeMap): Deleted. (JSC::clearLLIntGetByIdCache): Deleted. * bytecode/ExecutionCounter.h: * bytecode/Instruction.h: * bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp: (JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal): * bytecode/ObjectAllocationProfile.h: (JSC::ObjectAllocationProfile::isNull): (JSC::ObjectAllocationProfile::initialize): * bytecode/Opcode.h: (JSC::padOpcodeName): * bytecode/PolymorphicAccess.cpp: (JSC::AccessCase::generateImpl): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: * bytecode/PreciseJumpTargets.cpp: * bytecode/StructureStubInfo.cpp: * bytecode/StructureStubInfo.h: * bytecode/UnlinkedCodeBlock.cpp: (JSC::UnlinkedCodeBlock::vm): Deleted. * bytecode/UnlinkedCodeBlock.h: * bytecode/UnlinkedInstructionStream.cpp: * bytecode/UnlinkedInstructionStream.h: * dfg/DFGOperations.cpp: * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::emitAllocateRawObject): (JSC::DFG::SpeculativeJIT::compileMakeRope): (JSC::DFG::SpeculativeJIT::compileAllocatePropertyStorage): (JSC::DFG::SpeculativeJIT::compileReallocatePropertyStorage): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::emitAllocateJSCell): (JSC::DFG::SpeculativeJIT::emitAllocateJSObject): * dfg/DFGSpeculativeJIT32_64.cpp: (JSC::DFG::SpeculativeJIT::compile): (JSC::DFG::SpeculativeJIT::compileAllocateNewArrayWithSize): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::compile): (JSC::DFG::SpeculativeJIT::compileAllocateNewArrayWithSize): * dfg/DFGStrengthReductionPhase.cpp: (JSC::DFG::StrengthReductionPhase::handleNode): * ftl/FTLAbstractHeapRepository.h: * ftl/FTLCompile.cpp: * ftl/FTLJITFinalizer.cpp: * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToB3::compileCreateRest): (JSC::FTL::DFG::LowerDFGToB3::allocateArrayWithSize): (JSC::FTL::DFG::LowerDFGToB3::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToB3::compileMakeRope): (JSC::FTL::DFG::LowerDFGToB3::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToB3::initializeArrayElements): (JSC::FTL::DFG::LowerDFGToB3::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToB3::allocateHeapCell): (JSC::FTL::DFG::LowerDFGToB3::allocateCell): (JSC::FTL::DFG::LowerDFGToB3::allocateObject): (JSC::FTL::DFG::LowerDFGToB3::allocatorForSize): (JSC::FTL::DFG::LowerDFGToB3::allocateVariableSizedObject): (JSC::FTL::DFG::LowerDFGToB3::allocateJSArray): (JSC::FTL::DFG::LowerDFGToB3::compileAllocateArrayWithSize): Deleted. * ftl/FTLOutput.cpp: (JSC::FTL::Output::constBool): (JSC::FTL::Output::add): (JSC::FTL::Output::shl): (JSC::FTL::Output::aShr): (JSC::FTL::Output::lShr): (JSC::FTL::Output::zeroExt): (JSC::FTL::Output::equal): (JSC::FTL::Output::notEqual): (JSC::FTL::Output::above): (JSC::FTL::Output::aboveOrEqual): (JSC::FTL::Output::below): (JSC::FTL::Output::belowOrEqual): (JSC::FTL::Output::greaterThan): (JSC::FTL::Output::greaterThanOrEqual): (JSC::FTL::Output::lessThan): (JSC::FTL::Output::lessThanOrEqual): (JSC::FTL::Output::select): (JSC::FTL::Output::appendSuccessor): (JSC::FTL::Output::addIncomingToPhi): * ftl/FTLOutput.h: * ftl/FTLValueFromBlock.h: (JSC::FTL::ValueFromBlock::operator bool): (JSC::FTL::ValueFromBlock::ValueFromBlock): Deleted. * ftl/FTLWeightedTarget.h: (JSC::FTL::WeightedTarget::frequentedBlock): * heap/CellContainer.h: Added. (JSC::CellContainer::CellContainer): (JSC::CellContainer::operator bool): (JSC::CellContainer::isMarkedBlock): (JSC::CellContainer::isLargeAllocation): (JSC::CellContainer::markedBlock): (JSC::CellContainer::largeAllocation): * heap/CellContainerInlines.h: Added. (JSC::CellContainer::isMarked): (JSC::CellContainer::isMarkedOrNewlyAllocated): (JSC::CellContainer::noteMarked): (JSC::CellContainer::cellSize): (JSC::CellContainer::weakSet): (JSC::CellContainer::flipIfNecessary): * heap/ConservativeRoots.cpp: (JSC::ConservativeRoots::ConservativeRoots): (JSC::ConservativeRoots::~ConservativeRoots): (JSC::ConservativeRoots::grow): (JSC::ConservativeRoots::genericAddPointer): (JSC::ConservativeRoots::genericAddSpan): * heap/ConservativeRoots.h: (JSC::ConservativeRoots::roots): * heap/CopyToken.h: * heap/FreeList.cpp: Added. (JSC::FreeList::dump): * heap/FreeList.h: Added. (JSC::FreeList::FreeList): (JSC::FreeList::list): (JSC::FreeList::bump): (JSC::FreeList::operator==): (JSC::FreeList::operator!=): (JSC::FreeList::operator bool): (JSC::FreeList::allocationWillFail): (JSC::FreeList::allocationWillSucceed): * heap/GCTypeMap.h: Added. (JSC::GCTypeMap::operator[]): * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::lastChanceToFinalize): (JSC::Heap::finalizeUnconditionalFinalizers): (JSC::Heap::markRoots): (JSC::Heap::copyBackingStores): (JSC::Heap::gatherStackRoots): (JSC::Heap::gatherJSStackRoots): (JSC::Heap::gatherScratchBufferRoots): (JSC::Heap::clearLivenessData): (JSC::Heap::visitSmallStrings): (JSC::Heap::visitConservativeRoots): (JSC::Heap::removeDeadCompilerWorklistEntries): (JSC::Heap::gatherExtraHeapSnapshotData): (JSC::Heap::removeDeadHeapSnapshotNodes): (JSC::Heap::visitProtectedObjects): (JSC::Heap::visitArgumentBuffers): (JSC::Heap::visitException): (JSC::Heap::visitStrongHandles): (JSC::Heap::visitHandleStack): (JSC::Heap::visitSamplingProfiler): (JSC::Heap::traceCodeBlocksAndJITStubRoutines): (JSC::Heap::converge): (JSC::Heap::visitWeakHandles): (JSC::Heap::updateObjectCounts): (JSC::Heap::clearUnmarkedExecutables): (JSC::Heap::deleteUnmarkedCompiledCode): (JSC::Heap::collectAllGarbage): (JSC::Heap::collect): (JSC::Heap::collectWithoutAnySweep): (JSC::Heap::collectImpl): (JSC::Heap::suspendCompilerThreads): (JSC::Heap::willStartCollection): (JSC::Heap::flushOldStructureIDTables): (JSC::Heap::flushWriteBarrierBuffer): (JSC::Heap::stopAllocation): (JSC::Heap::prepareForMarking): (JSC::Heap::reapWeakHandles): (JSC::Heap::pruneStaleEntriesFromWeakGCMaps): (JSC::Heap::sweepArrayBuffers): (JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): (JSC::MarkedBlockSnapshotFunctor::operator()): (JSC::Heap::snapshotMarkedSpace): (JSC::Heap::deleteSourceProviderCaches): (JSC::Heap::notifyIncrementalSweeper): (JSC::Heap::writeBarrierCurrentlyExecutingCodeBlocks): (JSC::Heap::resetAllocators): (JSC::Heap::updateAllocationLimits): (JSC::Heap::didFinishCollection): (JSC::Heap::resumeCompilerThreads): (JSC::Zombify::visit): (JSC::Heap::forEachCodeBlockImpl): * heap/Heap.h: (JSC::Heap::allocatorForObjectWithoutDestructor): (JSC::Heap::allocatorForObjectWithDestructor): (JSC::Heap::allocatorForAuxiliaryData): (JSC::Heap::jitStubRoutines): (JSC::Heap::codeBlockSet): (JSC::Heap::storageAllocator): Deleted. * heap/HeapCell.h: (JSC::HeapCell::isZapped): Deleted. * heap/HeapCellInlines.h: Added. (JSC::HeapCell::isLargeAllocation): (JSC::HeapCell::cellContainer): (JSC::HeapCell::markedBlock): (JSC::HeapCell::largeAllocation): (JSC::HeapCell::heap): (JSC::HeapCell::vm): (JSC::HeapCell::cellSize): (JSC::HeapCell::allocatorAttributes): (JSC::HeapCell::destructionMode): (JSC::HeapCell::cellKind): * heap/HeapInlines.h: (JSC::Heap::heap): (JSC::Heap::isLive): (JSC::Heap::isMarked): (JSC::Heap::testAndSetMarked): (JSC::Heap::setMarked): (JSC::Heap::cellSize): (JSC::Heap::forEachCodeBlock): (JSC::Heap::allocateObjectOfType): (JSC::Heap::subspaceForObjectOfType): (JSC::Heap::allocatorForObjectOfType): (JSC::Heap::allocateAuxiliary): (JSC::Heap::tryAllocateAuxiliary): (JSC::Heap::tryReallocateAuxiliary): (JSC::Heap::isPointerGCObject): Deleted. (JSC::Heap::isValueGCObject): Deleted. * heap/HeapOperation.cpp: Added. (WTF::printInternal): * heap/HeapOperation.h: * heap/HeapUtil.h: Added. (JSC::HeapUtil::findGCObjectPointersForMarking): (JSC::HeapUtil::isPointerGCObjectJSCell): (JSC::HeapUtil::isValueGCObject): * heap/IncrementalSweeper.cpp: (JSC::IncrementalSweeper::sweepNextBlock): * heap/IncrementalSweeper.h: * heap/LargeAllocation.cpp: Added. (JSC::LargeAllocation::tryCreate): (JSC::LargeAllocation::LargeAllocation): (JSC::LargeAllocation::lastChanceToFinalize): (JSC::LargeAllocation::shrink): (JSC::LargeAllocation::visitWeakSet): (JSC::LargeAllocation::reapWeakSet): (JSC::LargeAllocation::flip): (JSC::LargeAllocation::isEmpty): (JSC::LargeAllocation::sweep): (JSC::LargeAllocation::destroy): (JSC::LargeAllocation::dump): * heap/LargeAllocation.h: Added. (JSC::LargeAllocation::fromCell): (JSC::LargeAllocation::cell): (JSC::LargeAllocation::isLargeAllocation): (JSC::LargeAllocation::heap): (JSC::LargeAllocation::vm): (JSC::LargeAllocation::weakSet): (JSC::LargeAllocation::clearNewlyAllocated): (JSC::LargeAllocation::isNewlyAllocated): (JSC::LargeAllocation::isMarked): (JSC::LargeAllocation::isMarkedOrNewlyAllocated): (JSC::LargeAllocation::isLive): (JSC::LargeAllocation::hasValidCell): (JSC::LargeAllocation::cellSize): (JSC::LargeAllocation::aboveLowerBound): (JSC::LargeAllocation::belowUpperBound): (JSC::LargeAllocation::contains): (JSC::LargeAllocation::attributes): (JSC::LargeAllocation::flipIfNecessary): (JSC::LargeAllocation::flipIfNecessaryConcurrently): (JSC::LargeAllocation::testAndSetMarked): (JSC::LargeAllocation::setMarked): (JSC::LargeAllocation::clearMarked): (JSC::LargeAllocation::noteMarked): (JSC::LargeAllocation::headerSize): * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::MarkedAllocator): (JSC::MarkedAllocator::isPagedOut): (JSC::MarkedAllocator::retire): (JSC::MarkedAllocator::filterNextBlock): (JSC::MarkedAllocator::setNextBlockToSweep): (JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl): (JSC::MarkedAllocator::tryAllocateWithoutCollecting): (JSC::MarkedAllocator::allocateSlowCase): (JSC::MarkedAllocator::tryAllocateSlowCase): (JSC::MarkedAllocator::allocateSlowCaseImpl): (JSC::blockHeaderSize): (JSC::MarkedAllocator::blockSizeForBytes): (JSC::MarkedAllocator::tryAllocateBlock): (JSC::MarkedAllocator::addBlock): (JSC::MarkedAllocator::removeBlock): (JSC::MarkedAllocator::stopAllocating): (JSC::MarkedAllocator::reset): (JSC::MarkedAllocator::lastChanceToFinalize): (JSC::MarkedAllocator::setFreeList): (JSC::isListPagedOut): Deleted. (JSC::MarkedAllocator::tryAllocateHelper): Deleted. (JSC::MarkedAllocator::tryPopFreeList): Deleted. (JSC::MarkedAllocator::tryAllocate): Deleted. (JSC::MarkedAllocator::allocateBlock): Deleted. * heap/MarkedAllocator.h: (JSC::MarkedAllocator::takeLastActiveBlock): (JSC::MarkedAllocator::offsetOfFreeList): (JSC::MarkedAllocator::offsetOfCellSize): (JSC::MarkedAllocator::tryAllocate): (JSC::MarkedAllocator::allocate): (JSC::MarkedAllocator::forEachBlock): (JSC::MarkedAllocator::offsetOfFreeListHead): Deleted. (JSC::MarkedAllocator::MarkedAllocator): Deleted. (JSC::MarkedAllocator::init): Deleted. (JSC::MarkedAllocator::stopAllocating): Deleted. * heap/MarkedBlock.cpp: (JSC::MarkedBlock::tryCreate): (JSC::MarkedBlock::Handle::Handle): (JSC::MarkedBlock::Handle::~Handle): (JSC::MarkedBlock::MarkedBlock): (JSC::MarkedBlock::Handle::specializedSweep): (JSC::MarkedBlock::Handle::sweep): (JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode): (JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): (JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated): (JSC::SetNewlyAllocatedFunctor::SetNewlyAllocatedFunctor): (JSC::SetNewlyAllocatedFunctor::operator()): (JSC::MarkedBlock::Handle::stopAllocating): (JSC::MarkedBlock::Handle::lastChanceToFinalize): (JSC::MarkedBlock::Handle::resumeAllocating): (JSC::MarkedBlock::Handle::zap): (JSC::MarkedBlock::Handle::forEachFreeCell): (JSC::MarkedBlock::flipIfNecessary): (JSC::MarkedBlock::Handle::flipIfNecessary): (JSC::MarkedBlock::flipIfNecessarySlow): (JSC::MarkedBlock::flipIfNecessaryConcurrentlySlow): (JSC::MarkedBlock::clearMarks): (JSC::MarkedBlock::assertFlipped): (JSC::MarkedBlock::needsFlip): (JSC::MarkedBlock::Handle::needsFlip): (JSC::MarkedBlock::Handle::willRemoveBlock): (JSC::MarkedBlock::Handle::didConsumeFreeList): (JSC::MarkedBlock::markCount): (JSC::MarkedBlock::Handle::isEmpty): (JSC::MarkedBlock::clearHasAnyMarked): (JSC::MarkedBlock::noteMarkedSlow): (WTF::printInternal): (JSC::MarkedBlock::create): Deleted. (JSC::MarkedBlock::destroy): Deleted. (JSC::MarkedBlock::callDestructor): Deleted. (JSC::MarkedBlock::specializedSweep): Deleted. (JSC::MarkedBlock::sweep): Deleted. (JSC::MarkedBlock::sweepHelper): Deleted. (JSC::MarkedBlock::stopAllocating): Deleted. (JSC::MarkedBlock::clearMarksWithCollectionType): Deleted. (JSC::MarkedBlock::lastChanceToFinalize): Deleted. (JSC::MarkedBlock::resumeAllocating): Deleted. (JSC::MarkedBlock::didRetireBlock): Deleted. * heap/MarkedBlock.h: (JSC::MarkedBlock::VoidFunctor::returnValue): (JSC::MarkedBlock::CountFunctor::CountFunctor): (JSC::MarkedBlock::CountFunctor::count): (JSC::MarkedBlock::CountFunctor::returnValue): (JSC::MarkedBlock::Handle::hasAnyNewlyAllocated): (JSC::MarkedBlock::Handle::isOnBlocksToSweep): (JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): (JSC::MarkedBlock::Handle::state): (JSC::MarkedBlock::needsDestruction): (JSC::MarkedBlock::handle): (JSC::MarkedBlock::Handle::block): (JSC::MarkedBlock::firstAtom): (JSC::MarkedBlock::atoms): (JSC::MarkedBlock::isAtomAligned): (JSC::MarkedBlock::Handle::cellAlign): (JSC::MarkedBlock::blockFor): (JSC::MarkedBlock::Handle::allocator): (JSC::MarkedBlock::Handle::heap): (JSC::MarkedBlock::Handle::vm): (JSC::MarkedBlock::vm): (JSC::MarkedBlock::Handle::weakSet): (JSC::MarkedBlock::weakSet): (JSC::MarkedBlock::Handle::shrink): (JSC::MarkedBlock::Handle::visitWeakSet): (JSC::MarkedBlock::Handle::reapWeakSet): (JSC::MarkedBlock::Handle::cellSize): (JSC::MarkedBlock::cellSize): (JSC::MarkedBlock::Handle::attributes): (JSC::MarkedBlock::attributes): (JSC::MarkedBlock::Handle::needsDestruction): (JSC::MarkedBlock::Handle::destruction): (JSC::MarkedBlock::Handle::cellKind): (JSC::MarkedBlock::Handle::markCount): (JSC::MarkedBlock::Handle::size): (JSC::MarkedBlock::atomNumber): (JSC::MarkedBlock::flipIfNecessary): (JSC::MarkedBlock::flipIfNecessaryConcurrently): (JSC::MarkedBlock::Handle::flipIfNecessary): (JSC::MarkedBlock::Handle::flipIfNecessaryConcurrently): (JSC::MarkedBlock::Handle::flipForEdenCollection): (JSC::MarkedBlock::assertFlipped): (JSC::MarkedBlock::Handle::assertFlipped): (JSC::MarkedBlock::isMarked): (JSC::MarkedBlock::testAndSetMarked): (JSC::MarkedBlock::Handle::isNewlyAllocated): (JSC::MarkedBlock::Handle::setNewlyAllocated): (JSC::MarkedBlock::Handle::clearNewlyAllocated): (JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated): (JSC::MarkedBlock::isMarkedOrNewlyAllocated): (JSC::MarkedBlock::Handle::isLive): (JSC::MarkedBlock::isAtom): (JSC::MarkedBlock::Handle::isLiveCell): (JSC::MarkedBlock::Handle::forEachCell): (JSC::MarkedBlock::Handle::forEachLiveCell): (JSC::MarkedBlock::Handle::forEachDeadCell): (JSC::MarkedBlock::Handle::needsSweeping): (JSC::MarkedBlock::Handle::isAllocated): (JSC::MarkedBlock::Handle::isMarked): (JSC::MarkedBlock::Handle::isFreeListed): (JSC::MarkedBlock::hasAnyMarked): (JSC::MarkedBlock::noteMarked): (WTF::MarkedBlockHash::hash): (JSC::MarkedBlock::FreeList::FreeList): Deleted. (JSC::MarkedBlock::allocator): Deleted. (JSC::MarkedBlock::heap): Deleted. (JSC::MarkedBlock::shrink): Deleted. (JSC::MarkedBlock::visitWeakSet): Deleted. (JSC::MarkedBlock::reapWeakSet): Deleted. (JSC::MarkedBlock::willRemoveBlock): Deleted. (JSC::MarkedBlock::didConsumeFreeList): Deleted. (JSC::MarkedBlock::markCount): Deleted. (JSC::MarkedBlock::isEmpty): Deleted. (JSC::MarkedBlock::destruction): Deleted. (JSC::MarkedBlock::cellKind): Deleted. (JSC::MarkedBlock::size): Deleted. (JSC::MarkedBlock::capacity): Deleted. (JSC::MarkedBlock::setMarked): Deleted. (JSC::MarkedBlock::clearMarked): Deleted. (JSC::MarkedBlock::isNewlyAllocated): Deleted. (JSC::MarkedBlock::setNewlyAllocated): Deleted. (JSC::MarkedBlock::clearNewlyAllocated): Deleted. (JSC::MarkedBlock::isLive): Deleted. (JSC::MarkedBlock::isLiveCell): Deleted. (JSC::MarkedBlock::forEachCell): Deleted. (JSC::MarkedBlock::forEachLiveCell): Deleted. (JSC::MarkedBlock::forEachDeadCell): Deleted. (JSC::MarkedBlock::needsSweeping): Deleted. (JSC::MarkedBlock::isAllocated): Deleted. (JSC::MarkedBlock::isMarkedOrRetired): Deleted. * heap/MarkedSpace.cpp: (JSC::MarkedSpace::initializeSizeClassForStepSize): (JSC::MarkedSpace::MarkedSpace): (JSC::MarkedSpace::~MarkedSpace): (JSC::MarkedSpace::lastChanceToFinalize): (JSC::MarkedSpace::allocate): (JSC::MarkedSpace::tryAllocate): (JSC::MarkedSpace::allocateLarge): (JSC::MarkedSpace::tryAllocateLarge): (JSC::MarkedSpace::sweep): (JSC::MarkedSpace::sweepLargeAllocations): (JSC::MarkedSpace::zombifySweep): (JSC::MarkedSpace::resetAllocators): (JSC::MarkedSpace::visitWeakSets): (JSC::MarkedSpace::reapWeakSets): (JSC::MarkedSpace::stopAllocating): (JSC::MarkedSpace::prepareForMarking): (JSC::MarkedSpace::resumeAllocating): (JSC::MarkedSpace::isPagedOut): (JSC::MarkedSpace::freeBlock): (JSC::MarkedSpace::freeOrShrinkBlock): (JSC::MarkedSpace::shrink): (JSC::MarkedSpace::clearNewlyAllocated): (JSC::VerifyMarked::operator()): (JSC::MarkedSpace::flip): (JSC::MarkedSpace::objectCount): (JSC::MarkedSpace::size): (JSC::MarkedSpace::capacity): (JSC::MarkedSpace::addActiveWeakSet): (JSC::MarkedSpace::didAddBlock): (JSC::MarkedSpace::didAllocateInBlock): (JSC::MarkedSpace::forEachAllocator): Deleted. (JSC::VerifyMarkedOrRetired::operator()): Deleted. (JSC::MarkedSpace::clearMarks): Deleted. * heap/MarkedSpace.h: (JSC::MarkedSpace::sizeClassToIndex): (JSC::MarkedSpace::indexToSizeClass): (JSC::MarkedSpace::version): (JSC::MarkedSpace::blocksWithNewObjects): (JSC::MarkedSpace::largeAllocations): (JSC::MarkedSpace::largeAllocationsNurseryOffset): (JSC::MarkedSpace::largeAllocationsOffsetForThisCollection): (JSC::MarkedSpace::largeAllocationsForThisCollectionBegin): (JSC::MarkedSpace::largeAllocationsForThisCollectionEnd): (JSC::MarkedSpace::largeAllocationsForThisCollectionSize): (JSC::MarkedSpace::forEachLiveCell): (JSC::MarkedSpace::forEachDeadCell): (JSC::MarkedSpace::allocatorFor): (JSC::MarkedSpace::destructorAllocatorFor): (JSC::MarkedSpace::auxiliaryAllocatorFor): (JSC::MarkedSpace::allocateWithoutDestructor): (JSC::MarkedSpace::allocateWithDestructor): (JSC::MarkedSpace::allocateAuxiliary): (JSC::MarkedSpace::tryAllocateAuxiliary): (JSC::MarkedSpace::forEachBlock): (JSC::MarkedSpace::forEachAllocator): (JSC::MarkedSpace::optimalSizeFor): (JSC::MarkedSpace::didAddBlock): Deleted. (JSC::MarkedSpace::didAllocateInBlock): Deleted. (JSC::MarkedSpace::objectCount): Deleted. (JSC::MarkedSpace::size): Deleted. (JSC::MarkedSpace::capacity): Deleted. * heap/SlotVisitor.cpp: (JSC::SlotVisitor::SlotVisitor): (JSC::SlotVisitor::didStartMarking): (JSC::SlotVisitor::reset): (JSC::SlotVisitor::append): (JSC::SlotVisitor::appendJSCellOrAuxiliary): (JSC::SlotVisitor::setMarkedAndAppendToMarkStack): (JSC::SlotVisitor::appendToMarkStack): (JSC::SlotVisitor::markAuxiliary): (JSC::SlotVisitor::noteLiveAuxiliaryCell): (JSC::SlotVisitor::visitChildren): * heap/SlotVisitor.h: * heap/WeakBlock.cpp: (JSC::WeakBlock::create): (JSC::WeakBlock::WeakBlock): (JSC::WeakBlock::visit): (JSC::WeakBlock::reap): * heap/WeakBlock.h: (JSC::WeakBlock::disconnectContainer): (JSC::WeakBlock::disconnectMarkedBlock): Deleted. * heap/WeakSet.cpp: (JSC::WeakSet::~WeakSet): (JSC::WeakSet::sweep): (JSC::WeakSet::shrink): (JSC::WeakSet::addAllocator): * heap/WeakSet.h: (JSC::WeakSet::container): (JSC::WeakSet::setContainer): (JSC::WeakSet::WeakSet): (JSC::WeakSet::visit): (JSC::WeakSet::shrink): Deleted. * heap/WeakSetInlines.h: (JSC::WeakSet::allocate): * inspector/InjectedScriptManager.cpp: * inspector/JSGlobalObjectInspectorController.cpp: * inspector/JSJavaScriptCallFrame.cpp: * inspector/ScriptDebugServer.cpp: * inspector/agents/InspectorDebuggerAgent.cpp: * interpreter/CachedCall.h: (JSC::CachedCall::CachedCall): * interpreter/Interpreter.cpp: (JSC::loadVarargs): (JSC::StackFrame::sourceID): Deleted. (JSC::StackFrame::sourceURL): Deleted. (JSC::StackFrame::functionName): Deleted. (JSC::StackFrame::computeLineAndColumn): Deleted. (JSC::StackFrame::toString): Deleted. * interpreter/Interpreter.h: (JSC::StackFrame::isNative): Deleted. * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::emitAllocateWithNonNullAllocator): (JSC::AssemblyHelpers::emitAllocate): (JSC::AssemblyHelpers::emitAllocateJSCell): (JSC::AssemblyHelpers::emitAllocateJSObject): (JSC::AssemblyHelpers::emitAllocateJSObjectWithKnownSize): (JSC::AssemblyHelpers::emitAllocateVariableSized): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): * jit/JIT.cpp: (JSC::JIT::compileCTINativeCall): (JSC::JIT::link): * jit/JIT.h: (JSC::JIT::compileCTINativeCall): Deleted. * jit/JITExceptions.cpp: (JSC::genericUnwind): * jit/JITExceptions.h: * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_new_object): (JSC::JIT::emitSlow_op_new_object): (JSC::JIT::emit_op_create_this): (JSC::JIT::emitSlow_op_create_this): * jit/JITOpcodes32_64.cpp: (JSC::JIT::emit_op_new_object): (JSC::JIT::emitSlow_op_new_object): (JSC::JIT::emit_op_create_this): (JSC::JIT::emitSlow_op_create_this): * jit/JITOperations.cpp: * jit/JITOperations.h: * jit/JITPropertyAccess.cpp: (JSC::JIT::emitWriteBarrier): * jit/JITThunks.cpp: * jit/JITThunks.h: * jsc.cpp: (functionDescribeArray): (main): * llint/LLIntData.cpp: (JSC::LLInt::Data::performAssertions): * llint/LLIntExceptions.cpp: * llint/LLIntThunks.cpp: * llint/LLIntThunks.h: * llint/LowLevelInterpreter.asm: * llint/LowLevelInterpreter.cpp: * llint/LowLevelInterpreter32_64.asm: * llint/LowLevelInterpreter64.asm: * parser/ModuleAnalyzer.cpp: * parser/NodeConstructors.h: * parser/Nodes.h: * profiler/ProfilerBytecode.cpp: * profiler/ProfilerBytecode.h: * profiler/ProfilerBytecodeSequence.cpp: * runtime/ArrayConventions.h: (JSC::indexingHeaderForArrayStorage): (JSC::baseIndexingHeaderForArrayStorage): (JSC::indexingHeaderForArray): Deleted. (JSC::baseIndexingHeaderForArray): Deleted. * runtime/ArrayPrototype.cpp: (JSC::arrayProtoFuncSplice): (JSC::concatAppendOne): (JSC::arrayProtoPrivateFuncConcatMemcpy): * runtime/ArrayStorage.h: (JSC::ArrayStorage::vectorLength): (JSC::ArrayStorage::totalSizeFor): (JSC::ArrayStorage::totalSize): (JSC::ArrayStorage::availableVectorLength): (JSC::ArrayStorage::optimalVectorLength): (JSC::ArrayStorage::sizeFor): Deleted. * runtime/AuxiliaryBarrier.h: Added. (JSC::AuxiliaryBarrier::AuxiliaryBarrier): (JSC::AuxiliaryBarrier::clear): (JSC::AuxiliaryBarrier::get): (JSC::AuxiliaryBarrier::slot): (JSC::AuxiliaryBarrier::operator bool): (JSC::AuxiliaryBarrier::setWithoutBarrier): * runtime/AuxiliaryBarrierInlines.h: Added. (JSC::AuxiliaryBarrier<T>::AuxiliaryBarrier): (JSC::AuxiliaryBarrier<T>::set): * runtime/Butterfly.h: * runtime/ButterflyInlines.h: (JSC::Butterfly::availableContiguousVectorLength): (JSC::Butterfly::optimalContiguousVectorLength): (JSC::Butterfly::createUninitialized): (JSC::Butterfly::growArrayRight): * runtime/ClonedArguments.cpp: (JSC::ClonedArguments::createEmpty): * runtime/CommonSlowPathsExceptions.cpp: * runtime/CommonSlowPathsExceptions.h: * runtime/DataView.cpp: * runtime/DirectArguments.h: * runtime/ECMAScriptSpecInternalFunctions.cpp: * runtime/Error.cpp: * runtime/Error.h: * runtime/ErrorInstance.cpp: * runtime/ErrorInstance.h: * runtime/Exception.cpp: * runtime/Exception.h: * runtime/GeneratorFrame.cpp: * runtime/GeneratorPrototype.cpp: * runtime/InternalFunction.cpp: (JSC::InternalFunction::InternalFunction): * runtime/IntlCollator.cpp: * runtime/IntlCollatorConstructor.cpp: * runtime/IntlCollatorPrototype.cpp: * runtime/IntlDateTimeFormat.cpp: * runtime/IntlDateTimeFormatConstructor.cpp: * runtime/IntlDateTimeFormatPrototype.cpp: * runtime/IntlNumberFormat.cpp: * runtime/IntlNumberFormatConstructor.cpp: * runtime/IntlNumberFormatPrototype.cpp: * runtime/IntlObject.cpp: * runtime/IteratorPrototype.cpp: * runtime/JSArray.cpp: (JSC::JSArray::tryCreateUninitialized): (JSC::JSArray::setLengthWritable): (JSC::JSArray::unshiftCountSlowCase): (JSC::JSArray::setLengthWithArrayStorage): (JSC::JSArray::appendMemcpy): (JSC::JSArray::setLength): (JSC::JSArray::pop): (JSC::JSArray::push): (JSC::JSArray::fastSlice): (JSC::JSArray::shiftCountWithArrayStorage): (JSC::JSArray::shiftCountWithAnyIndexingType): (JSC::JSArray::unshiftCountWithArrayStorage): (JSC::JSArray::fillArgList): (JSC::JSArray::copyToArguments): * runtime/JSArray.h: (JSC::createContiguousArrayButterfly): (JSC::createArrayButterfly): (JSC::JSArray::create): (JSC::JSArray::tryCreateUninitialized): Deleted. * runtime/JSArrayBufferView.h: * runtime/JSCInlines.h: * runtime/JSCJSValue.cpp: (JSC::JSValue::dumpInContextAssumingStructure): * runtime/JSCallee.cpp: (JSC::JSCallee::JSCallee): * runtime/JSCell.cpp: (JSC::JSCell::estimatedSize): * runtime/JSCell.h: (JSC::JSCell::cellStateOffset): Deleted. * runtime/JSCellInlines.h: (JSC::ExecState::vm): (JSC::JSCell::classInfo): (JSC::JSCell::callDestructor): (JSC::JSCell::vm): Deleted. * runtime/JSFunction.cpp: (JSC::JSFunction::create): (JSC::JSFunction::allocateAndInitializeRareData): (JSC::JSFunction::initializeRareData): (JSC::JSFunction::getOwnPropertySlot): (JSC::JSFunction::put): (JSC::JSFunction::deleteProperty): (JSC::JSFunction::defineOwnProperty): (JSC::JSFunction::setFunctionName): (JSC::JSFunction::reifyLength): (JSC::JSFunction::reifyName): (JSC::JSFunction::reifyLazyPropertyIfNeeded): (JSC::JSFunction::reifyBoundNameIfNeeded): * runtime/JSFunction.h: * runtime/JSFunctionInlines.h: (JSC::JSFunction::createWithInvalidatedReallocationWatchpoint): (JSC::JSFunction::JSFunction): * runtime/JSGenericTypedArrayViewInlines.h: (JSC::JSGenericTypedArrayView<Adaptor>::slowDownAndWasteMemory): * runtime/JSInternalPromise.cpp: * runtime/JSInternalPromiseConstructor.cpp: * runtime/JSInternalPromiseDeferred.cpp: * runtime/JSInternalPromisePrototype.cpp: * runtime/JSJob.cpp: * runtime/JSMapIterator.cpp: * runtime/JSModuleNamespaceObject.cpp: * runtime/JSModuleRecord.cpp: * runtime/JSObject.cpp: (JSC::JSObject::visitButterfly): (JSC::JSObject::notifyPresenceOfIndexedAccessors): (JSC::JSObject::createInitialIndexedStorage): (JSC::JSObject::createInitialUndecided): (JSC::JSObject::createInitialInt32): (JSC::JSObject::createInitialDouble): (JSC::JSObject::createInitialContiguous): (JSC::JSObject::createArrayStorage): (JSC::JSObject::createInitialArrayStorage): (JSC::JSObject::convertUndecidedToInt32): (JSC::JSObject::convertUndecidedToContiguous): (JSC::JSObject::convertUndecidedToArrayStorage): (JSC::JSObject::convertInt32ToDouble): (JSC::JSObject::convertInt32ToArrayStorage): (JSC::JSObject::convertDoubleToArrayStorage): (JSC::JSObject::convertContiguousToArrayStorage): (JSC::JSObject::putByIndexBeyondVectorLength): (JSC::JSObject::putDirectIndexBeyondVectorLength): (JSC::JSObject::getNewVectorLength): (JSC::JSObject::increaseVectorLength): (JSC::JSObject::ensureLengthSlow): (JSC::JSObject::growOutOfLineStorage): (JSC::JSObject::copyButterfly): Deleted. (JSC::JSObject::copyBackingStore): Deleted. * runtime/JSObject.h: (JSC::JSObject::globalObject): (JSC::JSObject::putDirectInternal): (JSC::JSObject::setStructureAndReallocateStorageIfNecessary): Deleted. * runtime/JSObjectInlines.h: * runtime/JSPromise.cpp: * runtime/JSPromiseConstructor.cpp: * runtime/JSPromiseDeferred.cpp: * runtime/JSPromisePrototype.cpp: * runtime/JSPropertyNameIterator.cpp: * runtime/JSScope.cpp: (JSC::JSScope::resolve): * runtime/JSScope.h: (JSC::JSScope::globalObject): (JSC::JSScope::vm): Deleted. * runtime/JSSetIterator.cpp: * runtime/JSStringIterator.cpp: * runtime/JSTemplateRegistryKey.cpp: * runtime/JSTypedArrayViewConstructor.cpp: * runtime/JSTypedArrayViewPrototype.cpp: * runtime/JSWeakMap.cpp: * runtime/JSWeakSet.cpp: * runtime/MapConstructor.cpp: * runtime/MapIteratorPrototype.cpp: * runtime/MapPrototype.cpp: * runtime/NativeErrorConstructor.cpp: * runtime/NativeStdFunctionCell.cpp: * runtime/Operations.h: (JSC::scribbleFreeCells): (JSC::scribble): * runtime/Options.h: * runtime/PropertyTable.cpp: * runtime/ProxyConstructor.cpp: * runtime/ProxyObject.cpp: * runtime/ProxyRevoke.cpp: * runtime/RegExp.cpp: (JSC::RegExp::match): (JSC::RegExp::matchConcurrently): (JSC::RegExp::matchCompareWithInterpreter): * runtime/RegExp.h: * runtime/RegExpConstructor.h: * runtime/RegExpInlines.h: (JSC::RegExp::matchInline): * runtime/RegExpMatchesArray.h: (JSC::tryCreateUninitializedRegExpMatchesArray): (JSC::createRegExpMatchesArray): * runtime/RegExpPrototype.cpp: (JSC::genericSplit): * runtime/RuntimeType.cpp: * runtime/SamplingProfiler.cpp: (JSC::SamplingProfiler::processUnverifiedStackTraces): * runtime/SetConstructor.cpp: * runtime/SetIteratorPrototype.cpp: * runtime/SetPrototype.cpp: * runtime/StackFrame.cpp: Added. (JSC::StackFrame::sourceID): (JSC::StackFrame::sourceURL): (JSC::StackFrame::functionName): (JSC::StackFrame::computeLineAndColumn): (JSC::StackFrame::toString): * runtime/StackFrame.h: Added. (JSC::StackFrame::isNative): * runtime/StringConstructor.cpp: * runtime/StringIteratorPrototype.cpp: * runtime/StructureInlines.h: (JSC::Structure::propertyTable): * runtime/TemplateRegistry.cpp: * runtime/TestRunnerUtils.cpp: (JSC::finalizeStatsAtEndOfTesting): * runtime/TestRunnerUtils.h: * runtime/TypeProfilerLog.cpp: * runtime/TypeSet.cpp: * runtime/VM.cpp: (JSC::VM::VM): (JSC::VM::ensureStackCapacityForCLoop): (JSC::VM::isSafeToRecurseSoftCLoop): * runtime/VM.h: * runtime/VMEntryScope.h: * runtime/VMInlines.h: (JSC::VM::ensureStackCapacityFor): (JSC::VM::isSafeToRecurseSoft): * runtime/WeakMapConstructor.cpp: * runtime/WeakMapData.cpp: * runtime/WeakMapPrototype.cpp: * runtime/WeakSetConstructor.cpp: * runtime/WeakSetPrototype.cpp: * testRegExp.cpp: (testOneRegExp): * tools/JSDollarVM.cpp: * tools/JSDollarVMPrototype.cpp: (JSC::JSDollarVMPrototype::isInObjectSpace): Source/WebCore: No new tests because no new WebCore behavior. Just rewiring #includes. * ForwardingHeaders/heap/HeapInlines.h: Added. * ForwardingHeaders/interpreter/Interpreter.h: Removed. * ForwardingHeaders/runtime/AuxiliaryBarrierInlines.h: Added. * Modules/indexeddb/IDBCursorWithValue.cpp: * Modules/indexeddb/client/TransactionOperation.cpp: * Modules/indexeddb/server/SQLiteIDBBackingStore.cpp: * Modules/indexeddb/server/UniqueIDBDatabase.cpp: * bindings/js/JSApplePayPaymentAuthorizedEventCustom.cpp: * bindings/js/JSApplePayPaymentMethodSelectedEventCustom.cpp: * bindings/js/JSApplePayShippingContactSelectedEventCustom.cpp: * bindings/js/JSApplePayShippingMethodSelectedEventCustom.cpp: * bindings/js/JSClientRectCustom.cpp: * bindings/js/JSDOMBinding.cpp: * bindings/js/JSDOMBinding.h: * bindings/js/JSDeviceMotionEventCustom.cpp: * bindings/js/JSDeviceOrientationEventCustom.cpp: * bindings/js/JSErrorEventCustom.cpp: * bindings/js/JSIDBCursorWithValueCustom.cpp: * bindings/js/JSIDBIndexCustom.cpp: * bindings/js/JSPopStateEventCustom.cpp: * bindings/js/JSWebGL2RenderingContextCustom.cpp: * bindings/js/JSWorkerGlobalScopeCustom.cpp: * bindings/js/WorkerScriptController.cpp: * contentextensions/ContentExtensionParser.cpp: * dom/ErrorEvent.cpp: * html/HTMLCanvasElement.cpp: * html/MediaDocument.cpp: * inspector/CommandLineAPIModule.cpp: * loader/EmptyClients.cpp: * page/CaptionUserPreferences.cpp: * page/Frame.cpp: * page/PageGroup.cpp: * page/UserContentController.cpp: * platform/mock/mediasource/MockBox.cpp: * testing/GCObservation.cpp: Source/WebKit2: Just rewiring some #includes. * UIProcess/ViewGestureController.cpp: * UIProcess/WebPageProxy.cpp: * UIProcess/WebProcessPool.cpp: * UIProcess/WebProcessProxy.cpp: * WebProcess/InjectedBundle/DOM/InjectedBundleRangeHandle.cpp: * WebProcess/Plugins/Netscape/JSNPObject.cpp: Source/WTF: I needed tryFastAlignedMalloc() so I added it. * wtf/FastMalloc.cpp: (WTF::tryFastAlignedMalloc): * wtf/FastMalloc.h: * wtf/ParkingLot.cpp: (WTF::ParkingLot::forEachImpl): (WTF::ParkingLot::forEach): Deleted. * wtf/ParkingLot.h: (WTF::ParkingLot::parkConditionally): (WTF::ParkingLot::unparkOne): (WTF::ParkingLot::forEach): * wtf/ScopedLambda.h: (WTF::scopedLambdaRef): * wtf/SentinelLinkedList.h: (WTF::SentinelLinkedList::forEach): (WTF::RawNode>::takeFrom): * wtf/SimpleStats.h: (WTF::SimpleStats::operator bool): (WTF::SimpleStats::operator!): Deleted. Tools: * DumpRenderTree/TestRunner.cpp: * DumpRenderTree/mac/DumpRenderTree.mm: (DumpRenderTreeMain): * Scripts/run-jsc-stress-tests: * TestWebKitAPI/Tests/WTF/Vector.cpp: (TestWebKitAPI::TEST): Canonical link: https://commits.webkit.org/179778@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205462 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-06 01:02:22 +00:00
template<typename FunctionType, typename Functor>
ScopedLambdaRefFunctor<FunctionType, Functor> scopedLambdaRef(const Functor& functor)
{
return ScopedLambdaRefFunctor<FunctionType, Functor>(functor);
}
Create a super rough prototype of B3 https://bugs.webkit.org/show_bug.cgi?id=150280 Reviewed by Benjamin Poulain. Source/JavaScriptCore: This changeset adds the basic scaffolding of the B3 compiler. B3 stands for Bare Bones Backend. It's a low-level SSA-based language-agnostic compiler. The basic structure allows for aggressive C-level optimizations and an awesome portable backend. The backend, called Air (Assembly IR), is a reflective abstraction over our MacroAssembler. The abstraction is defined using a spec file (AirOpcode.opcodes) which describes the various kinds of instructions that we wish to support. Then, the B3::LowerToAir phase, which does our instruction selection, reflectively selects Air opcodes by querying which instruction forms are possible. Air allows for optimal register allocation and stack layout. Currently the register allocator isn't written, but the stack layout is. Of course this isn't done yet. It can only compile simple programs, seen in the "test suite" called "testb3.cpp". There's a lot of optimizations that have to be written and a lot of stuff added to the instruction selector. But it's a neat start. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * assembler/MacroAssembler.cpp: (WTF::printInternal): * assembler/MacroAssembler.h: * b3: Added. * b3/B3AddressMatcher.patterns: Added. * b3/B3ArgumentRegValue.cpp: Added. (JSC::B3::ArgumentRegValue::~ArgumentRegValue): (JSC::B3::ArgumentRegValue::dumpMeta): * b3/B3ArgumentRegValue.h: Added. * b3/B3BasicBlock.cpp: Added. (JSC::B3::BasicBlock::BasicBlock): (JSC::B3::BasicBlock::~BasicBlock): (JSC::B3::BasicBlock::append): (JSC::B3::BasicBlock::addPredecessor): (JSC::B3::BasicBlock::removePredecessor): (JSC::B3::BasicBlock::replacePredecessor): (JSC::B3::BasicBlock::removeNops): (JSC::B3::BasicBlock::dump): (JSC::B3::BasicBlock::deepDump): * b3/B3BasicBlock.h: Added. (JSC::B3::BasicBlock::index): (JSC::B3::BasicBlock::begin): (JSC::B3::BasicBlock::end): (JSC::B3::BasicBlock::size): (JSC::B3::BasicBlock::at): (JSC::B3::BasicBlock::last): (JSC::B3::BasicBlock::values): (JSC::B3::BasicBlock::numPredecessors): (JSC::B3::BasicBlock::predecessor): (JSC::B3::BasicBlock::predecessors): (JSC::B3::BasicBlock::frequency): (JSC::B3::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::DeepBasicBlockDump::dump): (JSC::B3::deepDump): * b3/B3BasicBlockInlines.h: Added. (JSC::B3::BasicBlock::appendNew): (JSC::B3::BasicBlock::numSuccessors): (JSC::B3::BasicBlock::successor): (JSC::B3::BasicBlock::successors): (JSC::B3::BasicBlock::successorBlock): (JSC::B3::BasicBlock::successorBlocks): * b3/B3BasicBlockUtils.h: Added. (JSC::B3::addPredecessor): (JSC::B3::removePredecessor): (JSC::B3::replacePredecessor): (JSC::B3::resetReachability): (JSC::B3::blocksInPreOrder): (JSC::B3::blocksInPostOrder): * b3/B3BlockWorklist.h: Added. * b3/B3CheckSpecial.cpp: Added. (JSC::B3::Air::numB3Args): (JSC::B3::CheckSpecial::CheckSpecial): (JSC::B3::CheckSpecial::~CheckSpecial): (JSC::B3::CheckSpecial::hiddenBranch): (JSC::B3::CheckSpecial::forEachArg): (JSC::B3::CheckSpecial::isValid): (JSC::B3::CheckSpecial::admitsStack): (JSC::B3::CheckSpecial::generate): (JSC::B3::CheckSpecial::dumpImpl): (JSC::B3::CheckSpecial::deepDumpImpl): * b3/B3CheckSpecial.h: Added. * b3/B3CheckValue.cpp: Added. (JSC::B3::CheckValue::~CheckValue): (JSC::B3::CheckValue::dumpMeta): * b3/B3CheckValue.h: Added. * b3/B3Common.cpp: Added. (JSC::B3::shouldDumpIR): (JSC::B3::shouldDumpIRAtEachPhase): (JSC::B3::shouldValidateIR): (JSC::B3::shouldValidateIRAtEachPhase): (JSC::B3::shouldSaveIRBeforePhase): * b3/B3Common.h: Added. (JSC::B3::is64Bit): (JSC::B3::is32Bit): * b3/B3Commutativity.cpp: Added. (WTF::printInternal): * b3/B3Commutativity.h: Added. * b3/B3Const32Value.cpp: Added. (JSC::B3::Const32Value::~Const32Value): (JSC::B3::Const32Value::negConstant): (JSC::B3::Const32Value::addConstant): (JSC::B3::Const32Value::subConstant): (JSC::B3::Const32Value::dumpMeta): * b3/B3Const32Value.h: Added. * b3/B3Const64Value.cpp: Added. (JSC::B3::Const64Value::~Const64Value): (JSC::B3::Const64Value::negConstant): (JSC::B3::Const64Value::addConstant): (JSC::B3::Const64Value::subConstant): (JSC::B3::Const64Value::dumpMeta): * b3/B3Const64Value.h: Added. * b3/B3ConstDoubleValue.cpp: Added. (JSC::B3::ConstDoubleValue::~ConstDoubleValue): (JSC::B3::ConstDoubleValue::negConstant): (JSC::B3::ConstDoubleValue::addConstant): (JSC::B3::ConstDoubleValue::subConstant): (JSC::B3::ConstDoubleValue::dumpMeta): * b3/B3ConstDoubleValue.h: Added. (JSC::B3::ConstDoubleValue::accepts): (JSC::B3::ConstDoubleValue::value): (JSC::B3::ConstDoubleValue::ConstDoubleValue): * b3/B3ConstPtrValue.h: Added. (JSC::B3::ConstPtrValue::value): (JSC::B3::ConstPtrValue::ConstPtrValue): * b3/B3ControlValue.cpp: Added. (JSC::B3::ControlValue::~ControlValue): (JSC::B3::ControlValue::dumpMeta): * b3/B3ControlValue.h: Added. * b3/B3Effects.cpp: Added. (JSC::B3::Effects::dump): * b3/B3Effects.h: Added. (JSC::B3::Effects::mustExecute): * b3/B3FrequencyClass.cpp: Added. (WTF::printInternal): * b3/B3FrequencyClass.h: Added. * b3/B3FrequentedBlock.h: Added. * b3/B3Generate.cpp: Added. (JSC::B3::generate): (JSC::B3::generateToAir): * b3/B3Generate.h: Added. * b3/B3GenericFrequentedBlock.h: Added. (JSC::B3::GenericFrequentedBlock::GenericFrequentedBlock): (JSC::B3::GenericFrequentedBlock::operator==): (JSC::B3::GenericFrequentedBlock::operator!=): (JSC::B3::GenericFrequentedBlock::operator bool): (JSC::B3::GenericFrequentedBlock::block): (JSC::B3::GenericFrequentedBlock::frequency): (JSC::B3::GenericFrequentedBlock::dump): * b3/B3HeapRange.cpp: Added. (JSC::B3::HeapRange::dump): * b3/B3HeapRange.h: Added. (JSC::B3::HeapRange::HeapRange): (JSC::B3::HeapRange::top): (JSC::B3::HeapRange::operator==): (JSC::B3::HeapRange::operator!=): (JSC::B3::HeapRange::operator bool): (JSC::B3::HeapRange::begin): (JSC::B3::HeapRange::end): (JSC::B3::HeapRange::overlaps): * b3/B3IndexMap.h: Added. (JSC::B3::IndexMap::IndexMap): (JSC::B3::IndexMap::resize): (JSC::B3::IndexMap::operator[]): * b3/B3IndexSet.h: Added. (JSC::B3::IndexSet::IndexSet): (JSC::B3::IndexSet::add): (JSC::B3::IndexSet::contains): (JSC::B3::IndexSet::Iterable::Iterable): (JSC::B3::IndexSet::Iterable::iterator::iterator): (JSC::B3::IndexSet::Iterable::iterator::operator*): (JSC::B3::IndexSet::Iterable::iterator::operator++): (JSC::B3::IndexSet::Iterable::iterator::operator==): (JSC::B3::IndexSet::Iterable::iterator::operator!=): (JSC::B3::IndexSet::Iterable::begin): (JSC::B3::IndexSet::Iterable::end): (JSC::B3::IndexSet::values): (JSC::B3::IndexSet::indices): (JSC::B3::IndexSet::dump): * b3/B3InsertionSet.cpp: Added. (JSC::B3::InsertionSet::execute): * b3/B3InsertionSet.h: Added. (JSC::B3::InsertionSet::InsertionSet): (JSC::B3::InsertionSet::code): (JSC::B3::InsertionSet::appendInsertion): (JSC::B3::InsertionSet::insertValue): * b3/B3InsertionSetInlines.h: Added. (JSC::B3::InsertionSet::insert): * b3/B3LowerToAir.cpp: Added. (JSC::B3::Air::LowerToAir::LowerToAir): (JSC::B3::Air::LowerToAir::run): (JSC::B3::Air::LowerToAir::tmp): (JSC::B3::Air::LowerToAir::effectiveAddr): (JSC::B3::Air::LowerToAir::addr): (JSC::B3::Air::LowerToAir::loadAddr): (JSC::B3::Air::LowerToAir::imm): (JSC::B3::Air::LowerToAir::immOrTmp): (JSC::B3::Air::LowerToAir::appendBinOp): (JSC::B3::Air::LowerToAir::tryAppendStoreBinOp): (JSC::B3::Air::LowerToAir::moveForType): (JSC::B3::Air::LowerToAir::relaxedMoveForType): (JSC::B3::Air::LowerToAir::append): (JSC::B3::Air::LowerToAir::AddressSelector::AddressSelector): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRoot): (JSC::B3::Air::LowerToAir::AddressSelector::acceptRootLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternals): (JSC::B3::Air::LowerToAir::AddressSelector::acceptInternalsLate): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperands): (JSC::B3::Air::LowerToAir::AddressSelector::acceptOperandsLate): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift1): (JSC::B3::Air::LowerToAir::AddressSelector::tryAddShift2): (JSC::B3::Air::LowerToAir::AddressSelector::tryAdd): (JSC::B3::Air::LowerToAir::AddressSelector::tryDirect): (JSC::B3::Air::LowerToAir::acceptRoot): (JSC::B3::Air::LowerToAir::acceptRootLate): (JSC::B3::Air::LowerToAir::acceptInternals): (JSC::B3::Air::LowerToAir::acceptInternalsLate): (JSC::B3::Air::LowerToAir::acceptOperands): (JSC::B3::Air::LowerToAir::acceptOperandsLate): (JSC::B3::Air::LowerToAir::tryLoad): (JSC::B3::Air::LowerToAir::tryAdd): (JSC::B3::Air::LowerToAir::tryAnd): (JSC::B3::Air::LowerToAir::tryStoreAddLoad): (JSC::B3::Air::LowerToAir::tryStoreAndLoad): (JSC::B3::Air::LowerToAir::tryStore): (JSC::B3::Air::LowerToAir::tryTruncArgumentReg): (JSC::B3::Air::LowerToAir::tryTrunc): (JSC::B3::Air::LowerToAir::tryArgumentReg): (JSC::B3::Air::LowerToAir::tryConst32): (JSC::B3::Air::LowerToAir::tryConst64): (JSC::B3::Air::LowerToAir::tryIdentity): (JSC::B3::Air::LowerToAir::tryReturn): (JSC::B3::lowerToAir): * b3/B3LowerToAir.h: Added. * b3/B3LoweringMatcher.patterns: Added. * b3/B3MemoryValue.cpp: Added. (JSC::B3::MemoryValue::~MemoryValue): (JSC::B3::MemoryValue::dumpMeta): * b3/B3MemoryValue.h: Added. * b3/B3Opcode.cpp: Added. (WTF::printInternal): * b3/B3Opcode.h: Added. (JSC::B3::isCheckMath): * b3/B3Origin.cpp: Added. (JSC::B3::Origin::dump): * b3/B3Origin.h: Added. (JSC::B3::Origin::Origin): (JSC::B3::Origin::operator bool): (JSC::B3::Origin::data): * b3/B3PatchpointSpecial.cpp: Added. (JSC::B3::PatchpointSpecial::PatchpointSpecial): (JSC::B3::PatchpointSpecial::~PatchpointSpecial): (JSC::B3::PatchpointSpecial::forEachArg): (JSC::B3::PatchpointSpecial::isValid): (JSC::B3::PatchpointSpecial::admitsStack): (JSC::B3::PatchpointSpecial::generate): (JSC::B3::PatchpointSpecial::dumpImpl): (JSC::B3::PatchpointSpecial::deepDumpImpl): * b3/B3PatchpointSpecial.h: Added. * b3/B3PatchpointValue.cpp: Added. (JSC::B3::PatchpointValue::~PatchpointValue): (JSC::B3::PatchpointValue::dumpMeta): * b3/B3PatchpointValue.h: Added. (JSC::B3::PatchpointValue::accepts): (JSC::B3::PatchpointValue::PatchpointValue): * b3/B3PhaseScope.cpp: Added. (JSC::B3::PhaseScope::PhaseScope): (JSC::B3::PhaseScope::~PhaseScope): * b3/B3PhaseScope.h: Added. * b3/B3Procedure.cpp: Added. (JSC::B3::Procedure::Procedure): (JSC::B3::Procedure::~Procedure): (JSC::B3::Procedure::addBlock): (JSC::B3::Procedure::resetReachability): (JSC::B3::Procedure::dump): (JSC::B3::Procedure::blocksInPreOrder): (JSC::B3::Procedure::blocksInPostOrder): * b3/B3Procedure.h: Added. (JSC::B3::Procedure::size): (JSC::B3::Procedure::at): (JSC::B3::Procedure::operator[]): (JSC::B3::Procedure::iterator::iterator): (JSC::B3::Procedure::iterator::operator*): (JSC::B3::Procedure::iterator::operator++): (JSC::B3::Procedure::iterator::operator==): (JSC::B3::Procedure::iterator::operator!=): (JSC::B3::Procedure::iterator::findNext): (JSC::B3::Procedure::begin): (JSC::B3::Procedure::end): (JSC::B3::Procedure::ValuesCollection::ValuesCollection): (JSC::B3::Procedure::ValuesCollection::iterator::iterator): (JSC::B3::Procedure::ValuesCollection::iterator::operator*): (JSC::B3::Procedure::ValuesCollection::iterator::operator++): (JSC::B3::Procedure::ValuesCollection::iterator::operator==): (JSC::B3::Procedure::ValuesCollection::iterator::operator!=): (JSC::B3::Procedure::ValuesCollection::begin): (JSC::B3::Procedure::ValuesCollection::end): (JSC::B3::Procedure::ValuesCollection::size): (JSC::B3::Procedure::ValuesCollection::at): (JSC::B3::Procedure::ValuesCollection::operator[]): (JSC::B3::Procedure::values): (JSC::B3::Procedure::setLastPhaseName): (JSC::B3::Procedure::lastPhaseName): * b3/B3ProcedureInlines.h: Added. (JSC::B3::Procedure::add): * b3/B3ReduceStrength.cpp: Added. (JSC::B3::reduceStrength): * b3/B3ReduceStrength.h: Added. * b3/B3StackSlotKind.cpp: Added. (WTF::printInternal): * b3/B3StackSlotKind.h: Added. * b3/B3StackSlotValue.cpp: Added. (JSC::B3::StackSlotValue::~StackSlotValue): (JSC::B3::StackSlotValue::dumpMeta): * b3/B3StackSlotValue.h: Added. (JSC::B3::StackSlotValue::accepts): (JSC::B3::StackSlotValue::byteSize): (JSC::B3::StackSlotValue::kind): (JSC::B3::StackSlotValue::offsetFromFP): (JSC::B3::StackSlotValue::setOffsetFromFP): (JSC::B3::StackSlotValue::StackSlotValue): * b3/B3Stackmap.cpp: Added. (JSC::B3::Stackmap::Stackmap): (JSC::B3::Stackmap::~Stackmap): (JSC::B3::Stackmap::dump): * b3/B3Stackmap.h: Added. (JSC::B3::Stackmap::constrain): (JSC::B3::Stackmap::reps): (JSC::B3::Stackmap::clobber): (JSC::B3::Stackmap::clobbered): (JSC::B3::Stackmap::setGenerator): * b3/B3StackmapSpecial.cpp: Added. (JSC::B3::StackmapSpecial::StackmapSpecial): (JSC::B3::StackmapSpecial::~StackmapSpecial): (JSC::B3::StackmapSpecial::reportUsedRegisters): (JSC::B3::StackmapSpecial::extraClobberedRegs): (JSC::B3::StackmapSpecial::forEachArgImpl): (JSC::B3::StackmapSpecial::isValidImpl): (JSC::B3::StackmapSpecial::admitsStackImpl): (JSC::B3::StackmapSpecial::appendRepsImpl): (JSC::B3::StackmapSpecial::repForArg): * b3/B3StackmapSpecial.h: Added. * b3/B3SuccessorCollection.h: Added. (JSC::B3::SuccessorCollection::SuccessorCollection): (JSC::B3::SuccessorCollection::size): (JSC::B3::SuccessorCollection::at): (JSC::B3::SuccessorCollection::operator[]): (JSC::B3::SuccessorCollection::iterator::iterator): (JSC::B3::SuccessorCollection::iterator::operator*): (JSC::B3::SuccessorCollection::iterator::operator++): (JSC::B3::SuccessorCollection::iterator::operator==): (JSC::B3::SuccessorCollection::iterator::operator!=): (JSC::B3::SuccessorCollection::begin): (JSC::B3::SuccessorCollection::end): * b3/B3SwitchCase.cpp: Added. (JSC::B3::SwitchCase::dump): * b3/B3SwitchCase.h: Added. (JSC::B3::SwitchCase::SwitchCase): (JSC::B3::SwitchCase::operator bool): (JSC::B3::SwitchCase::caseValue): (JSC::B3::SwitchCase::target): (JSC::B3::SwitchCase::targetBlock): * b3/B3SwitchValue.cpp: Added. (JSC::B3::SwitchValue::~SwitchValue): (JSC::B3::SwitchValue::removeCase): (JSC::B3::SwitchValue::appendCase): (JSC::B3::SwitchValue::dumpMeta): (JSC::B3::SwitchValue::SwitchValue): * b3/B3SwitchValue.h: Added. (JSC::B3::SwitchValue::accepts): (JSC::B3::SwitchValue::numCaseValues): (JSC::B3::SwitchValue::caseValue): (JSC::B3::SwitchValue::caseValues): (JSC::B3::SwitchValue::fallThrough): (JSC::B3::SwitchValue::size): (JSC::B3::SwitchValue::at): (JSC::B3::SwitchValue::operator[]): (JSC::B3::SwitchValue::iterator::iterator): (JSC::B3::SwitchValue::iterator::operator*): (JSC::B3::SwitchValue::iterator::operator++): (JSC::B3::SwitchValue::iterator::operator==): (JSC::B3::SwitchValue::iterator::operator!=): (JSC::B3::SwitchValue::begin): (JSC::B3::SwitchValue::end): * b3/B3Type.cpp: Added. (WTF::printInternal): * b3/B3Type.h: Added. (JSC::B3::isInt): (JSC::B3::isFloat): (JSC::B3::pointerType): * b3/B3UpsilonValue.cpp: Added. (JSC::B3::UpsilonValue::~UpsilonValue): (JSC::B3::UpsilonValue::dumpMeta): * b3/B3UpsilonValue.h: Added. (JSC::B3::UpsilonValue::accepts): (JSC::B3::UpsilonValue::phi): (JSC::B3::UpsilonValue::UpsilonValue): * b3/B3UseCounts.cpp: Added. (JSC::B3::UseCounts::UseCounts): (JSC::B3::UseCounts::~UseCounts): * b3/B3UseCounts.h: Added. (JSC::B3::UseCounts::operator[]): * b3/B3Validate.cpp: Added. (JSC::B3::validate): * b3/B3Validate.h: Added. * b3/B3Value.cpp: Added. (JSC::B3::Value::~Value): (JSC::B3::Value::replaceWithIdentity): (JSC::B3::Value::replaceWithNop): (JSC::B3::Value::dump): (JSC::B3::Value::deepDump): (JSC::B3::Value::negConstant): (JSC::B3::Value::addConstant): (JSC::B3::Value::subConstant): (JSC::B3::Value::effects): (JSC::B3::Value::performSubstitution): (JSC::B3::Value::dumpMeta): (JSC::B3::Value::typeFor): * b3/B3Value.h: Added. (JSC::B3::DeepValueDump::DeepValueDump): (JSC::B3::DeepValueDump::dump): (JSC::B3::deepDump): * b3/B3ValueInlines.h: Added. (JSC::B3::Value::as): (JSC::B3::Value::isConstant): (JSC::B3::Value::hasInt32): (JSC::B3::Value::asInt32): (JSC::B3::Value::hasInt64): (JSC::B3::Value::asInt64): (JSC::B3::Value::hasInt): (JSC::B3::Value::asInt): (JSC::B3::Value::isInt): (JSC::B3::Value::hasIntPtr): (JSC::B3::Value::asIntPtr): (JSC::B3::Value::hasDouble): (JSC::B3::Value::asDouble): (JSC::B3::Value::stackmap): * b3/B3ValueRep.cpp: Added. (JSC::B3::ValueRep::dump): (WTF::printInternal): * b3/B3ValueRep.h: Added. (JSC::B3::ValueRep::ValueRep): (JSC::B3::ValueRep::reg): (JSC::B3::ValueRep::stack): (JSC::B3::ValueRep::stackArgument): (JSC::B3::ValueRep::constant): (JSC::B3::ValueRep::constantDouble): (JSC::B3::ValueRep::kind): (JSC::B3::ValueRep::operator bool): (JSC::B3::ValueRep::offsetFromFP): (JSC::B3::ValueRep::offsetFromSP): (JSC::B3::ValueRep::value): (JSC::B3::ValueRep::doubleValue): * b3/air: Added. * b3/air/AirAllocateStack.cpp: Added. (JSC::B3::Air::allocateStack): * b3/air/AirAllocateStack.h: Added. * b3/air/AirArg.cpp: Added. (JSC::B3::Air::Arg::dump): * b3/air/AirArg.h: Added. (JSC::B3::Air::Arg::isUse): (JSC::B3::Air::Arg::isDef): (JSC::B3::Air::Arg::typeForB3Type): (JSC::B3::Air::Arg::Arg): (JSC::B3::Air::Arg::imm): (JSC::B3::Air::Arg::imm64): (JSC::B3::Air::Arg::addr): (JSC::B3::Air::Arg::stack): (JSC::B3::Air::Arg::callArg): (JSC::B3::Air::Arg::isValidScale): (JSC::B3::Air::Arg::logScale): (JSC::B3::Air::Arg::index): (JSC::B3::Air::Arg::relCond): (JSC::B3::Air::Arg::resCond): (JSC::B3::Air::Arg::special): (JSC::B3::Air::Arg::operator==): (JSC::B3::Air::Arg::operator!=): (JSC::B3::Air::Arg::operator bool): (JSC::B3::Air::Arg::kind): (JSC::B3::Air::Arg::isTmp): (JSC::B3::Air::Arg::isImm): (JSC::B3::Air::Arg::isImm64): (JSC::B3::Air::Arg::isAddr): (JSC::B3::Air::Arg::isStack): (JSC::B3::Air::Arg::isCallArg): (JSC::B3::Air::Arg::isIndex): (JSC::B3::Air::Arg::isRelCond): (JSC::B3::Air::Arg::isResCond): (JSC::B3::Air::Arg::isSpecial): (JSC::B3::Air::Arg::isAlive): (JSC::B3::Air::Arg::tmp): (JSC::B3::Air::Arg::value): (JSC::B3::Air::Arg::pointerValue): (JSC::B3::Air::Arg::base): (JSC::B3::Air::Arg::hasOffset): (JSC::B3::Air::Arg::offset): (JSC::B3::Air::Arg::stackSlot): (JSC::B3::Air::Arg::scale): (JSC::B3::Air::Arg::isGPTmp): (JSC::B3::Air::Arg::isFPTmp): (JSC::B3::Air::Arg::isGP): (JSC::B3::Air::Arg::isFP): (JSC::B3::Air::Arg::hasType): (JSC::B3::Air::Arg::type): (JSC::B3::Air::Arg::isType): (JSC::B3::Air::Arg::isGPR): (JSC::B3::Air::Arg::gpr): (JSC::B3::Air::Arg::isFPR): (JSC::B3::Air::Arg::fpr): (JSC::B3::Air::Arg::isReg): (JSC::B3::Air::Arg::reg): (JSC::B3::Air::Arg::gpTmpIndex): (JSC::B3::Air::Arg::fpTmpIndex): (JSC::B3::Air::Arg::tmpIndex): (JSC::B3::Air::Arg::withOffset): (JSC::B3::Air::Arg::forEachTmpFast): (JSC::B3::Air::Arg::forEachTmp): (JSC::B3::Air::Arg::asTrustedImm32): (JSC::B3::Air::Arg::asTrustedImm64): (JSC::B3::Air::Arg::asTrustedImmPtr): (JSC::B3::Air::Arg::asAddress): (JSC::B3::Air::Arg::asBaseIndex): (JSC::B3::Air::Arg::asRelationalCondition): (JSC::B3::Air::Arg::asResultCondition): (JSC::B3::Air::Arg::isHashTableDeletedValue): (JSC::B3::Air::Arg::hash): (JSC::B3::Air::ArgHash::hash): (JSC::B3::Air::ArgHash::equal): * b3/air/AirBasicBlock.cpp: Added. (JSC::B3::Air::BasicBlock::addPredecessor): (JSC::B3::Air::BasicBlock::removePredecessor): (JSC::B3::Air::BasicBlock::replacePredecessor): (JSC::B3::Air::BasicBlock::dump): (JSC::B3::Air::BasicBlock::deepDump): (JSC::B3::Air::BasicBlock::BasicBlock): * b3/air/AirBasicBlock.h: Added. (JSC::B3::Air::BasicBlock::index): (JSC::B3::Air::BasicBlock::size): (JSC::B3::Air::BasicBlock::begin): (JSC::B3::Air::BasicBlock::end): (JSC::B3::Air::BasicBlock::at): (JSC::B3::Air::BasicBlock::last): (JSC::B3::Air::BasicBlock::appendInst): (JSC::B3::Air::BasicBlock::append): (JSC::B3::Air::BasicBlock::numSuccessors): (JSC::B3::Air::BasicBlock::successor): (JSC::B3::Air::BasicBlock::successors): (JSC::B3::Air::BasicBlock::successorBlock): (JSC::B3::Air::BasicBlock::successorBlocks): (JSC::B3::Air::BasicBlock::numPredecessors): (JSC::B3::Air::BasicBlock::predecessor): (JSC::B3::Air::BasicBlock::predecessors): (JSC::B3::Air::DeepBasicBlockDump::DeepBasicBlockDump): (JSC::B3::Air::DeepBasicBlockDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirCCallSpecial.cpp: Added. (JSC::B3::Air::CCallSpecial::CCallSpecial): (JSC::B3::Air::CCallSpecial::~CCallSpecial): (JSC::B3::Air::CCallSpecial::forEachArg): (JSC::B3::Air::CCallSpecial::isValid): (JSC::B3::Air::CCallSpecial::admitsStack): (JSC::B3::Air::CCallSpecial::reportUsedRegisters): (JSC::B3::Air::CCallSpecial::generate): (JSC::B3::Air::CCallSpecial::extraClobberedRegs): (JSC::B3::Air::CCallSpecial::dumpImpl): (JSC::B3::Air::CCallSpecial::deepDumpImpl): * b3/air/AirCCallSpecial.h: Added. * b3/air/AirCode.cpp: Added. (JSC::B3::Air::Code::Code): (JSC::B3::Air::Code::~Code): (JSC::B3::Air::Code::addBlock): (JSC::B3::Air::Code::addStackSlot): (JSC::B3::Air::Code::addSpecial): (JSC::B3::Air::Code::cCallSpecial): (JSC::B3::Air::Code::resetReachability): (JSC::B3::Air::Code::dump): (JSC::B3::Air::Code::findFirstBlockIndex): (JSC::B3::Air::Code::findNextBlockIndex): (JSC::B3::Air::Code::findNextBlock): * b3/air/AirCode.h: Added. (JSC::B3::Air::Code::newTmp): (JSC::B3::Air::Code::numTmps): (JSC::B3::Air::Code::callArgAreaSize): (JSC::B3::Air::Code::requestCallArgAreaSize): (JSC::B3::Air::Code::frameSize): (JSC::B3::Air::Code::setFrameSize): (JSC::B3::Air::Code::calleeSaveRegisters): (JSC::B3::Air::Code::size): (JSC::B3::Air::Code::at): (JSC::B3::Air::Code::operator[]): (JSC::B3::Air::Code::iterator::iterator): (JSC::B3::Air::Code::iterator::operator*): (JSC::B3::Air::Code::iterator::operator++): (JSC::B3::Air::Code::iterator::operator==): (JSC::B3::Air::Code::iterator::operator!=): (JSC::B3::Air::Code::begin): (JSC::B3::Air::Code::end): (JSC::B3::Air::Code::StackSlotsCollection::StackSlotsCollection): (JSC::B3::Air::Code::StackSlotsCollection::size): (JSC::B3::Air::Code::StackSlotsCollection::at): (JSC::B3::Air::Code::StackSlotsCollection::operator[]): (JSC::B3::Air::Code::StackSlotsCollection::iterator::iterator): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator*): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator++): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator==): (JSC::B3::Air::Code::StackSlotsCollection::iterator::operator!=): (JSC::B3::Air::Code::StackSlotsCollection::begin): (JSC::B3::Air::Code::StackSlotsCollection::end): (JSC::B3::Air::Code::stackSlots): (JSC::B3::Air::Code::SpecialsCollection::SpecialsCollection): (JSC::B3::Air::Code::SpecialsCollection::size): (JSC::B3::Air::Code::SpecialsCollection::at): (JSC::B3::Air::Code::SpecialsCollection::operator[]): (JSC::B3::Air::Code::SpecialsCollection::iterator::iterator): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator*): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator++): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator==): (JSC::B3::Air::Code::SpecialsCollection::iterator::operator!=): (JSC::B3::Air::Code::SpecialsCollection::begin): (JSC::B3::Air::Code::SpecialsCollection::end): (JSC::B3::Air::Code::specials): (JSC::B3::Air::Code::setLastPhaseName): (JSC::B3::Air::Code::lastPhaseName): * b3/air/AirFrequentedBlock.h: Added. * b3/air/AirGenerate.cpp: Added. (JSC::B3::Air::generate): * b3/air/AirGenerate.h: Added. * b3/air/AirGenerated.cpp: Added. * b3/air/AirGenerationContext.h: Added. * b3/air/AirHandleCalleeSaves.cpp: Added. (JSC::B3::Air::handleCalleeSaves): * b3/air/AirHandleCalleeSaves.h: Added. * b3/air/AirInsertionSet.cpp: Added. (JSC::B3::Air::InsertionSet::execute): * b3/air/AirInsertionSet.h: Added. (JSC::B3::Air::InsertionSet::InsertionSet): (JSC::B3::Air::InsertionSet::code): (JSC::B3::Air::InsertionSet::appendInsertion): (JSC::B3::Air::InsertionSet::insertInst): (JSC::B3::Air::InsertionSet::insert): * b3/air/AirInst.cpp: Added. (JSC::B3::Air::Inst::dump): * b3/air/AirInst.h: Added. (JSC::B3::Air::Inst::Inst): (JSC::B3::Air::Inst::opcode): (JSC::B3::Air::Inst::forEachTmpFast): (JSC::B3::Air::Inst::forEachTmp): * b3/air/AirInstInlines.h: Added. (JSC::B3::Air::ForEach<Tmp>::forEach): (JSC::B3::Air::ForEach<Arg>::forEach): (JSC::B3::Air::Inst::forEach): (JSC::B3::Air::Inst::hasSpecial): (JSC::B3::Air::Inst::extraClobberedRegs): (JSC::B3::Air::Inst::reportUsedRegisters): (JSC::B3::Air::isShiftValid): (JSC::B3::Air::isLshift32Valid): * b3/air/AirLiveness.h: Added. (JSC::B3::Air::Liveness::Liveness): (JSC::B3::Air::Liveness::liveAtHead): (JSC::B3::Air::Liveness::liveAtTail): (JSC::B3::Air::Liveness::LocalCalc::LocalCalc): (JSC::B3::Air::Liveness::LocalCalc::live): (JSC::B3::Air::Liveness::LocalCalc::takeLive): (JSC::B3::Air::Liveness::LocalCalc::execute): * b3/air/AirOpcode.opcodes: Added. * b3/air/AirPhaseScope.cpp: Added. (JSC::B3::Air::PhaseScope::PhaseScope): (JSC::B3::Air::PhaseScope::~PhaseScope): * b3/air/AirPhaseScope.h: Added. * b3/air/AirRegisterPriority.cpp: Added. (JSC::B3::Air::gprsInPriorityOrder): (JSC::B3::Air::fprsInPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirRegisterPriority.h: Added. (JSC::B3::Air::RegistersInPriorityOrder<GPRInfo>::inPriorityOrder): (JSC::B3::Air::RegistersInPriorityOrder<FPRInfo>::inPriorityOrder): (JSC::B3::Air::regsInPriorityOrder): * b3/air/AirSpecial.cpp: Added. (JSC::B3::Air::Special::Special): (JSC::B3::Air::Special::~Special): (JSC::B3::Air::Special::name): (JSC::B3::Air::Special::dump): (JSC::B3::Air::Special::deepDump): * b3/air/AirSpecial.h: Added. (JSC::B3::Air::DeepSpecialDump::DeepSpecialDump): (JSC::B3::Air::DeepSpecialDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirSpillEverything.cpp: Added. (JSC::B3::Air::spillEverything): * b3/air/AirSpillEverything.h: Added. * b3/air/AirStackSlot.cpp: Added. (JSC::B3::Air::StackSlot::setOffsetFromFP): (JSC::B3::Air::StackSlot::dump): (JSC::B3::Air::StackSlot::deepDump): (JSC::B3::Air::StackSlot::StackSlot): * b3/air/AirStackSlot.h: Added. (JSC::B3::Air::StackSlot::byteSize): (JSC::B3::Air::StackSlot::kind): (JSC::B3::Air::StackSlot::index): (JSC::B3::Air::StackSlot::alignment): (JSC::B3::Air::StackSlot::value): (JSC::B3::Air::StackSlot::offsetFromFP): (JSC::B3::Air::DeepStackSlotDump::DeepStackSlotDump): (JSC::B3::Air::DeepStackSlotDump::dump): (JSC::B3::Air::deepDump): * b3/air/AirTmp.cpp: Added. (JSC::B3::Air::Tmp::dump): * b3/air/AirTmp.h: Added. (JSC::B3::Air::Tmp::Tmp): (JSC::B3::Air::Tmp::gpTmpForIndex): (JSC::B3::Air::Tmp::fpTmpForIndex): (JSC::B3::Air::Tmp::operator bool): (JSC::B3::Air::Tmp::isGP): (JSC::B3::Air::Tmp::isFP): (JSC::B3::Air::Tmp::isGPR): (JSC::B3::Air::Tmp::isFPR): (JSC::B3::Air::Tmp::isReg): (JSC::B3::Air::Tmp::gpr): (JSC::B3::Air::Tmp::fpr): (JSC::B3::Air::Tmp::reg): (JSC::B3::Air::Tmp::hasTmpIndex): (JSC::B3::Air::Tmp::gpTmpIndex): (JSC::B3::Air::Tmp::fpTmpIndex): (JSC::B3::Air::Tmp::tmpIndex): (JSC::B3::Air::Tmp::isAlive): (JSC::B3::Air::Tmp::operator==): (JSC::B3::Air::Tmp::operator!=): (JSC::B3::Air::Tmp::isHashTableDeletedValue): (JSC::B3::Air::Tmp::hash): (JSC::B3::Air::Tmp::encodeGP): (JSC::B3::Air::Tmp::encodeFP): (JSC::B3::Air::Tmp::encodeGPR): (JSC::B3::Air::Tmp::encodeFPR): (JSC::B3::Air::Tmp::encodeGPTmp): (JSC::B3::Air::Tmp::encodeFPTmp): (JSC::B3::Air::Tmp::isEncodedGP): (JSC::B3::Air::Tmp::isEncodedFP): (JSC::B3::Air::Tmp::isEncodedGPR): (JSC::B3::Air::Tmp::isEncodedFPR): (JSC::B3::Air::Tmp::isEncodedGPTmp): (JSC::B3::Air::Tmp::isEncodedFPTmp): (JSC::B3::Air::Tmp::decodeGPR): (JSC::B3::Air::Tmp::decodeFPR): (JSC::B3::Air::Tmp::decodeGPTmp): (JSC::B3::Air::Tmp::decodeFPTmp): (JSC::B3::Air::TmpHash::hash): (JSC::B3::Air::TmpHash::equal): * b3/air/AirTmpInlines.h: Added. (JSC::B3::Air::Tmp::Tmp): * b3/air/AirValidate.cpp: Added. (JSC::B3::Air::validate): * b3/air/AirValidate.h: Added. * b3/air/opcode_generator.rb: Added. * b3/generate_pattern_matcher.rb: Added. * b3/testb3.cpp: Added. (JSC::B3::compileAndRun): (JSC::B3::test42): (JSC::B3::testLoad42): (JSC::B3::testArg): (JSC::B3::testAddArgs): (JSC::B3::testAddArgs32): (JSC::B3::testStore): (JSC::B3::testTrunc): (JSC::B3::testAdd1): (JSC::B3::testStoreAddLoad): (JSC::B3::testStoreAddAndLoad): (JSC::B3::testAdd1Uncommuted): (JSC::B3::testLoadOffset): (JSC::B3::testLoadOffsetNotConstant): (JSC::B3::testLoadOffsetUsingAdd): (JSC::B3::testLoadOffsetUsingAddNotConstant): (JSC::B3::run): (run): (main): * bytecode/CodeBlock.h: (JSC::CodeBlock::specializationKind): * jit/Reg.h: (JSC::Reg::index): (JSC::Reg::isSet): (JSC::Reg::operator bool): (JSC::Reg::isHashTableDeletedValue): (JSC::Reg::AllRegsIterable::iterator::iterator): (JSC::Reg::AllRegsIterable::iterator::operator*): (JSC::Reg::AllRegsIterable::iterator::operator++): (JSC::Reg::AllRegsIterable::iterator::operator==): (JSC::Reg::AllRegsIterable::iterator::operator!=): (JSC::Reg::AllRegsIterable::begin): (JSC::Reg::AllRegsIterable::end): (JSC::Reg::all): (JSC::Reg::invalid): (JSC::Reg::operator!): Deleted. * jit/RegisterAtOffsetList.cpp: (JSC::RegisterAtOffsetList::RegisterAtOffsetList): * jit/RegisterAtOffsetList.h: (JSC::RegisterAtOffsetList::clear): (JSC::RegisterAtOffsetList::size): (JSC::RegisterAtOffsetList::begin): (JSC::RegisterAtOffsetList::end): * jit/RegisterSet.h: (JSC::RegisterSet::operator==): (JSC::RegisterSet::hash): (JSC::RegisterSet::forEach): (JSC::RegisterSet::setAny): Source/WTF: * WTF.xcodeproj/project.pbxproj: * wtf/CMakeLists.txt: * wtf/HashSet.h: (WTF::copyToVector): (WTF::=): * wtf/ListDump.h: (WTF::PointerListDump::PointerListDump): (WTF::PointerListDump::dump): (WTF::MapDump::MapDump): (WTF::listDump): (WTF::pointerListDump): (WTF::sortedListDump): * wtf/MathExtras.h: (WTF::leftShiftWithSaturation): (WTF::rangesOverlap): * wtf/Platform.h: * wtf/ScopedLambda.h: Added. (WTF::scopedLambda): * wtf/SharedTask.h: (WTF::createSharedTask): Canonical link: https://commits.webkit.org/168813@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191705 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2015-10-28 23:57:04 +00:00
} // namespace WTF
using WTF::ScopedLambda;
using WTF::scopedLambda;
Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace and we should rewrite as much of the GC as needed to make this not a regression https://bugs.webkit.org/show_bug.cgi?id=160125 Reviewed by Geoffrey Garen and Keith Miller. JSTests: Most of the things I did properly covered by existing tests, but I found some simple cases of unshifting that had sketchy coverage. * stress/array-storage-array-unshift.js: Added. * stress/contiguous-array-unshift.js: Added. * stress/double-array-unshift.js: Added. * stress/int32-array-unshift.js: Added. Source/bmalloc: I needed to tryMemalign, so I added such a thing. * bmalloc/Allocator.cpp: (bmalloc::Allocator::allocate): (bmalloc::Allocator::tryAllocate): (bmalloc::Allocator::allocateImpl): * bmalloc/Allocator.h: * bmalloc/Cache.h: (bmalloc::Cache::tryAllocate): * bmalloc/bmalloc.h: (bmalloc::api::tryMemalign): Source/JavaScriptCore: In order to make the GC concurrent (bug 149432), we would either need to enable concurrent copying or we would need to not copy. Concurrent copying carries a 1-2% throughput overhead from the barriers alone. Considering that MarkedSpace does a decent job of avoiding fragmentation, it's unlikely that it's worth paying 1-2% throughput for copying. So, we want to get rid of copied space. This change moves copied space's biggest client over to marked space. Moving butterflies to marked space means having them use the new Auxiliary HeapCell allocation path. This is a fairly mechanical change, but it caused performance regressions everywhere, so this change also fixes MarkedSpace's performance issues. At a high level the mechanical changes are: - We use AuxiliaryBarrier instead of CopyBarrier. - We use tryAllocateAuxiliary instead of tryAllocateStorage. I got rid of the silly CheckedBoolean stuff, since it's so much more trouble than it's worth. - The JITs have to emit inlined marked space allocations instead of inline copy space allocations. - Everyone has to get used to zeroing their butterflies after allocation instead of relying on them being pre-zeroed by the GC. Copied space would zero things for you, while marked space doesn't. That's about 1/3 of this change. But this led to performance problems, which I fixed with optimizations that amounted to a major MarkedSpace rewrite: - MarkedSpace always causes internal fragmentation for array allocations because the vector length we choose when we resize usually leads to a cell size that doesn't correspond to any size class. I got around this by making array allocations usually round up vectorLength to the maximum allowed by the size class that we would have allocated in. Also, ensureLengthSlow() and friends first make sure that the requested length can't just be fulfilled with the current allocation size. This safeguard means that not every array allocation has to do size class queries. For example, the fast path of new Array(length) never does any size class queries, under the assumption that (1) the speed gained from avoiding an ensureLengthSlow() call, which then just changes the vectorLength by doing the size class query, is too small to offset the speed lost by doing the query on every allocation and (2) new Array(length) is a pretty good hint that resizing is not very likely. - Size classes in MarkedSpace were way too precise, which led to external fragmentation. This changes MarkedSpace size classes to use a linear progression for very small sizes followed by a geometric progression that naturally transitions to a hyperbolic progression. We want hyperbolic sizes when we get close to blockSize: for example the largest size we want is payloadSize / 2 rounded down, to ensure we get exactly two cells with minimal slop. The next size down should be payloadSize / 3 rounded down, and so on. After the last precise size (80 bytes), we proceed using a geometric progression, but round up each size to minimize slop at the end of the block. This naturally causes the geometric progression to turn hyperbolic for large sizes. The size class configuration happens at VM start-up, so it can be controlled with runtime options. I found that a base of 1.4 works pretty well. - Large allocations caused massive internal fragmentation, since the smallest large allocation had to use exactly blockSize, and the largest small allocation used blockSize / 2. The next size up - the first large allocation size to require two blocks - also had 50% internal fragmentation. This is because we required large allocations to be blockSize aligned, so that MarkedBlock::blockFor() would work. I decided to rewrite all of that. Cells no longer have to be owned by a MarkedBlock. They can now alternatively be owned by a LargeAllocation. These two things are abstracted as CellContainer. You know that a cell is owned by a LargeAllocation if the MarkedBlock::atomSize / 2 bit is set. Basically, large allocations are deliberately misaligned by 8 bytes. This actually works out great since (1) typed arrays won't use large allocations anyway since they have their own malloc fallback and (2) large array butterflies already have a 8 byte header, which means that the 8 byte base misalignment aligns the large array payload on a 16 byte boundary. I took extreme care to make sure that the isLargeAllocation bit checks are as rare as possible; for example, ExecState::vm() skips the check because we know that callees must be small allocations. It's also possible to use template tricks to do one check for cell container kind, and then invoke a function specialized for MarkedBlock or a function specialized for LargeAllocation. LargeAllocation includes stubs for all MarkedBlock methods that get used from functions that are template-specialized like this. That's mostly to speed up the GC marking code. Most other code can use CellContainer API or HeapCell API directly. That's another thing: HeapCell, the common base of JSCell and auxiliary allocations, is now smart enough to do a lot of things for you, like HeapCell::vm(), HeapCell::heap(), HeapCell::isLargeAllocation(), and HeapCell::cellContainer(). The size cutoff for large allocations is runtime-configurable, so long as you don't choose something so small that callees end up large. I found that 400 bytes is roughly optimal. This means that the MarkedBlock size classes end up being: 16, 32, 48, 64, 80, 112, 160, 224, 320 The next size class would have been 432, but that's above the 400 byte cutoff. All of this is configurable with --sizeClassProgression and --largeAllocationCutoff. You can see what size classes you end up with by doing --dumpSizeClasses=true. - Copied space uses 64KB blocks, while marked space used to use 16KB blocks. Allocating a lot of stuff in 16KB blocks was slower than allocating it in 64KB blocks because the GC had a lot of per-block overhead. I removed this overhead: It's now 2x faster to scan all MarkedBlocks because the list that contains the interesting meta-data is allocated on the side, for better locality during a sequential walk. It's no longer necessary to scan MarkedBlocks to find WeakSets, since the sets of WeakSets for eden scan and full scan are maintained on-the-fly. It's no longer necessary to scan all MarkedBlocks to clear mark bits because we now use versioned mark bits: to clear then, just increment the 64-bit heap version. It's no longer necessary to scan retired MarkedBlocks while allocating because marking retires them on-the-fly. It's no longer necessary to sort all blocks in the IncrementalSweeper's snapshot because blocks now know if they are in the snapshot. Put together, these optimizations allowed me to reduce block size to 16KB without losing much performance. There is some small perf loss on JetStream/splay, but not enough to hurt JetStream overall. I tried reducing block sizes further, to 4KB, since that is a progression on membuster. That's not possible yet, since there is still enough per-block overhead yet that such a reduction hurts JetStream too much. I filed a bug about improving this further: https://bugs.webkit.org/show_bug.cgi?id=161581. - Even after all of that, copying butterflies was still faster because it allowed us to skip sweeping dead space. A good GC allocates over dead bytes without explicitly freeing them, so the GC pause is O(size of live), not O(size of live + dead). O(dead) is usually much larger than O(live), especially in an eden collection. Copying satisfies this premise while mark+sweep does not. So, I invented a new kind of allocator: bump'n'pop. Previously, our MarkedSpace allocator was a freelist pop. That's simple and easy to inline but requires that we walk the block to build a free list. This means walking dead space. The new allocator allows totally free MarkedBlocks to simply set up a bump-pointer arena instead. The allocator is a hybrid of bump-pointer and freelist pop. It tries bump first. The bump pointer always bumps by cellSize, so the result of filling a block with bumping looks as if we had used freelist popping to fill it. Additionally, each MarkedBlock now has a bit to quickly tell if the block is entirely free. This makes sweeping O(1) whenever a MarkedBlock is completely empty, which is the common case because of the generational hypothesis: the number of objects that survive an eden collection is a tiny fraction of the number of objects that had been allocated, and this fraction is so small that there are typically fewer than one survivors per MarkedBlock. This change was enough to make this change a net win over tip-of-tree. - FTL now shares the same allocation fast paths as everything else, which is great, because bump'n'pop has gnarly control flow. We don't really want B3 to have to think about that control flow, since it won't be able to improve the machine code we write ourselves. GC fast paths are best written in assembly. So, I've empowered B3 to have even better support for Patchpoint terminals. It's now totally fine for a Patchpoint terminal to be non-Void. So, the new FTL allocation fast paths are just Patchpoint terminals that call through to AssemblyHelpers::emitAllocate(). B3 still reasons about things like constant-folding the size class calculation and constant-hoisting the allocator. Also, I gave the FTL the ability to constant-fold some allocator logic (in case we first assume that we're doing a variable-length allocation but then realize that the length is known). I think it makes sense to have constant folding rules in FTL::Output, or whatever the B3 IR builder is, since this makes lowering easier (you can constant fold during lowering more easily) and it reduces the amount of malloc traffic. In the future, we could teach B3 how to better constant-fold this code. That would require allowing loads to be constant-folded, which is doable but hella tricky. - It used to be that if a logical object allocation required two physical allocations (first the butterfly and then the cell), then the JIT would emit the code in such a way that a failure in the second fast path would cause us to forget the successful first physical allocation. This was pointlessly wasteful. It turns out that it's very cheap to devote a register to storing either the butterfly or null, because the butterfly register is anyway going to be free inside the first allocation. The only overhead here is zeroing the butterfly register. With that in place, we can just pass the butterfly-or-null to the slow path, which can then either allocate a butterfly or not. So now we never waste a successful allocation. This patch implements such a solution both in DFG (where it's easy to do this since we control registers already) and in FTL (where it's annoying, because mutable "butterfly-or-null" variables are hard to say in SSA; also I realized that we had code duplicated the JSArray allocation utility, so I deduplicated it). This came up because in one version of this patch, this wastage would resonate with some Kraken benchmark: the benchmark would always allocate N small things followed by one bigger thing. The problem was I accidentally adjusted the various fixed overheads in MarkedBlock in such a way that the JSObject size class, which both the small and big thing shared for their cell, could hold exactly N cells per MarkedBlock. Then the benchmark would always call slow path when it allocated the big thing. So, it would end up having to allocate the big thing's large butterfly twice, every single time! Ouch! - It used to be that we zeroed CopiedBlocks using memset, and so array allocations enjoyed amortization of the cost of zeroing. This doesn't work anymore - it's now up to the client of the allocator to initialize the object to whatever state they need. It used to be that we would just use a dumb loop. I initially changed this so that we would end up in memset for large allocations, but this didn't actually help performance that much. I got a much better result by playing with different memsets written in assembly. First I wrote one using non-temporal stores. That was a small speed-up over memset. Then I tried the classic "rep stos" approach, and holy cow that version was fast. It's a ~20% speed-up on array allocation microbenchmarks. So, this patch adds code paths to do "rep stos" on x86_64, or memset, or use a loop, as appropriate, for both "contiguous" arrays (holes are zero) and double arrays (holes are PNaN). Note that the JIT always emits either a loop or a flat slab of stores (if the size is known), but those paths in the JIT won't trigger for NewArrayWithSize() if the size is large, since that takes us to the operationNewArrayWithSize() slow path, which calls into JSArray::create(). That's why the optimizations here are all in JSArray::create() - that's the hot place for large arrays that need to be filled with holes. All of this put together gives us neutral perf on JetStream, membuster, and PLT3, a ~1% regression on Speedometer, and up to a 4% regression Kraken. The Kraken regression is because Kraken was allocating exactly 1024 element arrays at a rate of 400MB/sec. This is a best-case scenario for bump allocation. I think that we should fix bmalloc to make up the difference, but take the hit for now because it's a crazy corner case. By comparison, the alternative approach of using a copy barrier would have cost us 1-2%. That's the real apples-to-apples comparison if your premise is that we should have a concurrent GC. After we finish removing copied space, we will be barrier-ready for concurrent GC: we already have a marking barrier and we simply won't need a copying barrier. This change gets us there for the purposes of our benchmarks, since the remaining clients of copied space are not very important. On the other hand, if we keep copying, then getting barrier-ready would mean adding back the copy barrier, which costs more perf. We might get bigger speed-ups once we remove CopiedSpace altogether. That requires moving typed arrays and a few other weird things over to Aux MarkedSpace. This also includes some header sanitization. The introduction of AuxiliaryBarrier, HeapCell, and CellContainer meant that I had to include those files from everywhere. Fortunately, just including JSCInlines.h (instead of manually including the files that includes) is usually enough. So, I made most of JSC's cpp files include JSCInlines.h, which is something that we were already basically doing. In places where JSCInlines.h would be too much, I just included HeapInlines.h. This got weird, because we previously included HeapInlines.h from JSObject.h. That's bad because it led to some circular dependencies, so I fixed it - but that meant having to manually include HeapInlines.h from the places that previously got it implicitly via JSObject.h. But that led to more problems for some reason: I started getting build errors because non-JSC files were having trouble including Opcode.h. That's just silly, since Opcode.h is meant to be an internal JSC header. So, I made it an internal header and made it impossible to include it from outside JSC. This was a lot of work, but it was necessary to get the patch to build on all ports. It's also a net win. There were many places in WebCore that were transitively including a *ton* of JSC headers just because of the JSObject.h->HeapInlines.h edge and a bunch of dependency edges that arose from some public (for WebCore) JSC headers needing Interpreter.h or Opcode.h for bad reasons. * API/JSManagedValue.mm: (-[JSManagedValue initWithValue:]): * API/JSTypedArray.cpp: * API/ObjCCallbackFunction.mm: * API/tests/testapi.mm: (testObjectiveCAPI): (testWeakValue): Deleted. * CMakeLists.txt: * JavaScriptCore.xcodeproj/project.pbxproj: * Scripts/builtins/builtins_generate_combined_implementation.py: (BuiltinsCombinedImplementationGenerator.generate_secondary_header_includes): * Scripts/builtins/builtins_generate_internals_wrapper_implementation.py: (BuiltinsInternalsWrapperImplementationGenerator.generate_secondary_header_includes): * Scripts/builtins/builtins_generate_separate_implementation.py: (BuiltinsSeparateImplementationGenerator.generate_secondary_header_includes): * assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::JumpList::link): (JSC::AbstractMacroAssembler::JumpList::linkTo): * assembler/MacroAssembler.h: * assembler/MacroAssemblerARM64.h: (JSC::MacroAssemblerARM64::add32): * assembler/MacroAssemblerCodeRef.cpp: Added. (JSC::MacroAssemblerCodePtr::createLLIntCodePtr): (JSC::MacroAssemblerCodePtr::dumpWithName): (JSC::MacroAssemblerCodePtr::dump): (JSC::MacroAssemblerCodeRef::createLLIntCodeRef): (JSC::MacroAssemblerCodeRef::dump): * assembler/MacroAssemblerCodeRef.h: (JSC::MacroAssemblerCodePtr::createLLIntCodePtr): Deleted. (JSC::MacroAssemblerCodePtr::dumpWithName): Deleted. (JSC::MacroAssemblerCodePtr::dump): Deleted. (JSC::MacroAssemblerCodeRef::createLLIntCodeRef): Deleted. (JSC::MacroAssemblerCodeRef::dump): Deleted. * b3/B3BasicBlock.cpp: (JSC::B3::BasicBlock::appendBoolConstant): * b3/B3BasicBlock.h: * b3/B3DuplicateTails.cpp: * b3/B3StackmapGenerationParams.h: * b3/testb3.cpp: (JSC::B3::testPatchpointTerminalReturnValue): (JSC::B3::run): * bindings/ScriptValue.cpp: * bytecode/AdaptiveInferredPropertyValueWatchpointBase.cpp: * bytecode/BytecodeBasicBlock.cpp: * bytecode/BytecodeLivenessAnalysis.cpp: * bytecode/BytecodeUseDef.h: * bytecode/CallLinkInfo.cpp: (JSC::CallLinkInfo::callTypeFor): * bytecode/CallLinkInfo.h: (JSC::CallLinkInfo::callTypeFor): Deleted. * bytecode/CallLinkStatus.cpp: * bytecode/CodeBlock.cpp: (JSC::CodeBlock::finishCreation): (JSC::CodeBlock::clearLLIntGetByIdCache): (JSC::CodeBlock::predictedMachineCodeSize): * bytecode/CodeBlock.h: (JSC::CodeBlock::jitCodeMap): Deleted. (JSC::clearLLIntGetByIdCache): Deleted. * bytecode/ExecutionCounter.h: * bytecode/Instruction.h: * bytecode/LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp: (JSC::LLIntPrototypeLoadAdaptiveStructureWatchpoint::fireInternal): * bytecode/ObjectAllocationProfile.h: (JSC::ObjectAllocationProfile::isNull): (JSC::ObjectAllocationProfile::initialize): * bytecode/Opcode.h: (JSC::padOpcodeName): * bytecode/PolymorphicAccess.cpp: (JSC::AccessCase::generateImpl): (JSC::PolymorphicAccess::regenerate): * bytecode/PolymorphicAccess.h: * bytecode/PreciseJumpTargets.cpp: * bytecode/StructureStubInfo.cpp: * bytecode/StructureStubInfo.h: * bytecode/UnlinkedCodeBlock.cpp: (JSC::UnlinkedCodeBlock::vm): Deleted. * bytecode/UnlinkedCodeBlock.h: * bytecode/UnlinkedInstructionStream.cpp: * bytecode/UnlinkedInstructionStream.h: * dfg/DFGOperations.cpp: * dfg/DFGSpeculativeJIT.cpp: (JSC::DFG::SpeculativeJIT::emitAllocateRawObject): (JSC::DFG::SpeculativeJIT::compileMakeRope): (JSC::DFG::SpeculativeJIT::compileAllocatePropertyStorage): (JSC::DFG::SpeculativeJIT::compileReallocatePropertyStorage): * dfg/DFGSpeculativeJIT.h: (JSC::DFG::SpeculativeJIT::emitAllocateJSCell): (JSC::DFG::SpeculativeJIT::emitAllocateJSObject): * dfg/DFGSpeculativeJIT32_64.cpp: (JSC::DFG::SpeculativeJIT::compile): (JSC::DFG::SpeculativeJIT::compileAllocateNewArrayWithSize): * dfg/DFGSpeculativeJIT64.cpp: (JSC::DFG::SpeculativeJIT::compile): (JSC::DFG::SpeculativeJIT::compileAllocateNewArrayWithSize): * dfg/DFGStrengthReductionPhase.cpp: (JSC::DFG::StrengthReductionPhase::handleNode): * ftl/FTLAbstractHeapRepository.h: * ftl/FTLCompile.cpp: * ftl/FTLJITFinalizer.cpp: * ftl/FTLLowerDFGToB3.cpp: (JSC::FTL::DFG::LowerDFGToB3::compileCreateDirectArguments): (JSC::FTL::DFG::LowerDFGToB3::compileCreateRest): (JSC::FTL::DFG::LowerDFGToB3::allocateArrayWithSize): (JSC::FTL::DFG::LowerDFGToB3::compileNewArrayWithSize): (JSC::FTL::DFG::LowerDFGToB3::compileMakeRope): (JSC::FTL::DFG::LowerDFGToB3::compileMaterializeNewObject): (JSC::FTL::DFG::LowerDFGToB3::initializeArrayElements): (JSC::FTL::DFG::LowerDFGToB3::allocatePropertyStorageWithSizeImpl): (JSC::FTL::DFG::LowerDFGToB3::allocateHeapCell): (JSC::FTL::DFG::LowerDFGToB3::allocateCell): (JSC::FTL::DFG::LowerDFGToB3::allocateObject): (JSC::FTL::DFG::LowerDFGToB3::allocatorForSize): (JSC::FTL::DFG::LowerDFGToB3::allocateVariableSizedObject): (JSC::FTL::DFG::LowerDFGToB3::allocateJSArray): (JSC::FTL::DFG::LowerDFGToB3::compileAllocateArrayWithSize): Deleted. * ftl/FTLOutput.cpp: (JSC::FTL::Output::constBool): (JSC::FTL::Output::add): (JSC::FTL::Output::shl): (JSC::FTL::Output::aShr): (JSC::FTL::Output::lShr): (JSC::FTL::Output::zeroExt): (JSC::FTL::Output::equal): (JSC::FTL::Output::notEqual): (JSC::FTL::Output::above): (JSC::FTL::Output::aboveOrEqual): (JSC::FTL::Output::below): (JSC::FTL::Output::belowOrEqual): (JSC::FTL::Output::greaterThan): (JSC::FTL::Output::greaterThanOrEqual): (JSC::FTL::Output::lessThan): (JSC::FTL::Output::lessThanOrEqual): (JSC::FTL::Output::select): (JSC::FTL::Output::appendSuccessor): (JSC::FTL::Output::addIncomingToPhi): * ftl/FTLOutput.h: * ftl/FTLValueFromBlock.h: (JSC::FTL::ValueFromBlock::operator bool): (JSC::FTL::ValueFromBlock::ValueFromBlock): Deleted. * ftl/FTLWeightedTarget.h: (JSC::FTL::WeightedTarget::frequentedBlock): * heap/CellContainer.h: Added. (JSC::CellContainer::CellContainer): (JSC::CellContainer::operator bool): (JSC::CellContainer::isMarkedBlock): (JSC::CellContainer::isLargeAllocation): (JSC::CellContainer::markedBlock): (JSC::CellContainer::largeAllocation): * heap/CellContainerInlines.h: Added. (JSC::CellContainer::isMarked): (JSC::CellContainer::isMarkedOrNewlyAllocated): (JSC::CellContainer::noteMarked): (JSC::CellContainer::cellSize): (JSC::CellContainer::weakSet): (JSC::CellContainer::flipIfNecessary): * heap/ConservativeRoots.cpp: (JSC::ConservativeRoots::ConservativeRoots): (JSC::ConservativeRoots::~ConservativeRoots): (JSC::ConservativeRoots::grow): (JSC::ConservativeRoots::genericAddPointer): (JSC::ConservativeRoots::genericAddSpan): * heap/ConservativeRoots.h: (JSC::ConservativeRoots::roots): * heap/CopyToken.h: * heap/FreeList.cpp: Added. (JSC::FreeList::dump): * heap/FreeList.h: Added. (JSC::FreeList::FreeList): (JSC::FreeList::list): (JSC::FreeList::bump): (JSC::FreeList::operator==): (JSC::FreeList::operator!=): (JSC::FreeList::operator bool): (JSC::FreeList::allocationWillFail): (JSC::FreeList::allocationWillSucceed): * heap/GCTypeMap.h: Added. (JSC::GCTypeMap::operator[]): * heap/Heap.cpp: (JSC::Heap::Heap): (JSC::Heap::lastChanceToFinalize): (JSC::Heap::finalizeUnconditionalFinalizers): (JSC::Heap::markRoots): (JSC::Heap::copyBackingStores): (JSC::Heap::gatherStackRoots): (JSC::Heap::gatherJSStackRoots): (JSC::Heap::gatherScratchBufferRoots): (JSC::Heap::clearLivenessData): (JSC::Heap::visitSmallStrings): (JSC::Heap::visitConservativeRoots): (JSC::Heap::removeDeadCompilerWorklistEntries): (JSC::Heap::gatherExtraHeapSnapshotData): (JSC::Heap::removeDeadHeapSnapshotNodes): (JSC::Heap::visitProtectedObjects): (JSC::Heap::visitArgumentBuffers): (JSC::Heap::visitException): (JSC::Heap::visitStrongHandles): (JSC::Heap::visitHandleStack): (JSC::Heap::visitSamplingProfiler): (JSC::Heap::traceCodeBlocksAndJITStubRoutines): (JSC::Heap::converge): (JSC::Heap::visitWeakHandles): (JSC::Heap::updateObjectCounts): (JSC::Heap::clearUnmarkedExecutables): (JSC::Heap::deleteUnmarkedCompiledCode): (JSC::Heap::collectAllGarbage): (JSC::Heap::collect): (JSC::Heap::collectWithoutAnySweep): (JSC::Heap::collectImpl): (JSC::Heap::suspendCompilerThreads): (JSC::Heap::willStartCollection): (JSC::Heap::flushOldStructureIDTables): (JSC::Heap::flushWriteBarrierBuffer): (JSC::Heap::stopAllocation): (JSC::Heap::prepareForMarking): (JSC::Heap::reapWeakHandles): (JSC::Heap::pruneStaleEntriesFromWeakGCMaps): (JSC::Heap::sweepArrayBuffers): (JSC::MarkedBlockSnapshotFunctor::MarkedBlockSnapshotFunctor): (JSC::MarkedBlockSnapshotFunctor::operator()): (JSC::Heap::snapshotMarkedSpace): (JSC::Heap::deleteSourceProviderCaches): (JSC::Heap::notifyIncrementalSweeper): (JSC::Heap::writeBarrierCurrentlyExecutingCodeBlocks): (JSC::Heap::resetAllocators): (JSC::Heap::updateAllocationLimits): (JSC::Heap::didFinishCollection): (JSC::Heap::resumeCompilerThreads): (JSC::Zombify::visit): (JSC::Heap::forEachCodeBlockImpl): * heap/Heap.h: (JSC::Heap::allocatorForObjectWithoutDestructor): (JSC::Heap::allocatorForObjectWithDestructor): (JSC::Heap::allocatorForAuxiliaryData): (JSC::Heap::jitStubRoutines): (JSC::Heap::codeBlockSet): (JSC::Heap::storageAllocator): Deleted. * heap/HeapCell.h: (JSC::HeapCell::isZapped): Deleted. * heap/HeapCellInlines.h: Added. (JSC::HeapCell::isLargeAllocation): (JSC::HeapCell::cellContainer): (JSC::HeapCell::markedBlock): (JSC::HeapCell::largeAllocation): (JSC::HeapCell::heap): (JSC::HeapCell::vm): (JSC::HeapCell::cellSize): (JSC::HeapCell::allocatorAttributes): (JSC::HeapCell::destructionMode): (JSC::HeapCell::cellKind): * heap/HeapInlines.h: (JSC::Heap::heap): (JSC::Heap::isLive): (JSC::Heap::isMarked): (JSC::Heap::testAndSetMarked): (JSC::Heap::setMarked): (JSC::Heap::cellSize): (JSC::Heap::forEachCodeBlock): (JSC::Heap::allocateObjectOfType): (JSC::Heap::subspaceForObjectOfType): (JSC::Heap::allocatorForObjectOfType): (JSC::Heap::allocateAuxiliary): (JSC::Heap::tryAllocateAuxiliary): (JSC::Heap::tryReallocateAuxiliary): (JSC::Heap::isPointerGCObject): Deleted. (JSC::Heap::isValueGCObject): Deleted. * heap/HeapOperation.cpp: Added. (WTF::printInternal): * heap/HeapOperation.h: * heap/HeapUtil.h: Added. (JSC::HeapUtil::findGCObjectPointersForMarking): (JSC::HeapUtil::isPointerGCObjectJSCell): (JSC::HeapUtil::isValueGCObject): * heap/IncrementalSweeper.cpp: (JSC::IncrementalSweeper::sweepNextBlock): * heap/IncrementalSweeper.h: * heap/LargeAllocation.cpp: Added. (JSC::LargeAllocation::tryCreate): (JSC::LargeAllocation::LargeAllocation): (JSC::LargeAllocation::lastChanceToFinalize): (JSC::LargeAllocation::shrink): (JSC::LargeAllocation::visitWeakSet): (JSC::LargeAllocation::reapWeakSet): (JSC::LargeAllocation::flip): (JSC::LargeAllocation::isEmpty): (JSC::LargeAllocation::sweep): (JSC::LargeAllocation::destroy): (JSC::LargeAllocation::dump): * heap/LargeAllocation.h: Added. (JSC::LargeAllocation::fromCell): (JSC::LargeAllocation::cell): (JSC::LargeAllocation::isLargeAllocation): (JSC::LargeAllocation::heap): (JSC::LargeAllocation::vm): (JSC::LargeAllocation::weakSet): (JSC::LargeAllocation::clearNewlyAllocated): (JSC::LargeAllocation::isNewlyAllocated): (JSC::LargeAllocation::isMarked): (JSC::LargeAllocation::isMarkedOrNewlyAllocated): (JSC::LargeAllocation::isLive): (JSC::LargeAllocation::hasValidCell): (JSC::LargeAllocation::cellSize): (JSC::LargeAllocation::aboveLowerBound): (JSC::LargeAllocation::belowUpperBound): (JSC::LargeAllocation::contains): (JSC::LargeAllocation::attributes): (JSC::LargeAllocation::flipIfNecessary): (JSC::LargeAllocation::flipIfNecessaryConcurrently): (JSC::LargeAllocation::testAndSetMarked): (JSC::LargeAllocation::setMarked): (JSC::LargeAllocation::clearMarked): (JSC::LargeAllocation::noteMarked): (JSC::LargeAllocation::headerSize): * heap/MarkedAllocator.cpp: (JSC::MarkedAllocator::MarkedAllocator): (JSC::MarkedAllocator::isPagedOut): (JSC::MarkedAllocator::retire): (JSC::MarkedAllocator::filterNextBlock): (JSC::MarkedAllocator::setNextBlockToSweep): (JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl): (JSC::MarkedAllocator::tryAllocateWithoutCollecting): (JSC::MarkedAllocator::allocateSlowCase): (JSC::MarkedAllocator::tryAllocateSlowCase): (JSC::MarkedAllocator::allocateSlowCaseImpl): (JSC::blockHeaderSize): (JSC::MarkedAllocator::blockSizeForBytes): (JSC::MarkedAllocator::tryAllocateBlock): (JSC::MarkedAllocator::addBlock): (JSC::MarkedAllocator::removeBlock): (JSC::MarkedAllocator::stopAllocating): (JSC::MarkedAllocator::reset): (JSC::MarkedAllocator::lastChanceToFinalize): (JSC::MarkedAllocator::setFreeList): (JSC::isListPagedOut): Deleted. (JSC::MarkedAllocator::tryAllocateHelper): Deleted. (JSC::MarkedAllocator::tryPopFreeList): Deleted. (JSC::MarkedAllocator::tryAllocate): Deleted. (JSC::MarkedAllocator::allocateBlock): Deleted. * heap/MarkedAllocator.h: (JSC::MarkedAllocator::takeLastActiveBlock): (JSC::MarkedAllocator::offsetOfFreeList): (JSC::MarkedAllocator::offsetOfCellSize): (JSC::MarkedAllocator::tryAllocate): (JSC::MarkedAllocator::allocate): (JSC::MarkedAllocator::forEachBlock): (JSC::MarkedAllocator::offsetOfFreeListHead): Deleted. (JSC::MarkedAllocator::MarkedAllocator): Deleted. (JSC::MarkedAllocator::init): Deleted. (JSC::MarkedAllocator::stopAllocating): Deleted. * heap/MarkedBlock.cpp: (JSC::MarkedBlock::tryCreate): (JSC::MarkedBlock::Handle::Handle): (JSC::MarkedBlock::Handle::~Handle): (JSC::MarkedBlock::MarkedBlock): (JSC::MarkedBlock::Handle::specializedSweep): (JSC::MarkedBlock::Handle::sweep): (JSC::MarkedBlock::Handle::sweepHelperSelectScribbleMode): (JSC::MarkedBlock::Handle::sweepHelperSelectStateAndSweepMode): (JSC::MarkedBlock::Handle::unsweepWithNoNewlyAllocated): (JSC::SetNewlyAllocatedFunctor::SetNewlyAllocatedFunctor): (JSC::SetNewlyAllocatedFunctor::operator()): (JSC::MarkedBlock::Handle::stopAllocating): (JSC::MarkedBlock::Handle::lastChanceToFinalize): (JSC::MarkedBlock::Handle::resumeAllocating): (JSC::MarkedBlock::Handle::zap): (JSC::MarkedBlock::Handle::forEachFreeCell): (JSC::MarkedBlock::flipIfNecessary): (JSC::MarkedBlock::Handle::flipIfNecessary): (JSC::MarkedBlock::flipIfNecessarySlow): (JSC::MarkedBlock::flipIfNecessaryConcurrentlySlow): (JSC::MarkedBlock::clearMarks): (JSC::MarkedBlock::assertFlipped): (JSC::MarkedBlock::needsFlip): (JSC::MarkedBlock::Handle::needsFlip): (JSC::MarkedBlock::Handle::willRemoveBlock): (JSC::MarkedBlock::Handle::didConsumeFreeList): (JSC::MarkedBlock::markCount): (JSC::MarkedBlock::Handle::isEmpty): (JSC::MarkedBlock::clearHasAnyMarked): (JSC::MarkedBlock::noteMarkedSlow): (WTF::printInternal): (JSC::MarkedBlock::create): Deleted. (JSC::MarkedBlock::destroy): Deleted. (JSC::MarkedBlock::callDestructor): Deleted. (JSC::MarkedBlock::specializedSweep): Deleted. (JSC::MarkedBlock::sweep): Deleted. (JSC::MarkedBlock::sweepHelper): Deleted. (JSC::MarkedBlock::stopAllocating): Deleted. (JSC::MarkedBlock::clearMarksWithCollectionType): Deleted. (JSC::MarkedBlock::lastChanceToFinalize): Deleted. (JSC::MarkedBlock::resumeAllocating): Deleted. (JSC::MarkedBlock::didRetireBlock): Deleted. * heap/MarkedBlock.h: (JSC::MarkedBlock::VoidFunctor::returnValue): (JSC::MarkedBlock::CountFunctor::CountFunctor): (JSC::MarkedBlock::CountFunctor::count): (JSC::MarkedBlock::CountFunctor::returnValue): (JSC::MarkedBlock::Handle::hasAnyNewlyAllocated): (JSC::MarkedBlock::Handle::isOnBlocksToSweep): (JSC::MarkedBlock::Handle::setIsOnBlocksToSweep): (JSC::MarkedBlock::Handle::state): (JSC::MarkedBlock::needsDestruction): (JSC::MarkedBlock::handle): (JSC::MarkedBlock::Handle::block): (JSC::MarkedBlock::firstAtom): (JSC::MarkedBlock::atoms): (JSC::MarkedBlock::isAtomAligned): (JSC::MarkedBlock::Handle::cellAlign): (JSC::MarkedBlock::blockFor): (JSC::MarkedBlock::Handle::allocator): (JSC::MarkedBlock::Handle::heap): (JSC::MarkedBlock::Handle::vm): (JSC::MarkedBlock::vm): (JSC::MarkedBlock::Handle::weakSet): (JSC::MarkedBlock::weakSet): (JSC::MarkedBlock::Handle::shrink): (JSC::MarkedBlock::Handle::visitWeakSet): (JSC::MarkedBlock::Handle::reapWeakSet): (JSC::MarkedBlock::Handle::cellSize): (JSC::MarkedBlock::cellSize): (JSC::MarkedBlock::Handle::attributes): (JSC::MarkedBlock::attributes): (JSC::MarkedBlock::Handle::needsDestruction): (JSC::MarkedBlock::Handle::destruction): (JSC::MarkedBlock::Handle::cellKind): (JSC::MarkedBlock::Handle::markCount): (JSC::MarkedBlock::Handle::size): (JSC::MarkedBlock::atomNumber): (JSC::MarkedBlock::flipIfNecessary): (JSC::MarkedBlock::flipIfNecessaryConcurrently): (JSC::MarkedBlock::Handle::flipIfNecessary): (JSC::MarkedBlock::Handle::flipIfNecessaryConcurrently): (JSC::MarkedBlock::Handle::flipForEdenCollection): (JSC::MarkedBlock::assertFlipped): (JSC::MarkedBlock::Handle::assertFlipped): (JSC::MarkedBlock::isMarked): (JSC::MarkedBlock::testAndSetMarked): (JSC::MarkedBlock::Handle::isNewlyAllocated): (JSC::MarkedBlock::Handle::setNewlyAllocated): (JSC::MarkedBlock::Handle::clearNewlyAllocated): (JSC::MarkedBlock::Handle::isMarkedOrNewlyAllocated): (JSC::MarkedBlock::isMarkedOrNewlyAllocated): (JSC::MarkedBlock::Handle::isLive): (JSC::MarkedBlock::isAtom): (JSC::MarkedBlock::Handle::isLiveCell): (JSC::MarkedBlock::Handle::forEachCell): (JSC::MarkedBlock::Handle::forEachLiveCell): (JSC::MarkedBlock::Handle::forEachDeadCell): (JSC::MarkedBlock::Handle::needsSweeping): (JSC::MarkedBlock::Handle::isAllocated): (JSC::MarkedBlock::Handle::isMarked): (JSC::MarkedBlock::Handle::isFreeListed): (JSC::MarkedBlock::hasAnyMarked): (JSC::MarkedBlock::noteMarked): (WTF::MarkedBlockHash::hash): (JSC::MarkedBlock::FreeList::FreeList): Deleted. (JSC::MarkedBlock::allocator): Deleted. (JSC::MarkedBlock::heap): Deleted. (JSC::MarkedBlock::shrink): Deleted. (JSC::MarkedBlock::visitWeakSet): Deleted. (JSC::MarkedBlock::reapWeakSet): Deleted. (JSC::MarkedBlock::willRemoveBlock): Deleted. (JSC::MarkedBlock::didConsumeFreeList): Deleted. (JSC::MarkedBlock::markCount): Deleted. (JSC::MarkedBlock::isEmpty): Deleted. (JSC::MarkedBlock::destruction): Deleted. (JSC::MarkedBlock::cellKind): Deleted. (JSC::MarkedBlock::size): Deleted. (JSC::MarkedBlock::capacity): Deleted. (JSC::MarkedBlock::setMarked): Deleted. (JSC::MarkedBlock::clearMarked): Deleted. (JSC::MarkedBlock::isNewlyAllocated): Deleted. (JSC::MarkedBlock::setNewlyAllocated): Deleted. (JSC::MarkedBlock::clearNewlyAllocated): Deleted. (JSC::MarkedBlock::isLive): Deleted. (JSC::MarkedBlock::isLiveCell): Deleted. (JSC::MarkedBlock::forEachCell): Deleted. (JSC::MarkedBlock::forEachLiveCell): Deleted. (JSC::MarkedBlock::forEachDeadCell): Deleted. (JSC::MarkedBlock::needsSweeping): Deleted. (JSC::MarkedBlock::isAllocated): Deleted. (JSC::MarkedBlock::isMarkedOrRetired): Deleted. * heap/MarkedSpace.cpp: (JSC::MarkedSpace::initializeSizeClassForStepSize): (JSC::MarkedSpace::MarkedSpace): (JSC::MarkedSpace::~MarkedSpace): (JSC::MarkedSpace::lastChanceToFinalize): (JSC::MarkedSpace::allocate): (JSC::MarkedSpace::tryAllocate): (JSC::MarkedSpace::allocateLarge): (JSC::MarkedSpace::tryAllocateLarge): (JSC::MarkedSpace::sweep): (JSC::MarkedSpace::sweepLargeAllocations): (JSC::MarkedSpace::zombifySweep): (JSC::MarkedSpace::resetAllocators): (JSC::MarkedSpace::visitWeakSets): (JSC::MarkedSpace::reapWeakSets): (JSC::MarkedSpace::stopAllocating): (JSC::MarkedSpace::prepareForMarking): (JSC::MarkedSpace::resumeAllocating): (JSC::MarkedSpace::isPagedOut): (JSC::MarkedSpace::freeBlock): (JSC::MarkedSpace::freeOrShrinkBlock): (JSC::MarkedSpace::shrink): (JSC::MarkedSpace::clearNewlyAllocated): (JSC::VerifyMarked::operator()): (JSC::MarkedSpace::flip): (JSC::MarkedSpace::objectCount): (JSC::MarkedSpace::size): (JSC::MarkedSpace::capacity): (JSC::MarkedSpace::addActiveWeakSet): (JSC::MarkedSpace::didAddBlock): (JSC::MarkedSpace::didAllocateInBlock): (JSC::MarkedSpace::forEachAllocator): Deleted. (JSC::VerifyMarkedOrRetired::operator()): Deleted. (JSC::MarkedSpace::clearMarks): Deleted. * heap/MarkedSpace.h: (JSC::MarkedSpace::sizeClassToIndex): (JSC::MarkedSpace::indexToSizeClass): (JSC::MarkedSpace::version): (JSC::MarkedSpace::blocksWithNewObjects): (JSC::MarkedSpace::largeAllocations): (JSC::MarkedSpace::largeAllocationsNurseryOffset): (JSC::MarkedSpace::largeAllocationsOffsetForThisCollection): (JSC::MarkedSpace::largeAllocationsForThisCollectionBegin): (JSC::MarkedSpace::largeAllocationsForThisCollectionEnd): (JSC::MarkedSpace::largeAllocationsForThisCollectionSize): (JSC::MarkedSpace::forEachLiveCell): (JSC::MarkedSpace::forEachDeadCell): (JSC::MarkedSpace::allocatorFor): (JSC::MarkedSpace::destructorAllocatorFor): (JSC::MarkedSpace::auxiliaryAllocatorFor): (JSC::MarkedSpace::allocateWithoutDestructor): (JSC::MarkedSpace::allocateWithDestructor): (JSC::MarkedSpace::allocateAuxiliary): (JSC::MarkedSpace::tryAllocateAuxiliary): (JSC::MarkedSpace::forEachBlock): (JSC::MarkedSpace::forEachAllocator): (JSC::MarkedSpace::optimalSizeFor): (JSC::MarkedSpace::didAddBlock): Deleted. (JSC::MarkedSpace::didAllocateInBlock): Deleted. (JSC::MarkedSpace::objectCount): Deleted. (JSC::MarkedSpace::size): Deleted. (JSC::MarkedSpace::capacity): Deleted. * heap/SlotVisitor.cpp: (JSC::SlotVisitor::SlotVisitor): (JSC::SlotVisitor::didStartMarking): (JSC::SlotVisitor::reset): (JSC::SlotVisitor::append): (JSC::SlotVisitor::appendJSCellOrAuxiliary): (JSC::SlotVisitor::setMarkedAndAppendToMarkStack): (JSC::SlotVisitor::appendToMarkStack): (JSC::SlotVisitor::markAuxiliary): (JSC::SlotVisitor::noteLiveAuxiliaryCell): (JSC::SlotVisitor::visitChildren): * heap/SlotVisitor.h: * heap/WeakBlock.cpp: (JSC::WeakBlock::create): (JSC::WeakBlock::WeakBlock): (JSC::WeakBlock::visit): (JSC::WeakBlock::reap): * heap/WeakBlock.h: (JSC::WeakBlock::disconnectContainer): (JSC::WeakBlock::disconnectMarkedBlock): Deleted. * heap/WeakSet.cpp: (JSC::WeakSet::~WeakSet): (JSC::WeakSet::sweep): (JSC::WeakSet::shrink): (JSC::WeakSet::addAllocator): * heap/WeakSet.h: (JSC::WeakSet::container): (JSC::WeakSet::setContainer): (JSC::WeakSet::WeakSet): (JSC::WeakSet::visit): (JSC::WeakSet::shrink): Deleted. * heap/WeakSetInlines.h: (JSC::WeakSet::allocate): * inspector/InjectedScriptManager.cpp: * inspector/JSGlobalObjectInspectorController.cpp: * inspector/JSJavaScriptCallFrame.cpp: * inspector/ScriptDebugServer.cpp: * inspector/agents/InspectorDebuggerAgent.cpp: * interpreter/CachedCall.h: (JSC::CachedCall::CachedCall): * interpreter/Interpreter.cpp: (JSC::loadVarargs): (JSC::StackFrame::sourceID): Deleted. (JSC::StackFrame::sourceURL): Deleted. (JSC::StackFrame::functionName): Deleted. (JSC::StackFrame::computeLineAndColumn): Deleted. (JSC::StackFrame::toString): Deleted. * interpreter/Interpreter.h: (JSC::StackFrame::isNative): Deleted. * jit/AssemblyHelpers.h: (JSC::AssemblyHelpers::emitAllocateWithNonNullAllocator): (JSC::AssemblyHelpers::emitAllocate): (JSC::AssemblyHelpers::emitAllocateJSCell): (JSC::AssemblyHelpers::emitAllocateJSObject): (JSC::AssemblyHelpers::emitAllocateJSObjectWithKnownSize): (JSC::AssemblyHelpers::emitAllocateVariableSized): * jit/GCAwareJITStubRoutine.cpp: (JSC::GCAwareJITStubRoutine::GCAwareJITStubRoutine): * jit/JIT.cpp: (JSC::JIT::compileCTINativeCall): (JSC::JIT::link): * jit/JIT.h: (JSC::JIT::compileCTINativeCall): Deleted. * jit/JITExceptions.cpp: (JSC::genericUnwind): * jit/JITExceptions.h: * jit/JITOpcodes.cpp: (JSC::JIT::emit_op_new_object): (JSC::JIT::emitSlow_op_new_object): (JSC::JIT::emit_op_create_this): (JSC::JIT::emitSlow_op_create_this): * jit/JITOpcodes32_64.cpp: (JSC::JIT::emit_op_new_object): (JSC::JIT::emitSlow_op_new_object): (JSC::JIT::emit_op_create_this): (JSC::JIT::emitSlow_op_create_this): * jit/JITOperations.cpp: * jit/JITOperations.h: * jit/JITPropertyAccess.cpp: (JSC::JIT::emitWriteBarrier): * jit/JITThunks.cpp: * jit/JITThunks.h: * jsc.cpp: (functionDescribeArray): (main): * llint/LLIntData.cpp: (JSC::LLInt::Data::performAssertions): * llint/LLIntExceptions.cpp: * llint/LLIntThunks.cpp: * llint/LLIntThunks.h: * llint/LowLevelInterpreter.asm: * llint/LowLevelInterpreter.cpp: * llint/LowLevelInterpreter32_64.asm: * llint/LowLevelInterpreter64.asm: * parser/ModuleAnalyzer.cpp: * parser/NodeConstructors.h: * parser/Nodes.h: * profiler/ProfilerBytecode.cpp: * profiler/ProfilerBytecode.h: * profiler/ProfilerBytecodeSequence.cpp: * runtime/ArrayConventions.h: (JSC::indexingHeaderForArrayStorage): (JSC::baseIndexingHeaderForArrayStorage): (JSC::indexingHeaderForArray): Deleted. (JSC::baseIndexingHeaderForArray): Deleted. * runtime/ArrayPrototype.cpp: (JSC::arrayProtoFuncSplice): (JSC::concatAppendOne): (JSC::arrayProtoPrivateFuncConcatMemcpy): * runtime/ArrayStorage.h: (JSC::ArrayStorage::vectorLength): (JSC::ArrayStorage::totalSizeFor): (JSC::ArrayStorage::totalSize): (JSC::ArrayStorage::availableVectorLength): (JSC::ArrayStorage::optimalVectorLength): (JSC::ArrayStorage::sizeFor): Deleted. * runtime/AuxiliaryBarrier.h: Added. (JSC::AuxiliaryBarrier::AuxiliaryBarrier): (JSC::AuxiliaryBarrier::clear): (JSC::AuxiliaryBarrier::get): (JSC::AuxiliaryBarrier::slot): (JSC::AuxiliaryBarrier::operator bool): (JSC::AuxiliaryBarrier::setWithoutBarrier): * runtime/AuxiliaryBarrierInlines.h: Added. (JSC::AuxiliaryBarrier<T>::AuxiliaryBarrier): (JSC::AuxiliaryBarrier<T>::set): * runtime/Butterfly.h: * runtime/ButterflyInlines.h: (JSC::Butterfly::availableContiguousVectorLength): (JSC::Butterfly::optimalContiguousVectorLength): (JSC::Butterfly::createUninitialized): (JSC::Butterfly::growArrayRight): * runtime/ClonedArguments.cpp: (JSC::ClonedArguments::createEmpty): * runtime/CommonSlowPathsExceptions.cpp: * runtime/CommonSlowPathsExceptions.h: * runtime/DataView.cpp: * runtime/DirectArguments.h: * runtime/ECMAScriptSpecInternalFunctions.cpp: * runtime/Error.cpp: * runtime/Error.h: * runtime/ErrorInstance.cpp: * runtime/ErrorInstance.h: * runtime/Exception.cpp: * runtime/Exception.h: * runtime/GeneratorFrame.cpp: * runtime/GeneratorPrototype.cpp: * runtime/InternalFunction.cpp: (JSC::InternalFunction::InternalFunction): * runtime/IntlCollator.cpp: * runtime/IntlCollatorConstructor.cpp: * runtime/IntlCollatorPrototype.cpp: * runtime/IntlDateTimeFormat.cpp: * runtime/IntlDateTimeFormatConstructor.cpp: * runtime/IntlDateTimeFormatPrototype.cpp: * runtime/IntlNumberFormat.cpp: * runtime/IntlNumberFormatConstructor.cpp: * runtime/IntlNumberFormatPrototype.cpp: * runtime/IntlObject.cpp: * runtime/IteratorPrototype.cpp: * runtime/JSArray.cpp: (JSC::JSArray::tryCreateUninitialized): (JSC::JSArray::setLengthWritable): (JSC::JSArray::unshiftCountSlowCase): (JSC::JSArray::setLengthWithArrayStorage): (JSC::JSArray::appendMemcpy): (JSC::JSArray::setLength): (JSC::JSArray::pop): (JSC::JSArray::push): (JSC::JSArray::fastSlice): (JSC::JSArray::shiftCountWithArrayStorage): (JSC::JSArray::shiftCountWithAnyIndexingType): (JSC::JSArray::unshiftCountWithArrayStorage): (JSC::JSArray::fillArgList): (JSC::JSArray::copyToArguments): * runtime/JSArray.h: (JSC::createContiguousArrayButterfly): (JSC::createArrayButterfly): (JSC::JSArray::create): (JSC::JSArray::tryCreateUninitialized): Deleted. * runtime/JSArrayBufferView.h: * runtime/JSCInlines.h: * runtime/JSCJSValue.cpp: (JSC::JSValue::dumpInContextAssumingStructure): * runtime/JSCallee.cpp: (JSC::JSCallee::JSCallee): * runtime/JSCell.cpp: (JSC::JSCell::estimatedSize): * runtime/JSCell.h: (JSC::JSCell::cellStateOffset): Deleted. * runtime/JSCellInlines.h: (JSC::ExecState::vm): (JSC::JSCell::classInfo): (JSC::JSCell::callDestructor): (JSC::JSCell::vm): Deleted. * runtime/JSFunction.cpp: (JSC::JSFunction::create): (JSC::JSFunction::allocateAndInitializeRareData): (JSC::JSFunction::initializeRareData): (JSC::JSFunction::getOwnPropertySlot): (JSC::JSFunction::put): (JSC::JSFunction::deleteProperty): (JSC::JSFunction::defineOwnProperty): (JSC::JSFunction::setFunctionName): (JSC::JSFunction::reifyLength): (JSC::JSFunction::reifyName): (JSC::JSFunction::reifyLazyPropertyIfNeeded): (JSC::JSFunction::reifyBoundNameIfNeeded): * runtime/JSFunction.h: * runtime/JSFunctionInlines.h: (JSC::JSFunction::createWithInvalidatedReallocationWatchpoint): (JSC::JSFunction::JSFunction): * runtime/JSGenericTypedArrayViewInlines.h: (JSC::JSGenericTypedArrayView<Adaptor>::slowDownAndWasteMemory): * runtime/JSInternalPromise.cpp: * runtime/JSInternalPromiseConstructor.cpp: * runtime/JSInternalPromiseDeferred.cpp: * runtime/JSInternalPromisePrototype.cpp: * runtime/JSJob.cpp: * runtime/JSMapIterator.cpp: * runtime/JSModuleNamespaceObject.cpp: * runtime/JSModuleRecord.cpp: * runtime/JSObject.cpp: (JSC::JSObject::visitButterfly): (JSC::JSObject::notifyPresenceOfIndexedAccessors): (JSC::JSObject::createInitialIndexedStorage): (JSC::JSObject::createInitialUndecided): (JSC::JSObject::createInitialInt32): (JSC::JSObject::createInitialDouble): (JSC::JSObject::createInitialContiguous): (JSC::JSObject::createArrayStorage): (JSC::JSObject::createInitialArrayStorage): (JSC::JSObject::convertUndecidedToInt32): (JSC::JSObject::convertUndecidedToContiguous): (JSC::JSObject::convertUndecidedToArrayStorage): (JSC::JSObject::convertInt32ToDouble): (JSC::JSObject::convertInt32ToArrayStorage): (JSC::JSObject::convertDoubleToArrayStorage): (JSC::JSObject::convertContiguousToArrayStorage): (JSC::JSObject::putByIndexBeyondVectorLength): (JSC::JSObject::putDirectIndexBeyondVectorLength): (JSC::JSObject::getNewVectorLength): (JSC::JSObject::increaseVectorLength): (JSC::JSObject::ensureLengthSlow): (JSC::JSObject::growOutOfLineStorage): (JSC::JSObject::copyButterfly): Deleted. (JSC::JSObject::copyBackingStore): Deleted. * runtime/JSObject.h: (JSC::JSObject::globalObject): (JSC::JSObject::putDirectInternal): (JSC::JSObject::setStructureAndReallocateStorageIfNecessary): Deleted. * runtime/JSObjectInlines.h: * runtime/JSPromise.cpp: * runtime/JSPromiseConstructor.cpp: * runtime/JSPromiseDeferred.cpp: * runtime/JSPromisePrototype.cpp: * runtime/JSPropertyNameIterator.cpp: * runtime/JSScope.cpp: (JSC::JSScope::resolve): * runtime/JSScope.h: (JSC::JSScope::globalObject): (JSC::JSScope::vm): Deleted. * runtime/JSSetIterator.cpp: * runtime/JSStringIterator.cpp: * runtime/JSTemplateRegistryKey.cpp: * runtime/JSTypedArrayViewConstructor.cpp: * runtime/JSTypedArrayViewPrototype.cpp: * runtime/JSWeakMap.cpp: * runtime/JSWeakSet.cpp: * runtime/MapConstructor.cpp: * runtime/MapIteratorPrototype.cpp: * runtime/MapPrototype.cpp: * runtime/NativeErrorConstructor.cpp: * runtime/NativeStdFunctionCell.cpp: * runtime/Operations.h: (JSC::scribbleFreeCells): (JSC::scribble): * runtime/Options.h: * runtime/PropertyTable.cpp: * runtime/ProxyConstructor.cpp: * runtime/ProxyObject.cpp: * runtime/ProxyRevoke.cpp: * runtime/RegExp.cpp: (JSC::RegExp::match): (JSC::RegExp::matchConcurrently): (JSC::RegExp::matchCompareWithInterpreter): * runtime/RegExp.h: * runtime/RegExpConstructor.h: * runtime/RegExpInlines.h: (JSC::RegExp::matchInline): * runtime/RegExpMatchesArray.h: (JSC::tryCreateUninitializedRegExpMatchesArray): (JSC::createRegExpMatchesArray): * runtime/RegExpPrototype.cpp: (JSC::genericSplit): * runtime/RuntimeType.cpp: * runtime/SamplingProfiler.cpp: (JSC::SamplingProfiler::processUnverifiedStackTraces): * runtime/SetConstructor.cpp: * runtime/SetIteratorPrototype.cpp: * runtime/SetPrototype.cpp: * runtime/StackFrame.cpp: Added. (JSC::StackFrame::sourceID): (JSC::StackFrame::sourceURL): (JSC::StackFrame::functionName): (JSC::StackFrame::computeLineAndColumn): (JSC::StackFrame::toString): * runtime/StackFrame.h: Added. (JSC::StackFrame::isNative): * runtime/StringConstructor.cpp: * runtime/StringIteratorPrototype.cpp: * runtime/StructureInlines.h: (JSC::Structure::propertyTable): * runtime/TemplateRegistry.cpp: * runtime/TestRunnerUtils.cpp: (JSC::finalizeStatsAtEndOfTesting): * runtime/TestRunnerUtils.h: * runtime/TypeProfilerLog.cpp: * runtime/TypeSet.cpp: * runtime/VM.cpp: (JSC::VM::VM): (JSC::VM::ensureStackCapacityForCLoop): (JSC::VM::isSafeToRecurseSoftCLoop): * runtime/VM.h: * runtime/VMEntryScope.h: * runtime/VMInlines.h: (JSC::VM::ensureStackCapacityFor): (JSC::VM::isSafeToRecurseSoft): * runtime/WeakMapConstructor.cpp: * runtime/WeakMapData.cpp: * runtime/WeakMapPrototype.cpp: * runtime/WeakSetConstructor.cpp: * runtime/WeakSetPrototype.cpp: * testRegExp.cpp: (testOneRegExp): * tools/JSDollarVM.cpp: * tools/JSDollarVMPrototype.cpp: (JSC::JSDollarVMPrototype::isInObjectSpace): Source/WebCore: No new tests because no new WebCore behavior. Just rewiring #includes. * ForwardingHeaders/heap/HeapInlines.h: Added. * ForwardingHeaders/interpreter/Interpreter.h: Removed. * ForwardingHeaders/runtime/AuxiliaryBarrierInlines.h: Added. * Modules/indexeddb/IDBCursorWithValue.cpp: * Modules/indexeddb/client/TransactionOperation.cpp: * Modules/indexeddb/server/SQLiteIDBBackingStore.cpp: * Modules/indexeddb/server/UniqueIDBDatabase.cpp: * bindings/js/JSApplePayPaymentAuthorizedEventCustom.cpp: * bindings/js/JSApplePayPaymentMethodSelectedEventCustom.cpp: * bindings/js/JSApplePayShippingContactSelectedEventCustom.cpp: * bindings/js/JSApplePayShippingMethodSelectedEventCustom.cpp: * bindings/js/JSClientRectCustom.cpp: * bindings/js/JSDOMBinding.cpp: * bindings/js/JSDOMBinding.h: * bindings/js/JSDeviceMotionEventCustom.cpp: * bindings/js/JSDeviceOrientationEventCustom.cpp: * bindings/js/JSErrorEventCustom.cpp: * bindings/js/JSIDBCursorWithValueCustom.cpp: * bindings/js/JSIDBIndexCustom.cpp: * bindings/js/JSPopStateEventCustom.cpp: * bindings/js/JSWebGL2RenderingContextCustom.cpp: * bindings/js/JSWorkerGlobalScopeCustom.cpp: * bindings/js/WorkerScriptController.cpp: * contentextensions/ContentExtensionParser.cpp: * dom/ErrorEvent.cpp: * html/HTMLCanvasElement.cpp: * html/MediaDocument.cpp: * inspector/CommandLineAPIModule.cpp: * loader/EmptyClients.cpp: * page/CaptionUserPreferences.cpp: * page/Frame.cpp: * page/PageGroup.cpp: * page/UserContentController.cpp: * platform/mock/mediasource/MockBox.cpp: * testing/GCObservation.cpp: Source/WebKit2: Just rewiring some #includes. * UIProcess/ViewGestureController.cpp: * UIProcess/WebPageProxy.cpp: * UIProcess/WebProcessPool.cpp: * UIProcess/WebProcessProxy.cpp: * WebProcess/InjectedBundle/DOM/InjectedBundleRangeHandle.cpp: * WebProcess/Plugins/Netscape/JSNPObject.cpp: Source/WTF: I needed tryFastAlignedMalloc() so I added it. * wtf/FastMalloc.cpp: (WTF::tryFastAlignedMalloc): * wtf/FastMalloc.h: * wtf/ParkingLot.cpp: (WTF::ParkingLot::forEachImpl): (WTF::ParkingLot::forEach): Deleted. * wtf/ParkingLot.h: (WTF::ParkingLot::parkConditionally): (WTF::ParkingLot::unparkOne): (WTF::ParkingLot::forEach): * wtf/ScopedLambda.h: (WTF::scopedLambdaRef): * wtf/SentinelLinkedList.h: (WTF::SentinelLinkedList::forEach): (WTF::RawNode>::takeFrom): * wtf/SimpleStats.h: (WTF::SimpleStats::operator bool): (WTF::SimpleStats::operator!): Deleted. Tools: * DumpRenderTree/TestRunner.cpp: * DumpRenderTree/mac/DumpRenderTree.mm: (DumpRenderTreeMain): * Scripts/run-jsc-stress-tests: * TestWebKitAPI/Tests/WTF/Vector.cpp: (TestWebKitAPI::TEST): Canonical link: https://commits.webkit.org/179778@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@205462 268f45cc-cd09-0410-ab3c-d52691b4dbfc
2016-09-06 01:02:22 +00:00
using WTF::scopedLambdaRef;